I have 100 sentences of test data. I am trying to run sentiment analysis on them but no matter what input String I am using, I am only getting a positive estimation of the input string. Each sentence gets a return value of 1.0. Any idea why this might be happening? Even if I use negative example inputs from the .txt file, the result is a positive value.
public class StartSentiment
{
public static DoccatModel model = null;
public static String[] analyzedTexts = {"Good win"};
public static void main(String[] args) throws IOException {
// begin of sentiment analysis
trainModel();
for(int i=0; i<analyzedTexts.length;i++){
classifyNewText(analyzedTexts[i]);}
}
private static String readFile(String pathname) throws IOException {
File file = new File(pathname);
StringBuilder fileContents = new StringBuilder((int)file.length());
Scanner scanner = new Scanner(file);
String lineSeparator = System.getProperty("line.separator");
try {
while(scanner.hasNextLine()) {
fileContents.append(scanner.nextLine() + lineSeparator);
}
return fileContents.toString();
} finally {
scanner.close();
}
}
public static void trainModel() {
MarkableFileInputStreamFactory dataIn = null;
try {
dataIn = new MarkableFileInputStreamFactory(
new File("src\\sentiment\\Results.txt"));
ObjectStream<String> lineStream = null;
lineStream = new PlainTextByLineStream(dataIn, StandardCharsets.UTF_8);
ObjectStream<DocumentSample> sampleStream = new DocumentSampleStream(lineStream);
TrainingParameters tp = new TrainingParameters();
tp.put(TrainingParameters.CUTOFF_PARAM, "1");
tp.put(TrainingParameters.ITERATIONS_PARAM, "100");
DoccatFactory df = new DoccatFactory();
model = DocumentCategorizerME.train("en", sampleStream, tp, df);
} catch (Exception e) {
e.printStackTrace();
} finally {
if (dataIn != null) {
try {
} catch (Exception e2) {
e2.printStackTrace();
}
}
}
}
public static void classifyNewText(String text) throws IOException{
DocumentCategorizerME myCategorizer = new DocumentCategorizerME(model);
double[] outcomes = myCategorizer.categorize(text.split(" ") );
String category = myCategorizer.getBestCategory(outcomes);
if (category.equalsIgnoreCase("1")){
System.out.print("The text is positive");
} else {
System.out.print("The text is negative");
}
}
I have a huge CSV file ~800 K records and using that data I have to form a POST request and make a rest call.
Initially I tried WITHOUT using threads but it is taking very long time to process it.
So I thought of using threads to speed up the process and followed below approach.
divided file into relatively smaller files (Tried with 3 files with approx ~5K data each file). (I Did this Manually Before passing to application)
Created 3 threads (traditional way by extending Thread class)
Reads each file with each thread and form HashMap with details required to form request and added it to ArrayList of POST Request
Send Post request to in loop
List item
Get the Response and add it to Response List
Both above approach takes too long to even complete half (>3 hrs). Might be one of them would not be good approach.
Could anyone please provide suggestion which helps me speeding up the process ? It is NOT mandatory to use threading.
Just to add that my code is at the consumer end and does not have much control on producer end.
MultiThreadedFileRead (Main class that Extends Thread)
class MultiThreadedFileRead extends Thread
{
public static final String EMPTY = "";
public static Logger logger = Logger.getLogger(MultiThreadedFileRead.class);
BufferedReader bReader = null;
String filename = null;
int Request_Num = 0;
HashMap<Integer, String> filteredSrcDataMap = null;
List<BrandRQ> brandRQList = null;
List<BrandRS> brandRSList = null;
ReadDLShipFile readDLShipFile = new ReadDLShipFile();
GenerateAndProcessPostBrandAPI generateAndProcessPostBrandAPI = new GenerateAndProcessPostBrandAPI();
MultiThreadedFileRead()
{
}
MultiThreadedFileRead(String fname) throws Exception
{
filename = fname;
Request_Num = 0;
List<BrandRQ> brandRQList = new ArrayList<BrandRQ>();
List<BrandRS> brandRSList = new ArrayList<BrandRS>();
this.start();
}
public void run()
{
logger.debug("File *** : " + this.filename);
//Generate Source Data Map
HashMap<Integer, String> filteredSrcDataMap = (HashMap<Integer, String>) readDLShipFile.readSourceFileData(this.filename);
generateAndProcessPostBrandAPI.generateAndProcessPostRequest(filteredSrcDataMap, this);
}
public static void main(String args[]) throws Exception
{
String environment = ""; // TO BE UNCOMMENTED
String outPutfileName = ""; // TO BE UNCOMMENTED
BrandRetrivalProperties brProps = BrandRetrivalProperties.getInstance();
if(args != null && args.length == 2 && DLPremiumUtil.isNotEmpty(args[0]) && DLPremiumUtil.isNotEmpty(args[1])) // TO BE UNCOMMENTED
{
environment = args[0].trim();
outPutfileName = args[1].trim();
ApplicationInitializer applicationInitializer = new ApplicationInitializer(environment);
applicationInitializer.initializeParams();
logger.debug("Environment : " + environment);
logger.debug("Filename : " + outPutfileName);
// TO BE UNCOMMENTED - START
}else{
System.out.println("Invalid parameters passed. Expected paramenters: Environment");
System.out.println("Sample Request: java -jar BrandRetrival.jar [prd|int] brand.dat");
System.exit(1);
}
MultiThreadedFileRead multiThreadedFileRead = new MultiThreadedFileRead();
List<String> listOfFileNames = multiThreadedFileRead.getFileNames(brProps);
logger.debug("List : " + listOfFileNames);
int totalFileSize = listOfFileNames.size();
logger.debug("totalFileSize : " + totalFileSize);
MultiThreadedFileRead fr[]=new MultiThreadedFileRead[totalFileSize];
logger.debug("Size of Array : " + fr.length);
for(int i = 0; i < totalFileSize; i++)
{
logger.debug("\nFile Getting Added to Array : " + brProps.getCountFilePath()+listOfFileNames.get(i));
fr[i]=new MultiThreadedFileRead(brProps.getCountFilePath()+listOfFileNames.get(i));
}
}
public List<String> getFileNames(BrandRetrivalProperties brProps)
{
MultiThreadedFileRead multiThreadedFileRead1 = new MultiThreadedFileRead();
BufferedReader br = null;
String line = "";
String filename = multiThreadedFileRead1.getCounterFileName(brProps.getCountFilePath(), brProps.getCountFilePathStartsWith());
List<String> fileNameList = null;
logger.debug("filename : " + filename);
if(filename != null)
{
fileNameList = new ArrayList<String>();
try{
br = new BufferedReader(new FileReader(brProps.getCountFilePath()+filename));
while ((line = br.readLine()) != null)
{
fileNameList.add(line);
}
} catch (FileNotFoundException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
} catch (IOException e2) {
// TODO Auto-generated catch block ;
e2.printStackTrace();
} catch (Exception e3) {
// TODO Auto-generated catch block
e3.printStackTrace();
} finally
{
if (br != null)
{
try
{
br.close();
} catch (IOException e4)
{
e4.printStackTrace();
} catch (Exception e5)
{
e5.printStackTrace();
}
}
}
}
return fileNameList;
}
// Get the Name of the file which has name of individual CSV files to read to form the request
public String getCounterFileName(String sourceFilePath, String sourceFileNameStartsWith)
{
String fileName = null;
if(sourceFilePath != null && !sourceFilePath.equals(EMPTY) &&
sourceFileNameStartsWith != null && !sourceFileNameStartsWith.equals(EMPTY))
{
File filePath = new File(sourceFilePath);
File[] listOfFiles = filePath.listFiles();
for (int i = 0; i < listOfFiles.length; i++) {
if (listOfFiles[i].isFile() && listOfFiles[i].getName().startsWith(sourceFileNameStartsWith)) {
fileName = listOfFiles[i].getName();
} else if (listOfFiles[i].isDirectory()) {
fileName = null;
}
}
}
return fileName;
}
}
GenerateAndProcessPostBrandAPI
public class GenerateAndProcessPostBrandAPI {
public static Logger logger = Logger.getLogger(GenerateAndProcessPostBrandAPI.class);
List<BrandRQ> brandRQList = new ArrayList<BrandRQ>();
List<BrandRS> brandRSList = new ArrayList<BrandRS>();
DLPremiumClientPost dlPremiumclientPost = new DLPremiumClientPost();
JSONRequestGenerator jsonRequestGenerator = new JSONRequestGenerator();
public List<BrandRQ> getBrandRequstList(Map<Integer, String> filteredSrcDataMap)
{
if(filteredSrcDataMap != null)
{
brandRQList = jsonRequestGenerator.getBrandRQList(filteredSrcDataMap, brandRQList);
}
return brandRQList;
}
public void generateAndProcessPostRequest(Map<Integer, String> filteredSrcDataMap, MultiThreadedFileRead multiThreadedFileRead)
{
if(filteredSrcDataMap != null)
{
brandRQList = jsonRequestGenerator.getBrandRQList(filteredSrcDataMap, brandRQList);
if(brandRQList != null && brandRQList.size() > 0)
{
BrandRetrivalProperties brProps = BrandRetrivalProperties.getInstance();
String postBrandsEndpoint = brProps.getPostBrandsEndPoint();
for (BrandRQ brandRQ : brandRQList)
{
multiThreadedFileRead.Request_Num = multiThreadedFileRead.Request_Num + 1;
logger.debug("Brand Request - " + multiThreadedFileRead.Request_Num + " : " + ObjectToJSONCovertor.converToJSON(brandRQ));
BrandRS brandResponse = dlPremiumclientPost.loadBrandApiResponseJSON(brandRQ, postBrandsEndpoint, DLPremiumUtil.getTransactionID(), BrandConstants.AUTH_TOKEN);
logger.debug("Response - " + multiThreadedFileRead.Request_Num + " : " + ObjectToJSONCovertor.converToJSON(brandResponse));
brandRSList.add(brandResponse);
}
}
}
}
}
I'm making networking program with Java. As the title, the object which server is trying to send is changed in client which receives it. I'm trying to change the object which exists in client before I receive the new one from server.
Here's my codes. First one is Server.sendIdea and second is Client.rcvIdea.
void sendIdea(Idea _idea) throws IOException {
objectOS.flush();
Idea idea = _idea;
//when I look into 'idea' it's fine
objectOS.writeObject(idea);
}
..
Idea rcvIdea(int _ideaCode) throws ClassNotFoundException, IOException {
objectOS.writeObject("sendIdea");
objectOS.writeObject(_ideaCode);
Idea returnValue = (Idea) objectIS.readObject();
//when I look into 'returnValue', it is not the one 'sendIdea' has sent.
return returnValue;
}
As you can see, sendIdea(Idea _idea) is sending an object from the class Idea by using writeObject method. And rcvIdea() is receiving the object by using readObject() method. (I'm sure you don't have to know about class Idea in detail). The client actually received some Ideas at start of this program by this method and there was no problem. But when I try to receive the same but slightly changed object Idea by this method, in Client class the object does not change, not like in Server class where the object which is going to be sent by sendIdea method is changed correctly. I tried about 5 hours to solve this problem. I checked all the codes line by line and found nothing. I'm pretty sure that writeObject or readObject method have problem. I tried objectOS.flush() to make clear of the stream and many other trials. I hope that I can find the problem. Below is some codes in my program
Client.class
package ideaOcean;
import java.awt.HeadlessException;
import java.io.IOException;
import java.io.InputStream;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.net.Socket;
import java.net.UnknownHostException;
import java.util.ArrayList;
import javax.swing.JOptionPane;
import data.Idea;
import data.Opinion;
import data.Profile;
public class Client {
Socket socket;
OutputStream os;
ObjectOutputStream objectOS;
InputStream is;
ObjectInputStream objectIS;
MainWindow mainWindow;
int idCode;
String email, password;
Profile myProfile;
ArrayList<Idea> myIdeas;
ArrayList<Opinion> myOpinions;
ArrayList<Integer> newIdeasCodes, hotIdeasCodes;
ArrayList<Idea> newIdeas, hotIdeas;
String command;
static final String SERVER_IP = "127.0.0.1";//
static final int SERVER_PORT_NUM = 5000;
public static void main(String[] args) {
Client client = new Client();
client.mainWindow = new MainWindow();
client.mainWindow.setVisible(true);
client.mainWindow.showLoginPg();
try {
while (!client.loginCheck()) {// login
continue;
}
} catch (HeadlessException | NumberFormatException | ClassNotFoundException | IOException e) {
e.printStackTrace();
}
System.out.println("[login complete]");
try {
client.myProfile = client.rcvProfile(client.idCode);// get myProfile
int i;
for (i = 0; i < client.myProfile.myIdeaCode.size(); i++) {
client.myIdeas.add(client.rcvIdea(client.myProfile.myIdeaCode.get(i)));
}
for (i = 0; i < client.myProfile.myOpinionCode.size(); i++) {
client.myOpinions.add(client.rcvOpinion(client.myProfile.myOpinionCode.get(i)));
}
// ***************************
} catch (ClassNotFoundException | IOException e1) {
e1.printStackTrace();
}
try {
client.rcvNewIdeas(12);
client.mainWindow.newOcean.floatingIdeas = client.newIdeas;
client.mainWindow.newOcean.arrangeFloatingPanels();
client.rcvHotIdeas(12);
client.mainWindow.hotOcean.floatingIdeas = client.hotIdeas;
client.mainWindow.hotOcean.arrangeFloatingPanels();
} catch (ClassNotFoundException | IOException e) {
e.printStackTrace();
}
client.mainWindow.setMyPg(client.myProfile, client.myIdeas, client.myOpinions);
client.mainWindow.showMainPg();
client.start();
}
public Client() {
try {
socket = new Socket(SERVER_IP, SERVER_PORT_NUM);
System.out.println("Connected to Server!");
os = socket.getOutputStream();
objectOS = new ObjectOutputStream(os);
is = socket.getInputStream();
objectIS = new ObjectInputStream(is);
myIdeas = new ArrayList<>();
myOpinions = new ArrayList<>();
newIdeasCodes = new ArrayList<>();
hotIdeasCodes = new ArrayList<>();
newIdeas = new ArrayList<>();
hotIdeas = new ArrayList<>();
} catch (UnknownHostException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
void start() {
while (true) {
try {
if (mainWindow.newBtnClicked) {
rcvNewIdeas(12);
mainWindow.newOcean.floatingIdeas = newIdeas;
mainWindow.newOcean.arrangeFloatingPanels();
mainWindow.newBtnClicked = false;
} else if (mainWindow.hotBtnClicked) {
rcvHotIdeas(12);
mainWindow.hotOcean.floatingIdeas = hotIdeas;
mainWindow.hotOcean.arrangeFloatingPanels();
mainWindow.hotBtnClicked = false;
} else if (mainWindow.newOcean.detailBtnClicked) {
updateIdeaDetailFrame(mainWindow.newOcean.clickedIdea);
mainWindow.newOcean.detailBtnClicked = false;
} else if (mainWindow.hotOcean.detailBtnClicked) {
updateIdeaDetailFrame(mainWindow.hotOcean.clickedIdea);
mainWindow.hotOcean.detailBtnClicked = false;
} else if (mainWindow.ideaDetailFrame.saveOpinionBtnClicked) {
sendOpinion(mainWindow.ideaDetailFrame.newOpinion);
updateIdeaDetailMainPanel(rcvIdea(mainWindow.ideaDetailFrame.idea.ideaCode));
mainWindow.ideaDetailFrame.saveOpinionBtnClicked = false;
} else if (mainWindow.writeIdeaPg.postIdeaBtnClicked) {
sendIdea(mainWindow.writeIdeaPg.thisIdea);
mainWindow.writeIdeaPg.postIdeaBtnClicked = false;
} else if (mainWindow.newOcean.plusBtnClicked) {
objectOS.writeObject("plusBtnClicked");
objectOS.writeObject(mainWindow.newOcean.plusMinusClickedIdeaCode);
mainWindow.newOcean.plusBtnClicked = false;
} else if (mainWindow.newOcean.minusBtnClicked) {
objectOS.writeObject("minusBtnClicked");
objectOS.writeObject(mainWindow.newOcean.plusMinusClickedIdeaCode);
mainWindow.newOcean.minusBtnClicked = false;
} else if (mainWindow.hotOcean.plusBtnClicked) {
objectOS.writeObject("plusBtnClicked");
objectOS.writeObject(mainWindow.hotOcean.plusMinusClickedIdeaCode);
mainWindow.hotOcean.plusBtnClicked = false;
} else if (mainWindow.hotOcean.minusBtnClicked) {
objectOS.writeObject("minusBtnClicked");
objectOS.writeObject(mainWindow.hotOcean.plusMinusClickedIdeaCode);
mainWindow.hotOcean.minusBtnClicked = false;
} else if (mainWindow.myBtnClicked) {
mainWindow.setMyPg(myProfile, myIdeas, myOpinions);
mainWindow.myBtnClicked = false;
}
} catch (ClassNotFoundException | IOException e) {
e.printStackTrace();
}
}
}
int i = 0;
Idea rcvIdea(int _ideaCode) throws ClassNotFoundException, IOException {
objectOS.writeObject("sendIdea");
objectOS.writeObject(_ideaCode);
Idea returnValue = (Idea) objectIS.readObject();
return returnValue;
}
Opinion rcvOpinion(int _opinionCode) throws ClassNotFoundException, IOException {
objectOS.writeObject("sendOpinion");
objectOS.writeObject(_opinionCode);
return (Opinion) objectIS.readObject();
}
Profile rcvProfile(int _idCode) throws IOException, ClassNotFoundException {
objectOS.writeObject("sendProfile");
objectOS.writeObject(_idCode);
return (Profile) objectIS.readObject();
}
void rcvNewIdeasCodes() throws ClassNotFoundException, IOException {
objectOS.writeObject("sendNewIdeasCodes");
newIdeasCodes = (ArrayList<Integer>) objectIS.readObject();
}
void rcvHotIdeasCodes() throws IOException, ClassNotFoundException {
objectOS.writeObject("sendHotIdeasCodes");
hotIdeasCodes = (ArrayList<Integer>) objectIS.readObject();
}
void rcvNewIdeas(int num) throws ClassNotFoundException, IOException {
int i;
rcvNewIdeasCodes();
newIdeas = new ArrayList<>();
if (num <= newIdeasCodes.size()) {
for (i = 0; i < num; i++) {
newIdeas.add(rcvIdea(newIdeasCodes.get(i)));
}
} else {
for (i = 0; i < newIdeasCodes.size(); i++) {
newIdeas.add(rcvIdea(newIdeasCodes.get(i)));
}
}
}
void rcvHotIdeas(int num) throws ClassNotFoundException, IOException {
int i;
rcvHotIdeasCodes();
hotIdeas = new ArrayList<>();
if (num <= hotIdeasCodes.size()) {
for (i = 0; i < num; i++) {
hotIdeas.add(rcvIdea(hotIdeasCodes.get(i)));
}
} else {
for (i = 0; i < hotIdeasCodes.size(); i++) {
hotIdeas.add(rcvIdea(hotIdeasCodes.get(i)));
}
}
}
void sendIdea(Idea _idea) throws IOException {
objectOS.writeObject("rcvIdea");
objectOS.writeObject(_idea);
}
void sendOpinion(Opinion _opinion) throws IOException {
objectOS.writeObject("rcvOpinion");
objectOS.writeObject(_opinion);
}
void sendProfile(Profile _profile) throws IOException {
objectOS.writeObject(_profile);
}
boolean loginCheck() throws HeadlessException, NumberFormatException, IOException, ClassNotFoundException {
objectOS.writeObject("loginCheck");// send command
while (!mainWindow.loginBtnClicked) {
continue;
}
mainWindow.loginBtnClicked = false;
email = mainWindow.emailField.getText().trim();
password = mainWindow.passwordField.getText().trim();
objectOS.writeObject(email);
objectOS.writeObject(password);
boolean valid;
valid = (boolean) objectIS.readObject();
if (valid == false) {
JOptionPane.showMessageDialog(mainWindow, "ID or Password is not correct");
mainWindow.emailField.setText("");
mainWindow.passwordField.setText("");
return false;
} else if (valid == true) {
idCode = (int) objectIS.readObject();
return true;
} else {
return false;
}
}
void updateIdeaDetailMainPanel(Idea clickedIdea) throws ClassNotFoundException, IOException {
ArrayList<Opinion> opinions = new ArrayList<>();
for (int j = 0; j < clickedIdea.opinionCode.size(); j++) {
opinions.add(rcvOpinion(clickedIdea.opinionCode.get(j)));
}
mainWindow.ideaDetailFrame.updateMainPanel(opinions);
}
void updateIdeaDetailFrame(Idea clickedIdea) throws ClassNotFoundException, IOException {
ArrayList<Opinion> opinions = new ArrayList<>();
for (int j = 0; j < clickedIdea.opinionCode.size(); j++) {
opinions.add(rcvOpinion(clickedIdea.opinionCode.get(j)));
}
mainWindow.ideaDetailFrame = new IdeaDetailFrame(clickedIdea, opinions);
mainWindow.ideaDetailFrame.setVisible(true);
}
}
Idea.class
package data;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.io.Serializable;
import java.util.ArrayList;
import java.util.Date;
public class Idea implements Serializable {
private static final long serialVersionUID = 123123L;
public int idCode;
public int ideaCode;
public int plus = 0, minus = 0;
public String ideaName;
public String oneLineExp;
public String explanation;
public ArrayList<Integer> opinionCode;
public Date date;
public MyCanvas image;
int hotDegree;
public Idea(int _idCode,int _ideaCode, String _ideaName, String _oneLineExp, String _explanation, MyCanvas _image) {
this(_idCode,_ideaName,_oneLineExp,_explanation,_image);
ideaCode = _ideaCode;
}
public Idea(int _idCode, String _ideaName, String _oneLineExp, String _explanation, MyCanvas _image) {
this(_idCode,_ideaName,_oneLineExp,_explanation);
image = _image;
}
public Idea(int _idCode, String _ideaName, String _oneLineExp, String _explanation){
idCode = _idCode;
oneLineExp = new String(_oneLineExp);
ideaName = new String(_ideaName);
explanation = new String(_explanation);
date = new Date();
opinionCode = new ArrayList<>();
}
public void saveIdea() {
FileOutputStream fos = null;
ObjectOutputStream oos = null;
try {
fos = new FileOutputStream("Idea.dat");
oos = new ObjectOutputStream(fos);
oos.writeObject(this);
} catch (IOException e1) {
System.out.println("e1");
}
}
void addOpinionCode(int _opinion) {
opinionCode.add(opinionCode.size(), _opinion);
}
public void incPlus() {
plus++;
}
public void incMinus() {
minus++;
}
public int setHotDegree() {
hotDegree = plus - minus + opinionCode.size() * 2;
return hotDegree;
}
}
Opinion.class
package data;
import java.io.Serializable;
import java.util.Date;
public class Opinion implements Serializable{
int idCode;
public int opinionCode;//the intrinsic code of this opinion
public int commentedIdeaCode;
public String opinion;
public Date date;
int plus, minus;
public Opinion(int _idCode,int _commentedIdeaCode, String _opinion){
idCode = _idCode;
commentedIdeaCode = _commentedIdeaCode;
opinion = new String(_opinion);
date = new Date();
plus = 0;
minus = 0;
}// Opinion(int _idCode,int _commentedIdeaCode, String _opinion)
public Opinion(int _idCode,int _opinionCode,int _commentedIdeaCode, String _opinion){
this(_idCode, _commentedIdeaCode, _opinion);
opinionCode = _opinionCode;
}//Opinion(int _idCode,int _opinionCode,int _commentedIdeaCode, String _opinion)
void incPlus(){
plus++;
}
void incMinus(){
minus++;
}
}
ObjectOutputStream creates a graph of all objects already serialized, and uses references to previously serialized objects. Therefore, when you serialize an Idea instance multiple times, each time after the first, a reference to the first serialization is written instead of the full object.
You can use ObjectOutputStream.reset() after each serialization. This discards the object graph and forces ObjectOutputStream to create new object serializations, even for objects it had seen before.
Your sendIdea method should therefore look like this:
void sendIdea(Idea _idea) throws IOException {
objectOS.flush();
objectOS.reset();
Idea idea = _idea;
objectOS.writeObject(idea);
}
Very importantly, please note that after reset(), all object references are serialized anew. So if you have a complex object graph, you may end up with object duplicates after deserialization.
If you want to share transitive references for an object that is to be serialized multiple times, look into ObjectOutputStream.writeUnshared() instead.
I'm trying to fix a bug in the code I wrote which convert a srt file to dxfp.xml. it works fine but when there is a special character such as an ampersand it throws an java.lang.NumberFormatException error. I tried to use the StringEscapeUtils function from apache commons to solve it. Can someone explain to me what it is I am missing here? thanks in advance!
public class SRT_TO_DFXP_Converter {
File input_file;
File output_file;
ArrayList<CaptionLine> node_list;
public SRT_TO_DFXP_Converter(File input_file, File output_file) {
this.input_file = input_file;
this.output_file = output_file;
this.node_list = new ArrayList<CaptionLine>();
}
class CaptionLine {
int line_num;
String begin_time;
String end_time;
ArrayList<String> content;
public CaptionLine(int line_num, String begin_time, String end_time,
ArrayList<String> content) {
this.line_num = line_num;
this.end_time = end_time;
this.begin_time = begin_time;
this.content = content;
}
public String toString() {
return (line_num + ": " + begin_time + " --> " + end_time + "\n" + content);
}
}
private void readSRT() {
BufferedReader bis = null;
FileReader fis = null;
String line = null;
CaptionLine node;
Integer line_num;
String[] time_split;
String begin_time;
String end_time;
try {
fis = new FileReader(input_file);
bis = new BufferedReader(fis);
do {
line = bis.readLine();
line_num = Integer.valueOf(line);
line = bis.readLine();
time_split = line.split(" --> ");
begin_time = time_split[0];
begin_time = begin_time.replace(',', '.');
end_time = time_split[1];
end_time.replace(',', '.');
ArrayList<String> content = new ArrayList<String>();
while (((line = bis.readLine()) != null)
&& (!(line.trim().equals("")))) {
content.add(StringEscapeUtils.escapeJava(line));
//if (StringUtils.isEmpty(line)) break;
}
node = new CaptionLine(line_num, begin_time, end_time, content);
node_list.add(node);
} while (line != null);
} catch (Exception e) {
System.out.println(e);
}
finally {
if (bis != null) {
try {
bis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
if (fis != null) {
try {
fis.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
private String convertToXML() {
StringBuffer dfxp = new StringBuffer();
dfxp.append("<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<tt xml:lang=\"en\" xmlns=\"http://www.w3.org/2006/04/ttaf1\" xmlns:tts=\"http://www.w3.org/2006/04/ttaf1#styling\">\n\t<head>\n\t\t<styling>\n\t\t\t<style id=\"1\" tts:backgroundColor=\"black\" tts:fontFamily=\"Arial\" tts:fontSize=\"14\" tts:color=\"white\" tts:textAlign=\"center\" tts:fontStyle=\"Plain\" />\n\t\t</styling>\n\t</head>\n\t<body>\n\t<div xml:lang=\"en\" style=\"default\">\n\t\t<div xml:lang=\"en\">\n");
for (int i = 0; i < node_list.size(); i++) {
dfxp.append("\t\t\t<p begin=\"" + node_list.get(i).begin_time + "\" ")
.append("end=\"" + node_list.get(i).end_time
+ "\" style=\"1\">");
for (int k = 0; k < node_list.get(i).content.size(); k++) {
dfxp.append("" + node_list.get(i).content.get(k));
}
dfxp.append("</p>\n");
}
dfxp.append("\t\t</div>\n\t</body>\n</tt>\n");
return dfxp.toString();
}
private void writeXML(String dfxp) {
BufferedWriter out = null;
try {
out = new BufferedWriter(new FileWriter(output_file));
out.write(dfxp);
out.close();
} catch (IOException e) {
System.out.println("Error Writing To File:"+ input_file +'\n');
e.printStackTrace();
} finally {
if (out != null) {
try {
out.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
public static void main(String[] args) {
if ((args.length < 2) || (args[1].equals("-h"))) {
System.out.println("\n<--- SRT to DFXP Converter Usage --->");
System.out
.println("Conversion: java -jar SRT_TO_DFXP.jar <input_file> <output_file> [-d]");
System.out
.println("Conversion REQUIRES a input file and output file");
System.out.println("[-d] Will Display XML Generated In Console");
System.out.println("Help: java -jar SRT_TO_DFXP.jar -h");
} else if (!(new File(args[0]).exists())) {
System.out.println("Error: Input SubScript File Does Not Exist\n");
} else {
SRT_TO_DFXP_Converter converter = new SRT_TO_DFXP_Converter(
new File(args[0]), new File(args[1]));
converter.readSRT();
String dfxp = converter.convertToXML();
if ((args.length == 3) && (args[2].equals("-d")))
System.out.println("\n" + dfxp + "\n");
converter.writeXML(dfxp);
System.out.println("Conversion Complete");
}
}
here's part of the srt file that it is throwing an error when exported and run as a jar file.
1
00:20:43,133 --> 00:20:50,599
literature and paper by Liversmith & Newman, and I think the point is well made that a host of factors
I am trying to parse a csv and map the fields to a POJO class. However I can see that the mapping is not achieved correctly.
I am trying to map the header from a POJO file to the csv.
public class CarCSVFileInputBean {
private long Id;
private String shortName;
private String Name;
private String Type;
private String Environment;
//getter and setters
}
Can someone please take a look at my code:
public class carCSVUtil {
private static Log log = LogFactory.getLog(carCSVUtil.class);
private static final List<String> fileHeaderFields = new ArrayList<String>();
private static final String UTF8CHARSET = "UTF-8";
static {
for (Field f : carCSVFileInputBean.class.getDeclaredFields()) {
fileHeaderFields.add(f.getName());
}
}
public static List<carCSVFileInputBean> getCSVInputList(InputStream inputStream) {
CSVReader reader = null;
List<carCSVFileInputBean> csvList = null;
carCSVFileInputBean inputRecord = null;
String[] header = null;
String[] row = null;
try {
reader = new CSVReader(new InputStreamReader(inputStream, UTF8CHARSET));
csvList = new ArrayList<carCSVFileInputBean>();
header = reader.readNext();
boolean isEmptyLine = true;
while ((row = reader.readNext()) != null) {
isEmptyLine = true;
if (!(row.length == 1 && StringUtils.isBlank(row[0]))) { // not an empty line, not even containing ','
inputRecord = new carCSVFileInputBean();
isEmptyLine = populateFields(inputRecord, header, row);
if (!isEmptyLine)
csvList.add(inputRecord);
}
}
} catch (IOException e) {
log.debug("IOException while accessing carCSVFileInputBean: " + e);
return null;
} catch (IllegalAccessException e) {
log.debug("IllegalAccessException while accessing carCSVFileInputBean: " + e);
return null;
} catch (InvocationTargetException e) {
log.debug("InvocationTargetException while copying carCSVFileInputBean properties: " + e);
return null;
} catch (Exception e) {
log.debug("Exception while parsing CSV file: " + e);
return null;
} finally {
try {
if (reader != null)
reader.close();
} catch (IOException ioe) {}
}
return csvList;
}
protected static boolean populateFields(carCSVFileInputBean inputRecord, String[] header, String[] row) throws IllegalAccessException, InvocationTargetException {
boolean isEmptyLine = true;
for (int i = 0; i < row.length; i++) {
String val = row[i];
if (!StringUtils.isBlank(val)) {
BeanUtilsBean.getInstance().copyProperty(inputRecord, header[i], val);
isEmptyLine = false;
}
}
return isEmptyLine;
}
}
I found the solution - the headers in the csv file are expected to begin with a lowercase.