When I call my batch program I get a null pointer exception on this line:
String fileName = ((Map<String, MccFtpFile>) ec.get(AbstractSetupTasklet.BATCH_FTP_FILES)).get("scacFile").getLocalFile();
Here is the entire class
// Writer
#Bean(name = "clf010Writer")
#StepScope
public FlatFileItemWriter<Clf010Item> clfWriter(#Value("#{jobExecutionContext}") Map<Object, Object> ec, //
#Qualifier("clf010LineAggregator") FormatterLineAggregator<Clf010Item> lineAgg) throws IOException {
#SuppressWarnings("unchecked")
String fileName = ((Map<String, MccFtpFile>) ec.get(AbstractSetupTasklet.BATCH_FTP_FILES)).get("scacFile").getLocalFile();
//Ensure the file can exist.
PrintWriter fos = getIoHarness().getFileOutputStream(fileName);
fos.close();
FlatFileItemWriter<Clf010Item> writer = new FlatFileItemWriter<>();
writer.setResource(new FileSystemResource(fileName));
writer.setLineAggregator(lineAgg);
return writer;
}
I can't seem to figure out the problem. My guess is it's looking for a file it can't find? If anyone has some insight or information I would appreciate it a lot.
I didn't realize, but in my setup tasklet I needed to declare a new ftp file :) thanks for the help
Related
//I want to add configuration to input topic
public void createTopics(Properties envProps) {
Map<String, Object> config = new HashMap<>();
config.put("bootstrap.servers", envProps.getProperty("bootstrap.servers"));
AdminClient client = AdminClient.create(config);
List<NewTopic> topics = new ArrayList<>();
topics.add(new NewTopic(
envProps.getProperty("input.topic.name") ,
Integer.parseInt(envProps.getProperty("input.topic.partitions")),
Short.parseShort(envProps.getProperty("input.topic.replication.factor"))));
topics.add(new NewTopic(
envProps.getProperty("output.topic.name"),
Integer.parseInt(envProps.getProperty("output.topic.partitions")),
Short.parseShort(envProps.getProperty("output.topic.replication.factor"))));
client.createTopics(topics);
client.close();
}
public Properties loadEnvProperties(String fileName) throws IOException {
Properties envProps = new Properties();
FileInputStream input = new FileInputStream(fileName);
envProps.load(input);
input.close();
return envProps;
}
I am able to create the topics. But, I want to configure topic in this code ie., cleanup.policy. How can I incorporate this here?
You can find the source code for kafka-topics --create here
As you can see, it calls NewTopic#configs with a Map
In that map, you would have (cleanup.policy, compact)
You could also call TopicCommand.main on your own, rather than re-write most of the existing code.
How is a JSONObject from Twitter4J converted to an IOFile for use with the JsonParser from BaseX?
So, looking to use Twitter4J to get either a file or stream.
The JsonParser looks to work with a File fine:
JsonParser
public JsonParser(IO source,
MainOptions opts,
JsonParserOptions jopts)
throws java.io.IOException
Constructor.
Parameters:
source - document source
opts - database options
jopts - parser options
Throws:
java.io.IOException - I/O exception
although other IO works:
org.basex.io
Class IO
java.lang.Object
org.basex.io.IO
Direct Known Subclasses:
IOContent, IOFile, IOStream, IOUrl
How is a File acquired from a JSONObject here?
Snippet using Twitter4J:
private JSONObject jsonOps(Status status) throws JSONException, BaseXException {
String string = TwitterObjectFactory.getRawJSON(status);
JSONObject json = new JSONObject(string);
String language = json.getString("lang");
log.fine(language);
return json;
}
Writing a JSONObject to file file from Twitter4J seems to work, at least the file looks correct when opened manually:
public void writeJsonToFile(String fileName) throws FileNotFoundException, UnsupportedEncodingException, IOException {
FileOutputStream fileOutputStream = null;
OutputStreamWriter outputStreamWriter = null;
BufferedWriter bufferedWriter = null;
fileOutputStream = new FileOutputStream(fileName);
outputStreamWriter = new OutputStreamWriter(fileOutputStream, "UTF-8");
bufferedWriter = new BufferedWriter(outputStreamWriter);
bufferedWriter.write(tweets.toString());
bufferedWriter.close();
outputStreamWriter.close();
fileOutputStream.close();
}
And once the file is written, it seems quite easy to "parse" the JSON, or at least instantiate a parser, as:
private void parseJsonFile(String fileName) throws IOException {
JsonParser jsonParser = new JsonParser(new IOFile(fileName), new MainOptions());
}
full code on github.
In no way is this a complete solution. In fact, it seems a kludge to write the JSON to a file at all -- but there it is.
Seems the only way to move JSON data from Twitter4J to BaseX, or at least most pragmatic. Other solutions appreciated.
Error in code:
If that train is removed then also show compiler error as below. I tried a lot to remove that and classify sentence as positive or negative.
An InputStreamFactory in not the same as an InputStream. Here is a simple bit of code that will create an InputStreamFactory for you. (You might say it is an InputStreamFactoryFactory :-) )
public static InputStreamFactory getInputStreamFactory(final File file) throws IOException{
return new InputStreamFactory() {
#Override
public InputStream createInputStream() throws IOException {
return new FileInputStream(file);
}
};
}
see javadocs: https://opennlp.apache.org/documentation/1.7.2/apidocs/opennlp-tools/opennlp/tools/util/InputStreamFactory.html
I am new to opennlp, I am getting Missing the manifest.properties! exception when i excute the following code,please tell me suggestion to avoid this.
public class PrePostProcessing_Peregrine {
public Map<String,Set<String>> btntMap;
public Map<String, String> fishMap;
public SentenceModel sModel;
public SentenceDetectorME sentDet;
public Map<String,Set<String>> topBottomTermSet;
public PrePostProcessing_Peregrine() throws IOException {
FileInputStream str=new FileInputStream("/home/rajendraprasad.yk/Desktop/data/en-sent.bin");
System.out.println(str+"===================>");
SentenceModel sModel = new SentenceModel(str);
System.out.println("===================model =================>"+sModel);
sentDet = new SentenceDetectorME(sModel);
System.out.println("===================>sentDet "+sentDet);
System.err.println("Sentence Detector Initialized");
Exception is:
opennlp.tools.util.InvalidFormatException: Missing the manifest.properties!
at opennlp.tools.util.model.BaseModel.validateArtifactMap(BaseModel.java:217)
at opennlp.tools.sentdetect.SentenceModel.validateArtifactMap(SentenceModel.java:78)
at opennlp.tools.util.model.BaseModel.<init>(BaseModel.java:142)
at opennlp.tools.sentdetect.SentenceModel.<init>(SentenceModel.java:73)
at com.molcon.Text_Mining.PrePostProcessing_Peregrine.<init>(PrePostProcessing_Peregrine.java:66)
at com.molcon.Text_Mining.TextMining.peregrineRun(TextMining.java:207)
at com.molcon.Text_Mining.TextMining.process_journals_for_Mining(TextMining.java:108)
I made mistake at FileInputStream modelIn = new FileInputStream("/home/rajendraprasad.yk/Desktop/data/en-sent.bin"); now I changed to InputStream modelIn = new FileInputStream("/home/rajendraprasad.yk/Desktop/data/en-sent.bin"); from this changes am not getting any exception but not able to load file from SentenceModel.
when i execute this line SentenceModel sModel = new SentenceModel(modelIn); am not getting any response,please help
For what I know there are two possible causes:
the model you are using is corrupted, try to download it again
the version of the model and of the OpenNLP library that you are using are not matching. As I read from the official website models are version specific, so you should try to understand if this is you case, and act accordingly.
If you check this constructor you will see that manifest.properties is not a file, it is a set of hard-coded properties:
Properties manifest = new Properties();
manifest.setProperty(MANIFEST_VERSION_PROPERTY, "1.0");
...
artifactMap.put("manifest.properties", manifest);
When you compare it to the InputStream constructor you can see that manifest.properties is nowhere to be found, only loadModel(in) which leads to conclusion that manifest.properties should be in the model file.
Why this works in a standalone application and not inside Tomcat requires some debugging.
As #5agado suggested, your model's version might be different than the library's.
I have an application which throws this error (happens only with xlsx-files):
java.lang.NullPointerException
at java.io.File.<init>(File.java:222)
at de.mpicbg.tds.core.ExcelLayout.openWorkbook(ExcelLayout.java:75)
The method 'openWorkbook' looks like this:
private void openWorkbook() throws IOException {
File excelFile = new File(fileName);
timestamp = excelFile.lastModified();
// open excel file
if (fileName.endsWith(".xlsx")) {
InputStream excelStream = new BufferedInputStream(new FileInputStream(excelFile));
this.workbook = new XSSFWorkbook((excelStream));
} else {
this.workbook = new HSSFWorkbook(new POIFSFileSystem(new FileInputStream(excelFile)));
}
}
If I execute everything in debug mode, everything goes smoothly and the error message does not appear. I don't have any explanation for this behaviour nor any idea how to fix it.
Can anybody help?
The error message says your fileName is null
If you cannot reproduce this when debugging you can add a log message at the start of you method.
System.out.println("The fileName is `" + fileName+"`");
Instead of using a field which may or may not be set, I suggest you use a parameter.
private void openWorkbook(String fileName) throws IOException {
assert fileName != null;