We Keep Coding

Java Error while CSV to Json conversion. forType and readAll method

import java.io.*;
import java.util.*;
import com.fasterxml.jackson.databind.MappingIterator;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.dataformat.csv.CsvMapper;
import com.fasterxml.jackson.dataformat.csv.CsvSchema;
import com.fasterxml.jackson.databind.MappingIterator.*;
public class CsvtoJson {
public static void main(String args[]) throws Exception {
File input = new File("input.csv");
try {
CsvSchema csv = CsvSchema.emptySchema().withHeader();
CsvMapper csvMapper = new CsvMapper();
MappingIterator<Map<?, ?>> mappingIterator = csvMapper.reader().forType(Map.class).with(csv).readValues(input);
List<Map<?, ?>> list = mappingIterator.readAll();
System.out.println(list);
} catch(Exception e) {
e.printStackTrace();
}
}
}
Compilation Error:-
The method forType(Class) is undefined for the type ObjectReader
The method readAll() is undefined for the type MappingIterator<Map>
Trying to convert a big CSV file into JSON with Jackson library but I am stuck with this error. All jars all correctly added in the build path.

The code works fine for me (JDK13), I think you have dependency problem. Please make sure you have jars for these dependencies(versions are not much important):
com.fasterxml.jackson.core:jackson-annotations:2.11.1
com.fasterxml.jackson.core:jackson-core:2.11.1
com.fasterxml.jackson.core:jackson-databind:2.11.1
com.fasterxml.jackson.dataformat:jackson-dataformat-csv:2.11.1
Note: It is much easy to create a maven project with an IDE. When you create a maven project add this dependecy to your maven pom file.
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-csv</artifactId>
<version>2.11.1</version>
</dependency>

Properties file to complex JSON string [Java/Spring]

I'm creating a Spring application on backend and my main goal is to manage properties (add/update/delete) in *.properties file. I want to convert this file to JSON and then manipulate it from UI application.
Is there any possibility to convert structure like this:
a.x=1
a.y=2
b.z=3
To JSON like this:
{
"a": {
"x": 1,
"y": 2
},
"b": {
"z": 3
}
}
I found solution to use GSON library, but it creates for me flat structure, not hierarchical, code I used:
Properties props = new Properties();
try (FileInputStream in = new FileInputStream(classPathResource.getFile())) {
props.load(in);
}
String json = new GsonBuilder().enableComplexMapKeySerialization().create().toJson(props);
Is here someone who was facing same problem and maybe found a working project for this? Maybe GSON library can do that?

This solution does involve loads of work, but you will get what you want to achieve using the below code, basically, the idea is to split the key based on the single dot and then create a JsonObject if the same first key is found.
import java.io.File;
import java.io.FileReader;
import java.io.IOException;
import java.util.Iterator;
import java.util.Map.Entry;
import java.util.Properties;
import org.json.JSONObject;
import com.fasterxml.jackson.annotation.JsonIgnore;
public class SOTest {
public static void main(String[] args) throws IOException {
JSONObject jsonObject = new JSONObject();
FileReader fileReader = new FileReader(new File("C:\\Usrc\\main\\java\\Sample.properties"));
Properties properties = new Properties();
properties.load(fileReader);
Iterator<Entry<Object, Object>> iterator = properties.entrySet().iterator();
while (iterator.hasNext()) {
Entry<Object, Object> entry = iterator.next();
String value = (String) entry.getKey();
String[] values = value.split("\\.");
JSONObject opt = jsonObject.optJSONObject(values[0]);
if(opt!=null) {
opt.put(values[1],entry.getValue());
}else {
JSONObject object = new JSONObject();
object.put(values[1], entry.getValue());
jsonObject.put(values[0], object);
}
}
System.out.println(jsonObject.toString());
}
}
Output
{"a":{"x":"1","y":"3"},"b":{"z":"10"}}

convert json data to csv data using java

If our JSON data consists only of flat, one-dimensional data,
how can convert those data by iterating over your outer objects and building a CSV line by concating the string values of your inner objects.
Could anyone provide with example please?

You can try this :
import org.json.CDL;
import org.json.JSONArray;
import org.json.JSONException;
import org.json.JSONObject;
public class toCSV {
public static void main(String args[]){
String jsonString = "{\"infile\": [{\"field1\": 11,\"field2\": 12,\"field3\": 13},{\"field1\": 21,\"field2\": 22,\"field3\": 23},{\"field1\": 31,\"field2\": 32,\"field3\": 33}]}"
JSONObject output = new JSONObject(jsonString);
JSONArray docs = response.getJSONArray("infile");
File file=new File("C:/JsontoCSVExample.csv");
String csv = CDL.toString(docs);
FileUtils.writeStringToFile(file, csv);
}
}
The maven dependency was like,
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20090211</version>
</dependency>
Also, You can take ref. from here

How to parse items from JSON string in Java

I have this code and I tried to getting items from this JSON string but it failed.
I'm parsing the Json string from remote host.
package selectDB;
import java.io.*;
import java.net.URL;
import java.net.URLConnection;
import java.sql.*;
import java.text.DateFormat;
import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.Timer;
import java.util.TimerTask;
import org.json.simple.*;
public class selectDB
{
public static void main(String[] args) throws IOException, ParseException
{
String s = "";
URL u = new URL("http://192.168.3.1/android/select.php");
URLConnection c = u.openConnection();
InputStream r = c.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(r));
for(String line; (line = reader.readLine()) != null;)
{
s+=line;
}
System.out.println(s);
}
}
the result is
{"result" : "true" , "messages" : [{"id":"866343023633578","latitute":"27","longitude":"31","number_phone":"01113171374"},{"id":"352168066354050","latitute":"27","longitude":"31","number_phone":"202222"},{"id":"50","latitute":"50","longitude":"100","number_phone":"50"},{"id":"110","latitute":"50","longitude":"50","number_phone":"110"},{"id":"120","latitute":"27","longitude":"31","number_phone":"120"},{"id":"130","latitute":"28","longitude":"29","number_phone":"120"},{"id":"140","latitute":"30","longitude":"40","number_phone":"140"},{"id":"800","latitute":"60","longitude":"30","number_phone":"800"},{"id":"353629054230064","latitute":"70","longitude":"80","number_phone":"120"}]}
Please help!

U can use the JsonReader class.
try (JsonReader in = Json.createReader(r)) {
JsonObject jsonObject= in.readObject();
YourObject obj = new YourObject();
obj.setSomething(jsonObject.getString("something", null));
// "something" is the key in the json file, null is the default
// when "something" was not found
} catch (JsonException | ClassCastException ex) {
throw new BadRequestException("Invalid Json Input");
}

you can use the Google Library GSON as well, it is easy to use and self explaining.
https://code.google.com/p/google-gson/
Gson Goals
Provide simple toJson() and fromJson() methods to convert Java objects to JSON and vice-versa
Allow pre-existing unmodifiable objects to be converted to and from JSON
Extensive support of Java Generics
Allow custom representations for objects
Support arbitrarily complex objects (with deep inheritance hierarchies and extensive use of generic types)

How to use OpenNLP with Java?

I want to POStag an English sentence and do some processing. I would like to use openNLP. I have it installed
When I execute the command
I:\Workshop\Programming\nlp\opennlp-tools-1.5.0-bin\opennlp-tools-1.5.0>java -jar opennlp-tools-1.5.0.jar POSTagger models\en-pos-maxent.bin < Text.txt
It gives output POSTagging the input in Text.txt
Loading POS Tagger model ... done (4.009s)
My_PRP$ name_NN is_VBZ Shabab_NNP i_FW am_VBP 22_CD years_NNS old._.
Average: 66.7 sent/s
Total: 1 sent
Runtime: 0.015s
I hope it installed properly?
Now how do i do this POStagging from inside a java application? I have added the openNLPtools, jwnl, maxent jar to the project but how do i invoke the POStagging?

Here's some (old) sample code I threw together, with modernized code to follow:
package opennlp;
import opennlp.tools.cmdline.PerformanceMonitor;
import opennlp.tools.cmdline.postag.POSModelLoader;
import opennlp.tools.postag.POSModel;
import opennlp.tools.postag.POSSample;
import opennlp.tools.postag.POSTaggerME;
import opennlp.tools.tokenize.WhitespaceTokenizer;
import opennlp.tools.util.ObjectStream;
import opennlp.tools.util.PlainTextByLineStream;
import java.io.File;
import java.io.IOException;
import java.io.StringReader;
public class OpenNlpTest {
public static void main(String[] args) throws IOException {
POSModel model = new POSModelLoader().load(new File("en-pos-maxent.bin"));
PerformanceMonitor perfMon = new PerformanceMonitor(System.err, "sent");
POSTaggerME tagger = new POSTaggerME(model);
String input = "Can anyone help me dig through OpenNLP's horrible documentation?";
ObjectStream<String> lineStream =
new PlainTextByLineStream(new StringReader(input));
perfMon.start();
String line;
while ((line = lineStream.read()) != null) {
String whitespaceTokenizerLine[] = WhitespaceTokenizer.INSTANCE.tokenize(line);
String[] tags = tagger.tag(whitespaceTokenizerLine);
POSSample sample = new POSSample(whitespaceTokenizerLine, tags);
System.out.println(sample.toString());
perfMon.incrementCounter();
}
perfMon.stopAndPrintFinalResult();
}
}
The output is:
Loading POS Tagger model ... done (2.045s)
Can_MD anyone_NN help_VB me_PRP dig_VB through_IN OpenNLP's_NNP horrible_JJ documentation?_NN
Average: 76.9 sent/s
Total: 1 sent
Runtime: 0.013s
This is basically working from the POSTaggerTool class included as part of OpenNLP. The sample.getTags() is a String array that has the tag types themselves.
This requires direct file access to the training data, which is really, really lame.
An updated codebase for this is a little different (and probably more useful.)
First, a Maven POM:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>org.javachannel</groupId>
<artifactId>opennlp-example</artifactId>
<version>1.0-SNAPSHOT</version>
<dependencies>
<dependency>
<groupId>org.apache.opennlp</groupId>
<artifactId>opennlp-tools</artifactId>
<version>1.6.0</version>
</dependency>
<dependency>
<groupId>org.testng</groupId>
<artifactId>testng</artifactId>
<version>[6.8.21,)</version>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.1</version>
<configuration>
<source>1.8</source>
<target>1.8</target>
</configuration>
</plugin>
</plugins>
</build>
</project>
And here's the code, written as a test, therefore located in ./src/test/java/org/javachannel/opennlp/example:
package org.javachannel.opennlp.example;
import opennlp.tools.cmdline.PerformanceMonitor;
import opennlp.tools.postag.POSModel;
import opennlp.tools.postag.POSSample;
import opennlp.tools.postag.POSTaggerME;
import opennlp.tools.tokenize.WhitespaceTokenizer;
import org.testng.annotations.DataProvider;
import org.testng.annotations.Test;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.net.URL;
import java.nio.channels.Channels;
import java.nio.channels.ReadableByteChannel;
import java.util.stream.Stream;
public class POSTest {
private void download(String url, File destination) throws IOException {
URL website = new URL(url);
ReadableByteChannel rbc = Channels.newChannel(website.openStream());
FileOutputStream fos = new FileOutputStream(destination);
fos.getChannel().transferFrom(rbc, 0, Long.MAX_VALUE);
}
#DataProvider
Object[][] getCorpusData() {
return new Object[][][]{{{
"Can anyone help me dig through OpenNLP's horrible documentation?"
}}};
}
#Test(dataProvider = "getCorpusData")
public void showPOS(Object[] input) throws IOException {
File modelFile = new File("en-pos-maxent.bin");
if (!modelFile.exists()) {
System.out.println("Downloading model.");
download("http://opennlp.sourceforge.net/models-1.5/en-pos-maxent.bin", modelFile);
}
POSModel model = new POSModel(modelFile);
PerformanceMonitor perfMon = new PerformanceMonitor(System.err, "sent");
POSTaggerME tagger = new POSTaggerME(model);
perfMon.start();
Stream.of(input).map(line -> {
String whitespaceTokenizerLine[] = WhitespaceTokenizer.INSTANCE.tokenize(line.toString());
String[] tags = tagger.tag(whitespaceTokenizerLine);
POSSample sample = new POSSample(whitespaceTokenizerLine, tags);
perfMon.incrementCounter();
return sample.toString();
}).forEach(System.out::println);
perfMon.stopAndPrintFinalResult();
}
}
This code doesn't actually test anything - it's a smoke test, if anything - but it should serve as a starting point. Another (potentially) nice thing is that it downloads a model for you if you don't have it downloaded already.

The URL http://bulba.sdsu.edu/jeanette/thesis/PennTags.html does not work anymore. I found the below on the 14th slide at http://www.slideshare.net/gagan1667/opennlp-demo

The above answer does provide a way to use the existing models from OpenNLP but if you need to train your own model, maybe the below can help:
Here is a detailed tutorial with full code:
https://dataturks.com/blog/opennlp-pos-tagger-training-java-example.php
Depending upon your domain, you can build a dataset either automatically or manually. Building such a dataset manually can be really painful, tools like POS tagger can help make the process much easier.
Training data format
Training data is passed as a text file where each line is one data item. Each word in the line should be labeled in a format like "word_LABEL", the word and the label name is separated by an underscore '_'.
anki_Brand overdrive_Brand
just_ModelName dance_ModelName 2018_ModelName
aoc_Brand 27"_ScreenSize monitor_Category
horizon_ModelName zero_ModelName dawn_ModelName
cm_Unknown 700_Unknown modem_Category
computer_Category
Train model
The important class here is POSModel, which holds the actual model. We use class POSTaggerME to do the model building. Below is the code to build a model from training data file
public POSModel train(String filepath) {
POSModel model = null;
TrainingParameters parameters = TrainingParameters.defaultParams();
parameters.put(TrainingParameters.ITERATIONS_PARAM, "100");
try {
try (InputStream dataIn = new FileInputStream(filepath)) {
ObjectStream<String> lineStream = new PlainTextByLineStream(new InputStreamFactory() {
#Override
public InputStream createInputStream() throws IOException {
return dataIn;
}
}, StandardCharsets.UTF_8);
ObjectStream<POSSample> sampleStream = new WordTagSampleStream(lineStream);
model = POSTaggerME.train("en", sampleStream, parameters, new POSTaggerFactory());
return model;
}
}
catch (Exception e) {
e.printStackTrace();
}
return null;
}
Use model to do tagging.
Finally, we can see how the model can be used to tag unseen queries:
public void doTagging(POSModel model, String input) {
input = input.trim();
POSTaggerME tagger = new POSTaggerME(model);
Sequence[] sequences = tagger.topKSequences(input.split(" "));
for (Sequence s : sequences) {
List<String> tags = s.getOutcomes();
System.out.println(Arrays.asList(input.split(" ")) +" =>" + tags);
}
}