We Keep Coding

Create CSV file with columns and values from HashMap

Be gentle,
This is my first time using Apache Commons CSV 1.7.
I am creating a service to process some CSV inputs,
add some additional information from exterior sources,
then write out this CSV for ingestion into another system.
I store the information that I have gathered into a list of
HashMap<String, String> for each row of the final output csv.
The Hashmap contains the <ColumnName, Value for column>.
I have issues using the CSVPrinter to correctly assign the values of the HashMaps into the rows.
I can concatenate the values into a string with commas between the variables;
however,
this just inserts the whole string into the first column.
I cannot define or hardcode the headers since they are obtained from a config file and may change depending on which project uses the service.
Here is some of my code:
try (BufferedWriter writer = Files.newBufferedWriter(
Paths.get(OUTPUT + "/" + project + "/" + project + ".csv"));)
{
CSVPrinter csvPrinter = new CSVPrinter(writer,
CSVFormat.RFC4180.withFirstRecordAsHeader());
csvPrinter.printRecord(columnList);
for (HashMap<String, String> row : rowCollection)
{
//Need to map __record__ to column -> row.key, value -> row.value for whole map.
csvPrinter.printrecord(__record__);
}
csvPrinter.flush();
}
Thanks for your assistance.

You actually have multiple concerns with your technique;
How do you maintain column order?
How do you print the column names?
How do you print the column values?
Here are my suggestions.
Maintain column order.
Do not use HashMap,
because it is unordered.
Instead,
use LinkedHashMap which has a "predictable iteration order"
(i.e. maintains order).
Print column names.
Every row in your list contains the column names in the form of key values,
but you only print the column names as the first row of output.
The solution is to print the column names before you loop through the rows.
Get them from the first element of the list.
Print column values.
The "billal GHILAS" answer demonstrates a way to print the values of each row.
Here is some code:
try (BufferedWriter writer = Files.newBufferedWriter(
Paths.get(OUTPUT + "/" + project + "/" + project + ".csv"));)
{
CSVPrinter csvPrinter = new CSVPrinter(writer,
CSVFormat.RFC4180.withFirstRecordAsHeader());
// This assumes that the rowCollection will never be empty.
// An anonymous scope block just to limit the scope of the variable names.
{
HashMap<String, String> firstRow = rowCollection.get(0);
int valueIndex = 0;
String[] valueArray = new String[firstRow.size()];
for (String currentValue : firstRow.keySet())
{
valueArray[valueIndex++] = currentValue;
}
csvPrinter.printrecord(valueArray);
}
for (HashMap<String, String> row : rowCollection)
{
int valueIndex = 0;
String[] valueArray = new String[row.size()];
for (String currentValue : row.values())
{
valueArray[valueIndex++] = currentValue;
}
csvPrinter.printrecord(valueArray);
}
csvPrinter.flush();
}

for (HashMap<String,String> row : rowCollection) {
Object[] record = new Object[row.size()];
for (int i = 0; i < columnList.size(); i++) {
record[i] = row.get(columnList.get(i));
}
csvPrinter.printRecord(record);
}

How can I convert Json String to List<HashMap<String, String>>?

I creating compare data A and each data using Java.
First, I did extract array data from txt file (array type in file).
Second, I have a Json String data in my database (MySQL column type : JSON).
I parsed txt file and make List
FileInputStream fstream = new FileInputStream(Path);
List<HashMap<String, String>> list = list(fstream);
public List<HashMap<String, String>> list(FileInputStream fstream) {
BufferedReader buff = new BufferedReader(new InputStreamReader(fstream));
List<HashMap<String, String> list = new ArrayList<>();
try {
while ((strLine = buff.readLine()) != null) {
s = strLine.split(" ");
HashMap<String, String> map = new HashMap<>();
pts = s[4].split(":")[1];
ptstime = s[5].split(":")[1];
map.put("pts", pts);
map.put("ptstime", getDurationString(ptstime));
if (pts_itv.equals("0") && ptstime_itv.equals("00:00")) {
map.put("pts_itv", "0");
map.put("ptstime_itv", "00:00");
}
else {
map.put("pts_itv", String.valueOf(Long.parseLong(pts) - Long.parseLong(pts_itv)));
map.put("ptstime_itv", getDurationString(String.valueOf(String.format("%.4f", Double.parseDouble(ptstime) - Double.parseDouble(ptstime_itv)))));
}
pts_itv = pts;
ptstime_itv = ptstime;
list.add(map);
}
}
catch (IOException ex) {
LOG.error("ERROR : " + ex.getLocalizedMessage() + ", " + ex.getMessage());
}
finally {
try {
buff.close();
} catch (IOException ex) {}
}
return list;
}
Extraction data
[{pts_itv=0, ptstime=00:00:00.112792, pts=2707, ptstime_itv=00:00}, {pts_itv=192192, ptstime=00:00:08.12079, pts=194899, ptstime_itv=00:00:08.80}, {pts_itv=128128, ptstime=00:00:13.4595, pts=323027, ptstime_itv=00:00:05.3387}, {pts_itv=277277, ptstime=00:00:25.127, pts=600304, ptstime_itv=00:00:11.5532}]
I can get index key, object key.
list.get(0).get("pts_itv");
And Second data (SELECT Query from MySQL (Column JSON Type))
rs = Web.getInstance().getList(idx); //get data by sql query
jObj = (JSONObject) new JSONParser().parse(rs);
jArr = (JSONArray) jObj.get("data");
for (int i = 0; i < jArr.size(); i++) {
JSONObject Jarr = (JSONObject) jArr.get(i);
System.out.println(Jarr.get("PtsData"));
}
PtsData is
[{"pts": "81831", "pts_itv": "0", "ptstime": "00:00:03.40963", "ptstime_itv": "00:00"}, {"pts": "127877", "pts_itv": "46046", "ptstime": "00:00:05.32821", "ptstime_itv": "00:00:01.9186"}, {"pts": "157907", "pts_itv": "30030", "ptstime": "00:00:06.57946", "ptstime_itv": "00:00:01.2512"}]
java.lang.String
I want PtsData convert to first data type. How can I convert PtsData to List type?

Here are some suggestions to solve this problem.
Don't use String operations un-till they are really needed, you can avoid split by using JSON library to parse JSON data. More code you write, more bugs you introduce and more maintenance it need.
Use Object modelling. This would be more readable to use it in HashMap. Create an object model by defining peroperties pts, pts_itv, ptstime, ptstime_itv.Then choose your key and value. Define equals and hashcode. Then check equality of objects either with your own way to or use equals.

Java: Gson change property name without serialization

Is it possible to change the name of a Json property without serialization with Gson? For example, given this Json
{
"1": {
...
},
"2": {
...
}
}
could I change the "1" to a "3" without removing its contents. I know that the addProperty method adds a new property, or overwrites an existing property with a new value, but I want to change the name of a property without affecting its value. Also, pasting the existing value as the second argument of addProperty will not suffice.
EDIT: To add more context, I will explain the bigger picture. I have a JSON string that is a couple thousand lines long. I'm writing a program leveraging Gson in order to change the values in that JSON string. I am at a point where I not only want to change the values of properties, but the names of the properties themselves. I have done everything so far without serialization.
Here is a snippet of the Java I wrote:
String file = "\\temp.json";
FileReader reader = new FileReader(file);
JsonStreamParser parser = new JsonStreamParser(reader);
// Parse entire JSON
JsonElement element = parser.next();
// Get root element
JsonObject sites = element.getAsJsonObject();
// Get first child element
JsonObject site1 = sites.getAsJsonObject("1");
JsonObject clust1 = site1.getAsJsonObject("CLUST");
for(int i = 1; i < 12; i++) {
// "Dynamic" variable
String num = Integer.toString(i);
// Get property whose name is a number, has siblings
JsonObject one = custCluster1.getAsJsonObject(num);
one.getAsJsonObject().addProperty("name", "cluster" + i);
JsonObject subOne = one.getAsJsonObject("SUB");
subOne.getAsJsonObject().addProperty("name", "aName" + i);
for(int n = 1; n < 1002; n++) {
// "Dynamic" variable
String inst = Integer.toString(n);
// Get property whose name is a number, has siblings
JsonObject subSub = subOne.getAsJsonObject(inst);
// If the property doesn't exist, then don't execute
if(subSub != null) {
JsonArray subSubArray = subSub.getAsJsonArray("SUBSUB");
subSub.getAsJsonObject().remove("name");
int m = 0;
while(m < subSubArray.size()) {
subSubArray.get(m).getAsJsonObject().remove("SR");
subSubArray.get(m).getAsJsonObject().remove("FI");
subSubArray.get(m).getAsJsonObject().remove("IND");
subSubArray.get(m).getAsJsonObject().addProperty("ST", "1");
subSubArray.get(m).getAsJsonObject().addProperty("ID", "2");
subSubArray.get(m).getAsJsonObject().addProperty("DESCR", "hi");
m++;
}
m = 0;
}
}
}

Thanks to #mmcrae for helping and suggesting this method.
Since I'm already saving the (key, value) pairs in variables, you can remove the property whose name you want to change from the parent, and then add it back with a new name and the content that was already saved.
Like this:
JsonObject sites = element.getAsJsonObject();
JsonObject site1 = sites.getAsJsonObject("1");
JsonObject clust1 = site1.getAsJsonObject("CLUST");
site1.remove("CLUST");
site1.add("NEWCLUST", clust1);

JSON Parsing in Java Webservice of multiple JSONObjects in JSONObject

This is my JSON string,
{
"listmain":{
"16":[{"brandid":"186"},{"brandid":"146"},{"brandid":"15"}],
"17":[{"brandid":"1"}],
"18":[{"brandid":"12"},{"brandid":"186"}],
}
}
I need to get values in "16","17","18" tag and add values and ids("16","17","18") to two ArrayList.
What i meant is,
when we take "16", the following process should happen,
List<String> lsubid = new ArrayList<String>();
List<String> lbrandid = new ArrayList<String>();
for(int i=0;i<number of elements in "16";i++) {
lsubid.add("16");
lbrandid.add("ith value in tag "16" ");
}
finally the values in lsubid will be---> [16,16,16]
the values in lbrandid will be---> [186,146,15]
Can anyone please help me to complete this.

Use JSONObject keys() to get the key and then iterate each key to get to the dynamic value.
You can parse the JSON like this
JSONObject responseDataObj = new JSONObject(responseData);
JSONObject listMainObj = responseDataObj.getJSONObject("listmain");
Iterator keys = listMainObj.keys();
while(keys.hasNext()) {
// loop to get the dynamic key
String currentDynamicKey = (String)keys.next();
//store key in an arraylist which is 16,17,...
// get the value of the dynamic key
JSONArray currentDynamicValue = listMainObj.getJSONArray(currentDynamicKey);
int jsonrraySize = currentDynamicValue.length();
if(jsonrraySize > 0) {
for (int i = 0; i < jsonrraySize; i++) {
JSONObject brandidObj = currentDynamicValue.getJSONObject(i);
String brandid = brandidObj.getString("brandid");
System.out.print("Brandid = " + brandid);
//store brandid in an arraylist
}
}
}
Source of this answer

Convert JSON object with duplicate keys to JSON array

I have a JSON string that I get from a database which contains repeated keys. I want to remove the repeated keys by combining their values into an array.
For example
Input
{
"a":"b",
"c":"d",
"c":"e",
"f":"g"
}
Output
{
"a":"b",
"c":["d","e"],
"f":"g"
}
The actual data is a large file that may be nested. I will not know ahead of time what or how many pairs there are.
I need to use Java for this. org.json throws an exception because of the repeated keys, gson can parse the string but each repeated key overwrites the last one. I need to keep all the data.
If possible, I'd like to do this without editing any library code

As of today the org.json library version 20170516 provides accumulate() method that stores the duplicate key entries into JSONArray
JSONObject jsonObject = new JSONObject();
jsonObject.accumulate("a", "b");
jsonObject.accumulate("c", "d");
jsonObject.accumulate("c", "e");
jsonObject.accumulate("f", "g");
System.out.println(jsonObject);
Output:
{
"a":"b",
"c":["d","e"],
"f":"g"
}

I want to remove the repeated keys by combining their values into an array.
Think other than JSON parsing library. It's very simple Java Program using String.split() method that convert Json String into Map<String, List<String>> without using any library.
Sample code:
String jsonString = ...
// remove enclosing braces and double quotes
jsonString = jsonString.substring(2, jsonString.length() - 2);
Map<String, List<String>> map = new HashMap<String, List<String>>();
for (String values : jsonString.split("\",\"")) {
String[] keyValue = values.split("\":\"");
String key = keyValue[0];
String value = keyValue[1];
if (!map.containsKey(key)) {
map.put(key, new ArrayList<String>());
}
map.get(key).add(value);
}
output:
{
"f": ["g"],
"c": ["d","e"],
"a": ["b"]
}

In order to accomplish what you want, you need to create some sort of custom class since JSON cannot technically have 2 values at one key. Below is an example:
public class SomeClass {
Map<String, List<Object>> values = new HashMap<String, List<Object>>();
public void add(String key, Object o) {
List<Object> value = new ArrayList<Object>();
if (values.containsKey(key)) {
value = values.get(key);
}
value.add(o);
values.put(key, value);
}
public JSONObject toJson() throws JSONException {
JSONObject json = new JSONObject();
JSONArray tempArray = null;
for (Entry<String, List<Object>> en : values.entrySet()) {
tempArray = new JSONArray();
for (Object o : en.getValue()) {
tempArray.add(o);
}
json.put(en.getKey(), tempArray);
}
return json;
}
}
You can then retrieve the values from the database, call the .add(String key, Object o) function with the column name from the database, and the value (as the Object param). Then call .toJson() when you are finished.

Thanks to Mike Elofson and Braj for helping me in the right direction. I only wanted to have the keys with multiple values become arrays so I had to modify the code a bit. Eventually I want it to work for nested JSON as well, as it currently assumes it is flat. However, the following code works for what I need it for at the moment.
public static String repeatedKeysToArrays(String jsonIn) throws JSONException
{
//This assumes that the json is flat
String jsonString = jsonIn.substring(2, jsonIn.length() - 2);
JSONObject obj = new JSONObject();
for (String values : jsonString.split("\",\"")) {
String[] keyValue = values.split("\":\"");
String key = keyValue[0];
String value = "";
if (keyValue.length>1) value = keyValue[1];
if (!obj.has(key)) {
obj.put(key, value);
} else {
Object Oold = obj.get(key);
ArrayList<String> newlist = new ArrayList<String>();
//Try to cast as JSONArray. Otherwise, assume it is a String
if (Oold.getClass().equals(JSONArray.class)) {
JSONArray old = (JSONArray)Oold;
//Build replacement value
for (int i=0; i<old.length(); i++) {
newlist.add( old.getString(i) );
}
}
else if (Oold.getClass().equals(String.class)) newlist = new ArrayList<String>(Arrays.asList(new String[] {(String)Oold}));
newlist.add(value);
JSONArray newarr = new JSONArray( newlist );
obj.put(key,newarr);
}
}
return obj.toString();
}