I have JSON that looks similar to this:
{"test":{"red":"on","green":"off","yellow":"on"},"test1":{"red":"off","green":"on","yellow":"off"},"test2":{"red":"on","green":"off","yellow":"off"}}
I've iterating over this with the code below:
JSONObject t = JSON.parse(params.myObject)
t.each { id, data ->
println id
println data.red
println data.green
println data.yellow
}
However, at times I can have dynamically different values in the JSON Object. For example (new color added):
{"test":{"red":"on","green":"off","yellow":"on","pink":"on"},"test1":{"red":"off","green":"on","yellow":"off","pink":"on"},"test2":{"red":"on","green":"off","yellow":"off","pink":"on"}}
Question
Is there a way to iterate over all the json without hardcoding the colors in my code?
JSONObject t = JSON.parse(params.myObject)
t.each { id, data ->
println id
data.each { prop, value ->
println prop + " = " + value
}
}
I figured it out.
I can simply just iterate over the values in similar fashion:
t.each { id, data ->
println id
data.each {id1, d ->
println id1
println d
}
}
Related
I want to parse this object to a list of string. I do not need the key but just want the value as a list of string.
I cannot have a simple model classes because the keys object are more than 1000 in some responses and are random.
So please any idea how to parse it to list in kotlin or java?
{
"data": {
"21": "593754434425",
"22": "4560864343802",
"23": "7557134347529",
"24": "5937544344255",
"25": "45608643438024",
"26": "75571343475293"
}
}
You could first deserialize it as it is, and then convert to a list.
The JSON can be represented this way:
data class Response(val data: Map<String, String>)
You can mark this class #Serializable and use Kotlinx Serialization to deserialize it, or you can use other libraries like Moshi or Jackson (with jackson-module-kotlin).
Once it's deserialized, simply get the values of the map (it's a collection):
val response = Json.decodeFromString<Response>(yourJsonString)
// this is a Collection, not List, but it should be good enough
val stringValues = response.data.values
// if you really need a List<String>
val list = stringValues.toList()
If you want to get the values in the natural order of the keys, you can also use something like:
val values = response.data.toSortedMap(compareBy<String> { it.toInt() }).values
You can use this to parse your data:
val it: Iterator<String> = json.keys()
val arrayList = ArrayList<String>()
while (it.hasNext()) {
val key = it.next()
arrayList.add(json.get(key))
}
A better way is to change the json model, if you access it.
{
"data": [
"593754434425","4560864343802",
"7557134347529","5937544344255",
"45608643438024","75571343475293"
]
}
For this problem, its handy to use the libriary org.json.
See following code snippet:
import org.json.JSONObject;
import java.util.ArrayList;
public class Main {
public static void main(String[] args) {
// Defining the input
String input = "{\n" +
" \"data\": {\n" +
" \"21\": \"593754434425\",\n" +
" \"22\": \"4560864343802\",\n" +
" \"23\": \"7557134347529\",\n" +
" \"24\": \"5937544344255\",\n" +
" \"25\": \"45608643438024\",\n" +
" \"26\": \"75571343475293\"\n" +
" }\n" +
"}\n";
// Parsing it to a json object with org.json
JSONObject inputJson = new JSONObject(input);
// If inputJson does not contain the key data, we return
if(!inputJson.has("data")) return;
// Else we read this data object to a new JSONObject
JSONObject dataJson = inputJson.getJSONObject("data");
// Define an array list where all the values will be contained
ArrayList<String> values = new ArrayList<>();
// Get a key set of the dat json object. For each key we get its respective value and add it to our value array list
for (String key : dataJson.keySet()) values.add(dataJson.getString(key));
// Print all values
for (String value : values) System.out.println(value);
}
}
=>
4560864343802
7557134347529
5937544344255
45608643438024
75571343475293
593754434425
Installing org.json is the easiest with a package manager like maven or gradle.
Guys i have comeup with a similar solution for the problem here
this is my model class
data class UnVerifiedTagIds(
#SerializedName("data")
val data: Object
)
and this is how i parse the respone here
val values: ArrayList<String> = ArrayList()
val list_of_tag_ids: ArrayList<String> =response.data as ArrayList<String>
The ist one is the dataclass for the response
and the 2nd one is the ApiCallInterface m using Retrofit...
and the last one is the apicall itself
I am using Kotlin language
do class name with name like this data class Result(val data:Map<String,String>)
and using library GSON for convert string json to this model
val json = "{\n" +
" \"data\": {\n" +
" \"21\": \"593754434425\",\n" +
" \"22\": \"4560864343802\",\n" +
" \"23\": \"7557134347529\",\n" +
" \"24\": \"5937544344255\",\n" +
" \"25\": \"45608643438024\",\n" +
" \"26\": \"75571343475293\"\n" +
" }\n" +
"}"
val dat = Gson().fromJson(json,Result::class.java)
if (dat.data.isNotEmpty()){
val list= dat.data.values.toMutableList()
print(list)
}
that works fine with me
I need to write a code which would convert JSON file to CSV. The problem is in a format that the CSV file should look like.
Input json:
{
"strings":{
"1level1":{
"1level2":{
"1label1":"1value1",
"1label2":"1value2"
}
},
"2level1":{
"2level2":{
"2level3":{
"2label1":"2value1"
},
"2label2":"2value2"
}
}
}
}
And this is expected csv file for this json:
Keys,Default
1level1.1level2.1label1,1value1
1level1.1level2.1label2,1value2
2level1.2level2.2level3.2label1,2value1
2level1.2level2.2label2,2value2
I was trying to go through JSON file using recursion but this didn't work for me because of rewriting JSON object on each iteration and code was working only till the first value. Are there any suggestions about how can it be done?
Note: have tried to use different JSON libraries, so for now can be used any of them
UPDATE #1:
Non-working code example I was trying to use to go through JSON tree:
public static void jsonToCsv() throws JSONException {
InputStream is = MainClass.class.getResourceAsStream("/fromJson.json");
JSONTokener jsonTokener = new JSONTokener(is);
JSONObject jsonObject = new JSONObject(jsonTokener);
stepInto(jsonObject);
}
private static void stepInto(JSONObject jsonObject) {
JSONObject object = jsonObject;
try {
Set < String > keySet = object.keySet();
for (String key: keySet) {
object = object.getJSONObject(key);
stepInto(object);
}
} catch (JSONException e) {
Set < String > keySet = object.keySet();
for (String key: keySet) {
System.out.println(object.get(key));
}
e.printStackTrace();
}
}
UPDATE #2:
Another issue is that I will never know the names of the JSON object and count of child objects (update JSON and CSV examples as well to make the image more clear). All that is known, that it will always start with strings object.
Library used:
<dependency>
<groupId>org.json</groupId>
<artifactId>json</artifactId>
<version>20180813</version>
</dependency>
So found a solution by myself:
public static void jsonToCsv() throws JSONException, IOException {
InputStream is = MainClass.class.getResourceAsStream("/fromJson.json");
JSONTokener jsonTokener = new JSONTokener(is);
JSONObject jsonObject = new JSONObject(jsonTokener).getJSONObject("strings");
builder = new StringBuilder();
while (!jsonObject.isEmpty()) {
stepInto(jsonObject);
}
String[] lines = builder.toString().split("\n"); // builder lines are in reverse order from expected so this array is used to reverse them
FileWriter csvWriter = new FileWriter("src/main/resources/toCsv.csv");
csvWriter.append("Keys,Default (en_US)\n");
for (int i = lines.length - 1; i >= 0; i--) {
csvWriter.append(lines[i]).append("\n");
}
csvWriter.flush();
csvWriter.close();
}
private static void stepInto(JSONObject jsonObject) {
for (String key: jsonObject.keySet()) {
Object object = jsonObject.get(key);
if (object instanceof JSONObject) {
builder.append(key).append(".");
stepInto(jsonObject.getJSONObject(key));
} else {
builder.append(key).append(",").append(object).append("\n");
jsonObject.remove(key);
break;
}
if (jsonObject.getJSONObject(key).isEmpty()) {
jsonObject.remove(key);
}
break;
}
}
I think you just missed keeping track of your result, otherwise it looks good.
Let's say your result is a simple string. Then you have to concatenate all keys while traversing the json object until you reach a primitive value (like a number or a string).
(I am writing this out of my head, so please forgive me for incorrect syntax)
private static String stepInto(JSONObject jsonObject) { // we change "void" to "String" so we can record the results of each recursive "stepInto" call
//JSONObject object = jsonObject; // we don't need that. Both variables are the same object
String result ="";
try {
for (String key: jsonObject.keySet()) { // shorter version
Object object = jsonObject.get(key); // Attention! we get a simple Java Object
if(object instanceof JSONObject){
result+= key+"."+stepInto(jsonObject.getJSONObject(key)); // the recursive call, returning all keys concatenated to "level1.level2.level3" until we reach a primitive value
}
if(object instanceof JSONArray){
result+= key+", "+ ... // notice how we use the csv separator (comma) here, because we reached a value. For you to decide how you want to represent arrays
}
result+= key +", "+ object +"\n"; // here I am not sure. It may well be that you need to check if object is a String an Integer, Float or anything.
}
} catch (JSONException e) {
for (String key: jsonObject.keySet()) {
System.out.println(object.get(key));
}
e.printStackTrace();
result+= "\n"; // I added this fallback so your program can terminate even when an error occurs.
}
return result; // sorry, I forgot to accumulate all results and return them. So now we have only one actual "return" statement that will terminate the call and return all results.
}
As you can see, I didn't change much of your original method. The only difference is that now we keep track of the keys ("level1.level2...") for each recursive call.
EDIT
I added a +"\n"; so everytime we reach a value so we can terminate that "line".
AND more importantly, instead of returning everytime, I add the result of each call to a string, so we continue looping over the keys and concatenate all results. Each call of the method will return only once all keys are looped over. (sorry that missed that)
In your calling method you could print out the result, something like that:
JSONObject jsonObject = new JSONObject(jsonTokener);
String result = stepInto(jsonObject);
System.out.println(result);
I have been searching all over, but I still cannot find a solution to my problem. If there is a post made already, please tell me so I can visit it. I have seen similar posts, but they follow a different JSON format than mine, so I wanted to see if it is possible and how it is possible to make it using the JSON format that will be introduced below.
Basically, what I am trying to do is get every element in a JSON file, and retrieve each element's key name and value. Both the key and the value are String values. Here is an example JSON of how I want my JSON code to look like:
{
"Variable1":"-",
"Variable2":" Test "
}
I am using the org.json library, and I would like to know if this is possible, and if it is, how can I achieve it? What I tried to do originally was put the variables under an array named "Variables", but every time I tried getting that array, it gave me an error saying that JSONObject["Variables"] is not a JSONArray. Not sure if this is caused because of a problem in the JDK or because of a problem in my code. That is, of course, a thing to discuss in another thread. So far, this is what I have (FilePath is a String variable that contains the full path to the file):
String Contents = new String((Files.readAllBytes(Paths.get(FilePath))));
JSONObject JsonFile = new JSONObject(Contents);
JSONArray VariableList = JsonFile.getJSONArray("Variables");
for (Object Item: VariableList) {
Map.Entry Item2 = (Map.Entry)Item;
System.out.println("Key: " + Item2.getKey() + ", Value: " + Item2.getValue());
}
The above code should be working if the JSON looked something like this (yes, I said should because it does not work):
{
"Variables": {
"Variable1":"-",
"Variable2":" Test "
}
}
If it is possible, how would I be able to make get the key and value using the first JSON format? If not possible, then how would I do it in an alternative way? Keep in mind, the key name is never going to the same, as the key and value will be different depending on what the user wants them to be, so that is why it is important to be able to loop through every element and get both it's key and value.
Thank you for your time and effort.
"Variables" : { ... } is a JSONObject and not a JSONArray.
For package org.json
try {
String contents = "{\"Variables\":{\"Variable1\":\"-\",\"Variable2\":\" Test \"}}";
JSONObject jsonFile = new JSONObject(contents);
JSONObject variableList = jsonFile.getJSONObject("Variables"); // <-- use getJSONObject
JSONArray keys = variableList.names ();
for (int i = 0; i < keys.length (); ++i) {
String key = keys.getString(i);
String value = variableList.getString(key);
System.out.println("key: " + key + " value: " + value);
}
} catch (Exception e) {
e.printStackTrace();
}
For package JSON.simple
String contents = new String((Files.readAllBytes(Paths.get(FilePath))));
JSONObject jsonFile = new JSONObject(contents);
JSONObject variableList = jsonFile.getJSONObject("Variables"); // <-- use getJSONObject
variableList.keySet().forEach(key -> {
Object value = jsonObj.get(key);
System.out.println("key: "+ key + ", value: " + value);
});
I managed to update a json object using a jsonPath with this code
JSONObject json = new JSONObject("{\"data\":[{\"city\":\"New York\",\"name\":\"John\",\"age\":31},{\"city\":\"Paris\",\"name\":\"Jack\",\"age\":12}]}");
DocumentContext doc = JsonPath.parse(json.toString())
.set("$..name","newName");
System.out.println("doc.jsonString() = " + doc.jsonString());
outputs:
doc.jsonString() = {"data":[{"city":"New York","name":"newName","age":31},{"city":"Paris","name":"newName","age":12}]}
Now I would like to update the value depending on the old value (by applying a function on the old value)
Something like
DocumentContext doc = JsonPath.parse(json.toString())
.set("$..name",upper(oldValue))
.set("$..age", oldValue+10);
That would result in the following json
doc.jsonString() = doc.jsonString() = {"data":[{"city":"New York","name":"JOHN","age":41},{"city":"Paris","name":"JACK","age":22}]}
Does someone know how I can manage to reference the old value like that ?
Regards,
You can use the map function of the DocumentContext class like the example below:
DocumentContext json = JsonPath.using(configuration).parse(jsonStr);
DocumentContext result = json.map("$..name", (currentValue, configuration) -> {
return currentValue.toString().toUpperCase();
});
System.out.println(result.jsonString());
That example change the value to uppercase.
Try to check about it in the Jsonpath DocumentContext class documentation.
You can use parse and set functions of JsonPath.
example:
public static String replaceOldValueVithNewValueforGivenPath(String jsonBody, String path, String newValue)
{
// validateJsonInput(jsonBody);
try {
return JsonPath.parse(jsonBody).set(path,newValue).jsonString();
} catch (PathNotFoundException var3) {
throw new RuntimeException("No results for path: " + path);
}
}
I'm using Java to download HTML contents of websites whose URLs are stored in a database. I'd like to put their HTML into database, too.
I'm using Jsoup for this purpose:
public String downloadHTML(String byLink) {
String htmlInPage = "";
try {
Document doc = Jsoup.connect(byLink).get();
htmlInPage = doc.html();
} catch (org.jsoup.UnsupportedMimeTypeException e) {
// process this and some other exceptions
}
return htmlInPage;
}
I'd like to download websites concurrently and use this function:
public void downloadURL(int websiteId, String url,
String categoryName, ExecutorService executorService) {
executorService.submit((Runnable) () -> {
String htmlInPage = downloadHTML(url);
System.out.println("Category: " + categoryName + " " + websiteId + " " + url);
String insertQuery =
"INSERT INTO html_data (website_id, html_contents) VALUES (?,?)";
dbUtils.query(insertQuery, websiteId, htmlInPage);
});
}
dbUtils is my class based on Apache Commons DbUtils. Details are here: http://pastebin.com/iAKXchbQ
And I'm using everything mentioned above in a such way: (List<Object[]> details are explained on pastebin, too)
public static void main(String[] args) {
DbUtils dbUtils = new DbUtils("host", "db", "driver", "user", "pass");
List<String> categoriesList =
Arrays.asList("weapons", "planes", "cooking", "manga");
String sql = "SELECT lw.id, lw.website_url, category_name " +
"FROM list_of_websites AS lw JOIN list_of_categories AS lc " +
"ON lw.category_id = lc.id " +
"where category_name = ? ";
ExecutorService executorService = Executors.newFixedThreadPool(10);
for (String category : categoriesList) {
List<Object[]> sitesInCategory = dbUtils.select(sql, category );
for (Object[] entry : sitesInCategory) {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
}
}
executorService.shutdown();
}
I'm not sure if this solution is correct but it works. Now I want to modify code to save HTML not from all websites in my database, but only their fixed ammount in each category.
For example, download and save HTML of 50 websites from the "weapons" category, 50 from "planes", etc. I don't think it's necessary to use sql for this purpose: if we select 50 sites per category, it doesn't mean we save them all, because of possibly incorrect syntax and connection problems.
I've tryed to create separate class implementing Runnable with fields: counter and maxWebsitesPerCategory, but these variables aren't updated. Another idea was to create field Map<String,Integer> sitesInCategory instead of counter, put each category as a key there and increment its value until it reaches maxWebsitesPerCategory, but it didn't work, too. Please, help me!
P.S: I'll also be grateful for any recommendations connected with my realization of concurrent downloading (I haven't worked with concurrency in Java before and this is my first attempt)
How about this?
for (String category : categoriesList) {
dbUtils.select(sql, category).stream()
.limit(50)
.forEach(entry -> {
int websiteId = (int) entry[0];
String url = (String) entry[1];
String categoryName = (String) entry[2];
downloadURL(websiteId, url, categoryName, executorService);
});
}
sitesInCategory has been replaced with a stream of at most 50 elements, then your code is run on each entry.
EDIT
In regard to comments. I've gone ahead and restructured a bit, you can modify/implement the content of the methods I've suggested.
public void werk(Queue<Object[]> q, ExecutorService executorService) {
executorService.submit(() -> {
try {
Object[] o = q.remove();
try {
String html = downloadHTML(o); // this takes one of your object arrays and returns the text of an html page
insertIntoDB(html); // this is the code in the latter half of your downloadURL method
}catch (/*narrow exception type indicating download failure*/Exception e) {
werk(q, executorService);
}
}catch (NoSuchElementException e) {}
});
}
^^^ This method does most of the work.
for (String category : categoriesList) {
Queue<Object[]> q = new ConcurrentLinkedQueue<>(dbUtils.select(sql, category));
IntStream.range(0, 50).forEach(i -> werk(q, executorService));
}
^^^ this is the for loop in your main
Now each category tries to download 50 pages, upon failure of downloading a page it moves on and tries to download another page. In this way, you will either download 50 pages or have attempted to download all pages in the category.