We Keep Coding

How to convert a file to a String which is accepted in JSON?

I am trying to create gists in Github via REST ASSURED.
To create a gist a need to pass file names and their contents.
Now, the content of the file is something which is being rejected by the API.
Example:
{
"description": "Hello World Examples",
"public": true,
"files": {
"hello_world.rb": {
"content": "class HelloWorld\n def initialize(name)\n #name = name.capitalize\n end\n def sayHi\n puts \"Hello !\"\n end\nend\n\nhello = HelloWorld.new(\"World\")\nhello.sayHi"
},
"hello_world.py": {
"content": "class HelloWorld:\n\n def init(self, name):\n self.name = name.capitalize()\n \n def sayHi(self):\n print \"Hello \" + self.name + \"!\"\n\nhello = HelloWorld(\"world\")\nhello.sayHi()"
},
"hello_world_ruby.txt": {
"content": "Run ruby hello_world.rb to print Hello World"
},
"hello_world_python.txt": {
"content": "Run python hello_world.py to print Hello World"
}
}
This is how the the API wants the JSON to be, I could get this via my code:
{
"description": "Happy World",
"public": true,
"files": {
"sid.java": {
"content": "Ce4z5e22ta"
},
"siddharth.py": {
"content": "def a:
if sidh>kundu:
sid==kundu
else:
kundu==sid
"
}
}
}
So the change in the indentations is causing GitHUb API to fail this with 400 error. Can someone please help?

As pointed out in the comments, JSON does not allow control characters in strings. In the case of line breaks, these were encoded as \n in the example.
You should definitely consider using a proper library to create the JSON rather than handling the raw strings yourself.

Create a POJO which will represent your gist (i.e. object with fields like 'description', 'files' collection. And separate POJO for file containing string fields 'name' and 'content';
Do something like this to convert your gist:
try {
GistFile file new GistFile();// Assuming this is POJO for your file
//Set name and content
Gist gist = new Gist(); //Asuming this is a POJO for your gist
gist.addFile(file);
//Add more files if needed and set other properties
ObjectMapper mapper = new ObjectMapper();
String content = mapper.writeValueAsString(gist);
//Now you have valid JSON string
} catch (Exception e) {
e.printStackTrace();
}
This is for com.fasterxml.jackson.databind.ObjectMapper or use different JSON library
Actually there are GitHub specific libraries which do most of the job for you. Please refer to this question: How to connect to github using Java Program it might be helpful

Compare two json structure and get mismatch changes

Hi I have json structure #1 and #2 as follows. I would like to compare and capture the results.
Json #1.
{
"menu": {
"id": "file",
"popup": {
"menuitem": {
"menuitem-1": "sometext",
"menuitem-2": {
"menuitem-2.1": "sometext",
"menuitem-2.2": "sometext",
"menuitem-2.3": {
"menuitem-2.3.1": "sometext"
}
}
}
},
"value": "File"
}
}
Json #2
{
"menu": {
"id": "file",
"popup": {
"menuitem": {
"menuitem-2.3": {
"menuitem-2.3.1": "sometext"
}
"menuitem-1": "sometext",
"menuitem-2": {
"menuitem-2.1": "sometext",
"menuitem-2.2": "sometext"
},
}
},
"value": "File"
}
}
Am expecting that below JSON has been moved up in JSON #2. My goal here is identify any CREATE NEW / UPDATE / ADJUSTED / DELETE on JSON#2.
"menuitem-2.3": {
"menuitem-2.3.1": "sometext"
}
Is there any Spring / Java existing framework available to achieve above?

Use difference from org.apache.commons.lang.StringUtils.
Compares two Strings, and returns the portion where they differ. (More precisely, return the remainder of the second String, starting from where it's different from the first.)
For example,
difference("i am a machine", "i am a robot") -> "robot".
StringUtils.difference(null, null) = null
StringUtils.difference("", "") = ""
StringUtils.difference("", "abc") = "abc"
StringUtils.difference("abc", "") = ""
StringUtils.difference("abc", "abc") = ""
StringUtils.difference("ab", "abxyz") = "xyz"
StringUtils.difference("abcde", "abxyz") = "xyz"
StringUtils.difference("abcde", "xyz") = "xyz"
Parameters:
str1 - the first String, may be null
str2 - the second String, may be null

Try using Apache drill. It is easy to install and supports querying JSON. You can then execute a minus query and get the difference.
You can also query drill using java. Apache drill has a JDBC driver for that.
Hope it helps. :)

Improving processing time for mapping one json object to another

I am working on a module where i am getting a JSON response from a RESTful web service. The response is something like below.
[{
"orderNumber": "test order",
"orderDate": "2016 - 01 - 25",
"Billing": {
"Name": "Ron",
"Address": {
"Address1": "",
"City": ""
}
},
"Shipping": {
"Name": "Ron",
"Address": {
"Address1": "",
"City": ""
}
}
}]
This is not the complete response, but only with important elements just to elaborate the issue.
So what i need to do is, convert this JSON response into another JSON that my application understands and can process. Say the below for example.
{
"order_number": "test order",
"order_date": "2016-01-25",
"bill_to_name": "Ron",
"bill_to_address": "",
"bill_to_city": "",
"ship_from_name": "Ron",
"ship_from_Address": "",
"ship_from_city": ""
}
The idea that i had tried was to convert the JSONObject in the response i receive to a hashmap using JACKSON and then use StrSubstitutor to replace the placeholders in my application json with proper values from response json(My Application string with placeholders Shown below).
{"order_number":"${orderNumber}","order_date":"${orderDate}","bill_to_name":"${Billing.name}","bill_to_address":"${Billing.Address}","bill_to_city":"${Billing.City}","ship_from_name":"${Shipping.Name}","ship_from_Address":"${Shipping.Address}","ship_from_city":"${Shipping.City}"}
But the issue i faced was that
JSON to MAP didn't work with nested JSONOBJECT as shown in the response above.
Also to substitute Billing.Name/Shipping.Name etc, even if i extract the Shipping/Billing JSONObjects from the response, when i
would convert them to hashmap, they would give me Name, City,
Address1 as keys and not Billing.Name, Billing.City etc.
So as a solution i wrote the below piece of code which takes the response JSONObject(srcObject) and JSONObject of my application(destObject) as inputs, performs processing and fits in the values from the response JSON into my application JSON.
public void mapJsonToJson(final JSONObject srcObject, final JSONObject destObject){
for(String key : destObject.keys()){
String srcKey = destObject.getString(key)
if(srcKey.indexOf(".") != -1){
String[] jsonKeys = srcKey.split("\\.")
if(srcObject.has(jsonKeys[0])){
JSONObject tempJson
for(int i=0;i<jsonKeys.length - 1;i++){
if(i==0) {
tempJson = srcObject.getJSONObject(jsonKeys[i])
} else{
tempJson = tempJson.getJSONObject(jsonKeys[i])
}
}
destObject.put(key, tempJson.getString(jsonKeys[jsonKeys.length - 1]))
}
}else if(srcObject.has(srcKey)){
String value = srcObject.getString(srcKey)
destObject.put(key, value)
}
}
}
The issue with this piece of code is that it takes some time to process. I want to know is there a way i can implement this logic in a better way with less processing time?

You should create POJOs for your two data types, and then use Jackson's mapper to deserialize the REST data in as the first POJO, and then have a copy constructor on your second POJO that accepts the POJO from the REST service, and copies all the data to its fields. Then you can use Jackson's mapper to serialize the data back into JSON.
Only if the above still gives you performance issues would I start looking at faster but more difficult algorithms such as working with JsonParser/JsonGenerator directly to stream data.

I feel the standard approach will be to use XSLT equivalent for JSON. JOLT seems to be one such implementation. Demo page can be found here. Have a look at it.

Update nested field in an index of ElasticSearch with Java API

I am using Java API for CRUD operation on elasticsearch.
I have an typewith a nested field and I want to update this field.
Here is my mapping for the type:
"enduser": {
"properties": {
"location": {
"type": "nested",
"properties":{
"point":{"type":"geo_point"}
}
}
}
}
Of course my enduser type will have other parameters.
Now I want to add this document in my nested field:
"location":{
"name": "London",
"point": "44.5, 5.2"
}
I was searching in documentation on how to update nested document but I couldn't find anything. For example I have in a string the previous JSON obect (let's call this string json). I tried the following code but seems to not working:
params.put("location", json);
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location = location").setScriptParams(params).execute().actionGet();
I have got a parsing error from elasticsearch. Anyone knows what I am doing wrong ?

You don't need the script, just update it.
UpdateRequestBuilder br = client.prepareUpdate("index", "enduser", "1");
br.setDoc("{\"location\":{ \"name\": \"london\", \"point\": \"44.5,5.2\" }}".getBytes());
br.execute();

I tried to recreate your situation and i solved it by using an other way the .setScript method.
Your updating request now would looks like :
client.prepareUpdate(index, ElasticSearchConstants.TYPE_END_USER,id).setScript("ctx._source.location =" + json).execute().actionGet()
Hope it will help you.

I am not sure which ES version you were using, but the below solution worked perfectly for me on 2.2.0. I had to store information about named entities for news articles. I guess if you wish to have multiple locations in your case, it would also suit you.
This is the nested object I wanted to update:
"entities" : [
{
"disambiguated" : {
"entitySubTypes" : [],
"disambiguatedName" : "NameX"
},
"frequency" : 1,
"entityType" : "Organization",
"quotations" : ["...", "..."],
"name" : "entityX"
},
{
"disambiguated" : {
"entitySubType" : ["a", "b" ],
"disambiguatedName" : "NameQ"
},
"frequency" : 5,
"entityType" : "secondTypeTest",
"quotations" : [ "...", "..."],
"name" : "entityY"
}
],
and this is the code:
UpdateRequest updateRequest = new UpdateRequest();
updateRequest.index(indexName);
updateRequest.type(mappingName);
updateRequest.id(url); // docID is a url
XContentBuilder jb = XContentFactory.jsonBuilder();
jb.startObject(); // article
jb.startArray("entities"); // multiple entities
for ( /*each namedEntity*/) {
jb.startObject() // entity
.field("name", name)
.field("frequency",n)
.field("entityType", entityType)
.startObject("disambiguated") // disambiguation
.field("disambiguatedName", disambiguatedNameStr)
.field("entitySubTypes", entitySubTypeArray) // multi value field
.endObject() // disambiguation
.field("quotations", quotationsArray) // multi value field
.endObject(); // entity
}
jb.endArray(); // array of nested objects
b.endObject(); // article
updateRequest.doc(jb);
Blblblblblblbl's answer couldn't work for me atm, because scripts are not enabled in our server. I didn't try Bask's answer yet - Alcanzar's gave me a hard time, because I supposedly couldn't formulate the json string correctly that setDoc receives. I was constantly getting errors that either I am using objects instead of fields or vice versa. I also tried wrapping the json string with doc{} as indicated here, but I didn't manage to make it work. As you mentioned it is difficult to understand how to formulate a curl statement at ES's java API.

A simple way to update the arraylist and object value using Java API.
UpdateResponse update = client.prepareUpdate("indexname","type",""+id)
.addScriptParam("param1", arrayvalue)
.addScriptParam("param2", objectvalue)
.setScript("ctx._source.field1=param1;ctx._source.field2=param2").execute()
.actionGet();
arrayvalue-[
{
"text": "stackoverflow",
"datetime": "2010-07-27T05:41:52.763Z",
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
},
"id": 1,
}
object value -
"obj1": {
"id": 1,
"email": "sa#gmail.com",
"name": "bass"
}

How to create JSON Schema for Name/Value structure?

My problem is that i am serializing the content of map to JSON.
In the output (JSON), i have object that follow key/name syntax rule.
The key is created from map key, and the name from the value.
Model Example:
class Storage {
Map<String,String> values = new HashMap<>();
{
map.put("key1","key1");
map.put("key2","key2");
map.put("key3","key3");
}
}
JSON Example object:
{
key1=value1,
key2=value2,
key3=value3
}
JSON Schema:
{
"name": "storage",
"description": "Store of key values",
"properties": {
// How can we describe the properties if we do not know the name ?
}
}
The issue is that i do not know what the values will be but i know that they will be some.
Can you help me to provide me the full definition of schema?
Disclaimer:
I know that this can be also serialized as
{
values: [
{key="key1", value="value1"},
{key="key2", value="value2"},
{key="key3", value="value3"}
]
}
but is do not want to have array in the JSON.

Assuming your validator supports it you can use patternProperties.
For the schema...
{
"title": "Map<String,String>",
"type": "object",
"patternProperties": {
".{1,}": { "type": "string" }
}
}
...and the document...
{
"foo":"bar",
"baz":1
}
...the value of property foo is valid because it is a string but baz fails validation because it is a number.

I used the Solution suggested by #augurar
"additionalProperties": { "type": "string" }
for AWS API Gateway Model .... and the SDK was able to generate the Map variable as required in Java / Android SDK
#Arne Burmeister - in my case - Solution 1 didnt worked as needed - although it didnt gave any error in the Model (Schema Created)