I'm writing a tool to modify huge json file in groovy. I read this file, add new entry and save, but I'would like to avoid changes in places I didn't touch.
I'm using new JsonBuilder( o ).toPrettyString() to get pretty formatted json output, but this function gives me result like this:
{
"key": "Foo",
"items": [
{
"Bar1": 1
},
{
"Bar2": 2
}
]
}
when I need to get this:
{
"key": "Foo",
"items":
[
{
"Bar1": 1
},
{
"Bar2": 2
}
]
}
There should be newline before [.
It's important to me, because in other way I cannot find in GIT history, what I really changed.
Do you have any idea how to achieve this?
The JsonBuilder method toPrettyString() delegates directly to JsonOutput.prettyPrint() as follows:
public String toPrettyString() {
return JsonOutput.prettyPrint(toString());
}
The latter method is not really customizable at all. However, the source is freely available from any Maven central repository or mirror. I would suggest finding the source and creating your own variant of the method that behaves the way you would like it to. The source for JsonOutput.prettyPrint() is only about 65 lines long and shouldn't be that hard to change.
Related
I'm using https://github.com/json-path/JsonPath, I want to set fields that might not be yet set in the document. For example: doc.set(JsonPath.compile("$.some.array[0].value"), "abc"); on an {} document would result in:
{
"some": {
"array": [
{
"value": "abc"
}
]
}
}
I was thinking of trying to using the the compiled path, and get access to the path tokens to work my way through the document, but it doesn't seem that they are accessible. Is there any good way or alternative way of doing this?
Being able to access the path tokens in order to create fields that are needed
I have already a working synonym.txt in solr. Now I want to add that same txt file at elasticsearch. What can I do for it? At solr it was easy, I just kept that file in the system. At elasticsearch I added this and also run some command, but it is not working.
PUT /test_index
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"synonym": {
"tokenizer": "whitespace",
"filter": [ "synonym" ]
}
},
"filter": {
"synonym": {
"type": "synonym",
"synonyms_path": "analysis/synonym.txt"
}
}
}
}
}
}
What's wrong? Do I need to index it again or do I need to map this with any field? My search result depends on multiple fields.
Hope you have applied your synonym on your existing fields in your ES mapping, you have just provided your index setting, and you need to provide the index mapping to confirm it.
Also adding an analyzer to the existing field is a breaking change and you have to reindex the data again to see the updated tokens.
You must use Analyze API to see the updated tokens on your index, Also please cross-check if you have added the synonym.txt properly and there was no error while creating the index setting with this file.
I either don't know what I'm looking for or the documentation is lacking. The latter seems to be the case, given this:
http://spark.apache.org/docs/2.2.2/api/java/org/apache/spark/sql/functions.html#to_json-org.apache.spark.sql.Column-java.util.Map-
"options - options to control how the struct column is converted into a json string. accepts the same options and the json data source."
Great! So, what are my options?
I'm doing something like this:
Dataset<Row> formattedReader = reader
.withColumn("id", lit(id))
.withColumn("timestamp", lit(timestamp))
.withColumn("data", to_json(struct("record_count")));
...and I get this result:
{
"id": "ABC123",
"timestamp": "2018-11-16 20:40:26.108",
"data": "{\"record_count\": 989}"
}
I'd like this (remove back-slashes and quotes from "data"):
{
"id": "ABC123",
"timestamp": "2018-11-16 20:40:26.108",
"data": {"record_count": 989}
}
Is this one of the options by chance? Is there a better guide out there for Spark? The most frustrating part about Spark hasn't been getting it to do what I want, it's been a lack of good information on what it can do.
You are json encoding twice for the record_count field. Remove to_json. struct alone should be sufficient.
As in change your code to something like this.
Dataset<Row> formattedReader = reader
.withColumn("id", lit(id))
.withColumn("timestamp", lit(timestamp))
.withColumn("data", struct("record_count"));
I'm looking for potentially some design pattern advice regarding object traversal to dynamically construct objects based on the data being presented.
Below, I am manually constructing this object. Initially, the root node is a BinaryLogicOpType but could be a different object based on the rootNodeType.
My question is.. I need to dynamically construct these objects of differing types based on the string data in my lists. What is the best route in doing so?
I'm willing to refine this questions if it's confusing.
String rootNodeType = fN.type;
BinaryLogicOpType _blop = new BinaryLogicOpType();
JAXBElement<BinaryLogicOpType> root = constructRootElement(_blop, factory, rootNodeType);
/** LETS CREATE THE FIRST CHILD ELEMENT */
BinaryComparisonOpType attrName1GTEFive = new BinaryComparisonOpType();
_blop.getOps().add(factory.createPropertyIsEqualTo(attrName1GTEFive));
JAXBElement<String> attr1Property = factory.createValueReference(fN.nodes.get(0).getProperty());
LiteralType attr1ValueLiteral = new LiteralType();
attr1ValueLiteral.getContent().add(fN.nodes.get(0).getValue());
JAXBElement<LiteralType> attr1Value = factory.createLiteral(attr1ValueLiteral);
attrName1GTEFive.getExpression().add(attr1Property);
attrName1GTEFive.getExpression().add(attr1Value);
Sample JSON
{
"type": "AND",
"filters": [
{
"type": "=",
"value": "exampleid",
"property": "id"
},
{
"type": "ILIKE",
"value": "*",
"property": "metacard-tags"
},
{
"type": "OR",
"filters": [
{
"type": ">=",
"value": null,
"property": "benumber"
},
{
"type": "ILIKE",
"value": "redshirts",
"property": "title",
"isCaseSensitive": true
}
]
}
]
}
Sample XML
<?xml version="1.0" encoding="UTF-8"?><fes:Filter xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns:fes="http://www.opengis.net/fes/2.0" xmlns:gml="http://www.opengis.net/gml/3.2">
<fes:Or>
<fes:PropertyIsLessThan matchAction="ANY" matchCase="false">
<fes:ValueReference>name</fes:ValueReference>
<fes:Function name="sub">
<fes:Literal>my-id</fes:Literal>
</fes:Function>
</fes:PropertyIsLessThan>
<fes:And>
<fes:PropertyIsGreaterThanOrEqualTo matchAction="ANY" matchCase="false">
<fes:ValueReference>attName</fes:ValueReference>
<fes:Literal>5</fes:Literal>
</fes:PropertyIsGreaterThanOrEqualTo>
<fes:PropertyIsLike escapeChar="\" matchCase="false" singleChar="?" wildCard="*">
<fes:ValueReference>title</fes:ValueReference>
<fes:Literal>greetings</fes:Literal>
</fes:PropertyIsLike>
<fes:PropertyIsEqual>
<fes:ValueReference>be_number</fes:ValueReference>
<fes:Function>
<fes:Parameter>null</fes:Parameter>
<fes:Parameter>null</fes:Parameter>
</fes:Function>
</fes:PropertyIsEqual>
</fes:And>
</fes:Or>
</fes:Filter>
I think there is really no easy solution, but I'm pretty sure it is possible to implement conversion in quite an elegant manner with OOP.
Essentially, you're dealing with ASTs in both cases. And the their structures are not so different at all. So if you'd manage to implement conversion routine for each of the possible node types, you should be able to convert the whole AST.
First of all, I'd start by implementing a good Java model for your JSON structure. Classes like AndExpression, ComparisonExpression, LikeExpression, OrExpression etc. For inspiration, check the generated classes of the Filter schema. Good modelling is essential here.
Use poliformic deserialization so that you get the proper object structure after parsing Jackson. Now you "only" need to convert this structure into your JAXB structure.
The simplest would be to add conversion methods to your JSON model classes directly. I'd make them all implement an interface with a method like:
public Object convertToJAXB(ObjectFactory objectFactory);
In the case of "leaf" AST nodes like ILIKE you can just construct the appropriate object directly.
In the case of "composite" AST nodes like AND or OR you will need to call convertToJAXB on each of the filters and then do some more work by wrapping results into JAXBElements. You might need some instanceof checks here; it should be possible to come up with a more intelligent solution.
I have JSON that needs to be converted to a Java Object. The JSONs I need to handle can look like this:
{
"documents": [
{
"title": "Jobs",
"is-saved": false,
"abstract": "<span class=\"hit\">Jobs</span> may refer to:\n\n* Steve <span class=\"hit\">Jobs</span> (1955–2011), co-founder and former CEO of consumer electronics company...<br />",
"id": "Jobs",
"url": "http://en.wikipedia.org/wiki/Jobs"
}
],
"keywords_local": [
{
"keyword": "Jobs",
"interest": 1,
"angle": 0
}
],
"sessionid": "6cd6402e-1f67-45a8-b0fa-e79a5d0d50f4",
"q": "Jobs",
}
This JSON is returned when entering a search keyword on a searchengine, in this case "Jobs". I have not named these variables-to-be-created, this JSON was just "given" to me from a similar earlier app. So I'm obviously having trouble with variables is-saved and abstract. Abstract is a reserved keyword and everywhere I read a reserved keyword CANNOT be used as a variable name.
I do not have access to the previous app that I am sort of updating and I guess the point to that is that I need to figure it out by myself ;) But I am a bit of a stand still now, have no idea of how to move forward.
I'm a newbie, so do not give me hell if I'm asking a stupid question, it's my first time coding any sort of app! ;)
Thanks for any help!
If you use GSON for parsing you can name your members as you want and annotate them for mapping.
#SerializedName("abstract")
private String abstractText;
Another option you've got is to use Jackson, and use the #JsonProperty annotation..
#JsonProperty("abstract")
private String abstractText;
In fact, it depends on the tool you are using. With tools mapping directly to your custom POJO (like GSON, Jackson), you need to map your JSON field name with your Java correct and valid field name.
If you use a mors basic library such as JSON.org's, there is no need to do so because you parse it to specific object allowing you to handle it.
JSONObject obj = new JSONObject(" .... ");
JSONArray arr = obj.getJSONArray("documents");
String abstractValue = arr.getJSONObject(0).getString("abstract");