I have VertX application with a config.json file. I want to edit one of the properties in the file during an HTTP request.
Is it possible?
Json Config:
{
"REQUEST_OPTIONS": {
"dataFilter": [
{
"name": "xxx",
"values": [1, 2, 3] -> want to edit this specific field!
},
{
"name": "yyy",
"values": [4, 5, 6]
}
]
}
}
Start method:
#Override
public void start(Promise<Void> startPromise) {
CompositeFuture.all(
//Some other Futures ,
Future.future(ConfigRetriever.create(vertx)::getConfig)
.flatMap(config -> storeData(config)
.compose(maybeStoreData -> CompositeFuture.all(
matcherVerticle(subject).apply(maybeStoreData),
http(sharedData, maybeStoreData).apply(config)
))))
.<Void>mapEmpty()
.onComplete(startPromise);
Router code:
router.get("/api/action").handler(performAction(configJson));
Action request:
private Handler<RoutingContext> performAction(JsonObject config) {
return routingContext -> Try.of(() -> Json.decodeValue(routingContext.getBody(), RequestDto.class))
.onFailure(ex -> log.warn("Failed to process the RequestDto: ", ex))
.andThen(requestDto -> ???UPDATE CONFIG FILE according to requestDto ???)
.onFailure(ex -> log.warn("Failed to save request: ", ex))
.onFailure(ex -> routingContext.response().setStatusCode(HttpResponseStatus.INTERNAL_SERVER_ERROR.code()))
.onSuccess(handler -> routingContext.response().setStatusCode(HttpResponseStatus.OK.code()));
}
Given that you have your JsonObject already, you can write it to disk using fileSystem().writeFile().
Here's a full example:
Vertx vertx = Vertx.vertx();
var json = new JsonObject("{\n" +
" \"REQUEST_OPTIONS\": {\n" +
" \"dataFilter\": [\n" +
" {\n" +
" \"name\": \"xxx\",\n" +
" \"values\": [1, 2, 3]" +
" },\n" +
" {\n" +
" \"name\": \"yyy\",\n" +
" \"values\": [4, 5, 6]\n" +
" }\n" +
" ]\n" +
" }\n" +
"}");
json.copy()
.getJsonObject("REQUEST_OPTIONS")
// Accessing the dataFilter array
.getJsonArray("dataFilter")
// Getting the first element of dataFilter array
.getJsonObject(0)
// Accessing the values array
.getJsonArray("values")
.add(4);
vertx.fileSystem().writeFile("./new.json", json.toBuffer());
Related
This question already has answers here:
How to parse JSON in Java
(36 answers)
Closed 12 days ago.
I recently stumbled on an issue with parsing mapping values which are handed over via a List.
I receive a Json and within the JSON there is an extra field attributes. Which looks like this
"attributes": [
{
"id": "id",
"value": "12345677890124566"
},
{
"id": "Criticality",
"value": "medium"
},
{
"id": "type",
"value": "business"
},
{
"id": "active",
"value": "true"
}
],
I fetch it via parsing it into a List via (List<Map<String, String>>) request.get("attributes") attributes.
I parse through the list via : for (Map<String, String> attribute : attributes).
I am not able to get the value of any attribute. I tried stuff like get("active"), containsKey and much more the only result I get is null.
I tried parsing the value from the mapping for an attribute but received only null instead of the value.
You attributes data is not a map in the Java sense. It is an array of objects each with an id and a value.
This will work:
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.node.ArrayNode;
public class AttributeParser {
public static void main(String[] args) throws JsonMappingException, JsonProcessingException {
ObjectMapper objectMapper = new ObjectMapper() ;
String requestJSON = "{ \"attributes\": [\r\n"
+ " {\r\n"
+ " \"id\": \"id\",\r\n"
+ " \"value\": \"12345677890124566\"\r\n"
+ " },\r\n"
+ " {\r\n"
+ " \"id\": \"Criticality\",\r\n"
+ " \"value\": \"medium\"\r\n"
+ " },\r\n"
+ " {\r\n"
+ " \"id\": \"type\",\r\n"
+ " \"value\": \"business\"\r\n"
+ " },\r\n"
+ " {\r\n"
+ " \"id\": \"active\",\r\n"
+ " \"value\": \"true\"\r\n"
+ " }\r\n"
+ " ]}";
JsonNode request = objectMapper.readTree(requestJSON);
ArrayNode attributesNode = (ArrayNode) request.get("attributes");
attributesNode.forEach(jsonNode -> {
System.out.println("id: " + jsonNode.get("id").asText() + ", value: " + jsonNode.get("value").asText());
});
}
}
you try to access to map data with get("active")
but you should use get("value")
i hope this example can help :
for(Map<String, String> attribute: attributes){
String value = attribute.get("value");
system.out.print(value); }
I wonder if exists a quick way to deserialize a list of objects into a list of fields from that object?
I have such object:
#Data
public class Sample {
private List<String> ids;
}
And I have a json:
{
"ids": [
{
"id": "a"
},
{
"id": "b"
},
{
"id": "c"
}
]
}
I would like to parse such json object list into string list in "ids" field.
I know that I can do it using StdSerializer abstract class and implement deserialization logic for it but I wonder if exists something more smooth to make it flat.
One possible solution is to transform the JSON structure before deserialization.
https://github.com/octomix/josson
Josson josson = Josson.fromJsonString(
"{" +
" \"ids\": [" +
" {" +
" \"id\": \"a\"" +
" }," +
" {" +
" \"id\": \"b\"" +
" }," +
" {" +
" \"id\": \"c\"" +
" }" +
" ]" +
"}");
// Deserialize to Sample
JsonNode objectNode = josson.getNode("ids.id.toObject('ids')");
Sample sample = new ObjectMapper().readerFor(Sample.class).readValue(objectNode);
System.out.println(sample.ids.toString());
// Or deserialize to List<String> directly
JsonNode arrayNode = josson.getNode("ids.id");
List<String> ids = new ObjectMapper().readerFor(List.class).readValue(arrayNode);
System.out.println(ids.toString());
Output
[a, b, c]
My API version is 6.3 and I try to create index. The failure says "The mapping definition cannot be nested under a type [_doc] unless include_type_name is set to true" .Here is my code:
CreateIndexRequest indexRequest = new CreateIndexRequest("user");
request.source("{\n" +
" \"settings\" : {\n" +
" \"number_of_shards\" : 1,\n" +
" \"number_of_replicas\" : 0\n" +
" },\n" +
" \"mappings\" : {\n" +
" \"_doc\" : {\n" +
" \"properties\" : {\n" +
" \"message\" : { \"type\" : \"text\" }\n" +
" }\n" +
" }\n" +
" },\n" +
" \"aliases\" : {\n" +
" \"twitter_alias\" : {}\n" +
" }\n" +
"}", XContentType.JSON);
CreateIndexResponse createIndexResponse = restHighLevelClient.indices().create(indexRequest);
After I remove "_doc" as following, failure says "Failed to parse mapping [_doc]: No type specified for field [properties]". So what should I do now?
request.source("{\n" +
" \"settings\" : {\n" +
" \"number_of_shards\" : 1,\n" +
" \"number_of_replicas\" : 0\n" +
" },\n" +
" \"mappings\" : {\n" +
"
" \"properties\" : {\n" +
" \"message\" : { \"type\" : \"text\" }\n" +
" }\n" +
"
" },\n" +
" \"aliases\" : {\n" +
" \"twitter_alias\" : {}\n" +
" }\n" +
"}", XContentType.JSON);
_doc was a temporary type name which was kept for backward compatibility and to prepare for removing it, refer removal of type for more details on this.
Also refer schedule and how to handle this breaking change on your version
Indices created in 6.x only allow a single-type per index. Any name
can be used for the type, but there can be only one. The preferred
type name is _doc, so that index APIs have the same path as they
will have in 7.0: PUT {index}/_doc/{id} and POST {index}/_doc
Also please refer https://www.elastic.co/guide/en/elasticsearch/reference/7.x/removal-of-types.html#_migrating_multi_type_indices_to_single_type this section where its explained how to use it in 6.X.
I would suggest first try to create a index using REST API and see with the combination of include_type_name and index.mapping.single_type it works or not and accordingly add it in your code.
The reason was a mapping API query. I fixed as below.
# ES6
PUT myindex
{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 2
},
"mappings": {
"_doc": { <--------- Delete this layer
"properties": {
"blahblah": {
# ES7
PUT myindex
{
"settings": {
"number_of_shards": 5,
"number_of_replicas": 2
},
"mappings": {
"properties": {
"blahblah": {
Jest provides a brilliant async API for elasticsearch, we find it very usefull. However, sometimes it turns out that resulting requests are slightly different than what we would expect.
Usually we didn't care, since everything was working fine, but in this case it was not.
I want to create an index with a custom ngram analyzer. When I do this following the elasticsearch rest API docs, I call below:
curl -XPUT 'localhost:9200/test' --data '
{
"settings": {
"number_of_shards": 3,
"analysis": {
"filter": {
"keyword_search": {
"type": "edge_ngram",
"min_gram": 3,
"max_gram": 15
}
},
"analyzer": {
"keyword": {
"type": "custom",
"tokenizer": "whitespace",
"filter": [
"lowercase",
"keyword_search"
]
}
}
}
}
}'
and then I confirm the analyzer is configured properly using:
curl -XGET 'localhost:9200/test/_analyze?analyzer=keyword&text=Expecting many tokens
in response I receive multiple tokens like exp, expe, expec and so on.
Now using Jest client I put the config json to a file on my classpath, the content is exactly the same as the body of the PUT request above. I execute the Jest action constructed like this:
new CreateIndex.Builder(name)
.settings(
ImmutableSettings.builder()
.loadFromClasspath(
"settings.json"
).build().getAsMap()
).build();
In result
Primo - checked with tcpdump that what's actually posted to elasticsearch is (pretty printed):
{
"settings.analysis.filter.keyword_search.max_gram": "15",
"settings.analysis.filter.keyword_search.min_gram": "3",
"settings.analysis.analyzer.keyword.tokenizer": "whitespace",
"settings.analysis.filter.keyword_search.type": "edge_ngram",
"settings.number_of_shards": "3",
"settings.analysis.analyzer.keyword.filter.0": "lowercase",
"settings.analysis.analyzer.keyword.filter.1": "keyword_search",
"settings.analysis.analyzer.keyword.type": "custom"
}
Secundo - the resulting index settings is:
{
"test": {
"settings": {
"index": {
"settings": {
"analysis": {
"filter": {
"keyword_search": {
"type": "edge_ngram",
"min_gram": "3",
"max_gram": "15"
}
},
"analyzer": {
"keyword": {
"filter": [
"lowercase",
"keyword_search"
],
"type": "custom",
"tokenizer": "whitespace"
}
}
},
"number_of_shards": "3" <-- the only difference from the one created with rest call
},
"number_of_shards": "3",
"number_of_replicas": "0",
"version": {"created": "1030499"},
"uuid": "Glqf6FMuTWG5EH2jarVRWA"
}
}
}
}
Tertio - checking the analyzer with curl -XGET 'localhost:9200/test/_analyze?analyzer=keyword&text=Expecting many tokens I get just one token!
Question 1. What is the reason that Jest does not post my original settings json, but some processed one instead?
Question 2. Why the settings generated by Jest are not working?
Glad you found Jest useful, please see my answer below.
Question 1. What is the reason that Jest does not post my original
settings json, but some processed one instead?
It's not Jest but the Elasticsearch's ImmutableSettings doing that, see:
Map test = ImmutableSettings.builder()
.loadFromSource("{\n" +
" \"settings\": {\n" +
" \"number_of_shards\": 3,\n" +
" \"analysis\": {\n" +
" \"filter\": {\n" +
" \"keyword_search\": {\n" +
" \"type\": \"edge_ngram\",\n" +
" \"min_gram\": 3,\n" +
" \"max_gram\": 15\n" +
" }\n" +
" },\n" +
" \"analyzer\": {\n" +
" \"keyword\": {\n" +
" \"type\": \"custom\",\n" +
" \"tokenizer\": \"whitespace\",\n" +
" \"filter\": [\n" +
" \"lowercase\",\n" +
" \"keyword_search\"\n" +
" ]\n" +
" }\n" +
" }\n" +
" }\n" +
" }\n" +
"}").build().getAsMap();
System.out.println("test = " + test);
outputs:
test = {
settings.analysis.filter.keyword_search.type=edge_ngram,
settings.number_of_shards=3,
settings.analysis.analyzer.keyword.filter.0=lowercase,
settings.analysis.analyzer.keyword.filter.1=keyword_search,
settings.analysis.analyzer.keyword.type=custom,
settings.analysis.analyzer.keyword.tokenizer=whitespace,
settings.analysis.filter.keyword_search.max_gram=15,
settings.analysis.filter.keyword_search.min_gram=3
}
Question 2. Why the settings generated by Jest are not working?
Because your usage of settings JSON/map is not the intended case. I have created this test to reproduce your case (it's a bit long but bear with me):
#Test
public void createIndexTemp() throws IOException {
String index = "so_q_26949195";
String settingsAsString = "{\n" +
" \"settings\": {\n" +
" \"number_of_shards\": 3,\n" +
" \"analysis\": {\n" +
" \"filter\": {\n" +
" \"keyword_search\": {\n" +
" \"type\": \"edge_ngram\",\n" +
" \"min_gram\": 3,\n" +
" \"max_gram\": 15\n" +
" }\n" +
" },\n" +
" \"analyzer\": {\n" +
" \"keyword\": {\n" +
" \"type\": \"custom\",\n" +
" \"tokenizer\": \"whitespace\",\n" +
" \"filter\": [\n" +
" \"lowercase\",\n" +
" \"keyword_search\"\n" +
" ]\n" +
" }\n" +
" }\n" +
" }\n" +
" }\n" +
"}";
Map settingsAsMap = ImmutableSettings.builder()
.loadFromSource(settingsAsString).build().getAsMap();
CreateIndex createIndex = new CreateIndex.Builder(index)
.settings(settingsAsString)
.build();
JestResult result = client.execute(createIndex);
assertTrue(result.getErrorMessage(), result.isSucceeded());
GetSettings getSettings = new GetSettings.Builder().addIndex(index).build();
result = client.execute(getSettings);
assertTrue(result.getErrorMessage(), result.isSucceeded());
System.out.println("SETTINGS SENT AS STRING settingsResponse = " + result.getJsonString());
Analyze analyze = new Analyze.Builder()
.index(index)
.analyzer("keyword")
.source("Expecting many tokens")
.build();
result = client.execute(analyze);
assertTrue(result.getErrorMessage(), result.isSucceeded());
Integer actualTokens = result.getJsonObject().getAsJsonArray("tokens").size();
assertTrue("Expected multiple tokens but got " + actualTokens, actualTokens > 1);
analyze = new Analyze.Builder()
.analyzer("keyword")
.source("Expecting single token")
.build();
result = client.execute(analyze);
assertTrue(result.getErrorMessage(), result.isSucceeded());
actualTokens = result.getJsonObject().getAsJsonArray("tokens").size();
assertTrue("Expected single token but got " + actualTokens, actualTokens == 1);
admin().indices().delete(new DeleteIndexRequest(index)).actionGet();
createIndex = new CreateIndex.Builder(index)
.settings(settingsAsMap)
.build();
result = client.execute(createIndex);
assertTrue(result.getErrorMessage(), result.isSucceeded());
getSettings = new GetSettings.Builder().addIndex(index).build();
result = client.execute(getSettings);
assertTrue(result.getErrorMessage(), result.isSucceeded());
System.out.println("SETTINGS AS MAP settingsResponse = " + result.getJsonString());
analyze = new Analyze.Builder()
.index(index)
.analyzer("keyword")
.source("Expecting many tokens")
.build();
result = client.execute(analyze);
assertTrue(result.getErrorMessage(), result.isSucceeded());
actualTokens = result.getJsonObject().getAsJsonArray("tokens").size();
assertTrue("Expected multiple tokens but got " + actualTokens, actualTokens > 1);
}
When you run it you'll see that the case where settingsAsMap is used the actual settings is totally wrong (settings includes another settings which is your JSON but they should have been merged) and so the analyze fails.
Why is this not the intended usage?
Simply because that's how Elasticsearch behaves in this situation. If the settings data is flattened (as it is done by default by the ImmutableSettings class) then it should not have the top level element settings but it can have the same top level element if data is not flattened (and that's why the test case with settingsAsString works).
tl;dr:
Your settings JSON should not include the top level "settings" element (if you run it through ImmutableSettings).
Hello I'm trying to compare two json in java, each key can contain a json object or array of json objects, and each one of them can also be an array or json.
Here is an example of the Json:
{
"id": "123123asd123",
"attributes": [
{
"name": "apps",
"values": [
"111",
"222"
]
},
{
"name": "city",
"values": [
"NY"
]
}
]
}
I want to be able to get two json from this kind and compare them without caring about the order of the arrays.
As you can see the key 'attributes is an array of json so if i have another json like the one up here and the element in the array with key city is before apps i want the test to pass.
aswell of the numbers inside the apps values i dont care if it is 111,222 or 222,111
If anyone know any external java lib that is doing that ill be happy hear.
Or any idea how to implement this compare manually? or even an idea that will this kind of json and reorganize it so it will be easy to compare share that with me.
Take a look at this library she is cool i'm using it all the day for my Webservice Test:
https://github.com/jayway/JsonPath
#Test
public void test() {
String json = "{\n" +
" \"id\": \"123123asd123\",\n" +
" \"attributes\": [\n" +
" {\n" +
" \"name\": \"apps\",\n" +
" \"values\": [\n" +
" \"111\",\n" +
" \"222\"\n" +
" ]\n" +
" },\n" +
" {\n" +
" \"name\": \"city\",\n" +
" \"values\": [\n" +
" \"NY\"\n" +
" ]\n" +
" }\n" +
" ]\n" +
"}";
List<String> names = JsonPath.read(json, "$.attributes[*].name");
for(String name : names) {
//TODO assert that name is in other list from other json
}