We Keep Coding

Exists a quick way to deserialize a list of objects into a list of fields from that object?

I wonder if exists a quick way to deserialize a list of objects into a list of fields from that object?
I have such object:
#Data
public class Sample {
private List<String> ids;
}
And I have a json:
{
"ids": [
{
"id": "a"
},
{
"id": "b"
},
{
"id": "c"
}
]
}
I would like to parse such json object list into string list in "ids" field.
I know that I can do it using StdSerializer abstract class and implement deserialization logic for it but I wonder if exists something more smooth to make it flat.

One possible solution is to transform the JSON structure before deserialization.
https://github.com/octomix/josson
Josson josson = Josson.fromJsonString(
"{" +
" \"ids\": [" +
" {" +
" \"id\": \"a\"" +
" }," +
" {" +
" \"id\": \"b\"" +
" }," +
" {" +
" \"id\": \"c\"" +
" }" +
" ]" +
"}");
// Deserialize to Sample
JsonNode objectNode = josson.getNode("ids.id.toObject('ids')");
Sample sample = new ObjectMapper().readerFor(Sample.class).readValue(objectNode);
System.out.println(sample.ids.toString());
// Or deserialize to List<String> directly
JsonNode arrayNode = josson.getNode("ids.id");
List<String> ids = new ObjectMapper().readerFor(List.class).readValue(arrayNode);
System.out.println(ids.toString());
Output
[a, b, c]

How to parse only 1 variable from JSON HTTP response Spring Boot

Hi I would like to parse only 1 value from GET response in the most optimal way using Java (+Spring Boot).
{
"table": "A",
"currency": "usd",
"code": "USD",
"rates": [
{
"no": "073/A/NBP/2021",
"effectiveDate": "2021-04-16",
"mid": 3.7978
}
]
}
Im looking for way to parse "mid" value without creating DTO for this response. In the worst case I will do just a substring.

Try this.
String input = "{\r\n"
+ " \"table\": \"A\",\r\n"
+ " \"currency\": \"usd\",\r\n"
+ " \"code\": \"USD\",\r\n"
+ " \"rates\": [\r\n"
+ " {\r\n"
+ " \"no\": \"073/A/NBP/2021\",\r\n"
+ " \"effectiveDate\": \"2021-04-16\",\r\n"
+ " \"mid\": 3.7978\r\n"
+ " }\r\n"
+ " ]\r\n"
+ "}";
String midValue = input.replaceFirst("(?s).*\"mid\"\\s*:\\s*([-.\\d]+).*", "$1");
System.out.println(midValue);
output:
3.7978

How to filter list of data in list in java with sql or nested dynamic condition

How to filter data dynamically using java, Assume we have data (list of map / json array without pojo mapping).
[
{
"id": "1001",
"type": "Internal",
"status": "Closed"
},
{
"id": "1002",
"type": "External",
"status": "Closed"
},
{
"id": "1003",
"type": "Internal",
"status": "Open"
},
{
"id": "1004",
"type": "Internal",
"status": "Open"
}
]
Now we need out put filtered data as id > 1001 and ( type: 'External' or status: 'Open" )
[
{
"id": "1002",
"type": "External",
"status": "Closed"
},
{
"id": "1003",
"type": "Internal",
"status": "Open"
},
{
"id": "1004",
"type": "Internal",
"status": "Open"
}
]
Any Suggestions how to achieve this ?

Use JSON path , below are few JSONPath queries I have posted as an sample example
P.S :- Ur "Id" should be of type Integer for operation > or <
Code:-
public static void main(String[] args) {
String jsonData = "[\r\n" +
" {\r\n" +
" \"id\": 1001,\r\n" +
" \"type\": \"Internal\",\r\n" +
" \"status\": \"Closed\"\r\n" +
" },\r\n" +
" {\r\n" +
" \"id\": 1002,\r\n" +
" \"type\": \"External\",\r\n" +
" \"status\": \"Closed\"\r\n" +
" },\r\n" +
" {\r\n" +
" \"id\": 1003,\r\n" +
" \"type\": \"Internal\",\r\n" +
" \"status\": \"Open\"\r\n" +
" },\r\n" +
" {\r\n" +
" \"id\": 1004,\r\n" +
" \"type\": \"Internal\",\r\n" +
" \"status\": \"Open\"\r\n" +
" }\r\n" +
"]";
String filterId = "$.[?(#.id > 1001)]"; //For Id > 1001
String filterType = "$.[?(#.type in ['External'])]"; //for External type
String filterTypeAndId = "$.[?((#.id > 1001) && (#.type in ['Internal']))]"; //for External type with Id > 1001
String filterTypeAndId2 = "$.[?((#.id > 1001) && (#.type in ['Internal', 'External']))]"; //for External type with Id > 1001
DocumentContext documentContext = JsonPath.parse(jsonData);
System.out.println(documentContext.read(filterId).toString());
System.out.println(documentContext.read(filterType).toString());
System.out.println(documentContext.read(filterTypeAndId).toString());
System.out.println(documentContext.read(filterTypeAndId2).toString());
}
results are :-
[{"id":1002,"type":"External","status":"Closed"},{"id":1003,"type":"Internal","status":"Open"},{"id":1004,"type":"Internal","status":"Open"}]
[{"id":1002,"type":"External","status":"Closed"}]
[{"id":1003,"type":"Internal","status":"Open"},{"id":1004,"type":"Internal","status":"Open"}]
[{"id":1002,"type":"External","status":"Closed"},{"id":1003,"type":"Internal","status":"Open"},{"id":1004,"type":"Internal","status":"Open"}]

We can use javascript ability to resolve condition. We can use Java ScriptEngineManager to run javascript expression :
Expression can be type = 'Internal' AND status ='Open'
private String getResult(String expression) {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("JavaScript");
try {
return engine.eval(expression).toString();
} catch (Exception e) {
}
return null;
}

You can achieve this with the following approach :-
Convert json to string
Construct JSONArray >> JSONArray jsonArray = new JSONArray(converted_json_String)
pull out jsonArrayObject from JsonArray and construct a list
Use Collections.sort method and deploy your comparision logic in it on the JSON keys
and construct your sorted json.

try this code, I am assuming you have a list of objects.
List<YourCLass> filteredList = list
.stream()
.filter(obj -> obj.getId() > 1001 && (obj.getStatus().equals("Open") || obj.getType().equals("External")))
.collect(Collectors.toList());
filteredList have the list of objects you are expecting, using Object mapper you can get it as json format.
String filteredObjJson = new ObjectMapper().writeValuesAsString(filteredList);
add this dependency in pom.xml or download the lib and add it to your libraries
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
</dependency>

Troubles creating an index with custom analyzer using Jest

Jest provides a brilliant async API for elasticsearch, we find it very usefull. However, sometimes it turns out that resulting requests are slightly different than what we would expect.
Usually we didn't care, since everything was working fine, but in this case it was not.
I want to create an index with a custom ngram analyzer. When I do this following the elasticsearch rest API docs, I call below:
curl -XPUT 'localhost:9200/test' --data '
{
"settings": {
"number_of_shards": 3,
"analysis": {
"filter": {
"keyword_search": {
"type": "edge_ngram",
"min_gram": 3,
"max_gram": 15
}
},
"analyzer": {
"keyword": {
"type": "custom",
"tokenizer": "whitespace",
"filter": [
"lowercase",
"keyword_search"
]
}
}
}
}
}'
and then I confirm the analyzer is configured properly using:
curl -XGET 'localhost:9200/test/_analyze?analyzer=keyword&text=Expecting many tokens
in response I receive multiple tokens like exp, expe, expec and so on.
Now using Jest client I put the config json to a file on my classpath, the content is exactly the same as the body of the PUT request above. I execute the Jest action constructed like this:
new CreateIndex.Builder(name)
.settings(
ImmutableSettings.builder()
.loadFromClasspath(
"settings.json"
).build().getAsMap()
).build();
In result
Primo - checked with tcpdump that what's actually posted to elasticsearch is (pretty printed):
{
"settings.analysis.filter.keyword_search.max_gram": "15",
"settings.analysis.filter.keyword_search.min_gram": "3",
"settings.analysis.analyzer.keyword.tokenizer": "whitespace",
"settings.analysis.filter.keyword_search.type": "edge_ngram",
"settings.number_of_shards": "3",
"settings.analysis.analyzer.keyword.filter.0": "lowercase",
"settings.analysis.analyzer.keyword.filter.1": "keyword_search",
"settings.analysis.analyzer.keyword.type": "custom"
}
Secundo - the resulting index settings is:
{
"test": {
"settings": {
"index": {
"settings": {
"analysis": {
"filter": {
"keyword_search": {
"type": "edge_ngram",
"min_gram": "3",
"max_gram": "15"
}
},
"analyzer": {
"keyword": {
"filter": [
"lowercase",
"keyword_search"
],
"type": "custom",
"tokenizer": "whitespace"
}
}
},
"number_of_shards": "3" <-- the only difference from the one created with rest call
},
"number_of_shards": "3",
"number_of_replicas": "0",
"version": {"created": "1030499"},
"uuid": "Glqf6FMuTWG5EH2jarVRWA"
}
}
}
}
Tertio - checking the analyzer with curl -XGET 'localhost:9200/test/_analyze?analyzer=keyword&text=Expecting many tokens I get just one token!
Question 1. What is the reason that Jest does not post my original settings json, but some processed one instead?
Question 2. Why the settings generated by Jest are not working?

Glad you found Jest useful, please see my answer below.
Question 1. What is the reason that Jest does not post my original
settings json, but some processed one instead?
It's not Jest but the Elasticsearch's ImmutableSettings doing that, see:
Map test = ImmutableSettings.builder()
.loadFromSource("{\n" +
" \"settings\": {\n" +
" \"number_of_shards\": 3,\n" +
" \"analysis\": {\n" +
" \"filter\": {\n" +
" \"keyword_search\": {\n" +
" \"type\": \"edge_ngram\",\n" +
" \"min_gram\": 3,\n" +
" \"max_gram\": 15\n" +
" }\n" +
" },\n" +
" \"analyzer\": {\n" +
" \"keyword\": {\n" +
" \"type\": \"custom\",\n" +
" \"tokenizer\": \"whitespace\",\n" +
" \"filter\": [\n" +
" \"lowercase\",\n" +
" \"keyword_search\"\n" +
" ]\n" +
" }\n" +
" }\n" +
" }\n" +
" }\n" +
"}").build().getAsMap();
System.out.println("test = " + test);
outputs:
test = {
settings.analysis.filter.keyword_search.type=edge_ngram,
settings.number_of_shards=3,
settings.analysis.analyzer.keyword.filter.0=lowercase,
settings.analysis.analyzer.keyword.filter.1=keyword_search,
settings.analysis.analyzer.keyword.type=custom,
settings.analysis.analyzer.keyword.tokenizer=whitespace,
settings.analysis.filter.keyword_search.max_gram=15,
settings.analysis.filter.keyword_search.min_gram=3
}
Question 2. Why the settings generated by Jest are not working?
Because your usage of settings JSON/map is not the intended case. I have created this test to reproduce your case (it's a bit long but bear with me):
#Test
public void createIndexTemp() throws IOException {
String index = "so_q_26949195";
String settingsAsString = "{\n" +
" \"settings\": {\n" +
" \"number_of_shards\": 3,\n" +
" \"analysis\": {\n" +
" \"filter\": {\n" +
" \"keyword_search\": {\n" +
" \"type\": \"edge_ngram\",\n" +
" \"min_gram\": 3,\n" +
" \"max_gram\": 15\n" +
" }\n" +
" },\n" +
" \"analyzer\": {\n" +
" \"keyword\": {\n" +
" \"type\": \"custom\",\n" +
" \"tokenizer\": \"whitespace\",\n" +
" \"filter\": [\n" +
" \"lowercase\",\n" +
" \"keyword_search\"\n" +
" ]\n" +
" }\n" +
" }\n" +
" }\n" +
" }\n" +
"}";
Map settingsAsMap = ImmutableSettings.builder()
.loadFromSource(settingsAsString).build().getAsMap();
CreateIndex createIndex = new CreateIndex.Builder(index)
.settings(settingsAsString)
.build();
JestResult result = client.execute(createIndex);
assertTrue(result.getErrorMessage(), result.isSucceeded());
GetSettings getSettings = new GetSettings.Builder().addIndex(index).build();
result = client.execute(getSettings);
assertTrue(result.getErrorMessage(), result.isSucceeded());
System.out.println("SETTINGS SENT AS STRING settingsResponse = " + result.getJsonString());
Analyze analyze = new Analyze.Builder()
.index(index)
.analyzer("keyword")
.source("Expecting many tokens")
.build();
result = client.execute(analyze);
assertTrue(result.getErrorMessage(), result.isSucceeded());
Integer actualTokens = result.getJsonObject().getAsJsonArray("tokens").size();
assertTrue("Expected multiple tokens but got " + actualTokens, actualTokens > 1);
analyze = new Analyze.Builder()
.analyzer("keyword")
.source("Expecting single token")
.build();
result = client.execute(analyze);
assertTrue(result.getErrorMessage(), result.isSucceeded());
actualTokens = result.getJsonObject().getAsJsonArray("tokens").size();
assertTrue("Expected single token but got " + actualTokens, actualTokens == 1);
admin().indices().delete(new DeleteIndexRequest(index)).actionGet();
createIndex = new CreateIndex.Builder(index)
.settings(settingsAsMap)
.build();
result = client.execute(createIndex);
assertTrue(result.getErrorMessage(), result.isSucceeded());
getSettings = new GetSettings.Builder().addIndex(index).build();
result = client.execute(getSettings);
assertTrue(result.getErrorMessage(), result.isSucceeded());
System.out.println("SETTINGS AS MAP settingsResponse = " + result.getJsonString());
analyze = new Analyze.Builder()
.index(index)
.analyzer("keyword")
.source("Expecting many tokens")
.build();
result = client.execute(analyze);
assertTrue(result.getErrorMessage(), result.isSucceeded());
actualTokens = result.getJsonObject().getAsJsonArray("tokens").size();
assertTrue("Expected multiple tokens but got " + actualTokens, actualTokens > 1);
}
When you run it you'll see that the case where settingsAsMap is used the actual settings is totally wrong (settings includes another settings which is your JSON but they should have been merged) and so the analyze fails.
Why is this not the intended usage?
Simply because that's how Elasticsearch behaves in this situation. If the settings data is flattened (as it is done by default by the ImmutableSettings class) then it should not have the top level element settings but it can have the same top level element if data is not flattened (and that's why the test case with settingsAsString works).
tl;dr:
Your settings JSON should not include the top level "settings" element (if you run it through ImmutableSettings).

Compare JSON with nested arrays and jsons (Array order does not matter)

Hello I'm trying to compare two json in java, each key can contain a json object or array of json objects, and each one of them can also be an array or json.
Here is an example of the Json:
{
"id": "123123asd123",
"attributes": [
{
"name": "apps",
"values": [
"111",
"222"
]
},
{
"name": "city",
"values": [
"NY"
]
}
]
}
I want to be able to get two json from this kind and compare them without caring about the order of the arrays.
As you can see the key 'attributes is an array of json so if i have another json like the one up here and the element in the array with key city is before apps i want the test to pass.
aswell of the numbers inside the apps values i dont care if it is 111,222 or 222,111
If anyone know any external java lib that is doing that ill be happy hear.
Or any idea how to implement this compare manually? or even an idea that will this kind of json and reorganize it so it will be easy to compare share that with me.

Take a look at this library she is cool i'm using it all the day for my Webservice Test:
https://github.com/jayway/JsonPath
#Test
public void test() {
String json = "{\n" +
" \"id\": \"123123asd123\",\n" +
" \"attributes\": [\n" +
" {\n" +
" \"name\": \"apps\",\n" +
" \"values\": [\n" +
" \"111\",\n" +
" \"222\"\n" +
" ]\n" +
" },\n" +
" {\n" +
" \"name\": \"city\",\n" +
" \"values\": [\n" +
" \"NY\"\n" +
" ]\n" +
" }\n" +
" ]\n" +
"}";
List<String> names = JsonPath.read(json, "$.attributes[*].name");
for(String name : names) {
//TODO assert that name is in other list from other json
}