We Keep Coding

Updating elements in a Map which has immutable objects

I have a Java Map which is deserialized into immutable objects from a json similar to below and has multiple levels of nesting. I have a second Map containing alternate values for some of the strings. The second map has the value appended to the key for the nested levels to differentiate between different elements in an array. I would like to replace the values in the first map using the values found in the second map. Are there any suggestions for the approach that could used to solve the below?
{
"product": "Internet",
"version": 3.1,
"releaseDate": "2014-06-25T00:00:00.000Z",
"person": {
"id": 12345,
"name": "John Doe",
"phones": {
"home": "800-123-4567",
"mobile": "877-123-1234"
},
"email": [
"jd#example.com",
"jd#example.org"
],
"dateOfBirth": "1980-01-02T00:00:00.000Z",
"registered": true,
"emergencyContacts": [
{
"name": "Jane Doe",
"phone": "888-555-1212",
"relationship": "spouse"
},
{
"name": "Justin Doe",
"phone": "877-123-1212",
"relationship": "parent"
}
]
}
}
Map containing alternate values
{
"product": "Strong Internet",
"person.name=John Doe" : "Barry Richards",
"person.emergencyContacts.name=Jane Doe" : "Jane Richards"
}
Resulting Map
{
"product": "Strong Internet",
"version": 3.1,
"releaseDate": "2014-06-25T00:00:00.000Z",
"person": {
"id": 12345,
"name": "Barry Richards",
"phones": {
"home": "800-123-4567",
"mobile": "877-123-1234"
},
"email": [
"jd#example.com",
"jd#example.org"
],
"dateOfBirth": "1980-01-02T00:00:00.000Z",
"registered": true,
"emergencyContacts": [
{
"name": "Jane Richards",
"phone": "888-555-1212",
"relationship": "spouse"
},
{
"name": "Justin Doe",
"phone": "877-123-1212",
"relationship": "parent"
}
]
}
}

Concatenating a string to a JOLT value output

I using JOLT to transform and extract a URL of an object, but it's only the URL slug. Is it possible to concatenate a static string to that slug to have a full URL?
The current output is:
{
"title" : "Where is deleted user data visible, and for how long is it there?",
"externalId" : "where_is_deleted_user_data_visible__and_for_how_lo",
"summary" : "Deleted user data is stored in LP and visible for 13 months following deletion.",
"detail" : "<p>Deleted user data is stored in LP and visible for <strong>13 months following deletion.</strong></p>\n<p>Deleted agents are displayed with a '(deleted)' indicator in the Open Conversations, All Conversations, Web History lists, and the Conversation History widget.</p>\n<object type=\"application/kenticocloud\" data-type=\"item\" data-rel=\"component\" data-codename=\"n6fdd091f_fcb5_012a_8745_c1671c268e6b\"></object>\n<p>Filter by deleted agents is available in the Open Conversations, All Conversations, and web history lists.</p>\n<p>In the visitor conversation window, the name of the deleted agent will be displayed in the window header (with no indication that they have been deleted).</p>\n<p>The LP user limit '<code>collections.maxElements</code>' will not count the number of deleted users in the total.</p>\n<object type=\"application/kenticocloud\" data-type=\"item\" data-rel=\"component\" data-codename=\"n3f1dea42_ac2e_01c6_cbd6_3f802d4e8fb3\"></object>",
"url" : "faqs-data-reporting1.html"
}
And the expected output is:
{
"title" : "Where is deleted user data visible, and for how long is it there?",
"externalId" : "where_is_deleted_user_data_visible__and_for_how_lo",
"summary" : "Deleted user data is stored in LP and visible for 13 months following deletion.",
"detail" : "<p>Deleted user data is stored in LP and visible for <strong>13 months following deletion.</strong></p>\n<p>Deleted agents are displayed with a '(deleted)' indicator in the Open Conversations, All Conversations, Web History lists, and the Conversation History widget.</p>\n<object type=\"application/kenticocloud\" data-type=\"item\" data-rel=\"component\" data-codename=\"n6fdd091f_fcb5_012a_8745_c1671c268e6b\"></object>\n<p>Filter by deleted agents is available in the Open Conversations, All Conversations, and web history lists.</p>\n<p>In the visitor conversation window, the name of the deleted agent will be displayed in the window header (with no indication that they have been deleted).</p>\n<p>The LP user limit '<code>collections.maxElements</code>' will not count the number of deleted users in the total.</p>\n<object type=\"application/kenticocloud\" data-type=\"item\" data-rel=\"component\" data-codename=\"n3f1dea42_ac2e_01c6_cbd6_3f802d4e8fb3\"></object>",
"url" : "https://www.static-url.com/faqs-data-reporting1.html"
}
The JOLT Spec is:
[
{
"operation": "shift",
"spec": {
"item": {
"system": {
"name": "title",
"codename": "externalId"
},
"elements": {
"short_answer": {
"value": ["summary"]
},
"long_answer": {
"value": "detail"
}
}
},
"modular_content": {
"*": {
"elements": {
"permalink": {
"value": "url"
}
}
}
}
}
}
]
The JSON input is:
{
"item": {
"system": {
"id": "6e55g261aee-34d9-47be-8830-145ec71b564f",
"name": "Where is deleted user data visible, and for how long is it there?",
"codename": "where_is_deleted_user_data_visible__and_for_how_lo",
"language": "en-US",
"type": "faq",
"collection": "default",
"sitemap_locations": [],
"last_modified": "2022-04-11T15:32:08.0844256Z",
"workflow_step": "published"
},
"elements": {
"question": {
"type": "text",
"name": "Question",
"value": "Where is deleted user data visible, and for how long is it there?"
},
"short_answer": {
"type": "text",
"name": "Bot Teaser Answer",
"value": "Deleted user data is stored in LP and visible for 13 months following deletion."
},
"long_answer": {
"type": "rich_text",
"name": "Full Answer",
"images": {},
"links": {},
"modular_content": [
"n6fdd091f_fcb5_012a_87545_c1671c268e6b",
"n3f1dea42_ac2e_01cdf6_cbd6_3f802d4e8fb3"
],
"value": "<p>Deleted user data is stored in LP and visible for <strong>13 months following deletion.</strong></p>\n<p>Deleted agents are displayed with a '(deleted)' indicator in the Open Conversations, All Conversations, Web History lists, and the Conversation History widget.</p>\n<object type=\"application/kenticocloud\" data-type=\"item\" data-rel=\"component\" data-codename=\"n6fdd091f_fcb5_012a_8745_c1671c268e6b\"></object>\n<p>Filter by deleted agents is available in the Open Conversations, All Conversations, and web history lists.</p>\n<p>In the visitor conversation window, the name of the deleted agent will be displayed in the window header (with no indication that they have been deleted).</p>\n<p>The LP user limit '<code>collections.maxElements</code>' will not count the number of deleted users in the total.</p>\n<object type=\"application/kenticocloud\" data-type=\"item\" data-rel=\"component\" data-codename=\"n3f1dea42_ac2e_01c6_cbd6_3f802d4e8fb3\"></object>"
},
"faq_page": {
"type": "modular_content",
"name": "This FAQ item's FAQ page(s)",
"value": [
"data___reporting_9658b36"
]
},
"related_article": {
"type": "modular_content",
"name": "Related article",
"value": []
}
}
},
"modular_content": {
"data___reporting_9658b36": {
"system": {
"id": "9658bddf362-735b-47fd3c-9232-26de14835d8f",
"name": "Data & Reporting",
"codename": "data___reporting_9658b36",
"language": "en-US",
"type": "kc_faqs",
"collection": "default",
"sitemap_locations": [],
"last_modified": "2022-04-13T16:18:12.2538592Z",
"workflow_step": "published"
},
"elements": {
"pagename": {
"type": "text",
"name": "Title",
"value": "Data & Reporting"
},
"faq_items": {
"type": "modular_content",
"name": "FAQ Items",
"value": [
"how_do_i_find_chats_that_only_contain_private_mess",
"how_do_i_understand_why_a_chat_ended_",
"how_do_i_change_the_email_address_for_scheduled_re",
"how_do_i_retrieve_data_that_has_been_masked_",
"where_is_deleted_user_data_visible__and_for_how_lo"
]
},
"related_articles": {
"type": "modular_content",
"name": "Related Articles",
"value": []
},
"permalink": {
"type": "url_slug",
"name": "URL slug",
"value": "faqs-data-reporting1.html"
},
"redirects": {
"type": "text",
"name": "redirects",
"value": "faqs-data-reporting1.html"
}
}
},
"n3f1dea42_ac2e_01c6_cbd6_3f802d4e8fb3": {
"system": {
"id": "3f1dea42-ac2e-01c6-cbd6-3f802d4e8fb3",
"name": "3f1dea42-ac2e-01c6-cbd6-3f802d4e8fb3",
"codename": "n3f1dea42_ac2e_01c6_cbd6_3f802d4e8fb3",
"language": "en-US",
"type": "contentbox",
"collection": "default",
"sitemap_locations": [],
"last_modified": "2022-04-11T15:32:08.0844256Z"
},
"elements": {
"type": {
"type": "multiple_choice",
"name": "Type",
"value": [
{
"name": "Warning (orange)",
"codename": "warning"
}
]
},
"notice_text": {
"type": "rich_text",
"name": "Text",
"images": {},
"links": {},
"modular_content": [],
"value": "<p>A user who joined a chat and then was deleted won’t be reflected as deleted in the history.</p>"
}
}
},
"n6fdd091f_fcb5_012a_8745_c1671c268e6b": {
"system": {
"id": "6fdd091f-fcb5-012a-8745-c1671c268e6b",
"name": "6fdd091f-fcb5-012a-8745-c1671c268e6b",
"codename": "n6fdd091f_fcb5_012a_8745_c1671c268e6b",
"language": "en-US",
"type": "contentbox",
"collection": "default",
"sitemap_locations": [],
"last_modified": "2022-04-11T15:32:08.0844256Z"
},
"elements": {
"type": {
"type": "multiple_choice",
"name": "Type",
"value": [
{
"name": "Notice (blue)",
"codename": "notice"
}
]
},
"notice_text": {
"type": "rich_text",
"name": "Text",
"images": {},
"links": {},
"modular_content": [],
"value": "<p>Before deleting a user, ensure that the agent doesn’t own any open conversations (if so, these conversations should be transferred to the queue) and that the user is logged out of LP.</p>"
}
}
}
}
}
Any option to add the static URL link to the output?

You can add a modify spec with concat function after the current shift transformation spec such as
{
"operation": "modify-overwrite-beta",
"spec": {
"url": "=concat('https://www.static-url.com/',#(1,&))" // first argument is static, and the second one is dynamic(directly inherited from the current value of "url") content
}
}

Elasticsearch nested sort - mismatch between document and nested object used for sorting

I've been developing a new search API with AWS Elasticsearch (version 6.2) as backend.
Right now, I'm trying to support "sort" options for the API.
My mapping is as follows (unrelated fields not included):
{
"properties": {
"id": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
},
"description": {
"type": "text"
},
"materialDefinitionProperties": {
"type": "nested",
"properties": {
"id": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
},
"analyzer": "case_sensitive_analyzer"
},
"value" : {
"type": "nested",
"properties": {
"valueString": {
"type": "text",
"fields": {
"raw": {
"type": "keyword"
}
}
}
}
}
}
}
}
}
I'm attempting to allow the users sort by property value (path: materialDefinitionProperties.value.valueLong.raw).
Note that it's inside 2 levels of nested objects (materialDefinitionProperties and materialDefinitionProperties.value are nested objects).
To sort the results by the value of property with ID "PART NUMBER", my request for sorting is:
{
"fieldName": "materialDefinitionProperties.value.valueString.raw",
"nestedSort": {
"path": "materialDefinitionProperties",
"filter": {
"fieldName": "materialDefinitionProperties.id",
"value": "PART NUMBER",
"slop": 0,
"boost": 1
},
"nestedSort": {
"path": "materialDefinitionProperties.value"
}
},
"order": "ASC"
}
However, as I examined the response, the "sort" field does not match with document's property value:
{
"_index": "material-definition-index-v2",
"_type": "default",
"_id": "development_LITL4ZCNE",
"_source": {
"id": "LITL4ZCNE",
"description": [
"CPU, Intel, Cascade Lake, 8259CL, 24C, 210W, B1 Prod"
]
"materialDefinitionProperties": [
{
"id": "PART NUMBER",
"description": [],
"value": [
{
"valueString": "202-001193-001",
"isOriginal": true
}
]
}
]
},
"sort": [
"100-000018"
]
},
The document's PART NUMBER property is "202-001193-001", the "sort" field says "100-000018", which is the part number of another document.
It seems that there's a mismatch between the master document and nested object used for sorting.
This request worked well when there's only a small number of documents in the cluster. But once I backfill the cluster with ~1 million of records, the symptom appears. I've also tried creating a new ES cluster but the results are the same.
Sorting by other non-nested attributes worked well.
Did I misunderstand the concept of nested objects, or misuse the nested sort feature?
Any ideas appreciated!

This is a bug in Elasticsearch. Upgrading to 6.4.0 fixed the issue.
Issue tracker: https://github.com/elastic/elasticsearch/pull/32204
Release note: https://www.elastic.co/guide/en/elasticsearch/reference/current/release-notes-6.4.0.html

Json Format error, ask for array when he already gets one

I have a small issues when I try to parse the below JSON arrays
{
"category": [
{
"id": "1",
"idRef": "1",
"name": "projet",
"products": [
{
"category": "1",
"content": "produit du projet , et sa description",
"id": "5",
"name": "ProdProj",
"price": "10000.0"
},
{
"category": "1",
"content": "application de pokemon potoo",
"id": "7",
"name": "pokedex",
"price": "10000.0"
},
{
"category": "1",
"content": "description du projet n2",
"id": "8",
"name": "projet2",
"price": "100.0"
},
{
"category": "1",
"content": "Construisez de vos reve",
"id": "9",
"name": "Pokedex Dream",
"price": "100.0"
}
]
},
{
"id": "2",
"idRef": "2",
"name": "jeux video",
"products": [
{
"category": "2",
"content": "Description du projet de de tout ce qui suit",
"id": "6",
"name": "Jeux video Project",
"price": "10000.0"
},
{
"category": "2",
"content": "The description pokedex is relouu",
"id": "10",
"name": "thePokedex",
"price": "100.0"
}
]
},
{
"id": "3",
"idRef": "3",
"name": "apps mobil",
"products": [
{
"category": "3",
"content": "description de l'application numéro2",
"id": "11",
"name": "application2",
"price": "100.0"
},
{
"category": "3",
"content": "azerazeraze mobil",
"id": "12",
"name": "azerMobil",
"price": "100.0"
}
]
}
]
}
with this code:
public class GsonFormatter {
private static Gson gson = new GsonBuilder()
.registerTypeAdapter(User.class, new UserDeserializer())
.registerTypeAdapter(Category.class, new CategoryDeserializer())
.registerTypeAdapter(Project.class, new ProjectDeserializer())
.create();
public static Gson getGson() {
return gson;
}
}
And I have this method that makes an error:
"Expected BEGIN_ARRAY but was BEGIN_OBJECT at line 1 column 2"
Public ArrayList<Category>() {
return GsonFormatter.getGson().fromJson(json,
new TypeToken<List<Category>>() {}.getType());
So I have been trying all night long. But I cannot figure out what is going wrong. This is my first JSON use, so i'm sorry for this stupid question...

The error is telling you that while parsing the JSON and mapping it to your Category class, there was a case where your class is expecting an object, but the JSON is an array.
Looking at your JSON, it's easy to see where that is. You have an array of Category objects. In one of them, products is an object, and in another it's an array:
... \"products\":[{\"category\":\"1\",\"content\":\"Jeux ...
... \"products\":{\"category\":\"3\",\"content\":\"application ...
Edit in response to comments: IMHO, yes, the way the server is producing the JSON is broken. What it appears is that if there is only one product in a category, it returns the single product as an object, but if there are multiple products you get an array of them. Ideally you would want the server to always return an array even when there's only one product.
If you can't get the server changed, what you would need to do is handle this in your deserializer. You'd have to manually check the products field and see if it's an object and convert it to an array for your class.

Your above-mentioned method only apples for get a json array without attributes to be extracted.
My following method is suggested to take a json object with an array associated with.
Gson gson = new Gson();
String jsonOutput = "{ \"array\": [{ \"id\" : \"3\", \"name\" : \"hello_world\" }, { \"id\" : \"2\", \"name\" : \"hello_world\" }] }";
JsonObject root = new JsonParser().parse(jsonOutput).getAsJsonObject();
JsonArray terms = root.getAsJsonArray("array");
//JSONArray cms = jsonObject.getJSONArray("array");
MyModel[] arr = gson.fromJson(terms.toString(), MyModel[].class);

REST hypermedia/links to collections

Part of REST best practice is to make use of links in the responses to allow clients to navigate from one entity to another.
For example if I had a customer object type which has child account. If I was to request a customer using /customers/1 then I might provide the following response
{
"self": "http://localhost:43002/rest/v1/customers/1",
"id": 1,
"name": "Isabella Button",
"number": "000001",
"forename": "Isabella",
"surname": "Button",
"accounts": [
{
"self": "http://localhost:43002/rest/v1/accounts/1",
"id": 1,
"name": "Main Account",
"number": "000001",
"currency": "GBP",
"fromDate": "2013-01-01",
"toDate": "9999-01-01",
"createdDttm": "2013-01-01T00:00:00.000"
}
]
}
Note the self property holds the links.
However let's say I didn't want to return the accounts in the customer query, perhaps the number of accounts might be very large so I don't want to return them by default.
{
"self": "http://localhost:43002/rest/v1/customers/1",
"id": 1,
"name": "Isabella Button",
"number": "000001",
"forename": "Isabella",
"surname": "Button"
}
A resource URL for a customer's accounts could be /customers/1/accounts
However with the customer response above the client would be unable to discover the /customers/1/accounts link.
Is there a best practice for providing hyperlinks in a response that point to "child" collections of the returned resource?

One practice is to use a links element like this:
{
"self": "http://localhost:43002/rest/v1/customers/1",
"id": 1,
"name": "Isabella Button",
"number": "000001",
"forename": "Isabella",
"surname": "Button",
"links" : [
{
"rel" : "http://www.yourapi.com/rels/accounts",
"href" : "http://localhost:43002/rest/v1/customers/1/accounts"
},
{
"rel" : "http://www.yourapi.com/rels/someOtherCollection",
"href" : "http://localhost:43002/rest/v1/customers/1/someOtherCollection",
}
]
}
Or, if you find easier to construct/read the response, you can put the same links as Link http headers.

provide a links attribute like in this example http://caines.ca/blog/programming/json-is-under-defined-for-rest/
{
"links": {
"self" : { "href": "{id}" },
"up" : { "href": "{upId}" },
"children" : { "href": "{id}/children" }
}
}