We Keep Coding

Google DLP - Can I use a delimiter to instruct DLP infotype detectors to search only inside that for sensitive text?

I have an issue while trying to deidentify some data with DLP using an object mapper to parse the object into string - send it to DLP for deidentification - getting back the deidentified string and using the object mapper to parse the string back to the initial object. Sometimes DLP will return a string that cannot be parsed back to the initial object (it breaks the json format of the object mapper)
I use an objectMapper to parse an Address object to string like this:
Address(
val postal_code: String,
val street: String,
val city: String,
val provence: String
)
and my objectmapper will transform this object into a string eg: "{\"postal_code\":\"123ABC\",\"street\":\"Street Name\",\"city\":\"My City\",\"provence\":\"My Provence\"}" which is sent to DLP and deidentified (using LOCATION or STREET_ADDRESS detectors).
The issue is that my object mapper would expect to take back the deidentified string and parse it back to my Address object using the same json format eg:
"{\"postal_code\":\"LOCATION_TOKEN(10):asdf\",\"street\":\"LOCATION_TOKEN(10):asdf\",\"city\":\"LOCATION_TOKEN(10):asdf\",\"provence\":\"LOCATION_TOKEN(10):asdf\"}"
But there are a lot of times that DLP will return something like
"{"LOCATION_TOKEN(25):asdfasdfasdf)\",\"provence\":\"LOCATION_TOKEN(10):asdf\"}" - basically breaking the json format and i am unable to parse back the string from DLP to my initial object
Is there a way to instruct DLP infotype detectors to keep the json format, or to look for sensitive text only inside \" * \"?
Thanks

There are some options here using a custom regex and a detection ruleset in order to define a boundary on matches.
The general idea is that you require that findings must match both an infoType (e.g. STREET_ADDRESS, LOCATION, PERSON_NAME, etc.) and your custom infoType before reporting as a finding or for redaction. By requiring that both match, you can set bounds on where the infoType can detect.
Here is an example.
{
"item": {
"value": "{\"postal_code\":\"123ABC\",\"street\":\"Street Name\",\"city\":\"My City\",\"provence\":\"My Provence\"}"
},
"inspectConfig": {
"customInfoTypes": [
{
"infoType": {
"name": "CUSTOM_BLOCK"
},
"regex": {
"pattern": "(:\")([^,]*)(\")",
"groupIndexes": [
2
]
},
"exclusionType": "EXCLUSION_TYPE_EXCLUDE"
}
],
"infoTypes": [
{
"name": "EMAIL_ADDRESS"
},
{
"name": "LOCATION"
},
{
"name": "PERSON_NAME"
}
],
"ruleSet": [
{
"infoTypes": [
{
"name": "LOCATION"
}
],
"rules": [
{
"exclusionRule": {
"excludeInfoTypes": {
"infoTypes": [
{
"name": "CUSTOM_BLOCK"
}
]
},
"matchingType": "MATCHING_TYPE_INVERSE_MATCH"
}
}
]
}
]
},
"deidentifyConfig": {
"infoTypeTransformations": {
"transformations": [
{
"primitiveTransformation": {
"replaceWithInfoTypeConfig": {}
}
}
]
}
}
}
Example output:
"item": {
"value": "{\"postal_code\":\"123ABC\",\"street\":\"Street Name\",\"city\":\"My City\",\"provence\":\"My [LOCATION]\"}"
},
By setting "groupIndexes" to 2 we are indicating that we only want the custom infoType to match the middle (or second) regex group and not allow the :" or " to be part of the match. Also, in this example we mark the custom infoType as EXCLUSION_TYPE_EXCLUDE so that it does not report itself:
"exclusionType": "EXCLUSION_TYPE_EXCLUDE"
If you remove this line, anything matching your infoType could also get redacted. This can be useful for testing though - example output:
"item": {
"value": "{\"postal_code\":\"[CUSTOM_BLOCK]\",\"street\":\"[CUSTOM_BLOCK]\",\"city\":\"[CUSTOM_BLOCK]\",\"provence\":\"[CUSTOM_BLOCK][LOCATION]\"}"
},
...
Hope this helps.

Get json object in Array based on key and value in Java

I have a Json body like the example below. I need to extract the value from a key that has another key with a specific value in an array. I am passing in a JsonNode with everything in the detail component of the message, I can easily extract from each level, however, I'm struggling with the array.
In this case, I need to extract the value of "value" (Police/Fire/Accident Report) from the object in the array which has a key/value pair of "name":"documentTitle". I understand this is a JSONArray, but I can't find a good example that shows me how to extract the values for an object in the array that contains a certain key/value pair, I don't think I can rely on getting the object in position [2] in the array as the same objects may not always be present in the additionalMetadata array.
Sample Json:
"sourceVersion": "1.0",
"eventId": "8d74b892-810a-47c3-882b-6e641fd509eb",
"clientRequestId": "b84f3a7b-03cc-4848-a1e8-3519106c6fcb",
"detail": {
"stack": "corona",
"visibilityIndicator": null,
"documentUid": "b84f3a7b-03cc-4848-a1e8-3519106c6fcb",
"additionalMetadata": [
{
"name": "lastModifiedDate",
"value": "2021-05-21T04:53:53Z"
},
{
"name": "documentName",
"value": "Police/Fire Report, 23850413, 2021-05-20 14:51:23"
},
{
"name": "documentTitle",
"value": "Police/Fire/Accident Report"
},
{
"name": "documentAuthor",
"value": "System Generated"
},
{
"name": "lastModifiedBy",
"value": "System Updated"
},
{
"name": "createdBy",
"value": "System Generated"
},
{
"name": "documentDescription",
"value": "Police/Fire Report received"
},
{
"name": "organizationCode",
"value": "Claims"
}
]
}
}```

Loop through the json array and extract the json object with name documentTitile. From that json object you can get the value

Well, either the JSON framework you're using supports this out of the box (check the documentation) or you could convert it manually to a map:
List<AdditionalMetadataEntry> additionalMetadata;
[...]
Map<String, String> additionalMetadataMap = additionalMetadata.stream().collect(Collectors.toMap(AdditionalMetadataEntry::getName, AdditionalMetadataEntry::getValue));

I was able to figure it out. I created a new node off the existing notificationBody JsonNode, then parsed through the metadata key/value pairs:
String docTitle = "";
JsonNode additionalMetadata = notificationBody.get("detail").get("additionalMetadata");
for (JsonNode node: additionalMetadata) {
String name = node.get("name").asText();
String value = node.get("value").asText();
if(name.equals("documentTitle")){
docTitle = value;
}
}

Trim redundant attributes in json objects using java

I would like to trim the below json object. That is a json object I built on top of what mongoDB responded. What I want to do is to remove just $oid because they are redundant attributes and keep the value inside (_id or $id ) without Curley braces and simply call the attribute id.
so what I need is just "id": "2283cef627ff2cc33ad5990"
Could you please help me I am struggling with json.
{
"_id": {
"$oid": "22383cef627ff2cc33ad5990"
},
"name": "data1",
"users": [
{
"$ref": "user",
"$id": {
"$oid": "16a5fbcee4b0c2c2da3017ef"
}
},
{
"$ref": "user",
"$id": {
"$oid": "1795ff86e4b09fc66416cd2f"
}
},
],
},

a) You can use a mapper to convert your JSON to an object and then call the desired attribute, like Jackson:
ObjectMapper mapper = new ObjectMapper();
String jsonInString = YOUR_STRING;
//from String to MyClass
MyClass object = mapper.readValue(jsonInString, MyClass.class);
In this example you have to define a class MyClass with all the attributes you need (e.g. _id, name, users, etc).
b) If you want to implement a quicker solution you can manipulate directly the string; if you know that the oid is always 24 characters you can do something like
String c = str.substring(str.indexOf("\"", str.indexOf("$oid")+6)+1, str.indexOf("\"", str.indexOf("$oid")+6)+25);
but I highly recommend to take a look to Jackson and give it a try. A solution like this is very fragile and every change in the JSON will lead to a wrong result.

LibGDX Json parser - parse specific node/object

I'm currently implementing JSON parsing in my game but I've encountered a problem that I can't find a solution for: how do you parse a specific node/object (not sure what to call it) from a JSON file? Let's say my JSON looks like this:
{
"intro/credits": { //A node/object.
"title": "Intro music / Credits music",
"authors": [
{
"name": "Vindsvept",
"links": {
"YouTube": "https://www.youtube.com/channel/UCfSUheoljDlGDjerRylO4Nw",
"Bandcamp": "https://vindsvept.bandcamp.com/"
}
}
]
},
"extra": { //Another node/object.
"title": "extra",
"authors": [
{
"name": "extra",
"links": {
"linkTest": "linkTest"
}
}
]
}
}
With that JSON in mind, how would I do something like this?:
MyObject myObj = json.fromJson(parse.object.called.extra);

Thanks to Underbalanced I can now answer my own question: to exctract an object called extra you would do something like this:
Json json = new Json();
JsonValue root = new JsonReader().parse(Gdx.files.internal("path/to/your/file.json"));
JsonValue extra = root.get("extra"); //Replace 'extra' with whatever your object is called.
MyObject myObj = json.fromJson(MyObject.class, extra.toString());

Converting a specific JSON to java object format? [duplicate]

This question already has answers here:
How to parse JSON in Java
(36 answers)
Closed 9 years ago.
Iam new to java script and JSON, please help me in solving my problem. Below is the structure of my JSON in JavaScript
{
"name": "sample",
"def": [
{
"setId": 1,
"setDef": [
{
"name": "ABC",
"type": "STRING"
},
{
"name": "XYZ",
"type": "STRING"
}
]
},
{
"setId": 2,
"setDef": [
{
"name": "abc",
"type": "STRING"
},
{
"name": "xyz",
"type": "STRING"
}
]
}
]
}
in the backend, what should be the synatx of java method to receive this data
public void getJsonData(****){
}
How to parse this JSON data in java and what should be the syntax of method parameter ?
update 1: Edited the json format to make it valid

First create a class that will map your json object and give a name something like "DataObject". Then use the gson library and do the following:
String s = "";
DataObject obj = gson.fromJson(s, DataObject.class);

Your JSON is invalid, but assuming you fix that then you are looking for a library in Java which will serialize an annotated Java class to JSON, or deserialize JSON data to an annotated Java class.
There is a whole list of suitable libraries here:
http://json.org/java/