I have an issue while trying to deidentify some data with DLP using an object mapper to parse the object into string - send it to DLP for deidentification - getting back the deidentified string and using the object mapper to parse the string back to the initial object. Sometimes DLP will return a string that cannot be parsed back to the initial object (it breaks the json format of the object mapper)
I use an objectMapper to parse an Address object to string like this:
Address(
val postal_code: String,
val street: String,
val city: String,
val provence: String
)
and my objectmapper will transform this object into a string eg: "{\"postal_code\":\"123ABC\",\"street\":\"Street Name\",\"city\":\"My City\",\"provence\":\"My Provence\"}" which is sent to DLP and deidentified (using LOCATION or STREET_ADDRESS detectors).
The issue is that my object mapper would expect to take back the deidentified string and parse it back to my Address object using the same json format eg:
"{\"postal_code\":\"LOCATION_TOKEN(10):asdf\",\"street\":\"LOCATION_TOKEN(10):asdf\",\"city\":\"LOCATION_TOKEN(10):asdf\",\"provence\":\"LOCATION_TOKEN(10):asdf\"}"
But there are a lot of times that DLP will return something like
"{"LOCATION_TOKEN(25):asdfasdfasdf)\",\"provence\":\"LOCATION_TOKEN(10):asdf\"}" - basically breaking the json format and i am unable to parse back the string from DLP to my initial object
Is there a way to instruct DLP infotype detectors to keep the json format, or to look for sensitive text only inside \" * \"?
Thanks
There are some options here using a custom regex and a detection ruleset in order to define a boundary on matches.
The general idea is that you require that findings must match both an infoType (e.g. STREET_ADDRESS, LOCATION, PERSON_NAME, etc.) and your custom infoType before reporting as a finding or for redaction. By requiring that both match, you can set bounds on where the infoType can detect.
Here is an example.
{
"item": {
"value": "{\"postal_code\":\"123ABC\",\"street\":\"Street Name\",\"city\":\"My City\",\"provence\":\"My Provence\"}"
},
"inspectConfig": {
"customInfoTypes": [
{
"infoType": {
"name": "CUSTOM_BLOCK"
},
"regex": {
"pattern": "(:\")([^,]*)(\")",
"groupIndexes": [
2
]
},
"exclusionType": "EXCLUSION_TYPE_EXCLUDE"
}
],
"infoTypes": [
{
"name": "EMAIL_ADDRESS"
},
{
"name": "LOCATION"
},
{
"name": "PERSON_NAME"
}
],
"ruleSet": [
{
"infoTypes": [
{
"name": "LOCATION"
}
],
"rules": [
{
"exclusionRule": {
"excludeInfoTypes": {
"infoTypes": [
{
"name": "CUSTOM_BLOCK"
}
]
},
"matchingType": "MATCHING_TYPE_INVERSE_MATCH"
}
}
]
}
]
},
"deidentifyConfig": {
"infoTypeTransformations": {
"transformations": [
{
"primitiveTransformation": {
"replaceWithInfoTypeConfig": {}
}
}
]
}
}
}
Example output:
"item": {
"value": "{\"postal_code\":\"123ABC\",\"street\":\"Street Name\",\"city\":\"My City\",\"provence\":\"My [LOCATION]\"}"
},
By setting "groupIndexes" to 2 we are indicating that we only want the custom infoType to match the middle (or second) regex group and not allow the :" or " to be part of the match. Also, in this example we mark the custom infoType as EXCLUSION_TYPE_EXCLUDE so that it does not report itself:
"exclusionType": "EXCLUSION_TYPE_EXCLUDE"
If you remove this line, anything matching your infoType could also get redacted. This can be useful for testing though - example output:
"item": {
"value": "{\"postal_code\":\"[CUSTOM_BLOCK]\",\"street\":\"[CUSTOM_BLOCK]\",\"city\":\"[CUSTOM_BLOCK]\",\"provence\":\"[CUSTOM_BLOCK][LOCATION]\"}"
},
...
Hope this helps.
I have some ugly JSON that I need to deserialize which looks like the following:
"ContainerValues": [
{
"ParentAttribute": "QuantityContained",
"RowList": [
{
"Values": [
{
"Name": "Code",
"ValuesByLocale": {
"en-US": "GRM"
},
},
{
"Name": "Value",
"ValuesByLocale": {
"en-US": "4.0"
},
}
],
}
],
}
],
This is just a sample of the JSON I have. All I need to do is to get this into a POJO which looks like something like the following:
Class POJO{
String grmValue; // This is the "Value" for the GRM "Code" above, i.e. "4.0"
...
}
Any idea how I might be able to assign the value of grmValue based on the JSON above using Jackson? I'm starting to think I'll need to write a custom deserializer.
First You have to deserialize to class similar to your JSON, then transform to your POJO format :)
I've tried a few different way of doing this. I have a JSON string that I'm parsing with JSONSlurper that looks like this, and I want to get the value with a key==StoreID2.
{
"EmailData": {
"MessageRecords": [
{
"To": "this#that.com",
"FieldData": [
{
"key": "StoreName",
"value": "Livonia"
},
{
"key": "StoreID2",
"value": "7017"
}
]
}
]
}
}
With JSONPath I can get the value like this: $.EmailData.MessageRecords[0].FieldData[?(#.key=="StoreID2")].value But it doesn't look like Groovy has JSONPath available, without loading a non-native library (com.jayway.jsonpath.JsonPath.parse), which I'm trying to avoid.
This is actually the solution:
def jsonSlurper = new JsonSlurper();
def jsonObject = jsonSlurper.parseText(inputSB.toString());
logger.info("Store: " + jsonObject.EmailData.MessageRecords[0].FieldData.find{it.key=='StoreID2'}.value);
Thanks
I have looking to similar questions but i was not able to solve my problem .I have tried parsing my json response both into a Json array and Json object but every time , i am getting this exception of string to json object.
I guess may be the problem is with my Json response.
results = { "type": 1, "user_log": "ahsan.tahir.92_/var/www/html/2014-10-11__15-54-52__50sec.txt_res", "freq": 0.01, "coordina": [ [-37], [-9], [-20], [-12], [-22], [-9], [-22], [-15], [-25], [-7], [-20], [-12], [-20], [-9], [-25], [8], [-23], [-11], [-18], [-13], [-19], [-10], [-21], [-12], [-25], [-11], [-17], [-12], [-22], [-13], [-21], [3] ], "tot_time": 40.11, "tot_distace": 100, "stroke_each_pool": [ [16], [16] ], "tot_stroke": [ [16], [16] ], "split": [ [20.22], [19.89] ], "timing_turn": 7.04, "cycle_Rate_l": [ [2.22], [2.2680001] ], "cycle_Rate_r": [ [2.224], [2.27] ], "mean_velocity": [ [2.4727993], [2.5138261] ], "stroke_length": [ [3.125], [3.125] ], "stroke_freq": [ [79.077431], [77.419357] ], "roll_peaks": [ [-44.10043335], [55.79428101], [-61.51541138], [54.7466507], [-62.09820557], [55.01488495], [-62.48770142], [53.44023132], [-70.32449341], [51.8399353], [-65.84837341], [53.5617981], [-63.50210571], [55.9821167], [-62.37905121], [39.42669678], [-43.44207764], [63.20912933], [-59.19660187], [50.6708374], [-63.8214798], [54.57595062], [-63.31864166], [53.82037354], [-66.93650818], [52.36277008], [-65.23461151], [52.89829254], [-62.78508759], [51.17367554], [-62.87123108], [59.13114929] ], "mean_roll_dx": [ [52.475822], [54.73027] ], "mean_roll_sx": [ [61.531971], [60.950779] ], "std_roll_dx": [ [5.4471908], [4.3127728] ], "std_roll_sx": [ [7.6123171], [7.4134283] ], "mean_roll": [ [57.003899], [57.840527] ], "std_roll": [ [7.9220791], [6.6817732] ], "mean_pitch": [ [-5.5227709], [-5.2282872] ], "std_pitch": [ [-5.5227709], [-5.2282872] ], "clean_stroke_time": [ [15.92], [16.84] ], "errore": 227, "fatal_error": { "_ArrayType_": "double", "_ArraySize_": [0,0], "_ArrayData_": null } }
and i am parsing it like this :
JSONObject reader = new JSONObject(in);
JSONObject sys = reader.getJSONObject("results");
Any idea what am i missing here ?
My exception is as follows :
org.json.JSONException: Value type of type java.lang.String cannot be converted to JSONObject
There is no results field in the results string you have posted, is that the issue. In any case, please go ahead and reformat your question to more accurately reflect what you are doing, and include the error message you are seeing.
Actually, are you referring to the error message in the JSON output? That seems to be a JSON-encoded error message that was output by whatever service you called. There seems to be some issue with the data you are passing to that service.
This:
results = {rest of response here}
is not valid JSON.
Perhaps you meant to return this:
{"results": {rest of response here}}
... or just
{rest of response here}
I may have Misunderstood your question, but where are you parsing the Json response using something like
JSONObject jsonObject = (JSONObject) jsonParser.parse(strippedJSON);
where strippedJSON the response received from API.
Your reader object is your results object itself.
JSONObject resultsJSON = new JSONObject(in);
System.out.println("User Log: " + resultsJSON.get("user_log"));
You're getting the exception because there's no results object in your input JSON string i.e. you're receiving your results object without a JSON wrapper around it.
If you are using javascript
try this
var parsed_data = JSON && JSON.parse(results) || $.parseJSON(results);
{
"vers": 0.01,
"config": {
"rate": "perhr",
"valueColumns": [
"vCPU",
"ECU",
"memoryGiB",
"storageGB",
"linux"
],
"currencies": [
"USD"
],
"regions": [
{
"region": "us-east",
"instanceTypes": [
{
"type": "generalCurrentGen",
"sizes": [
{
"size": "t2.micro",
"vCPU": "1",
"ECU": "variable",
"memoryGiB": "1",
"storageGB": "ebsonly",
"valueColumns": [
{
"name": "linux",
"prices": {
"USD": "0.013"
}
}
]
},
{
"size": "t2.small",
"vCPU": "1",
"ECU": "variable",
"memoryGiB": "2",
"storageGB": "ebsonly",
"valueColumns": [
{
"name": "linux",
"prices": {
"USD": "0.026"
}
}
]
}
]
}
]
}
]
}
}
Hi, i wanted to read this json file. I tried various ways from google but getting a null at valuesColumns. I have to read sizes array and have to put in list.
I think it will help your cause if you format your json. As it is it's quite hard to read. Googling for a JSON beautifier quickly found me this one.
When working with JSON your browser console provides a nice environment for inspecting and playing with the data. I pasted it into the browser console and did (hit enter after each line):
var x = { ... paste JSON here ... }
x
x.config
x.config.valueColumns
This tells me that x is a JSON object, config is a JSON object and valueColumns is a JSON array
Now to java. Grab yourself a json library, and accessing valueColumns will be something like:
JSONObject x = new JSONObject("{ ... JSON string ... }");
JSONObject config = x.getJSONObject("config");
JSONArray valueColumns = config.getJSONArray("valueColumns");
You can then iterate over valueColumns and pull out what you need.
Note that the above only gets you to the first valueColumns array under config. By following the same principle you can go deeper into the structure and get out the valueColumns for the objects in the sizes array if that's what you're really after.
For parsing json to java there are simple step.
Below code snippet may help you.
// parse json to java
Object obj = parser.parse(s);
JSONObject json = (JSONObject) obj;
JSONObject o = (JSONObject) json.get("config");
//get json array.
JSONArray array = (JSONArray) o.get("valueColumns");
System.out.println(array.toJSONString());
//access element by index
System.out.println(array.get(0));