I would like to extract all the status codes from a JSON response (Elasticsearch response to a bulk request) so that I can count how many documents have been created and how many errored.
Which regex should I use in the following code?
import java.util.regex.Matcher;
import java.util.regex.Pattern;
...
List<String> allCodes = new ArrayList<String>();
Matcher m = Pattern.compile("regex").matcher(jsonResponseString);
while (m.find()) {
allCodes.add(m.group());
}
Example of JSON response:
{
"took": 9,
"errors": false,
"items": [
{
"index": {
"_index": "movies",
"_type": "drama",
"_id": "123",
"_version": 68,
"result": "updated",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": false,
"status": 200
}
},
{
"index": {
"_index": "movies",
"_type": "drama",
"_id": "456",
"_version": 1,
"result": "created",
"_shards": {
"total": 2,
"successful": 1,
"failed": 0
},
"created": true,
"status": 201
}
}
]
}
Thanks!
List<String> allCodes = new ArrayList<String>();
Matcher m = Pattern.compile("\"status\": (\\d.*)").matcher(YOUR_TEXT);
while (m.find()) {
allCodes.add(m.group(1));
}
System.out.println(allCodes);
But i would create a Pojo with just the information you want:
e.g
public class Response {
int took;
List<Item> itemList;
class Item{
int status;
}
}
and then use jackson to convert.
By the way there is a java api for elasticsearch: https://www.elastic.co/guide/en/elasticsearch/client/java-api/current/index.html so you don't need to handle parsing etc.
If status is just a numbers, you can use something like this \"status\":\s+(\d+). Also if you will use group() you will get all matched sequences: "status": 200, "status": 201
List<String> allCodes = new ArrayList<>();
Matcher m = Pattern.compile("\"status\":\\s+(\\d+)").matcher(jsonResponseString);
while (m.find()) {
allCodes.add(m.group(1));
}
System.out.println(allCodes);
As suggested in the comment, you shouldn't use a regex. If you think parsing the full JSON graph is too memory-consuming and don't want to use a Elastic search Java client, you may want to consider:
https://github.com/jayway/JsonPath
Lightweight library which is simmilar to XPath for XML, but for JSON. It doesn't parse the full model into memory but streams it and only extracts the matching elements, based on the path you supply. In this case this path would be something like:
$.items[?(#status = 200)]
I think you can even have the expression immediatly return the count you're looking for by using .length(), without the need to count the matching elements manually...
Related
we have some requests which have a lot of experiments. I just want to count the no experiments. If it's greater than some number then I will block those requests
{
"context": {
"requestId": "",
"locale": "",
"deviceId": "",
"currency": "",
"memberId": 0,
"cmsOrigin":,
"experiments": {
"forceByVariant":,
"forceByExperiment": [
{
"id": "test",
"variant": "A"
}
]
}
}
In this request, I just want to check how many id and variant inside the forceByExperiment. I have tried to do using regular expression but not able to do it. Anyone do it before similar thing.
I just split the string with variant and count them. Not sure good idea, but the end goal is to figure out that the request have a lot of experiments.
Using the circe library and Scala, here is an easy solution:
import io.circe._, io.circe.parser._
val jsonString = """{
"context": {
"requestId": "",
"locale": "",
"deviceId": "",
"currency": "",
"memberId": 0,
"cmsOrigin": "foo",
"experiments": {
"forceByVariant": [],
"forceByExperiment": [
{
"id": "test",
"variant": "A"
}
]
}
}
}"""
val parseResult = parse(jsonString)
val nElems = for {
json <- parse(jsonString)
array <- json.hcursor.downField("context").downField("experiments").downField("forceByExperiment").as[Seq[Json]]
} yield array.length
println(nElems) // Right(1)
If you really want to use regex, and if there is no id field in your json structure, you can use the following expression "id": "(\w+)" and count the number of match.
Example: https://regex101.com/r/GCeByw/1/
I'm having some trouble with GSON in regards to printing. GSON has two options when it comes to printing.
Pretty Printing
Compact Printing
I intend to use a modified form of Pretty Printing and even though the documentation says JsonPrintFormatter is the class which is used to modify the output format. I can't find that class in the GSON repository!
Any ideas on why this is the case or anyway I can modify the GSON printing?
Apart from that, any libraries used to modify spacing or formatting of JSON in the Java language would also be helpful.
Pretty Print:
{
"classname": "something",
"type": "object",
"version": 1,
"properties": [
{
"propertyname": "something1",
"type": "String",
"length": 255
},
{
"propertyname": "something2",
"type": "Date",
"length": 10
}
]
}
Compact Print:
{"classname":"something","type":"object","version":1,"properties":[{"propertyname":"something1","type":"String","length":255},{"propertyname":"something2","type":"Date","length":10}]}
My Print Style:
{
"classname": "something",
"type": "object",
"version": 1,
"properties": [
{"propertyname": "something1","type": "String","length": 255},
{"propertyname": "something2","type": "Date","length": 10}
]
}
Well, it's just work in progress for now, but this should do the trick for strings with only one array. Will look into to making it more stable and able to handle more complex structures.
private static String reformat(String og){
String reformattable = og;
String[] parts = reformattable.split("\\[",2);
String arrayPart = parts[1];
String arrayOnly = arrayPart.split("]",2)[0];
reformattable = arrayOnly.replaceAll("\\{\n","{");
reformattable = reformattable.replaceAll("\",\n", "\\\",");
reformattable = reformattable.replaceAll(" +"," ");
reformattable = reformattable.replaceAll("\\{ "," {");
reformattable = reformattable.replaceAll("\n }","}");
return og.replace(arrayOnly,reformattable);
}
Result should look like this (at least for my simple class):
{
"classname": "test",
"properties": [
{"propertyname": "1", "length": 1},
{"propertyname": "1", "length": 1}
]
}
I have a json file as below which I am getting as a response from rest API:
{
"label": " MARA LEYZIN",
"ClassCode": "PROFESSIONAL",
"actvFlg": "A",
"name": "MARA LEYZIN",
"Typ": {
"label": "C_TYP_LU",
"TypCode": "PROFESSIONAL "
},
"Address": {
"link": [],
"firstRecord": 1,
"pageSize": 10,
"searchToken": "multi",
"item": [
{
"label": "Address",
"addrTypFk": {
"label": "C_ADDRESS_TYPE_LU",
"addrTypCd": "INDUSTRY",
"addrTypDesc": "Industry"
}
}
]
}
I am trying to parse this in Java and to remove some unwanted json objects. Like I want the following string to be replaced by blank:
"link": [],
"firstRecord": 1,
"pageSize": 10,
"searchToken": "multi",
"item":
To achieve this I am trying the following approach:
String jsonStr = new String(Files.readAllBytes(Paths.get(inputFile)));
System.out.println(jsonStr);
jsonStr.replaceAll("link", "");
But it is not replacing the required string with blanks. Please help me in this.
string object is immutable , so basically if do you want to replace something
System.out.println(jsonStr.replaceAll("link", "")); this will print the replaced string but it will not affect the original string, however if you do this
jsonStr=jsonStr.replaceAll("link", "");
System.out.println(jsonStr); this will print the replaced string
First of all:
Your JSON is not validate. You're missing a closing curly bracket at the end of it.
{
"label": " MARA LEYZIN",
"ClassCode": "PROFESSIONAL",
"actvFlg": "A",
"name": "MARA LEYZIN",
"Typ": {
"label": "C_TYP_LU",
"TypCode": "PROFESSIONAL "
},
"Address": {
"link": [],
"firstRecord": 1,
"pageSize": 10,
"searchToken": "multi",
"item": [{
"label": "Address",
"addrTypFk": {
"label": "C_ADDRESS_TYPE_LU",
"addrTypCd": "INDUSTRY",
"addrTypDesc": "Industry"
}
}]
}
}
Second of all you should just change order of your commands to this:
jsonStr.replaceAll("link", "");
System.out.println(jsonStr);
Important addition:
And I would suggest you to use org.json library or even better JACKSON to parse JSON files.
Here's tutorial how to use jackson and it's my warmest suggestion.
You will save a lot of time and you can do whatever you like.
I have an Elasticsearch index which has _timestamp populated on every record. Using Marvel or curl I can get the _timestamp in the "fields" part of the result for example:
GET index/type/_search?fields=_timestamp,_source
{
"took": 11,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"failed": 0
},
"hits": {
"total": 116888,
"max_score": 1,
"hits": [
{
"_index": "index",
"_type": "type",
"_id": "mXJdWqSLSfykbMtChiCRjA",
"_score": 1,
"_source": {
"results": "example",
},
"fields": {
"_timestamp": 1443618319514
}
},...
However when doing a search using the Java API I cant get it to return the _timestamp.
SearchRequestBuilder builder= client.prepareSearch(index)
.addFacet(facet)
.setFrom(start)
.setSize(limit);
SearchResponse response = builder.execute().actionGet();
Can anyone tell me how to ask for _timestamp too?
You simply need to use the setFields() method like this:
SearchRequestBuilder builder= client.prepareSearch(index)
.setType(type)
.addFacet(facet)
.setFields("_timestamp") <--- add this line
.setFrom(start)
.setSize(limit);
SearchResponse response = builder.execute().actionGet();
I have a JSON file that has multiple entries inside of an array. Each of those entries needs to be mapped to a java object. Here is the JSON string I am testing with (http://jsonlint.com/ validated the JSON below, but I had to erase all of the escape characters which I use in the actual Java test),
[{
"LocId":99,
"typeId":99,
"name":"foo",
"parentId":99,
"geoCode":
{
"type":"foo",
"coordinates":
[{
"latitude":99.0,
"longitude":99.0
}]
}
,
"LocId":8,
"typeId":99,
"name":"foo",
"parentId":99,
"geoCode":
{
"type":"foo",
"coordinates":
[{
"latitude":99.0,
"longitude":99.0
}]
}
}]
I read this string in like this,
String str = "The JSON string shown above";
InputStream is = new ByteArrayInputStream(str.getBytes());
BufferedReader br = new BufferedReader(new InputStreamReader(is));
LocIdClass[] locations = new Gson().fromJson(br, LocIdClass[].class);
But the size of my locations array is always one and only the last entry in the JSON string is stored.
for(int i=0; i<locations.length; i++)
System.out.println(locations[i]);
System.out.println("The size of the array is " + locations.length);
I'm not sure why only the last entry in the JSON string would be retrieved but the others are skipped. I referred to this SO post to figure out the POJO array Jackson - Json to POJO With Multiple Entries.
you have an error in your current json payload, try this:
[
{
"LocId": 99,
"typeId": 99,
"name": "foo",
"parentId": 99,
"geoCode": {
"type": "foo",
"coordinates": [
{
"latitude": 99,
"longitude": 99
}
]
}
},
{
"LocId": 8,
"typeId": 99,
"name": "foo",
"parentId": 99,
"geoCode": {
"type": "foo",
"coordinates": [
{
"latitude": 99,
"longitude": 99
}
]
}
}
]
Edit: To avoid this issue in the future, you can use jsonlint.com to check your json. Make sure it is what you expect it to be.