I am trying to read a response from an API endpoint. It works in Postman and appears to work when debugging my java code.
However it will not read any objects that are in the nested array.
I've followed this solution (in the comments someone lays out their solution to accessing objects in a nested array) yet that does not seem to have solved anything as the mapper consistently shows nothing inside the json list.
Here's my code:
try {
response = httpClient.execute(httpPost);
InputStream inputStream = response.getEntity().getContent();
final ObjectMapper mapper = new ObjectMapper();
final JsonNode node = mapper.readTree(inputStream);
final JsonNode value = node.get("value");
final JsonNode index = value.get(1);
final String status = index.get("status").asText();
return status;`
}
Here's the response that I'm trying to unpack. I'm trying to get the value of "status"
{
"value": [
{
"activityRunEnd": "2023-01-23T23:22:01.4234985Z",
},
{
"status": "Succeeded"
}
]
}
This is the response that I'm currently getting when I readTree(inputStream)
{"value":[]}
InputStream does not seem to read any values inside the list
Related
I am trying to get a large json response using a recursive REST call like below:
private List<MyPojo> recursiveRestCallMethod(String folderId) throws IOException {
List<MyPojo> mypojoList = new ArrayList<>();
String hugeJson = webClient.get()
.uri("/my/rest/api/accepting/" + folderId
+ "/and/producing/huge/jsonresponse/for/all/files/recursively")
.retrieve().bodyToMono(String.class).block();
byte[] bytes = hugeJson.getBytes("UTF-8");
String json = new String(bytes, StandardCharsets.UTF_8);
ObjectMapper objectMapper = new ObjectMapper();
ObjectNode node = objectMapper.readValue(json, ObjectNode.class);
objectMapper.configure(
DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
if (node.get("list").get("entries").isArray()) {
for (JsonNode jsonNode : node.get("list").get("entries")) {
MyPojo pojo = new MyPojo();
JsonNode mainNode = jsonNode.get("entry");
if (mainNode.get("isFile").asBoolean()) {
JsonNode nameNode = mainNode.get("name");
pojo.setNodename(nameNode.toString());
// and 20 more fields
mypojoList.add(pojo);
}
if (mainNode.get("isFolder").asBoolean()) {
mypojoList.addAll(recursiveRestCallMethod(mainNode.get("id").toString().replaceAll("\"", "").trim()));
}
}
return mypojoList;
}
return null;
}
Now everytime the json returned has 4193150 characters and it throws exception - Unexpected end-of-input: expected close marker for Object as reported here and some other SO threads (obviously, the json is not complete and valid).
The incomplete json I am getting looks something like:
{"list":{"pagination":{"count":6097,"hasMoreItems":false,"totalItems":6097,"skipCount":0,"maxItems":10000},"entries":[{"entry":{"....
From above, as you can see I should get 6097 objects, but I am getting only 2024 entry array items. And after that json ends abruptly. i.e. invalid json string.
However, for smaller response, where I have 20/30 entry array items, it works as expected.
Note: I am using Spring-Boot 2.4.5 and hence Jackson 2.12.4
Question: Even though I am using .block(), why the response stops at 4193150 characters? What I am doing wrong here?
Not sure what was wrong using String but when I switched to DataBuffer, it worked fine.
Here is the snippet for what I used:
final Flux<DataBuffer> hugeJson = webClient.get()
.uri("/my/rest/api/accepting/" + folderId
+ "/and/producing/huge/jsonresponse/for/all/files/recursively")
.accept(MediaType.ALL)
.retrieve()
.bodyToFlux(DataBuffer.class);
I am trying to learn JAVA and build a plugin for Minecraft; I have been successfully able to get the JSON data from my api endpoint however, the issue I am facing right now is an llegalFormatConversionException: d !=java.lang.String which means that the format I am trying to make into string isn't equal to the type of string that it's looking for.
I am trying to access a JSON element from my endpoint called condition
{
"todaysdate": "2021-02-12",
"temperature": 25,
"description": [
"Overcast"
],
"condition": 122,
}
Coming from C#; I know there's a website called json2sharp where you can create a root class for the JSON. I'm not sure how it would be applied in Java but currently, my code looks like this.
private String fetchWeather() throws IOException, InvalidConfigurationException {
// Download
final URL url = new URL(API);
final URLConnection request = url.openConnection();
// Set HEADER
request.setRequestProperty("x-api-key", plugin.apiKey);
request.setConnectTimeout(5000);
request.setReadTimeout(5000);
request.connect();
// Convert to a JSON object to print data
JsonParser jp = new JsonParser(); //from gson
JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent())); //Convert the input stream to a json element
JsonObject rootobj = root.getAsJsonObject();
//JsonElement code = rootobj.get("condition");
String condition_code = rootobj.get("condition").toString();
plugin.getLogger().fine(String.format(
"[%s] Weather is %d",
world.getName(), condition_code
));
return condition_code;
}
If I call
private JsonObject fetchWeather() throws IOException, InvalidConfigurationException {
// Download
final URL url = new URL(API);
final URLConnection request = url.openConnection();
// Set HEADER
request.setRequestProperty("x-api-key", plugin.apiKey);
request.setConnectTimeout(5000);
request.setReadTimeout(5000);
request.connect();
// Convert to a JSON object to print data
JsonParser jp = new JsonParser(); //from gson
JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent())); //Convert the input stream to a json element
JsonObject rootobj = root.getAsJsonObject();
return rootobj;
}
state = fetchWeather();
plugin.getLogger().warning(state.toString());
I do get the actual JSON with all of the elements so I know the URL and accessing it is completely working, but I get an llegalexception for format if I try to call and print with logger the condition code.
So, if I change the return type in fetchWeather() to be the root json object and then try to print it with the state variable above it works; but if I try to return the condition json element and print it it gives me an iilegal exception.
Now before I posted this question I did read some other questions people had but I couldn't get a working solution from their suggested answers. So, I am hoping someone can point me out on what I'm doing wrong because I know I'm messing up somewhere with the variable format.
Thanks.
Line 37: state = fetchWeather();
Line 103:
plugin.getLogger().fine(String.format(
"[%s] Weather is %d",
world.getName(), bId
));
condition_code variable is String. You should use the %s format specifier.
EDIT:
I am not sure how much the WHY matters here. There are lot of reasons I am doing what I am doing in a certain way not feasible to explain in one post.
But I will still give some context, the service that stores my json files gives them back as byte arrays, so i start with byte[] arrays, each byte array is a json structure.
From this json i need to convert to a matching JAVA POJO this is POJO serves as a validator and then the final step is using Freemarker generating HTML files out of this POJO.
My scenario is that i get potentially get multiple byte arrays. Each can be a maximum of 500mb. I need to combine them, convert to JSON string and then convert to JAVA POJO.
There are couple of problems i am trying to address:
I see the jsonData only having 1000 records in all.
How do I combine the two jsons into one single json to be then able to convert to JAVA POJO?
Here's my JSON structure:
{
"Header": {
"id": 123
},
"Data": [
{
"key": "123
}
]
}
Here's my current code stripped to bare which I use for BYTE[] -> JSON STRING - > POJO
byte[] dataFile = null;
for (String fileName : getFileNames()) {
try {
byte[] currentFile = null;
currentFile = SomeService.getByteFile(fileName);
dataFile = ArrayUtils.addAll(dataFile, currentFile);
}
String jsonData = new JSONObject(new String(dataFile)).toString();
ObjectMapper mapper = new ObjectMapper();
className classForMapping = mapper.readValue(jsonData, className.class));
I'm trying to parse the JSON from the following API:
https://opentdb.com/api.php?amount=1
but when I'm trying to get the question value, I get the following error:
Exception in thread "main" java.lang.NullPointerException
I use this code:
public static void main(String[] args) throws IOException {
String question;
String sURL = "https://opentdb.com/api.php?amount=1"; //just a string
// Connect to the URL using java's native library
URL url = new URL(sURL);
URLConnection request = url.openConnection();
request.connect();
// Convert to a JSON object to print data
JsonParser jp = new JsonParser(); //from gson
JsonElement root = jp.parse(new InputStreamReader((InputStream) request.getContent())); //Convert the input stream to a json element
JsonObject rootobj = root.getAsJsonObject(); //May be an array, may be an object.
question = rootobj.get("question").getAsString(); //grab the question
}
Hope someone can tell me what I am doing wrong.
Thanks in advance!
When I look at the JSON you are trying to interpret, I get:
{
"response_code": 0,
"results":[
{
"category": "Entertainment: Board Games",
"type": "multiple",
"difficulty": "medium",
"question": "Who is the main character in the VHS tape included in the board game Nightmare?",
"correct_answer": "The Gatekeeper",
"incorrect_answers":["The Kryptkeeper","The Monster","The Nightmare"]
}
]
}
This JSON doesn't contain a root member "question". This makes rootobj.get("question") return null, and therefore calling getAsString on it throws the NullPointerException.
So instead of rootobj.get("question"), you would have to walk through the hierarchy: "results" -> first array member -> "question":
rootobj.getAsJsonArray("result").getAsJsonObject(0).get("question")
Try this one
question = rootobj.getAsJsonArray("results").get(0).getAsJsonObject().get("question").getAsString();
The JSON has no direct "question" field.
call question = rootobj.get("result").get(0).get("question").getAsString(); instead
I have been doing some work with apache CXF(version 2.2.2) JAX-RS. I am trying to introduce data validation layer in CXF request handler before business method be invoked. Fortunately :), I am encountering an issue on input parameter handling in request handler(DataValidationHandler). I can read the JSON Object manually by following code lines in request handler. But it's duplicated with JSONProvider registered in CXF framework. Because JSON object input stream only can be read once, otherwise we will meet exception "java.io.EOFException: No content to map to Object due to end of input". Moreover, duplicated JSON object deserializing will impacts performance. Following code is sample for your reference.
Read JSON Object from HTTP body manually:
OperationResourceInfo ori = paramMessage.getExchange().get(OperationResourceInfo.class);
MultivaluedMap<String, String> values = new MetadataMap<String, String>();
List<Object> objList = JAXRSUtils.processParameters(ori, values, paramMessage);
Register JSONProvider in CXF JAX-RS framework:
<bean id="JSONProvider" class="com.accela.govxml2.jaxrs.util.JSONProvider"></bean>
Read JSON Object to Java Object from input stream:
public Object readFrom(......){
ObjectMapper objectMapper = new ObjectMapper();
Object result = objectMapper.readValue(entityStream, TypeFactory.defaultInstance().constructType(genericType));
Return result;
}
I am dealing with Path Parameter manually by following code lines.
OperationResourceInfo ori = paramMessage.getExchange().get(OperationResourceInfo.class);
URITemplate t1 = ori.getClassResourceInfo().getURITemplate();
URITemplate t2 = ori.getURITemplate();
UriInfo uriInfo = new UriInfoImpl(paramMessage, null);
MultivaluedMap<String, String> map = new MetadataMap<String, String>();
t1.match(uriInfo.getPath(), map);
String str = map.get(URITemplate.FINAL_MATCH_GROUP).get(0);
t2.match(str, map);
String pathParameter= null;
if (map.containsKey("pathParam") && !ValidationUtil.isEmpty(map.get("pathParam")))
{
pathParameter= map.get("pathParam").get(0);
}
My questions are here:
How to deal with POST/PUT input parameter of http body in request handler in general?
How to avoid performance issue to read input parameter efficiently?
Is there any way to inject the validation (handler/interceptor) layer between parameter reading by CXF(JSONProvider) and business method invoking?
Is there any elegant way to deal with path parameter?
Thanks for your help. Any comments & suggestions will be appreciated.
Regards,
Dylan
I have found another way to inject DataValidation Interceptor into reading parameter phase. We can reuse deserialized input model from message content, which be deserialized by JSONProvider registered in framework. It can improve performance, because only deserialize input model once.
public class DataValidationInInterceptor extends AbstractPhaseInterceptor<Message>
{
public DataValidationInInterceptor()
{
super(Phase.READ);
}
#Override
public void handleMessage(Message message)
{
OperationResourceInfo ori = message.getExchange().get(OperationResourceInfo.class);
Method method = ori.getMethodToInvoke();
Class<?>[] types = method.getParameterTypes();
Type[] genericParameterTypes = method.getGenericParameterTypes();
for (int i = 0; i < types.length; i++)
{
Class<?> type = types[i];
List obj = (List) message.getContent(List.class);
System.out.println(obj);
System.out.println(type);
}
}
}
After researching, I can read the input stream twice based on the following question's answer ( Read stream twice ).
However, JSON object deserializing performance is still my concern. Who has better solution for it?
Intercept request and change message content from CoyoteInputStream to ByteArrayInputStream, so I can read the InputStream twice.
InputStream in = message.getContent(InputStream.class);
if (in != null)
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
IOUtils.copy(in, baos);
byte[] bytes = baos.toByteArray();
ByteArrayInputStream bais = new ByteArrayInputStream(bytes);
message.setContent(InputStream.class, bais);
}
Reset ByteArrayInputStream before Read JSON Object to Java Object from input stream:
public Object readFrom(......){
ObjectMapper objectMapper = new ObjectMapper();
if (entityStream.markSupported())
{
entityStream.reset();
}
Object result = objectMapper.readValue(entityStream, TypeFactory.defaultInstance().constructType(genericType));
return result;
}