Extract data from YouTube Data API with Java / Sprint Boot - java

I am trying to extract title, videoId, and description data of all the videos I got from using YT Data API. I was able to return the data in what looks like JSON format (but it is actually LinkedHashMap when I check with getClass()) in Postman and Chrome browser, but was unable to extract the value from specific keys I have mentioned above.
I tried:
System.out.println((rest.getForObject(url, Object.class, params)).get("items"));
It asks me to cast to JSONObject
Overall, I need to extract the data and convert into xml before sending it over to ActiveMQ
Edit:
Format I get in Postman/Chrome browser
{
"kind": "youtube#searchListResponse",
"etag": "3LV4enCWAzOaiqJb_cMIUVklXJY",
"nextPageToken": "CAUQAA",
"regionCode": "CA",
"pageInfo": {
"totalResults": 478712,
"resultsPerPage": 5
},
"items": [
{
"kind": "youtube#searchResult",
"etag": "gt6x7J2XpzU8Mpb3yv9_HmNTzWY",
"id": {
"kind": "youtube#video",
"videoId": "I_ZK0t9-llo"
},
"snippet": {
"publishedAt": "2021-03-01T18:15:13Z",
"channelId": "UC2WHjPDvbE6O328n17ZGcfg",
"title": "Stack Overflow is full of idiots.",
"description": "The Stack Overflow culture needs to be fixed. The overall gatekeeping & elitism in computer science & programming - as a whole ...",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/I_ZK0t9-llo/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/I_ZK0t9-llo/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/I_ZK0t9-llo/hqdefault.jpg",
"width": 480,
"height": 360
}
},
"channelTitle": "ForrestKnight",
"liveBroadcastContent": "none",
"publishTime": "2021-03-01T18:15:13Z"
}
},
{
"kind": "youtube#searchResult",
"etag": "RwmGP1qMzUCzK6oV82s0NUEOETw",
"id": {
"kind": "youtube#video",
"videoId": "sMIslcynm0Q"
},
"snippet": {
"publishedAt": "2021-03-07T16:00:13Z",
"channelId": "UCXwjZTvpFiBl93ACCUh1NXQ",
"title": "How To Use Stack Overflow (no, ForrestKnight, it's not full of idiots)",
"description": "Hey everyone, I can't believe I have to make this video. Unfortunately ForrestKnight recently made a video saying Stack Overflow ...",
"thumbnails": {
...
Edit2: I tried
JSONObject jsonObject = new JSONObject( rest.getForObject(url, Object.class, params));
JSONArray array = jsonObject.getJSONArray("items" );
for(int i=0;i<array.length();i++){
JSONObject snippet =array.getJSONObject(i);
System.out.println(snippet.getJSONObject("snippet").get("title"));
}
JSONObject["items"] not found.
Edit3:
I also tried without Edit2 and just
JSONObject jsonObject = new JSONObject( rest.getForObject(url, Object.class, params));
System.out.println(jsonObject);
No error and I was about to fetch data shown in Postman. But in console printed {}, meaning I might not have gotten the data and won't be able to extract.
I tried extracting using .get System.out.println(jsonObject.get("items"));
and I got the same error JSONObject["items"] not found.
Seems like it could be a format error, again it looks like json already and to use .get, I placed it inside jsonObject but found nothing
Edit4: I can confirm the data I get back is Json (like it says on the doc) using
try {
new JSONObject(rest.getForObject(url, Object.class, params));
} catch (JSONException ex) {
// edited, to include #Arthur's comment
// e.g. in case JSONArray is valid as well...
try {
new JSONArray(rest.getForObject(url, Object.class, params));
} catch (JSONException ex1) {
return false;
}
}
return true;
Returned true
Edit5: Seems like the keys is null after trying
String[] keys = JSONObject.getNames(jsonObject);
// iterate over them
for (String key1 : keys) {
// retrieve the values
Object value = jsonObject.get(key1);
// if you just have strings:
String value1 = (String) jsonObject.get(key1);
System.out.println(value);
System.out.println(value1);
}
Cannot read the array length because "keys" is null

The RestTemplate delegates to the underlying Jackson to deserialize the JSON string to a java object .If you get the HTTP response as a generic object type , it will deserialize it into a Map.
You can simply create a POJO with the structure that is the same as the expected JSON response, and then use the Jackson annotations to configure how to deserialize the JSON into this POJO , and get the HTTP response as this POJO class.
Something like :
public class Result {
#JsonProperty("items")
private List<Item> items = new ArrayList<>();
}
public class Item {
#JsonProperty("kind")
private String kind;
#JsonProperty("etag")
private String etag;
#JsonProperty("snippet")
private List<Snippet> snippets = new ArrayList<>();
}
public class Snippet {
#JsonProperty("channelId")
private String channelId;
#JsonProperty("title")
private String title;
}
Then use the following to get the response :
Result result = rest.getForObject(url, Result.class, params);
Once you get the Result POJO , use the favourite library to serialize it into XML.
P.S I just show you the ideas. You have to fine tune the POJO structure

Related

How to update Rest API request Body array

I am trying to automate one API using rest API JAVA. I have kept sample request json in one file and before passing that json I want to modify some values. New values are in Map.
For example I want to modify 'company' value to 'Microsoft'(which is in hashmap newInputs). Problem is my code is adding new 'company' at the top of body instead of updating it. How do i update specific values in Items array.
json body in file
{
"Items": [
{
"newValue": "Q3",
"company": "Test",
"oldValue": "Q2",
"Date": "2022-03-27"
}
]
}
What i want to pass to API
{
"Items": [
{
"newValue": "Q3",
"company": "Microsoft",
"oldValue": "Q2",
"Date": "2022-03-27"
}
]
}
What my code is passing
{
"company": "Microsoft",
"Items": [
{
"newValue": "Q3",
"company": "Test",
"oldValue": "Q2",
"Date": "2022-03-27"
}
]
}
My code
public static String createItem(Map<String, String> newInputs) {
JSONArray jsonArray = null;
JSONObject jsonObject;
try {
jsonArray = new JSONArray(new String(
Files.readAllBytes(Paths.get("src", "test", "resources", "Payloads", "testdata.json"))));
jsonObject = new JSONObject(jsonArray.get(0).toString());
for (String key : newInputs.keySet()) {
jsonObject.put(key, newInputs.get(key));
}
jsonArray.put(0, jsonObject);
}

How to make com.fasterxml.jackson print array vertically? [duplicate]

I have data that looks like this:
{
"status": "success",
"data": {
"irrelevant": {
"serialNumber": "XYZ",
"version": "4.6"
},
"data": {
"lib": {
"files": [
"data1",
"data2",
"data3",
"data4"
],
"another file": [
"file.jar",
"lib.jar"
],
"dirs": []
},
"jvm": {
"maxHeap": 10,
"maxPermSize": "12"
},
"serverId": "134",
"version": "2.3"
}
}
}
Here is the function I'm using to prettify the JSON data:
public static String stringify(Object o, int space) {
ObjectMapper mapper = new ObjectMapper();
try {
return mapper.writerWithDefaultPrettyPrinter().writeValueAsString(o);
} catch (Exception e) {
return null;
}
}
I am using the Jackson JSON Processor to format JSON data into a String.
For some reason the JSON format is not in the format that I need. When passing the data to that function, the format I'm getting is this:
{
"status": "success",
"data": {
"irrelevant": {
"serialNumber": "XYZ",
"version": "4.6"
},
"another data": {
"lib": {
"files": [ "data1", "data2", "data3", "data4" ],
"another file": [ "file.jar", "lib.jar" ],
"dirs": []
},
"jvm": {
"maxHeap": 10,
"maxPermSize": "12"
},
"serverId": "134",
"version": "2.3"
}
}
}
As you can see under the "another data" object, the arrays are displayed as one whole line instead of a new line for each item in the array. I'm not sure how to modify my stringify function for it to format the JSON data correctly.
You should check how DefaultPrettyPrinter looks like. Really interesting in this class is the _arrayIndenter property. The default value for this property is FixedSpaceIndenter class. You should change it with Lf2SpacesIndenter class.
Your method should looks like this:
public static String stringify(Object o) {
try {
ObjectMapper mapper = new ObjectMapper();
DefaultPrettyPrinter printer = new DefaultPrettyPrinter();
printer.indentArraysWith(new Lf2SpacesIndenter());
return mapper.writer(printer).writeValueAsString(o);
} catch (Exception e) {
return null;
}
}
I don't have enough reputation to add the comment, but referring to the above answer Lf2SpacesIndenter is removed from the newer Jackson's API (2.7 and up), so instead use:
printer.indentArraysWith(DefaultIndenter.SYSTEM_LINEFEED_INSTANCE);
Source of the solution

Spring RestTemplate parse JSON object with variable keyname

I have a REST API call that returns the following JSON object. I need to parse this with Spring's RestTemplate. The problem is that the first key ISBN:0132856204 is variable (the numbers change depending on the book). How would I go about doing this?
{
"ISBN:0132856204": {
"publishers": [
{
"name": "Pearson"
}
],
"pagination": "xxiv, 862p",
"identifiers": {
"isbn_13": [
"978-0-13-285620-1"
],
"openlibrary": [
"OL25617855M"
]
},
"weight": "1340 grams",
"title": "Computer networking",
"url": "https://openlibrary.org/books/OL25617855M/Computer_networking",
"number_of_pages": 862,
"cover": {
"small": "https://covers.openlibrary.org/b/id/7290810-S.jpg",
"large": "https://covers.openlibrary.org/b/id/7290810-L.jpg",
"medium": "https://covers.openlibrary.org/b/id/7290810-M.jpg"
},
"publish_date": "2013",
"key": "/books/OL25617855M",
"authors": [
{
"url": "https://openlibrary.org/authors/OL31244A/James_F._Kurose",
"name": "James F. Kurose"
},
{
"url": "https://openlibrary.org/authors/OL658909A/Keith_W._Ross",
"name": "Keith W. Ross"
}
],
"subtitle": "A Top-Down Approach"
}
}
In here "ISBN:0132856204" is a value and also a key for your business.
To get ISBN first, what about wrapping json content with 1 more closure?
{
"yourAwesomePlaceHolderKey" :
{
"ISBN:0132856204": {
......
}
}
}
First get the ISBN key as a value, then your ISBN value can be used as a key to get related content.
First goal will be extracting -String1,Object1- pair where String1 is "yourAwesomePlaceholderKey" and second goal will be again extracting -String2,Object2- from Object1 where String2 is your ISBN key.
This is the way I solved it, using JsonPath for getting the book out of the JSON object and Jackson for mapping it to a Book object:
RestTemplate restTemplate = new RestTemplate();
String isbn = "0132856204";
String endpoint = "https://openlibrary.org/api/books?jscmd=data&format=json&bibkeys=ISBN:{isbn}";
//Get JSON as String
String jsonString = restTemplate.getForObject(endpoint, String.class, isbn);
//Configure JsonPath to use Jackson for mapping
Configuration.setDefaults(new Configuration.Defaults() {
private final JsonProvider jsonProvider = new JacksonJsonProvider();
private final MappingProvider mappingProvider = new JacksonMappingProvider();
#Override
public JsonProvider jsonProvider() {
return jsonProvider;
}
#Override
public MappingProvider mappingProvider() {
return mappingProvider;
}
#Override
public Set<Option> options() {
return EnumSet.noneOf(Option.class);
}
});
//Parse the JSON as a book
Book book = JsonPath.parse(jsonString).read("$.ISBN:" + isbn, Book.class);
You can use JsonProperty to solve
#JsonProperty("ISBN:0132856204")

Parsing JSON in java: Malformed JSON: Unexpected 'C'

Good day,
I am parsing some JSON in java (a notification from a cisco CMX system). I've parsed a lot of JSON in my time but this one refuses to get parsed. I've tried several methods: a reader with lenient mode, plain gson, etc.
The JSON I parse is valid according to jsonlint, which leads me to believe it is a problem with the parser, or maybe some hidden characters that I am unable to sanitize out. This is the JSON I receive:
{
"startTime": "08:00",
"previousEndDate": null,
"startDate": "2016-02-17",
"title": "Visitors",
"executionTime": 29,
"value": {
"primary": {
"title": "TotalVisitors",
"value": 16,
"peakValue": 0,
"breakdown": [{
"title": "RepeatVisitors",
"value": 11
}, {
"title": "NewVisitors",
"value": 5
}]
},
"average": {
"title": "TotalVisitors",
"value": 19,
"peakValue": 0,
"breakdown": [{
"title": "RepeatVisitors",
"value": 15
}, {
"title": "NewVisitors",
"value": 4
}]
},
"previousTimeRange": {
"title": "TotalVisitors",
"value": 23,
"peakValue": 0,
"breakdown": [{
"title": "RepeatVisitors",
"value": 19
}, {
"title": "NewVisitors",
"value": 4
}]
}
},
"areas": [{
"id": 20,
"name": "CineCitta"
}],
"previousStartDate": "2016-02-16",
"endDate": null,
"endTime": "09:29"
}
It seems valid to me, and the object I try to parse it into has the correct fields.
I've tried filtering out \r \t \n \0 and some combinations between them.
The code I currently have in java is:
String result = "{\"startTime\":\"08:00\",\"previousEndDate\":null,\"startDate\":\"2016-02-17\",\"title\":\"Visitors\",\"executionTime\":29,\"value\":{\"primary\":{\"title\":\"TotalVisitors\",\"value\":16,\"peakValue\":0,\"breakdown\":[{\"title\":\"RepeatVisitors\",\"value\":11},{\"title\":\"NewVisitors\",\"value\":5}]},\"average\":{\"title\":\"TotalVisitors\",\"value\":19,\"peakValue\":0,\"breakdown\":[{\"title\":\"RepeatVisitors\",\"value\":15},{\"title\":\"NewVisitors\",\"value\":4}]},\"previousTimeRange\":{\"title\":\"TotalVisitors\",\"value\":23,\"peakValue\":0,\"breakdown\":[{\"title\":\"RepeatVisitors\",\"value\":19},{\"title\":\"NewVisitors\",\"value\":4}]}},\"areas\":[{\"id\":20,\"name\":\"CineCitta\"}],\"previousStartDate\":\"2016-02-16\",\"endDate\":null,\"endTime\":\"09:29\"}";
JsonReader reader = new JsonReader(new StringReader(result));
reader.setLenient(true);
Gson gson = new Gson();
ClientInfo info = gson.fromJson(reader, ClientInfo.class);
The question is: Does anyone know how to debug a problem like this? are there sanitization techniques I can use? Other parsers?
EDIT: The code to clientinfo as requested (using project lombok, all fields are public):
#ToString
#FieldDefaults(level = AccessLevel.PUBLIC)
public class ClientInfo {
String startTime;
String previousEndDate;
String startDate;
String title;
Integer executionTime;
Value value;
Area [] areas;
String previousStartDate;
String endDate;
String endTime;
}
public class Value {
public Visitors primary;
public Visitors average;
public Visitors previousTimeRange;
}
#FieldDefaults(level = AccessLevel.PUBLIC)
public class Area {
Integer id;
String name;
}
#FieldDefaults(level = AccessLevel.PUBLIC)
public class Visitors {
String title;
Integer value;
Integer peakValue;
Record [] breakdown;
}
public class Record {
public String title;
public Integer value;
}
Thanks and good day
String result = "{\"startTime\":\"08:00\",\"previousEndDate\":null,\"startDate\":\"2016-02-17\",\"title\":\"Visitors\",\"executionTime\":29,\"value\":{\"primary\":{\"title\":\"TotalVisitors\",\"value\":16,\"peakValue\":0,\"breakdown\":[{\"title\":\"RepeatVisitors\",\"value\":11},{\"title\":\"NewVisitors\",\"value\":5}]},\"average\":{\"title\":\"TotalVisitors\",\"value\":19,\"peakValue\":0,\"breakdown\":[{\"title\":\"RepeatVisitors\",\"value\":15},{\"title\":\"NewVisitors\",\"value\":4}]},\"previousTimeRange\":{\"title\":\"TotalVisitors\",\"value\":23,\"peakValue\":0,\"breakdown\":[{\"title\":\"RepeatVisitors\",\"value\":19},{\"title\":\"NewVisitors\",\"value\":4}]}},\"areas\":[{\"id\":20,\"name\":\"CineCitta\"}],\"previousStartDate\":\"2016-02-16\",\"endDate\":null,\"endTime\":\"09:29\"}";
Gson gson = new Gson();
JsonParser parser = new JsonParser();
JsonObject jsonObj = parser.parse(result).getAsJsonObject();
ClientInfo info = gson.fromJson( jsonObj , ClientInfo.class);
You can give a try with above code.
I am a dumdum.
I assumed the parsing was what went wrong, but it was actually that my service was supposed to produce a JSON, where I did not return it as json, but the object itself resulting in a bad parsing.
Because I was so focused on the parsing I was doing I did not realize the error itself occurred on the implicit json parsing of the restful service.
That'll teach me not to test in a simple environment before moving my code to the a restful client.
Thanks and dumb greetings,
Dries

Create object from JSON with Parse & Java for Android

I have recently started doing Android development, I am using Parse.com and native Java for building the app. What I am trying to do is figure out how to create objects that are created from the JSON result via a callback from Parse Parse.com.
{
"result":
{
"count": 3,
"lang": "en-US",
"tasks": [
{
"ID": "0123",
"Name": "Task 1",
"AssingedTo": "Darxval",
"Scope": "Home"
},
{
"ID": "0124",
"Name": "Task 2",
"AssingedTo": "Darxval",
"Scope": "Home"
},
{
"ID": "0125",
"Name": "Task 3",
"AssingedTo": "Darxval",
"Scope": "Home"
}
]
}
}
The method I call via Parse is shown below.
(see documentation for more info on the method here):Parse Documentation
ParseCloud.callFunctionInBackground("RetrieveTasks", new HashMap<String, Object>(), new FunctionCallback<Object>() {
void done(Object result, ParseException e) {
if (e == null) {
// do stuff with object
}
}
});
I have tried to change the Function callback to a class that takes in the json described as a bunch of strings and a ListArray of Task objects. But it does not create the object. It always want to use HashMap. Is there a way to turn the data into the object I want instead of a HashMap? Is there a interface I need to "implement" on my objects to allow the creation of the object on callback?
Any help in this would be greatly appreciated.
What object is added to the HashMap? Is it a JSONObject?
If so, you can convert a JSONObject to whatever custom object you are looking for using a constructor, and the data from the JSONObject.
For example, if your JSONObject that is returned is called "output":
try {
JSONObject result = output.getJSONObject("result");
int count = result.getInt("count");
String lang = result.getString("lang");
JSONArray tasks = result.getJSONArray("tasks");
JSONObject firstObjectInArray = tasks.getJSONObject(0);
String firstID = firstObjectInArray.getString("ID");
} catch (JSONException e) {
e.printStackTrace();
}
etc.
Then use your constructor for your Object to create a new object:
Object obj = new Object(count, lang, firstID);
etc.
For more information

Categories