I am new to Scala. I was trying to parse an API response in Scala. The API response is in the format:
{"items":[{"name":"john", "time":"2017-05-11T13:51:34.037232", "topic":"india", "reviewer":{"id":"12345","name":"jack"}},
{"name":"Mary", "time":"2017-05-11T13:20:26.001496", "topic":"math", "reviewer":{"id":"5678","name":"Tom"}}]}
My target is to populate a list of reviewer id's from the JSON response. I tried to create a JSON object from the response by
val jsonObject= parse(jsonResponse.getContentString()).getOrElse(Json.empty)
but couldn't get the reviewer ids from the json object. Even tried to iterate the JSON object, but didn't work.
I am not familiar with circe but here is how you would do it with spray-json
import spray.json._
import DefaultJsonProtocol._
val jsonResponse = """{"items":[{"name":"john", "time":"2017-05-11T13:51:34.037232", "topic":"india", "reviewer":{"id":"12345","name":"jack"}},{"name":"Mary", "time":"2017-05-11T13:20:26.001496", "topic":"math", "reviewer":{"id":"5678","name":"Tom"}}]}"""
Need to define the schema using case classes:
case class Reviewer(id: String, name: String)
case class Item(name: String, time: String, topic: String, reviewer: Reviewer)
case class Items(items: Array[Item])
And their implicit conversion:
implicit val reviewerImp: RootJsonFormat[Reviewer] = jsonFormat2(Reviewer)
implicit val itemConverted: RootJsonFormat[Item] = jsonFormat4(Item)
implicit val itemsConverted: RootJsonFormat[Items] = jsonFormat1(Items)
Then it's very simple, parsing is just this:
val obj = jsonResponse.parseJson.convertTo[Items]
At last, get the ids for the reviewers:
val reviewers = obj.items.map(it => it.reviewer.id)
You mentioned play, so here's how you could do it in Play
case class Reviewer(id:Long, name:String)
object Reviewer { implicit val format = Json.format[Reviewer] }
Once you have those set up you could either
val json:JsValue = Json.toJson(reviewerObject)
val json:JsObject = Json.toJson(reviewerObject).as[JsObject]
val json:String = Json.toJson(reviewerObject).toString // Valid json string
Or
val reviewer:Reviewer = Json.parse(reviewerJsonString).as[Reviewer]
val validates:Boolean = Json.parse(reviewerJsonString).validates[Reviewer]
Related
I am using aws dynamodb akka persistence API https://github.com/akka/akka-persistence-dynamodb which doesn't have a read journal API like Cassandra (Akka Persistence Query).
I can write journal data to dynamodb the event column is in string java object format my next task is to build CQRS using aws lambda or AWS Java API to read dynamodb, which has to convert the event data to human readble format.
Event Data:-
rO0ABXNyAD9jb20uY2Fwb25lLmJhbmsuYWN0b3JzLlBlcnNpc3RlbnRCYW5rQWNjb3VudCRCYW5rQWNjb3VudENyZWF0ZWQrGoMniq0AywIAAUwAC2JhbmtBY2NvdW50dAA6TGNvbS9jYXBvbmUvYmFuay9hY3RvcnMvUGVyc2lzdGVudEJhbmtBY2NvdW50JEJhbmtBY2NvdW50O3hwc3IAOGNvbS5jYXBvbmUuYmFuay5hY3RvcnMuUGVyc2lzdGVudEJhbmtBY2NvdW50JEJhbmtBY2NvdW5011CikshX3ysCAAREAAdiYWxhbmNlTAAIY3VycmVuY3l0ABJMamF2YS9sYW5nL1N0cmluZztMAAJpZHEAfgAETAAEdXNlcnEAfgAEeHBAj0AAAAAAAHQAA0VVUnQAJDM5M2M2NmRiLTJhYmItNDEwNS04NWUyLWMwZjc3MzExMDNlM3QAB3JjYXJkaW4=
I want to know how to convert the above Java Object string value to human-reable format ? I tried using Java objectinputstream but I think I am doing something wrong.
Scala example:-
val eventData:String = "rO0ABXNyAD9jb20uY2Fwb25lLmJhbmsuYWN0b3JzLlBlcnNpc3RlbnRCYW5rQWNjb3VudCRCYW5rQWNjb3VudENyZWF0ZWQrGoMniq0AywIAAUwAC2JhbmtBY2NvdW50dAA6TGNvbS9jYXBvbmUvYmFuay9hY3RvcnMvUGVyc2lzdGVudEJhbmtBY2NvdW50JEJhbmtBY2NvdW50O3hwc3IAOGNvbS5jYXBvbmUuYmFuay5hY3RvcnMuUGVyc2lzdGVudEJhbmtBY2NvdW50JEJhbmtBY2NvdW5011CikshX3ysCAAREAAdiYWxhbmNlTAAIY3VycmVuY3l0ABJMamF2YS9sYW5nL1N0cmluZztMAAJpZHEAfgAETAAEdXNlcnEAfgAEeHBAj0AAAAAAAHQAA0VVUnQAJDM5M2M2NmRiLTJhYmItNDEwNS04NWUyLWMwZjc3MzExMDNlM3QAB3JjYXJkaW4="
??? (and then what how to convert above string value to human reable format)
Thanks
Sri
ok was able to deserialize the object string data and convert it to json below is an example
object DeserializeData extends App {
import java.io.ByteArrayInputStream
import java.io.InputStream
import java.io.ObjectInputStream
import java.util.Base64
import com.google.gson.Gson
val base64encodedString = "rO0ABXNyAD9jb20uY2Fwb25lLmJhbmsuYWN0b3JzLlBlcnNpc3RlbnRCYW5rQWNjb3VudCRCYW5rQWNjb3VudENyZWF0ZWQrGoMniq0AlM3QAB3JjYXJkaW4="
println("Base64 encoded string :" + base64encodedString)
// Decode
val base64decodedBytes = Base64.getDecoder.decode(base64encodedString)
val in = new ByteArrayInputStream(base64decodedBytes)
val obin = new ObjectInputStream(in)
val `object` = obin.readObject
println("Deserialised data: \n" + `object`.toString)
// You could also try...
println("Object class is " + `object`.getClass.toString)
val json = new Gson();
val resp = json.toJson(`object`)
println(resp)
}
A feature to read aws Dynamodb read journal is now implemented no need of any kind of clunky code https://github.com/akka/akka-persistence-dynamodb/pull/114/files thank you Lightbend
I have a string that i'd like to stringify in Kotlin (Android), but it seems that org.json.* doesn't support taking a string and re-stringifying it, instead it always tries to parse it first.
val str = "test=\"123\""
val stringified = JSONObject(str).toString() // JSONException: Value a of type java.lang.String cannot be converted to `JSONObject`
The use case for this ability is passing data to JS inside a Webview in a secure manner.
val json = "test=\"123\""
webview.evaluateJavascript("window.onData(${json})")
// on the JS side it will look like this: window.onData(test="123")
// this is both invalid and insecure since it opens the door for raw JS injection
Any attempt to do it manually will result in an insecure and possibly invalid JS string
This example should NOT be used:
val insecureJSON = "'${str.replace("\\", "\\\\").replace("\"", "\\\"").replace("'", "\'")}'"
The desired behavior:
val json = jsonStringifyString("test=\"123\"")
webview.evaluateJavascript("window.onData(${json})")
// rendered JS: window.onData("test=\"123\"")
Is there an easy method for stringifying a string in Android?
Ended up using the JSONArray class and removing the array wrapping to trick the class to stringify a plain string
fun jsonStringifyString(str: String): String {
val jsonStr = JSONArray().put(str).toString()
return jsonStr.substring(1, jsonStr.length - 1) // remove first and last char
}
val serializedData = jsonStringifyString("test=\"123\"");
webview.evaluateJavascript("window.onData(${serializedData})")
// rendered JS: window.onData("test=\"123\"")
Recently, I was trying to do a simple Scala code to analyse COVID - 19 data, I called one API and cast structure of that JSON API call to my Scala case classes. If I do not apply filters code is working as expected, When I try to apply filter it do not work, Little bit confused. Why it is not working.
case class RegionData(region: Option[String], totalInfected: Option[String], recovered: Option[String], deceased: Option[String])
case class Data(activeCases: String, recovered: String, deaths: String, totalCases: String, sourceUrl: String, lastUpdatedAtApify: String, readMe: String, regionData: Option[List[RegionData]])
These are my case classes for Scala.
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.module.scala.DefaultScalaModule
import scala.io.Source
object CovidAnalytics {
val objectMapper: ObjectMapper = new ObjectMapper()
objectMapper.registerModule(DefaultScalaModule)
def main(args: Array[String]): Unit = {
val getData: String = Source.fromURL("https://api.apify.com/v2/datasets/58a4VXwBBF0HtxuQa/items?format=json&clean=1").mkString
val data: List[Data] = objectMapper.readValue(getData, classOf[List[Data]])
//This is working
println(data)
val filter = data.filter(e => e.deaths != "")
//This is not working (Confused!!!)
println(filter)
}
}
Exception in thread "main" java.lang.ClassCastException:
scala.collection.immutable.HashMap$HashTrieMap cannot be cast to
com.example.analytics.Data at
com.example.analytics.CovidAnalytics$$anonfun$1.apply(CovidAnalytics.scala:18)
at
scala.collection.TraversableLike$$anonfun$filterImpl$1.apply(TraversableLike.scala:248)
at scala.collection.immutable.List.foreach(List.scala:392) at
scala.collection.TraversableLike$class.filterImpl(TraversableLike.scala:247)
at
scala.collection.TraversableLike$class.filter(TraversableLike.scala:259)
at scala.collection.AbstractTraversable.filter(Traversable.scala:104)
at
com.example.analytics.CovidAnalytics$.main(CovidAnalytics.scala:18)
at com.example.analytics.CovidAnalytics.main(CovidAnalytics.scala)
This is happening because of the type erasure. You can replace List by Array.
val getData: String = Source.fromURL("https://api.apify.com/v2/datasets/58a4VXwBBF0HtxuQa/items?format=json&clean=1").mkString
val data = objectMapper.readValue(getData, classOf[Array[Data]]).toList //.toList if you need it as List
println(data)
val filter = data.filter(e => e.deaths != "")
println(filter)
I am currently using JsonObject and JsonParser of com.google.gson api (using gson-2.8.5 version) to parse and read the value form input JSON.
I have JSON filed like , smaple "resultCode":"SUCCESS", when I try to read the same value from json it gives the result as ""SUCCESS"" .
Every value I am reading, getting with double "" not sure why ? You can refer below screen of my debugging screen.
I am new to Json and parser, is that default behavior ?
I am expecting "SUCCESS", "S", "00000000" not like ""SUCCESS"" or ""S""
or ""00000000""
same I have highlighted in the below image .
Please share any idea how we can get apbsolute vlaue of string without """" double quote string it causing my string comparison fail.
String response_result = "{\"response\": {\"head\": {\"function\": \"acquiring.order.create\",\"version\": \"2.0\",\"clientId\": \"201810300000\",\"reqMsgId\": \"56805892035\",\"respTime\": \"2019-09-13T13:18:08+08:00\"},\"body\": {\"resultInfo\": {\"resultCode\": \"SUCCESS\",\"resultCodeId\": \"00000000\",\"resultStatus\": S,\"resultMsg\": \"SUCCESS\"},\"acquirementId\": \"2018080834569894848930\",\"merchantTransId\": \"5683668701112717398\",\"checkoutUrl\": \"http://localhost:8081/crm/operator/operator-search-init.action\"}},\"signature\":\"d+TUYLvt1a491R1e6aO8i9VwXWzVhfNgnhD0Du74f4RgBQ==\"}";
HttpInvoker.Result result = i.new Result(200, response_result);
JsonObject jo = new JsonParser().parse(response_result).getAsJsonObject();
String resultCode = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().get("resultCode").toString();
String resultCodeId = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().get("resultCodeId").toString();
String resultStatus = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().get("resultStatus").toString();
String checkoutUrl = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("checkoutUrl").toString();
if ( RESULT_CODE_GCASH_SUCCESS.equals(resultCode)
&& RESULT_STATUS_SUCCESS.equals(resultStatus)
&& StringUtils.isNotEmpty(checkoutUrl)) {
log.error("Testing ".concat(resultCode).concat(resultStatus).concat(checkoutUrl));
}
log.error("Testing ".concat(resultCode).concat(resultStatus).concat(checkoutUrl));
}
This is my input JSON
{
"response":{
"head":{
"function":"acquiring.order.create",
"version":"2.0",
"clientId":"201810300000",
"reqMsgId":"56805892035",
"respTime":"2019-09-13T13:18:08+08:00"
},
"body":{
"resultInfo":{
"resultCode":"SUCCESS",
"resultCodeId":"00000000",
"resultStatus":"S",
"resultMsg":"SUCCESS"
},
"acquirementId":"2018080834569894848930",
"merchantTransId":"5683668701112717398",
"checkoutUrl":"http://localhost:8081/crm/operator/operator-search-init.action"
}
},
"signature":"d+TUYLvtI38YL2hresd98Ixu1BXccvvh1IQMiHuMXUEeW/N5exUsW491R1e6aO8i9VwXWzVhfNgnhD0Du74f4RgBQ=="
}
JsonParser parses your json into JsonElement structure. The behaviour that you see is a normal since you are using toString method of JsonElement. To achieve your goal just use JsonElement::getAsString method :
String resultCode = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().get("resultCode").getAsString();
which gives SUCCESS instead of "SUCCESS"
Note that JsonElement is an abstract class and classes, that extend this class, will override those helper getAs... methods. In your case JsonPrimitive::getAsString will be invoked.
Also you could create a POJO class for your json and use Gson::fromJson to parse json into object of your POJO class.
With the input from #Michalk:
I understand that easy way to read JSON data is using Gson::fromJson and creating POJO class for out json.
I have generated POJO Classes supplying my sample input JSON using this link
and Now I have POJO Classes called : CreateOrderJSONResponse
Gson::fromJson
Sample :
Gson gson = new Gson();
CreateOrderJSONResponse responseJson = gson.fromJson(inputJSON, CreateOrderJSONResponse.class);
Accessubg data :
String resultCodeText = responseJson.getResponse().getBody().getResultInfo().getResultCode();
String resultCodeId = responseJson.getResponse().getBody().getResultInfo().getResultCodeId();
String resultStatus = responseJson.getResponse().getBody().getResultInfo().getResultStatus();
String checkoutUrl = responseJson.getResponse().getBody().getCheckoutUrl();
Above Gson::fromJson example works smooth and it looks neat compare to direct accessing the filed with below sample code :
JsonObject jo = parser.parse(inputJSON).getAsJsonObject();
String resultCodeText = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().getAsJsonPrimitive("resultCode").getAsString();
String resultCodeId = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().getAsJsonPrimitive("resultCodeId").getAsString();
String resultStatus = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().get("resultInfo").getAsJsonObject().getAsJsonPrimitive("resultStatus").getAsString();
String checkoutUrl = jo.get("response").getAsJsonObject().get("body").getAsJsonObject().getAsJsonPrimitive("checkoutUrl").getAsString();
Note :
I have found this link of JSON or JAVA, SCALA, POJO generator tools as GitHub access you can access here
I'm playing around with Kotlin and Spark, creating a RESTful web service. However I'm struggling to parse a JSON POST request. I have the following endpoint...
post("") { req, res ->
var objectMapper = ObjectMapper()
println(req.body())
val data = objectMapper.readValue(req.body(), User::class.java)
usersDao.save(data.name, data.email, data.age)
res.status(201)
"okies"
}
However I'm getting a 500 error, it's not actually printing an error, just returning a 500.
It seems to be this line val data = objectMapper.readValue(req.body(), User::class.java). I'm attempting to convert the json body into a user object. Here's my user object...
data class User(val name: String, val email: String, val age: Int, val id: Int)