Previously I was reading json data in the following format:
JSON
{
"CreationTime":"2018-01-12T12:32:31",
"Id":"08f81fd7-21f1-48ba-a991-08d559b88cc5",
"Operation":"AddedToGroup",
"RecordType":14,
"UserType":0,
"Version":1,
"Workload":"OneDrive",
"ClientIP":"115.186.129.229",
"UserId":"omaji7#emumbaa10.onmicrosoft.com",
"EventSource":"SharePoint",
"ItemType":"Web"
}
I am reading this json data from a kafka topic and doing some stream processing on it and passing it onto another topic. In processing I have created two json objects, send and received.
Using this code:
final StreamsBuilder builder = new StreamsBuilder();
KStream<String, String> source_o365_user_activity = builder.stream("o365_user_activity");
source_o365_user_activity.flatMapValues(new ValueMapper<String, Iterable<String>>() {
#Override
public Iterable<String> apply(String value) {
System.out.println("========> o365_user_activity_by_date Log: " + value);
ArrayList<String> keywords = new ArrayList<String>();
try {
JSONObject send = new JSONObject();
JSONObject received = new JSONObject(value);
send.put("current_date", getCurrentDate().toString()); // UTC TIME
send.put("activity_time", received.get("CreationTime")); // CONSTANTS FINAL STATIC(Topic Names, Cassandra keys)
send.put("user_id", received.get("UserId"));
send.put("operation_type", received.get("Operation"));
send.put("app_name", received.get("Workload"));
keywords.add(send.toString());
// apply regex to value and for each match add it to keywords
} catch (Exception e) {
// TODO: handle exception
System.err.println("Unable to convert to json");
e.printStackTrace();
}
return keywords;
}
}).to("o365_user_activity_by_date");
This was fairly simple. Now I have a json data with lists in them.
JSON
{
"CreationTime":"2017-12-27T07:47:46",
"Id":"10ee505b-90a4-4ac1-b96f-a6dbca939694",
"Operation":"Add member to role.",
"OrganizationId":"2f88f444-62da-4aae-b8af-8331a6915801",
"RecordType":8,
"ResultStatus":"success",
"UserKey":"10030000A656FE5B#emumbaa10.onmicrosoft.com",
"UserType":0,
"Version":1,
"Workload":"AzureActiveDirectory",
"ObjectId":"mustafa#emumbaa10.onmicrosoft.com",
"UserId":"omaji7#emumbaa10.onmicrosoft.com",
"AzureActiveDirectoryEventType":1,
"ExtendedProperties":[
{
"Name":"Role.ObjectID",
"Value":"b0f54661-2d74-4c50-afa3-1ec803f12efe"
},
{
"Name":"Role.DisplayName",
"Value":"Billing Administrator"
},
{
"Name":"Role.TemplateId",
"Value":"b0f54661-2d74-4c50-afa3-1ec803f12efe"
},
{
"Name":"Role.WellKnownObjectName",
"Value":"BillingAdmins"
}
],
"Actor":[
{
"ID":"omaji7#emumbaa10.onmicrosoft.com",
"Type":5
},
{
"ID":"10030000A656FE5B",
"Type":3
},
{
"ID":"User_d03ca514-adfa-4585-a8bd-7182a9a086c7",
"Type":2
}
],
"ActorContextId":"2f88f444-62da-4aae-b8af-8331a6915801",
"InterSystemsId":"6d402a5b-c5de-4d9f-a805-9371c109e55f",
"IntraSystemId":"a5568d01-f100-497a-b88b-c9731ff31248",
"Target":[
{
"ID":"User_8f77c311-3ea0-4146-9f7d-db21bd052d3d",
"Type":2
},
{
"ID":"mustafa#emumbaa10.onmicrosoft.com",
"Type":5
},
{
"ID":"1003BFFDA67CCA03",
"Type":3
}
],
"TargetContextId":"2f88f444-62da-4aae-b8af-8331a6915801"
}
How can I go about doing the same thing in my Stream processing?
I want to be able to read JSON data against some keys (including the list data keys).
Why not convert JSON to Object and then filter against using the field in Object?
Can't you do like this?
send.put("target_0_id", received.get("Target").getJSONObject(0).get("ID"));
You can use gson library and can convert the json to object and using getter and setter you can build your desired output JSON. You can also parse the input JSON to fetch the JSONArray details. Following is the code how you can do it using POJO.
Input class:
public class Input {
private String UserType;
private String TargetContextId;
private String RecordType;
private String Operation;
private String Workload;
private String UserId;
private String OrganizationId;
private String InterSystemsId;
private ExtendedProperties[] ExtendedProperties;
private String ActorContextId;
private String CreationTime;
private String IntraSystemId;
private Target[] Target;
private Actor[] Actor;
private String Id;
private String Version;
private String ResultStatus;
private String ObjectId;
private String AzureActiveDirectoryEventType;
private String UserKey;
public String getUserType ()
{
return UserType;
}
public void setUserType (String UserType)
{
this.UserType = UserType;
}
public String getTargetContextId ()
{
return TargetContextId;
}
public void setTargetContextId (String TargetContextId)
{
this.TargetContextId = TargetContextId;
}
public String getRecordType ()
{
return RecordType;
}
public void setRecordType (String RecordType)
{
this.RecordType = RecordType;
}
public String getOperation ()
{
return Operation;
}
public void setOperation (String Operation)
{
this.Operation = Operation;
}
public String getWorkload ()
{
return Workload;
}
public void setWorkload (String Workload)
{
this.Workload = Workload;
}
public String getUserId ()
{
return UserId;
}
public void setUserId (String UserId)
{
this.UserId = UserId;
}
public String getOrganizationId ()
{
return OrganizationId;
}
public void setOrganizationId (String OrganizationId)
{
this.OrganizationId = OrganizationId;
}
public String getInterSystemsId ()
{
return InterSystemsId;
}
public void setInterSystemsId (String InterSystemsId)
{
this.InterSystemsId = InterSystemsId;
}
public ExtendedProperties[] getExtendedProperties ()
{
return ExtendedProperties;
}
public void setExtendedProperties (ExtendedProperties[] ExtendedProperties)
{
this.ExtendedProperties = ExtendedProperties;
}
public String getActorContextId ()
{
return ActorContextId;
}
public void setActorContextId (String ActorContextId)
{
this.ActorContextId = ActorContextId;
}
public String getCreationTime ()
{
return CreationTime;
}
public void setCreationTime (String CreationTime)
{
this.CreationTime = CreationTime;
}
public String getIntraSystemId ()
{
return IntraSystemId;
}
public void setIntraSystemId (String IntraSystemId)
{
this.IntraSystemId = IntraSystemId;
}
public Target[] getTarget ()
{
return Target;
}
public void setTarget (Target[] Target)
{
this.Target = Target;
}
public Actor[] getActor ()
{
return Actor;
}
public void setActor (Actor[] Actor)
{
this.Actor = Actor;
}
public String getId ()
{
return Id;
}
public void setId (String Id)
{
this.Id = Id;
}
public String getVersion ()
{
return Version;
}
public void setVersion (String Version)
{
this.Version = Version;
}
public String getResultStatus ()
{
return ResultStatus;
}
public void setResultStatus (String ResultStatus)
{
this.ResultStatus = ResultStatus;
}
public String getObjectId ()
{
return ObjectId;
}
public void setObjectId (String ObjectId)
{
this.ObjectId = ObjectId;
}
public String getAzureActiveDirectoryEventType ()
{
return AzureActiveDirectoryEventType;
}
public void setAzureActiveDirectoryEventType (String AzureActiveDirectoryEventType)
{
this.AzureActiveDirectoryEventType = AzureActiveDirectoryEventType;
}
public String getUserKey ()
{
return UserKey;
}
public void setUserKey (String UserKey)
{
this.UserKey = UserKey;
}
#Override
public String toString()
{
return "ClassPojo [UserType = "+UserType+", TargetContextId = "+TargetContextId+", RecordType = "+RecordType+", Operation = "+Operation+", Workload = "+Workload+", UserId = "+UserId+", OrganizationId = "+OrganizationId+", InterSystemsId = "+InterSystemsId+", ExtendedProperties = "+ExtendedProperties+", ActorContextId = "+ActorContextId+", CreationTime = "+CreationTime+", IntraSystemId = "+IntraSystemId+", Target = "+Target+", Actor = "+Actor+", Id = "+Id+", Version = "+Version+", ResultStatus = "+ResultStatus+", ObjectId = "+ObjectId+", AzureActiveDirectoryEventType = "+AzureActiveDirectoryEventType+", UserKey = "+UserKey+"]";
}}
Target class:
public class Target {
private String Type;
private String ID;
public String getType() {
return Type;
}
public void setType(String Type) {
this.Type = Type;
}
public String getID() {
return ID;
}
public void setID(String ID) {
this.ID = ID;
}
#Override
public String toString() {
return "ClassPojo [Type = " + Type + ", ID = " + ID + "]";
}}
Actor class :
public class Actor {
private String Type;
private String ID;
public String getType() {
return Type;
}
public void setType(String Type) {
this.Type = Type;
}
public String getID() {
return ID;
}
public void setID(String ID) {
this.ID = ID;
}
#Override
public String toString() {
return "ClassPojo [Type = " + Type + ", ID = " + ID + "]";
}}
ExtendedProperties class :
public class ExtendedProperties {
private String Name;
private String Value;
public String getName() {
return Name;
}
public void setName(String Name) {
this.Name = Name;
}
public String getValue() {
return Value;
}
public void setValue(String Value) {
this.Value = Value;
}
#Override
public String toString() {
return "ClassPojo [Name = " + Name + ", Value = " + Value + "]";
}}
Main class :
public class Stack {
public static void main(String[] args) {
doIt();
}
private static void doIt() {
String received = "{\"CreationTime\":\"2017-12-27T07:47:46\",\"Id\":\"10ee505b-90a4-4ac1-b96f-a6dbca939694\",\"Operation\":\"Add member to role.\",\"OrganizationId\":\"2f88f444-62da-4aae-b8af-8331a6915801\",\"RecordType\":8,\"ResultStatus\":\"success\",\"UserKey\":\"10030000A656FE5B#emumbaa10.onmicrosoft.com\",\"UserType\":0,\"Version\":1,\"Workload\":\"AzureActiveDirectory\",\"ObjectId\":\"mustafa#emumbaa10.onmicrosoft.com\",\"UserId\":\"omaji7#emumbaa10.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":1,\"ExtendedProperties\":[{\"Name\":\"Role.ObjectID\",\"Value\":\"b0f54661-2d74-4c50-afa3-1ec803f12efe\"},{\"Name\":\"Role.DisplayName\",\"Value\":\"Billing Administrator\"},{\"Name\":\"Role.TemplateId\",\"Value\":\"b0f54661-2d74-4c50-afa3-1ec803f12efe\"},{\"Name\":\"Role.WellKnownObjectName\",\"Value\":\"BillingAdmins\"}],\"Actor\":[{\"ID\":\"omaji7#emumbaa10.onmicrosoft.com\",\"Type\":5},{\"ID\":\"10030000A656FE5B\",\"Type\":3},{\"ID\":\"User_d03ca514-adfa-4585-a8bd-7182a9a086c7\",\"Type\":2}],\"ActorContextId\":\"2f88f444-62da-4aae-b8af-8331a6915801\",\"InterSystemsId\":\"6d402a5b-c5de-4d9f-a805-9371c109e55f\",\"IntraSystemId\":\"a5568d01-f100-497a-b88b-c9731ff31248\",\"Target\":[{\"ID\":\"User_8f77c311-3ea0-4146-9f7d-db21bd052d3d\",\"Type\":2},{\"ID\":\"mustafa#emumbaa10.onmicrosoft.com\",\"Type\":5},{\"ID\":\"1003BFFDA67CCA03\",\"Type\":3}],\"TargetContextId\":\"2f88f444-62da-4aae-b8af-8331a6915801\"}";
JSONObject send = new JSONObject();
Gson gson = new Gson();
Input inputObject = gson.fromJson(received, Input.class);
// you can add values here and customize the output JSON
send.put("userId", inputObject.getUserId());
send.put("Workload", inputObject.getWorkload());
// read Actor list
Actor[] arr = inputObject.getActor();
for (int i = 0; i < arr.length; i++) {
// write your logic here how you want to handle the Actor list
// values
System.out.println(arr[i].getID() + " : " + arr[i].getType());
}
// read ExtendedProperties list
ExtendedProperties[] extendedProperties = inputObject.getExtendedProperties();
for (int j = 0; j < extendedProperties.length; j++) {
// write your logic here how you want to handle the
// ExtendedProperties list values
System.out.println(extendedProperties[j].getName() + " : " + extendedProperties[j].getValue());
}
System.out.println("*************");
}}
alternate main class without using POJO. Here org.json library have been used to parse the input JSON.
public class Test {
public static void main(String[] args) {
doIt();
}
private static void doIt() {
String received = "{\"CreationTime\":\"2017-12-27T07:47:46\",\"Id\":\"10ee505b-90a4-4ac1-b96f-a6dbca939694\",\"Operation\":\"Add member to role.\",\"OrganizationId\":\"2f88f444-62da-4aae-b8af-8331a6915801\",\"RecordType\":8,\"ResultStatus\":\"success\",\"UserKey\":\"10030000A656FE5B#emumbaa10.onmicrosoft.com\",\"UserType\":0,\"Version\":1,\"Workload\":\"AzureActiveDirectory\",\"ObjectId\":\"mustafa#emumbaa10.onmicrosoft.com\",\"UserId\":\"omaji7#emumbaa10.onmicrosoft.com\",\"AzureActiveDirectoryEventType\":1,\"ExtendedProperties\":[{\"Name\":\"Role.ObjectID\",\"Value\":\"b0f54661-2d74-4c50-afa3-1ec803f12efe\"},{\"Name\":\"Role.DisplayName\",\"Value\":\"Billing Administrator\"},{\"Name\":\"Role.TemplateId\",\"Value\":\"b0f54661-2d74-4c50-afa3-1ec803f12efe\"},{\"Name\":\"Role.WellKnownObjectName\",\"Value\":\"BillingAdmins\"}],\"Actor\":[{\"ID\":\"omaji7#emumbaa10.onmicrosoft.com\",\"Type\":5},{\"ID\":\"10030000A656FE5B\",\"Type\":3},{\"ID\":\"User_d03ca514-adfa-4585-a8bd-7182a9a086c7\",\"Type\":2}],\"ActorContextId\":\"2f88f444-62da-4aae-b8af-8331a6915801\",\"InterSystemsId\":\"6d402a5b-c5de-4d9f-a805-9371c109e55f\",\"IntraSystemId\":\"a5568d01-f100-497a-b88b-c9731ff31248\",\"Target\":[{\"ID\":\"User_8f77c311-3ea0-4146-9f7d-db21bd052d3d\",\"Type\":2},{\"ID\":\"mustafa#emumbaa10.onmicrosoft.com\",\"Type\":5},{\"ID\":\"1003BFFDA67CCA03\",\"Type\":3}],\"TargetContextId\":\"2f88f444-62da-4aae-b8af-8331a6915801\"}";
JSONObject send = new JSONObject();
JSONObject input = new JSONObject(received);
// you can add values here and customize the output JSON
send.put("userId", input.getString("UserId"));
send.put("Workload", input.getString("Workload"));
// read Actor list
JSONArray actorArray = input.getJSONArray("Actor");
for (int i = 0; i < actorArray.length(); i++) {
// write your logic here how you want to handle the Actor list
// values
System.out.println(
actorArray.getJSONObject(i).getString("ID") + ":" + actorArray.getJSONObject(i).getInt("Type"));
}
// read ExtendedProperties list
JSONArray extendedProperties = input.getJSONArray("ExtendedProperties");
for (int j = 0; j < extendedProperties.length(); j++) {
// write your logic here how you want to handle the
// ExtendedProperties list values
System.out.println(extendedProperties.getJSONObject(j).getString("Name") + " : "
+ extendedProperties.getJSONObject(j).getString("Value"));
}
System.out.println("*************");
}}
I'm facing this task:
I have class A and class B. These two classes are different but almost the same.
I need to somehow merge them into 1 Single array of objects so I will be able to use them later in a list that combines both classes.
Class A:
public class Followers {
private String request_id;
private String number_sender;
private String state;
public String getRequest_id() {
return request_id;
}
public String getNumber_sender() {
return number_sender;
}
public String getState() {
return state;
}
}
Class B:
public class Following {
private String name;
private String state;
private String request_id;
public String getRequest_id() {
return request_id;
}
public String getName() {
return name;
}
public String getState() {
return state;
}
}
I've tried doing this next move:
Object[] obj1 = (Object[]) followers;
Object[] obj2 = (Object[]) followings;
Object[] completeArray = ArrayUtils.addAll(obj1, obj2);
Where followers and followings are both arrays of the corresponding classes. Then in my list adapter I use:
if (values[currentItem] instanceof Followers) { BLA BLA BLA}
else if (values[currentItem] instanceof Following) { BLA BLA BLA}
But I get this exception:
Caused by: java.lang.ArrayStoreException: source[0] of type json.objects.Following cannot be stored in destination array of type json.objects.Followers[]
What will be the best way to merge two arrays of different objects into one array?
Will just implementing the same interface between them do the job and then they will basically be in an array of the interface type?
what other ways do you recommend?
Try this
Object[] completeArray = new Object[0];
completeArray = ArrayUtils.addAll(completeArray, obj1);
completeArray = ArrayUtils.addAll(completeArray, obj2);
If you make both classes implement a common interface you can manipulate arrays/lists of them as if they contains instances of the interface.
public interface Follow {
public String getRequest_id();
public String getState();
}
public class Follower implements Follow {
private String request_id;
private String number_sender;
private String state;
public String getRequest_id() {
return request_id;
}
public String getNumber_sender() {
return number_sender;
}
public String getState() {
return state;
}
}
public class Following implements Follow {
private String name;
private String state;
private String request_id;
public String getRequest_id() {
return request_id;
}
public String getName() {
return name;
}
public String getState() {
return state;
}
}
public void test() {
List<Follow> all = new ArrayList<>();
all.add(new Following());
all.add(new Follower());
for ( Follow f : all ) {
String id = f.getRequest_id();
String state = f.getState();
}
}
Alternatively you could put them in a hierarchy:
public class Entity {
private String request_id;
private String state;
public String getRequest_id() {
return request_id;
}
public String getState() {
return state;
}
}
public class Follower extends Entity {
private String number_sender;
public String getNumber_sender() {
return number_sender;
}
}
public class Following extends Entity {
private String name;
public String getName() {
return name;
}
}
public void test() {
List<Entity> all = new ArrayList<>();
all.add(new Following());
all.add(new Follower());
for ( Entity f : all ) {
String id = f.getRequest_id();
String state = f.getState();
}
}
Or you could make the extra fields into attributes.
enum Attribute {
Follows,
Followed;
}
public static class Entity {
private String request_id;
private String state;
EnumMap<Attribute, String> attributes = new EnumMap<>(Attribute.class);
public String getRequest_id() {
return request_id;
}
public String getState() {
return state;
}
// Factory to make entities.
static Entity make(Attribute attribute, String value) {
Entity e = new Entity();
e.attributes.put(attribute, value);
return e;
}
}
public void test() {
List<Entity> all = new ArrayList<>();
all.add(Entity.make(Attribute.Follows, "Fred"));
all.add(Entity.make(Attribute.Followed, "Gill"));
for (Entity f : all) {
String id = f.getRequest_id();
String state = f.getState();
}
}
There are an infinite number of possibilities.
USE concat
var combined= obj1.concat(obj2); // Merges both arrays
Try this.
private Object[] appendObj(Object[] obj, Object newObj) {
ArrayList<Object> temp = new ArrayList<Object>(Arrays.asList(obj));
temp.add(newObj);
return temp.toArray();
}
i can't parse JSON object to corresponding formats in my Java class. Problems were began when i try to parse MySql data type in Java Data type and MySQL time type to java data type. I was tried few different solutions from stackoverflow but i cant resolve problem. Here is my DATABASE TABLE, JSON function and JAVA Class
DATABASE TABLE
filmID int(11)
naziv varchar(50)
datum date
trajanje time
cijenaKarte float
salaID int(11)
JSON Object mapper
/**************************************************************************/
public static filmovi jsonToFilmovi(JSONObject jsonObject) {
filmovi Filmovi = null;
try {
Filmovi = new filmovi(jsonObject.getJSONArray("korisnik").getJSONObject(0).getInt("filmID"),
jsonObject.getJSONArray("filmovi").getJSONObject(0).getString("naziv"),
jsonObject.getJSONArray("filmovi").getJSONObject(0).get("datum").toString()),
jsonObject.getJSONArray("filmovi").getJSONObject(0).getString("trajanje"),
Float.parseFloat(jsonObject.getJSONArray("filmovi").getJSONObject(0).get("cijenaKarte").toString()),
jsonObject.getJSONArray("filmovi").getJSONObject(0).getInt("salaID"));
} catch (Exception e) {
Log.e("jsontToFilmovi", "JSON TO FILMOVI ERROR: " + e.getMessage());
}
return Filmovi;
}
/********************************************************************************/
JAVA CLASS
public class filmovi {
#Expose
private Integer filmID;
#Expose
private String naziv;
#Expose
private Date datum;
#Expose
private Date trajanje;
#Expose
private Float cijenaKarte;
#Expose
private Integer salaID;
public filmovi(Integer filmID, String naziv, Date datum, Date trajanje, Float cijenaKarte, Integer salaID) {
this.filmID = filmID;
this.naziv = naziv;
this.datum = datum;
this.trajanje = trajanje;
this.cijenaKarte = cijenaKarte;
this.salaID = salaID;
}
public Integer getFilmID() {
return filmID;
}
public void setFilmID(Integer filmID) {
this.filmID = filmID;
}
public String getNaziv() {
return naziv;
}
public void setNaziv(String naziv) {
this.naziv = naziv;
}
public Date getDatum() {
return datum;
}
public void setDatum(Date datum) {
this.datum = datum;
}
public Date getTrajanje() {
return trajanje;
}
public void setTrajanje(Date trajanje) {
this.trajanje = trajanje;
}
public Float getCijenaKarte() {
return cijenaKarte;
}
public void setCijenaKarte(Float cijenaKarte) {
this.cijenaKarte = cijenaKarte;
}
public Integer getSalaID() {
return salaID;
}
public void setSalaID(Integer salaID) {
this.salaID = salaID;
}
I'm being given a Json file with the form:
{
"descriptions": {
"desc1": "someString",
"desc2": {"name":"someName", "val": 7.0}
}
}
I have the POJO:
public class CustomClass {
Map<String, Object> descriptions;
public static class NameVal{
String name;
double val;
public NameVal(String name, double val){...}
}
}
I can recreate the json file with the code:
CustomClass a = new CustomClass();
a.descriptions = new HashMap<String, Object>();
a.descriptions.put("desc1", "someString");
a.descriptions.put("desc2", new CustomClass.NameVal("someName", 7.0));
new ObjectMapper().writeValue(new File("testfile"), a);
But, when I read the object back in using:
CustomClass fromFile = new ObjectMapper().readValue(new File("testfile"), CustomClass.class);
then fromFile.descriptions.get("desc2") is of type LinkedHashMap instead of type CustomClass.NameVal.
How can I get Jackson to properly parse the type of the CustomClass.NameVal descriptors (other than making some class that wraps the parsing and explicitly converts the LinkedHashMap after Jackson reads the file)?
Try this. Create a class Description with name and value attributes:
public class Description {
private String name;
private double val;
}
Now in your CustomClass do this:
public class CustomClass {
List<Description> descriptions;
}
And that's it. Remember to create getters and setters because Jackson needs it.
You could try something like this:
public class DescriptionWrapper {
private Description descriptions;
public Description getDescriptions() {
return descriptions;
}
public void setDescriptions(Description descriptions) {
this.descriptions = descriptions;
}
}
public class Description {
private String desc1;
private NameValue desc2;
public String getDesc1() {
return desc1;
}
public void setDesc1(String desc1) {
this.desc1 = desc1;
}
public NameValue getDesc2() {
return desc2;
}
public void setDesc2(NameValue desc2) {
this.desc2 = desc2;
}
}
public class NameValue {
private String name;
private double val;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public double getVal() {
return val;
}
public void setVal(double val) {
this.val = val;
}
}