I am using following class to create bean from Spark Encoders
Class OuterClass implements Serializable {
int id;
ArrayList<InnerClass> listofInner;
public int getId() {
return id;
}
public void setId (int num) {
this.id = num;
}
public ArrayList<InnerClass> getListofInner() {
return listofInner;
}
public void setListofInner(ArrayList<InnerClass> list) {
this.listofInner = list;
}
}
public static class InnerClass implements Serializable {
String streetno;
public void setStreetno(String streetno) {
this.streetno= streetno;
}
public String getStreetno() {
return streetno;
}
}
Encoder<OuterClass> outerClassEncoder = Encoders.bean(OuterClass.class);
Dataset<OuterClass> ds = spark.createDataset(Collections.singeltonList(outerclassList), outerClassEncoder)
And I am getting the following error
Exception in thread "main" java.lang.UnsupportedOperationException: Cannot infer type for class OuterClass$InnerClass because it is not bean-compliant
How can I implement this type of use case for Spark in Java? This worked fine if I remove the inner class. But I need to have an inner class for my use case.
Your JavaBean class should have a public no-argument constructor, getter and setters and it should implement Serializable interface. Spark SQL works on valid JavaBean class.
EDIT : Adding working sample with inner class
OuterInnerDF.java
package com.abaghel.examples;
import java.util.ArrayList;
import java.util.Collections;
import org.apache.spark.sql.Dataset;
import org.apache.spark.sql.Encoder;
import org.apache.spark.sql.Encoders;
import org.apache.spark.sql.SparkSession;
import com.abaghel.examples.OuterClass.InnerClass;
public class OuterInnerDF {
public static void main(String[] args) {
SparkSession spark = SparkSession
.builder()
.appName("OuterInnerDF")
.config("spark.sql.warehouse.dir", "/file:C:/temp")
.master("local[2]")
.getOrCreate();
System.out.println("====> Create DataFrame");
//Outer
OuterClass us = new OuterClass();
us.setId(111);
//Inner
OuterClass.InnerClass ic = new OuterClass.InnerClass();
ic.setStreetno("My Street");
//list
ArrayList<InnerClass> ar = new ArrayList<InnerClass>();
ar.add(ic);
us.setListofInner(ar);
//DF
Encoder<OuterClass> outerClassEncoder = Encoders.bean(OuterClass.class);
Dataset<OuterClass> ds = spark.createDataset(Collections.singletonList(us), outerClassEncoder);
ds.show();
}
}
OuterClass.java
package com.abaghel.examples;
import java.io.Serializable;
import java.util.ArrayList;
public class OuterClass implements Serializable {
int id;
ArrayList<InnerClass> listofInner;
public int getId() {
return id;
}
public void setId(int num) {
this.id = num;
}
public ArrayList<InnerClass> getListofInner() {
return listofInner;
}
public void setListofInner(ArrayList<InnerClass> list) {
this.listofInner = list;
}
public static class InnerClass implements Serializable {
String streetno;
public void setStreetno(String streetno) {
this.streetno = streetno;
}
public String getStreetno() {
return streetno;
}
}
}
Console Output
====> Create DataFrame
16/08/28 18:02:55 INFO CodeGenerator: Code generated in 32.516369 ms
+---+-------------+
| id| listofInner|
+---+-------------+
|111|[[My Street]]|
+---+-------------+
Related
I'm using the Gson library and jakarta. Although I have been able to use the conversion in CarrinhoResource.java as below, my ClienteTest.java cannot use the String content (already in json) inside the cart. I cant run my test a just only message into my intellij is (Cannot resolve method 'fromJson(java.lang.String)').
Can someone help me?
Class CarrinhoResource.java
package br.com.alura.loja.resource;
import br.com.alura.loja.dao.CarrinhoDAO;
import br.com.alura.loja.modelo.Carrinho;
import jakarta.ws.rs.GET;
import jakarta.ws.rs.Path;
import jakarta.ws.rs.Produces;
import jakarta.ws.rs.core.MediaType;
#Path("/v1/carrinhos")
public class CarrinhoResource {
#GET
#Produces(MediaType.APPLICATION_JSON)
public String busca(){
Carrinho carrinho = new CarrinhoDAO().busca(1L);
return carrinho.toJson();
}
}
Carrinho.java
package br.com.alura.loja.modelo;
import java.util.ArrayList;
import java.util.Iterator;
import java.util.List;
import com.google.gson.Gson;
public class Carrinho {
private List<Produto> produtos = new ArrayList<Produto>();
private String rua;
private String cidade;
private long id;
public Carrinho adiciona(Produto produto) {
produtos.add(produto);
return this;
}
public Carrinho para(String rua, String cidade) {
this.rua = rua;
this.cidade = cidade;
return this;
}
public Carrinho setId(long id) {
this.id = id;
return this;
}
public String getRua() {
return rua;
}
public void setRua(String rua) {
this.rua = rua;
}
public void setCidade(String cidade) {
this.cidade = cidade;
}
public long getId() {
return id;
}
public void remove(long id) {
for (Iterator iterator = produtos.iterator(); iterator.hasNext();) {
Produto produto = (Produto) iterator.next();
if(produto.getId() == id) {
iterator.remove();
}
}
}
public void troca(Produto produto) {
remove(produto.getId());
adiciona(produto);
}
public void trocaQuantidade(Produto produto) {
for (Iterator iterator = produtos.iterator(); iterator.hasNext();) {
Produto p = (Produto) iterator.next();
if(p.getId() == produto.getId()) {
p.setQuantidade(produto.getQuantidade());
return;
}
}
}
public List<Produto> getProdutos() {
return produtos;
}
public String toJson() {
return new Gson().toJson(this);
}
}
ClienteTest.java
package br.com.alura.loja;
import br.com.alura.loja.modelo.Carrinho;
import com.google.gson.*;
import jakarta.ws.rs.client.Client;
import jakarta.ws.rs.client.ClientBuilder;
import jakarta.ws.rs.client.WebTarget;
import org.junit.Assert;
import org.junit.Test;
public class ClienteTest {
#Test
public void testaConexaoServidor() {
Client client = ClientBuilder.newClient();
WebTarget target = client.target("http://localhost:8085");
String conteudo = target.path("/v1/carrinhos").request().get(String.class);
Carrinho carrinho = (Carrinho) new Gson().fromJson(conteudo); **//Cannot resolve method 'fromJson(java.lang.String)'/**
System.out.println(carrinho);
Assert.assertEquals("Rua Vergueiro, 3185", carrinho.getRua());
}
}
Carrinho carrinho = (Carrinho) new Gson().fromJson(conteudo); **//Cannot resolve method 'fromJson(java.lang.String)'/**
The reason for this is that there is no Gson.fromJson(String) method, see the Gson class documentation. For deserialization Gson needs to know which type you are expecting, so all fromJson methods have a second parameter representing the type.
You can simply change your code to:
Carrinho carrinho = new Gson().fromJson(conteudo, Carrinho.class);
I have a problem when i want deserialize the object "ObjectAbstract1". I have this error :
I tried several solutions but without success ... If someone has an idea? The problem is located on object ObjectAbstract2 but I do not see what jackson anotation to add
-- serializing --
{"object1Type":"INSTANCE_OBJECT1","autor":"John","objectAbstract2":{"nameProc":"Processus transfert","identifiant":null,"nomObject3":"Demande de transfert"},"state":"En cours","priority":null}
-- deserializing --
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Can not construct instance of fr.test.jakson.ObjectAbstract2, problem: abstract types either need to be mapped to concrete types, have custom deserializer, or be instantiated with additional type information
at [Source: {"object1Type":"INSTANCE_OBJECT1","autor":"John","objectAbstract2":{"nameProc":"Processus transfert","identifiant":null,"nomObject3":"Demande de transfert"},"state":"En cours","priority":null}; line: 1, column: 49] (through reference chain: fr.test.jakson.Object1["objectAbstract2"])
at com.fasterxml.jackson.databind.JsonMappingException.from(JsonMappingException.java:148)
at com.fasterxml.jackson.databind.DeserializationContext.instantiationException(DeserializationContext.java:892)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserialize(AbstractDeserializer.java:139)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:520)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:258)
at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:161)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:136)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer._deserializeTypedForId(AsPropertyTypeDeserializer.java:122)
at com.fasterxml.jackson.databind.jsontype.impl.AsPropertyTypeDeserializer.deserializeTypedFromObject(AsPropertyTypeDeserializer.java:93)
at com.fasterxml.jackson.databind.deser.AbstractDeserializer.deserializeWithType(AbstractDeserializer.java:131)
at com.fasterxml.jackson.databind.deser.impl.TypeWrappedDeserializer.deserialize(TypeWrappedDeserializer.java:42)
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:3736)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2726)
at fr.test.jakson.MainTest.main(MainTest.java:33)
ObjectAbstract1 :
package fr.test.jakson;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonSubTypes.Type;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = "object1Type")
#JsonSubTypes({ #Type(value = Object1.class, name = "INSTANCE_OBJECT1") })
public abstract class ObjectAbstract1 implements Serializable {
private static final long serialVersionUID = 1L;
private String autor;
private ObjectAbstract2 objectAbstract2;
public ObjectAbstract1() {
}
public String getAutor() {
return autor;
}
public void setAutor(String autor) {
this.autor = autor;
}
public ObjectAbstract2 getObjectAbstract2() {
return objectAbstract2;
}
public void setObjectAbstract2(ObjectAbstract2 objectAbstract2) {
this.objectAbstract2 = objectAbstract2;
}
}
Object1 :
package fr.test.jakson;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonTypeName;
#JsonTypeName("INSTANCE_OBJECT1")
public class Object1 extends ObjectAbstract1 implements Serializable {
private static final long serialVersionUID = 1L;
private String state;
private String priority;
public Object1() {
}
public String getState() {
return state;
}
public void setState(String state) {
this.state = state;
}
public String getPriority() {
return priority;
}
public void setPriority(String priority) {
this.priority = priority;
}
}
ObjectAbstract2 :
package fr.test.jakson;
import java.io.Serializable;
public abstract class ObjectAbstract2 implements Serializable {
private static final long serialVersionUID = 1L;
private String nameProc;
public ObjectAbstract2() {
}
public String getNameProc() {
return nameProc;
}
public void setNameProc(String nameProc) {
this.nameProc = nameProc;
}
}
ObjectAbstract3 :
package fr.test.jakson;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonSubTypes;
import com.fasterxml.jackson.annotation.JsonSubTypes.Type;
import com.fasterxml.jackson.annotation.JsonTypeInfo;
#JsonTypeInfo(use = JsonTypeInfo.Id.NAME, include = JsonTypeInfo.As.PROPERTY, property = "objectAbstract3Type")
#JsonSubTypes({ #Type(value = Object3.class, name = "Object3"), #Type(value = Object2.class, name = "Object2") })
public abstract class ObjectAbstract3 extends ObjectAbstract2 implements Serializable {
private static final long serialVersionUID = 1L;
private String identifiant;
public ObjectAbstract3() {
}
public String getIdentifiant() {
return identifiant;
}
public void setIdentifiant(String identifiant) {
this.identifiant = identifiant;
}
}
Object2 :
package fr.test.jakson;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonTypeName;
#JsonTypeName("Object2")
public class Object2 extends ObjectAbstract3 implements Serializable {
private static final long serialVersionUID = 1L;
private String nomObject2;
public Object2() {
}
public String getNomObject2() {
return nomObject2;
}
public void setNomObject2(String nomObject2) {
this.nomObject2 = nomObject2;
}
}
Object 3 :
package fr.test.jakson;
import java.io.Serializable;
import com.fasterxml.jackson.annotation.JsonTypeName;
#JsonTypeName("Object3")
public class Object3 extends ObjectAbstract3 implements Serializable {
private static final long serialVersionUID = 1L;
private String nomObject3;
public Object3() {
}
public String getNomObject3() {
return nomObject3;
}
public void setNomObject3(String nomObject3) {
this.nomObject3 = nomObject3;
}
}
Main Class test :
package fr.test.jakson;
import java.io.IOException;
import com.fasterxml.jackson.core.JsonParseException;
import com.fasterxml.jackson.databind.JsonMappingException;
import com.fasterxml.jackson.databind.ObjectMapper;
public class MainTest {
public static void main(String[] args) throws JsonParseException, JsonMappingException, IOException {
Object1 object1 = new Object1();
object1.setAutor("John");
object1.setState("En cours");
Object3 object3 = new Object3();
object3.setNomObject3("Demande de transfert");
ObjectAbstract2 objectAbstract2 = (ObjectAbstract2) object3;
objectAbstract2.setNameProc("Processus transfert");
ObjectAbstract1 objectAbstract1 = (ObjectAbstract1) object1;
objectAbstract1.setObjectAbstract2(objectAbstract2);
System.out.println("-- serializing --");
ObjectMapper om = new ObjectMapper();
String s = om.writeValueAsString(objectAbstract1);
System.out.println(s);
System.out.println("-- deserializing --");
ObjectAbstract1 instanceTacheDeserializing;
ObjectMapper om1 = new ObjectMapper();
String jsonTest = "{\"object1Type\":\"INSTANCE_OBJECT1\",\"autor\":\"John\",\"objectAbstract2\":{\"nameProc\":\"Processus transfert\",\"identifiant\":null,\"nomObject3\":\"Demande de transfert\"},\"state\":\"En cours\",\"priority\":null}";
instanceTacheDeserializing = om1.readValue(jsonTest, ObjectAbstract1.class);
}
}
ObjectAbstract2 cannot be an Abstract class. Deserializer requires concrete class to convert into java object. Change ObjectAbstract2 class like below
package fr.test.jakson;
import java.io.Serializable;
public class ObjectAbstract2 implements Serializable {
private static final long serialVersionUID = 1L;
private String nameProc;
public ObjectAbstract2() {
}
public String getNameProc() {
return nameProc;
}
public void setNameProc(String nameProc) {
this.nameProc = nameProc;
}
}
I'm trying to deserialize some xml with nested properties with the same name, but the wrapper name is unique for each property. Example XML below.
I've tried playing with switching wrapper and property names but doesn't seem to work.
<response>
<string>
<item>Sample string.</item>
<item>Another sample string.</item>
</string>
<number>
<item>123123123</item>
<item>900912</item>
</number>
</response>
I'm trying to deserialize the above XML into a List<String> and List<Integer> variable.
I managed to make it creating a pair or wrappers of ArrayLists as inner classes:
import java.io.Serializable;
import java.util.ArrayList;
import java.util.List;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlElementWrapper;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlProperty;
import com.fasterxml.jackson.dataformat.xml.annotation.JacksonXmlRootElement;
#JacksonXmlRootElement(localName="response")
public class ResponseObjectList implements Serializable {
#JacksonXmlProperty(localName = "string")
private StringArrayListContainer string;
#JacksonXmlProperty(localName = "number")
private IntegerArrayListContainer number;
public ResponseObjectList() {
super();
}
public ResponseObjectList(List<String> stringItems, List<Integer> intItems) {
super();
this.string = new StringArrayListContainer(stringItems);
this.number = new IntegerArrayListContainer(intItems);
}
public StringArrayListContainer getString() {
return string;
}
public void setString(StringArrayListContainer string) {
this.string = string;
}
public IntegerArrayListContainer getNumber() {
return number;
}
public void setNumber(IntegerArrayListContainer number) {
this.number = number;
}
public static class StringArrayListContainer extends ArrayListContainer<String>{
public StringArrayListContainer() {
super();
}
public StringArrayListContainer(List<String> item) {
super(item);
}
}
public static class IntegerArrayListContainer extends ArrayListContainer<Integer>{
public IntegerArrayListContainer() {
super();
}
public IntegerArrayListContainer(List<Integer> item) {
super(item);
}
}
public static class ArrayListContainer<T extends Serializable>{
#JacksonXmlElementWrapper(useWrapping=false)
#JacksonXmlProperty(localName="item")
private List<T> item;
public ArrayListContainer(List<T> item) {
super();
this.item = item;
}
public ArrayListContainer() {
super();
}
public List<T> getItem() {
return item;
}
public void setItem(List<T> item) {
this.item = item;
}
}
}
Tests looked good:
#Test
public void test3() throws JsonProcessingException {
ResponseObjectList response = new ResponseObjectList(
Arrays.asList(new String[] {"Sample string.","Another sample string"}),
Arrays.asList(new Integer[] {123123123,900912})
);
XmlMapper xmlMapper = new XmlMapper();
String content = xmlMapper.writeValueAsString(response);
this.logger.debug("content: " + content);
// content: <response xmlns=""><string><item>Sample string.</item><item>Another sample string</item></string><number><item>123123123</item><item>900912</item></number></response>
}
#Test
public void test4() throws JsonParseException, JsonMappingException, IOException {
String xml =
"<response>"
+ "<string>"
+ "<item>Sample string.</item>"
+ "<item>Another sample string</item>"
+ "</string>"
+ "<number>"
+ "<item>123123123</item>"
+ "<item>900912</item>"
+ "</number>"
+ "</response>";
XmlMapper xmlMapper = new XmlMapper();
ResponseObjectList object = xmlMapper.readValue(xml, ResponseObjectList.class);
Assert.assertFalse(object.getString().getItem().isEmpty());
Assert.assertFalse(object.getNumber().getItem().isEmpty());
}
I used Lists instead of ArrayLists both for the wrappers and the tests
For version 2.9.9 simple POJO with JacksonXmlElementWrapper annotation works as expected:
class Response {
#JacksonXmlElementWrapper(localName = "string")
private List<String> strings;
#JacksonXmlElementWrapper(localName = "number")
private List<Integer> numbers;
// getters, setters
}
My java class is throwing some error. In my class i am using this to get my data.
((myDataDetails) Names.get(0)).InputParamNames().add("SomeValue");
But it is throwing error
Here is my Pohjo Class.
package common.pojo;
import java.util.Date;
import java.util.List;
public class myDataDetails
{
private String myID;
private List<String> InputParamNames;
private List InputParamData;
public String getmyID() {
return this.myID;
}
public void setmyID(String myID) {
this.myID = myID;
}
public List<String> getInputParamNames() {
return this.InputParamNames;
}
public void setInputParamNames(List<String> InputParamNames) {
this.InputParamNames = InputParamNames;
}
public List getInputParamData() {
return this.InputParamData;
}
public void setInputParamData(List InputParamData) {
this.InputParamData = InputParamData;
}
}
What should I need to change in pojo to avoid this exception.
Your class 'myDataDetails' needs to extend from LinkedHashMap in order to cast it.
What you have right now is a regular POJO class that is not an instance of LinkedHashMap, so you can't cast it as such.
EDIT: It should look like this
package common.pojo;
import java.util.Date;
import java.util.List;
import java.util.LinkedHashMap;
public class myDataDetails extends LinkedHashMap<Object, Object>
{
private String myID;
private List<String> InputParamNames;
private List InputParamData;
public String getmyID() {
return this.myID;
}
public void setmyID(String myID) {
this.myID = myID;
}
public List<String> getInputParamNames() {
return this.InputParamNames;
}
public void setInputParamNames(List<String> InputParamNames) {
this.InputParamNames = InputParamNames;
}
public List getInputParamData() {
return this.InputParamData;
}
public void setInputParamData(List InputParamData) {
this.InputParamData = InputParamData;
}
}
I have a base class called GenericOrder that can be used to create an order with any type of products, then I have subclasses of that order that are more specific. My problem is with my ComputerOrder class and a method that I'm overriding. Here's the base class code.
import java.util.List;
import java.util.ArrayList;
public class GenericOrder<T> {
private long orderNumber;
private List<T> products;
private T theClass;
public GenericOrder()
{
products = new ArrayList<T>();
orderNumber = System.currentTimeMillis();
}
public long getOrderNumber() {
return orderNumber;
}
public void addProduct(T newProduct) {
products.add(newProduct);
}
public int getNumberOfProducts() {
return products.size();
}
public List<T> getProducts()
{
return products;
}
public void setProducts(List<T> products)
{
this.products = products;
}
public T get()
{
return theClass;
}
public void set(T theClass)
{
this.theClass = theClass;
}
}
And here is my subClass code. The getProducts is the method I'm having trouble with.
import java.util.ArrayList;
import java.util.List;
public class ComputerOrder<T> extends GenericOrder<T> {
private List<ComputerPart> computerParts = new ArrayList<ComputerPart>();
private String orderType = "Computer Parts";
public ComputerOrder() {
super();
}
public void addProduct(ComputerPart newProduct) {
computerParts.add(newProduct);
}
public String getOrderType() {
return orderType;
}
public int getNumberOfProducts() {
return computerParts.size();
}
public List<T> getProducts()
{
return computerParts;
}
}
The Error I get says cannot convert from List(ComputerPart) to List<T>
The error is pretty clear: getProducts() is declared to return a List<T> yet you're returning a List<ComputerPart>. I think we agree that these two are not equivalent.
Looking at your code it looks like that you actually don't want a generic class since ComputerOrder only accepts ComputerParts. What you want is something like the following:
public class ComputerOrder extends GenericOrder<ComputerPart> {
#Override
public List<ComputerPart> getProducts() {
return computerParts;
}
}
Design wise, I think you should reconsider whether products should be in the GenericOrder class. If GenericOrder is meant only to handle the orders, then it might not make sense to have any product related methods or fields defined there. As it is now you have a products List array in GenericOrder that is not being used because you have defined computerParts List array in ComputerOrder. This makes for bad code. In this case your classes would look like:
public class GenericOrder<T> {
private long orderNumber;
private String orderType;
private T theClass;
public GenericOrder(String orderType) {
this.orderType = orderType;
orderNumber = System.currentTimeMillis();
}
public String getOrderType() {
return orderType;
}
public long getOrderNumber() {
return orderNumber;
}
public T get() {
return theClass;
}
public void set(T theClass) {
this.theClass = theClass;
}
}
and
import java.util.ArrayList;
import java.util.List;
public class PartOrder<T> extends GenericOrder<T> {
private List<T> parts = new ArrayList<T>();
public PartOrder(String orderType) {
super(orderType);
}
public void addProduct(T newProduct) {
parts.add(newProduct);
}
public int getNumberOfProducts() {
return parts.size();
}
public List<T> getProducts() {
return parts;
}
}
and you would have a ComputerPartOrder class like so:
public class ComputerPartOrder extends PartOrder<ComputerPart> {
public ComputerPartOrder() {
super("Computer Parts");
}
}
Otherwise, you might also define the GenericOrder.getProducts method as abstract as per this stackoverflow post.
It actually looks like you don't want your ComputerOrder to be generic.
While a GenericOrder<T> is generic and can be an order of anything, ComputerOrder seems specific to ComputerPart(s) and should extend GenericOrder<ComputerPart>.
This way you will only have to implement List<ComputerPart> getProducts() and your code will be fine.
As your ComputerOrders class looks more specific, consider refactoring your code as below :
public class ComputerOrder extends GenericOrder<ComputerPart> {
#Override
public List<ComputerPart> getProducts() { return computerParts; }
}