I have a flink project that will be inserting data in a cassandra table as a batch job. I already have a flink stream project where it is writing a pojo to the same cassandra table, but cassandraOutputFormat needs the data as a Tuple (hope that is changed to accept pojos like CassandraSink does at some point). So here is the pojo that I have that:
#Table(keyspace="mykeyspace", name="mytablename")
public class AlphaGroupingObject implements Serializable {
#Column(name = "jobId")
private String jobId;
#Column(name = "datalist")
#Frozen("list<frozen<dataobj>")
private List<CustomDataObj> dataobjs;
#Column(name = "userid")
private String userid;
//Getters and Setters
}
And the dataset of tuple I am making from this pojo:
DataSet<Tuple3<String, List<CustomDataObj>, String>> outputDataSet = listOfAlphaGroupingObject.map(new AlphaGroupingObjectToTuple3Mapper());
And here is the line that triggers the output as well:
outputDataSet.output(new CassandraOutputFormat<>("INSERT INTO mykeyspace.mytablename (jobid, datalist, userid) VALUES (?,?,?);", clusterThatWasBuilt));
Now the issue that I have is when I try to run this, I get this error when it tries to output it to the cassandra table:
Caused by: com.datastax.driver.core.exceptions.CodecNotFoundException:
Codec not found for requested operation: [frozen<mykeyspace.dataobj> <-> flink.custom.data.CustomDataObj]
So I know when it was a pojo, I just had to add the #Frozen annotation to the field, but I don't know how to do that for a tuple. What is the best/proper way to fix this? Or am I doing something unnecessary because there is actually a way to send pojos through the cassandraOutputFormat I just haven't found?
Thanks for any and all help in advance!
EDIT:
Here is the code for the CustomDataObj class too:
#UDT(name="dataobj", keyspace = "mykeyspace")
public class CustomDataObj implements Serializable {
#Field(name = "userid")
private String userId;
#Field(name = "groupid")
private String groupId;
#Field(name = "valuetext")
private String valueText;
#Field(name = "comments")
private String comments;
//Getters and setters
}
EDIT 2
Including the table schema in cassandra that the CustomDataObj is tied to and the mytablename schema.
CREATE TYPE mykeyspace.dataobj (
userid text,
groupid text,
valuetext text,
comments text
);
CREATE TABLE mykeyspace.mytablename (
jobid text,
datalist list<frozen<dataobj>>,
userid text,
PRIMARY KEY (jobid, userid)
);
Add UDT Annotation on CustomDataObj class
#UDT(name = "dataobj")
public class CustomDataObj {
//......
}
Edited
Change jobid Annotation to #Column(name = "jobid") and dataobjs Frozen Annotation to #Frozen
#Table(keyspace="mykeyspace", name="mytablename")
public class AlphaGroupingObject implements Serializable {
#Column(name = "jobid")
private String jobId;
#Column(name = "datalist")
#Frozen
private List<CustomDataObj> dataobjs;
#Column(name = "userid")
private String userid;
//Getters and Setters
}
I believe I have found a better way than having to provide a tuple to the cassandraOutputFormat, but it technically still doesn't answer this question so I won't mark this as the answer. I ended up using cassandra's object mapper so I can just send the pojo to the table. Still need to validate that data got to the table successfully and that everything is working properly with the way it is implemented, but I felt this would help anyone who is facing a similar problem.
Here is the doc that outlines the solution: http://docs.datastax.com/en/developer/java-driver/2.1/manual/object_mapper/using/
Related
I am trying to read data from another table based on the value of one of the fields in the current entity. But somehow I am facing an issue selecting multiple fields inside the formula.
#Entity
#Table(name = "contacts")
#Data
#Builder
#NoArgsConstructor
#AllArgsConstructor
public class ContactInfo {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
#Column(name = "id")
private Long id;
#Column(name = "name")
private String name;
#Column(name = "url")
private String imageUrl;
//Not working
#Formula("(select code,area from areas where area_id=id)")
private Map<String, String> vals;
//working
#Formula("(select code from areas where area_id=id)")
private String someVal;
}
is there any way that I can use the formula for retrieving multiple columns of data with multiple rows?
Thanks for your help.
I'm not sure you can retrieve like Map<string, String>. Suppose you declare another field how you going to retrieve. you can receive as a List instead of Map<st..., str..>.
Instead of #Formulae, you can use #OneToMany & #ManyToOne. through a Bi-direction connection, you achieve your scenario.
This looks more like it should be using JPA's ElementCollection mapping to the Area table, with a MapKeyColumn mapping:
public class ContactInfo {
..
#ElementCollection
#MapKeyColumn(name="code")//key
#Column(name="area") //value
#CollectionTable(name="areas",
joinColumns=#JoinColumn(name="area_id"))//fk from Areas->contacts.id
private Map<String, String> vals;
..
See this for some more information and examples.
i have a column in Database with a String, i.e. '{es:title_es,en:title_en}'
So, I need read it as a Map<String,String>, it will be for I18N.
I need the map with the language as key and label as value.
I use Hiberante 3.3.2 and Java 8 and SQL Server.
Is any chance to obtain this map in the EntityClass? This will be usefull because any relationship with this entity will have the correct representation and not a simple string.
And the behavior keep encapsulated.
Table :
OID CODE TITLES
1 XXX '{es:title_es,en:title_en}'
Entity :
#Entity
#Table(name = "TABLE_NAME")
public class EntityClass {
private Long oid;
private String code;
private String titles; ---> i have this but i want private Map<String, String>
#Column(name = "CODE", nullable = false)
public String getCode() {
return code;
}
#Column(name = "TITLES", nullable = false)
public String getTitles() {
return descriptions;
} ----> this work fine, but here i want to get directly the Map,
}```
So, I have one #Document class which has a embedded pojo field which I want it to be unique for the document based on a key in the pojo class. I tried using #CompoundIndex & #Indexed to mark it as unique but it doesn't seem to work.
#Document
public class Project {
private String id;
private String name;
private List<Details> details = new ArrayList<>();
}
public class Details{
private String key;
private String description;
}
What I want to achieve is that a project document should have unique details field in it with it's key being unique. But when I have the
#CompoundIndexes({ #CompoundIndex(name = "details_key", def = "{'details.key':1}", unique = true) }) on the Project class it doesn't work. Which I thought it should. Or am I wrong somewhere with my understanding. As I am new to this.
I am trying to use hibernate annotations for getting data from a MySQL database table which doesn't have a primary key defined.
However the fact is 2 columns of that table together are unique in the table. How can I achieve the same using hibernate annotation?
This is my code
#Entity
#Table(name = "coc_order_view")
public class CoCOrderDetailsTest {
#EmbeddedId
private MyJoinClassKey key;
#Column(name = "coupon_code")
private String couponCode;
some other columns and their getters and setters .....
#Embeddable
public class MyJoinClassKey implements Serializable {
private static final long serialVersionUID = -5L;
#Column(name = "product_id")
private int productId;
#Column(name = "order_id")
private int orderId;
gettes and setters....
And here is my criteria query
Criteria criteria = getHibernatetemplate().getSession().createCriteria(CoCOrderDetailsTest.class);
criteria.add(Restrictions.eq("status", "New"));
ArrayList<CoCOrderDetailsTest> orderDet = (ArrayList<CoCOrderDetailsTest>) getHibernatetemplate().get(criteria);
I am unable to get all the values from db. Kindly suggest some solutions.
After reading through your question again not sure this will help. You can't have a table without primary key(s). Read the first couple of paragraphs in this article
That said, if you can alter the table and add primary keys on those fields you need to add #IdClass annotation to your class signature for CoCOrderDetailsTest and then get rid of the #embeddable and #embeddedId notation in your classes.
Another alternative, if you can add a field to the table, would be to use an #GeneratedValue on that added primary key field and of course annotate it with #Id.
If you can't alter the table then you can't use JPA and you'll have to use JDBC.
See http://docs.oracle.com/javaee/5/api/javax/persistence/IdClass.html
A working example:
#Entity
#Table(name = "player_game_log")
#IdClass(PlayerGameLogId.class)
public class PlayerGameLog {
#Id
#Column(name = "PLAYER_ID")
private Integer playerId;
#Id
#Column(name = "GAME_ID")
private String gameId;
....
and the id class (note there are no annotations on the id class)....
public class PlayerGameLogId implements Serializable {
private static final long serialVersionUID = 1L;
private Integer playerId;
private String gameId;
Try:
String hql = "FROM CoCOrderDetailsTest WHERE status = :status";
Query query = session.createQuery(hql);
query.setParameter("status","New");
List results = query.list();
I usually use EntityManager rather than session so I'm not familiar with this syntax - and I have typically added a type to the list to be returned - like:
List<CoCOrderDetailsTest> results = query.list();
I'm trying to save a nested object using hibernate and I receive could not execute statement; SQL [n/a] Exception
CODE
#Entity
#Table(name = "listing")
#Inheritance(strategy = InheritanceType.JOINED)
public class Listing implements Serializable {
#Id
#Column(name = "listing_id")
private String listingId;
#Column(name = "property_type")
private PropertyType propertyType;
#Column(name = "category")
private Category category;
#Column(name = "price_currency")
private String priceCurrency;
#Column(name = "price_value")
private Double priceValue;
#Column(name = "map_point")
private MapPoint mapPoint;
#Column(name = "commission_fee_info")
private CommissionFeeInfo commissionFeeInfo;
}
public class MapPoint implements Serializable {
private final float latitude;
private final float longitude;
}
public class CommissionFeeInfo implements Serializable {
private String agentFeeInfo;
private CommissionFeeType commissionFeeType;
private Double value;
private Double commissionFee;
}
public enum CommissionFeeType implements Serializable { }
Using RazorSQL I saw that hibernate defines MapPoint and CommissionFee as VARBINARY
What I can't understand, is the fact that hibernate manages to save it when commissionFeeInfo is not present. It has no problem with saving MapPoint
Does anyone have an idea about what I do wrong?
UPDATE
I found out that if all attributes of CommissionFeeInfo excepting agentFeeInfoare null, the object will be saved without problems. If one of the other attributes is != null, the errors occur.
UPDATE 2
I changed the type of all attributes of CommissionFeeInfo into String and the object will be saved without problem, but I can't let the attributes as String.
I solved the problem by adding setting
#Column(name = "commission_fee_info", columnDefinition = "LONGVARBINARY")
as annotation for the field commisionFeeInfo in the class Listing
For me,
#Column(columnDefinition="text")
solves my problem.
That solution could help for a different reason. One other reason could be Column length. Check your column length. I had the same error the reason was my data exceed the size of the column.
setSignInProvider("String length > 15 ")
Before
#Column(name = "sing_in_provider", length = 15)
and then
#Column(name = "sing_in_provider", length = 100)
I was also facing the same issue . and then I solved the problem
spring.jpa.hibernate.ddl-auto=update
For me I'm using current_date for a field in my sql table. but this is a keyword in SQL so I can't use this name. I changed the field name to current_date_and_time it works for me. also I added the column name on my entity class.
#Column(name = "current_date_and_time")