Im using loacalstack with dynamodb setup. Iv a table with two columns, an Id and a name column. Im struggling to query the table by 'name' using DynamoDBMapper. Below is a snippet of my setup
entity >
#AllArgsConstructor
#NoArgsConstructor
#DynamoDBTable(tableName = "my-table")
public class Table {
private String id;
private String name;
#DynamoDBHashKey(attributeName = "id")
#DynamoDBAutoGeneratedKey
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
#DynamoDBAttribute
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
#query >
public Table getByName(String name) {
return dynamoDBMapper.load(Table.class, name);
}
aws dynamodb create-table --endpoint-url=http://localstack:4569 --table-name my-table \
--attribute-definitions AttributeName=id,AttributeType=S AttributeName=name,AttributeType=S \
--key-schema AttributeName=id,KeyType=HASH AttributeName=name,KeyType=RANGE \
Any help appreciated
Obviously you cannot query the table by name as it is not the partition/hash key. Either you can use scan or you should state name as a GSI field and then query it. Scan command would be like:
dynamoDBMapper.scan(Table.class, new DynamoDBScanExpression());
The above command will scan the entire table. Read about DynamoDBScanExpression here to see how you can filter it with a specific name.
You can add a filter like on name like:
public Table getByName(String name) {
DynamoDBScanExpression scanExpression = new DynamoDBScanExpression();
scanExpression.addFilterCondition("name", new Condition()
.withComparisonOperator(ComparisonOperator.EQ)
.withAttributeValueList(new AttributeValue().withS(name)));
dynamoDBMapper.scan(Table.class, scanExpression);
}
Related
#Transactionalshould itself reflect the changes made to the entity in the database.
I'm creating an application where the client can create a Car entity that looks like this (the update method is later used by PUT, do not pay attention to the brand property):
#Entity
#Table(name = "cars")
public class Car {
#Id
#GeneratedValue(generator = "inc")
#GenericGenerator(name = "inc", strategy = "increment")
private int id;
#NotBlank(message = "car name`s must be not empty")
private String name;
private LocalDateTime productionYear;
private boolean tested;
public Car() {
}
public Car(#NotBlank(message = "car name`s must be not empty") String name, LocalDateTime productionYear) {
this.name = name;
this.productionYear = productionYear;
}
#ManyToOne
#JoinColumn(name = "brand_id")
private Brand brand;
public int getId() {
return id;
}
void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public LocalDateTime getProductionYear() {
return productionYear;
}
public void setProductionYear(LocalDateTime productionYear) {
this.productionYear = productionYear;
}
public boolean isTested() {
return tested;
}
public void setTested(boolean tested) {
this.tested = tested;
}
public Brand getBrand() {
return brand;
}
void setBrand(Brand brand) {
this.brand = brand;
}
public Car update(final Car source) {
this.productionYear = source.productionYear;
this.brand = source.brand;
this.tested = source.tested;
this.name = source.name;
return this;
}
}
In my application, the client can create a new Car or update an existing one with the PUT method.
My controller:
#RestController
public class CarController {
private Logger logger = LoggerFactory.getLogger(CarController.class);
private CarRepository repository;
public CarController(CarRepository repository) {
this.repository = repository;
}
//The client can create a new resource or update an existing one via PUT
#Transactional
#PutMapping("/cars/{id}")
ResponseEntity<?> updateCar(#PathVariable int id, #Valid #RequestBody Car source) {
//update
if(repository.existsById(id)) {
repository.findById(id).ifPresent(car -> {
car.update(source); //it doesn`t work
//Snippet below works
//var updated = car.update(source);
//repository.save(updated);
});
return ResponseEntity.noContent().build();
}
//create
else {
var result = repository.save(source);
return ResponseEntity.created(URI.create("/" + id)).body(result);
}
}
}
When I create a new Car, it works. However as described in the code, when there is no save method the entity is not changed although I get the status 204 (no content). When there is a save method, it works fine.
Do you know why this is so?
One of the users asked me for a Brand entity. I haven't created any Brand object so far but essentially Car can belong to a specific Brand in my app. So far, no Car belongs to any Brand. Here is this entity:
#Entity
#Table(name = "brands")
public class Brand {
#Id
#GeneratedValue(generator = "i")
#GenericGenerator(name = "i", strategy = "increment")
private int id;
#NotBlank(message = "brand name`s must be not empty")
private String name;
private LocalDateTime productionBrandYear;
#OneToMany(fetch = FetchType.LAZY, cascade = CascadeType.ALL, mappedBy = "brand")
private Set<Car> cars;
#ManyToOne
#JoinColumn(name = "factory_id")
private Factory factory;
public Brand() {
}
public int getId() {
return id;
}
void setId(int id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public LocalDateTime getProductionBrandYear() {
return productionBrandYear;
}
public void setProductionBrandYear(LocalDateTime productionBrandYear) {
this.productionBrandYear = productionBrandYear;
}
public Set<Car> getCars() {
return cars;
}
public void setCars(Set<Car> cars) {
this.cars = cars;
}
public Factory getFactory() {
return factory;
}
public void setFactory(Factory factory) {
this.factory = factory;
}
}
I tried your entities with same use case locally and found out everything is working fine, I am writing here my findings and configurations so that you can verify what's going on wrong for you.
So, when I issue a PUT call providing id but Car entity doesn't exist into table, it gets created and I receive 201 response (I guess you are getting the same)
you can see that row with value got inserted into table as well
and these are the query logs printed
- [nio-8080-exec-8] org.hibernate.SQL: select count(*) as col_0_0_ from car car0_ where car0_.id=?
[nio-8080-exec-8] org.hibernate.SQL: select car0_.id as id1_1_0_, car0_.brand_id as brand_id5_1_0_, car0_.name as name2_1_0_, car0_.production_year as producti3_1_0_, car0_.tested as tested4_1_0_ from car car0_ where car0_.id=?
[nio-8080-exec-8] org.hibernate.SQL: insert into car (brand_id, name, production_year, tested) values (?, ?, ?, ?)
Now, let's come to updating the same entity, when issued PUT request for same id with changed values notice that values changes in table and update queries in log
You can see that got same 204 response with empty body, let's look the table entry
So changes got reflected in DB, let's look at the SQL logs for this operation
select count(*) as col_0_0_ from car car0_ where car0_.id=?
[nio-8080-exec-1] org.hibernate.SQL: select car0_.id as id1_1_0_, car0_.brand_id as brand_id5_1_0_, car0_.name as name2_1_0_, car0_.production_year as producti3_1_0_, car0_.tested as tested4_1_0_, brand1_.id as id1_0_1_, brand1_.name as name2_0_1_, brand1_.production_year as producti3_0_1_ from car car0_ left outer join brand brand1_ on car0_.brand_id=brand1_.id where car0_.id=?
[nio-8080-exec-1] org.hibernate.SQL: update car set brand_id=?, name=?, production_year=?, tested=? where id=?
So, I am not sure, how you verified and what you verified but your entities must work, I have used same controller function as yours
#RestController
class CarController {
private final CarRepository repository;
public CarController(CarRepository repository) {
this.repository = repository;
}
#PutMapping("/car/{id}")
#Transactional
public ResponseEntity<?> updateCar(#PathVariable Integer id, #RequestBody Car source) {
if(repository.existsById(id)) {
repository.findById(id).ifPresent(car -> car.update(source));
return ResponseEntity.noContent().build();
}else {
Car created = repository.save(source);
return ResponseEntity.created(URI.create("/" + created.getId())).body(created);
}
}
}
Possible differences from your source code could be as follow:
I used IDENTITY generator to generate the PRIMARY KEY, instead of the one you have on your entity as it was easy for me to test.
I provided ObjectMapper bean to serialize/deserialize the request body to Car object to support Java 8 LocalDateTime conversion, you may have your way to send datetime values, so that it converts to Car Object.
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private int id;
// And Object mapper bean
#Bean
public static ObjectMapper objectMapper() {
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new JavaTimeModule());
mapper.disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS);
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
return mapper;
}
However, these differences should not matter.
application.properties
To print query logs to verify if queries are fired or not
spring.datasource.url=jdbc:mysql://localhost:3306/test
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
spring.datasource.username=test
spring.datasource.password=test
spring.datasource.jpa.show-sql=true
spring.jpa.open-in-view=false
logging.level.org.hibernate.SQL=DEBUG
The fact that you are updating the car object doesn't mean it updates the value in the DB. You always need to call repository.save() method to persist your changes in the DB.
This is maybe a beginner question on hibernate. I am doing my first steps, I designed a simple datamodel consisting of about 10 entities and I use hibernate to persist them to my Oracle XE database. Now I am facing the following problem: First time, when I do a transaction to persist some entities, they are persisted properly. I verify, that the data exists in the database and then I delete all the entries from all database tables. I verify that all tables are empty again. Then I run my program again to persist some new entities - and here happens something really strange: Afterwards I find in my databse the new entries as well as the old ones, which were persisted last time and which I had deleted! They contained the old IDs and the old data fields! How can this be? This happens even if I shut down my computer after the first time the program runs! How does it remember the old entries and where are they saved? Do you have any ideas?
Some information, that might be useful:
I am using annotations (instead of config files) for the mapping.
Following you see the classes used for persisting as well as one example of an entity (I am showing only one entity to avoid making the question too long).
As you see, I am using FetchType = EAGER on my MANY to MANY mappings (as I understand, this makes sure, that all related entities are loaded immediately together with any loaded entity). Can this have any impact?
Thanks for any help!
public class PersistenceManager {
private static final SessionFactory factory = new Configuration().configure().buildSessionFactory();
public static void sampleData() {
try(Session session = factory.openSession()) {
SampleDataLoader.loadSampleData(session);
} catch(HibernateException e) {
System.out.println("Exception during persisting! Message: " + e.getMessage());
e.printStackTrace();
}
}
}
public class SampleDataLoader {
static void loadSampleData(Session session) {
Language french = new Language("French");
Language german = new Language("German");
Noun garcon = new Noun(french, "garcon", false);
Noun junge = new Noun(german, "Junge", false);
junge.addTranslation(garcon);
ZUser user = new ZUser("Daniel", "password");
user.setOwnLanguage(german);
user.setEmail("abc#somemail.de");
user.setDateRegistered(LocalDateTime.now());
user.addForeignLanguage(french);
Transaction transaction = session.beginTransaction();
session.save(user);
session.save(french);
session.save(german);
session.save(junge);
transaction.commit();
}
}
#Entity
public class ZUser {
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
#Column(name = "id")
private int id;
#Column
private String name;
#Column
private String password;
#Column
private String email;
#Column
private String picturePath;
#Column
private LocalDateTime dateRegistered;
#ManyToOne(fetch=FetchType.EAGER)
#JoinColumn(name="OWNLANGUAGE_ID")
private Language ownLanguage;
#ManyToMany(cascade = { CascadeType.ALL })
#JoinTable(name="USER_LANGUAGE",
joinColumns=#JoinColumn(name="USER_ID"),
inverseJoinColumns=#JoinColumn(name="LANGUAGE_ID")
)
private Set<Language> foreignLanguages = new HashSet<>();
public ZUser() { }
public ZUser(String n, String p) {
name = n;
password = p;
}
public int getId() { return id; }
public void setId(int id) { this.id = id; }
public String getName() { return name; }
public void setName(String name) { this.name = name; }
public String getPassword() { return password; }
public void setPassword(String password) { this.password = password; }
public String getEmail() { return email; }
public void setEmail(String email) { this.email = email; }
public String getPicturePath() { return picturePath; }
public void setPicturePath(String picturePath) { this.picturePath = picturePath; }
public LocalDateTime getDateRegistered() { return dateRegistered; }
public void setDateRegistered(LocalDateTime dateRegistered) { this.dateRegistered = dateRegistered; }
public Language getOwnLanguage() { return ownLanguage; }
public void setOwnLanguage(Language ownLanguage) { this.ownLanguage = ownLanguage; }
public void addForeignLanguage(Language language) {foreignLanguages.add(language);}
public Set<Language> getForeignLanguages() {return Collections.unmodifiableSet(foreignLanguages); }
}
Clarified by the comment of Jagger (see comments). Indeed, I was using Oracle SQL command line to delete the entries and I had rgotten, that I need to explicitely commit after deleting. The solution can be so easy :)
I have the following user info object mapped to a table "user_info" in my keyspace "data_collection". I have created the "user_info" table in my cassandra database. I am using spring data cassandra for connecting to cassandra database from JAVA and the spring annotations as below.
#Table(name="user_info",keyspace="data_collection", caseSensitiveKeyspace = false,caseSensitiveTable = false)
public class UserInfo {
#PartitionKey
private UUID id;
#PrimaryKeyColumn
private String email;
private int phone;
private String name;
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
public String getEmail() {
return email;
}
public void setEmail(String email) {
this.email = email;
}
public int getPhone() {
return phone;
}
public void setPhone(int phone) {
this.phone = phone;
}
}
I am using the following code to insert a record into my "user_info" table.
#Autowired
CassandraTemplate cassandraTemplate;
public void saveUserInfo(UserInfo userInfo){
logger.debug("userInfo "+new Gson().toJson(userInfo));
String email = userInfo.getEmail();
Select select = QueryBuilder.select().from("user_info");
select.where(QueryBuilder.eq("email", email));
logger.debug("Query "+select.toString());
UserInfo existingUser = cassandraTemplate.selectOne(select, UserInfo.class);
if(existingUser!=null){
cassandraTemplate.update(userInfo);
}
else{
cassandraTemplate.insert(userInfo);
}
}
My selectOne is working properly whereas during insert I am getting the following exception. I have clearly mapped the UserInfo.java class to the table name "user_info" using annotation above. I don't know why the insert is trying to happen to the table "userinfo".
org.springframework.cassandra.support.exception.CassandraInvalidQueryException: unconfigured columnfamily userinfo; nested exception is com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured columnfamily userinfo
at org.springframework.cassandra.support.CassandraExceptionTranslator.translateExceptionIfPossible(CassandraExceptionTranslator.java:128)
at org.springframework.cassandra.core.CqlTemplate.potentiallyConvertRuntimeException(CqlTemplate.java:946)
at org.springframework.cassandra.core.CqlTemplate.translateExceptionIfPossible(CqlTemplate.java:930)
at org.springframework.cassandra.core.CqlTemplate.translateExceptionIfPossible(CqlTemplate.java:912)
at org.springframework.cassandra.core.CqlTemplate.doExecute(CqlTemplate.java:278)
at org.springframework.cassandra.core.CqlTemplate.doExecute(CqlTemplate.java:559)
at org.springframework.cassandra.core.CqlTemplate.execute(CqlTemplate.java:1333)
at org.springframework.data.cassandra.core.CassandraTemplate.doUpdate(CassandraTemplate.java:895)
at org.springframework.data.cassandra.core.CassandraTemplate.update(CassandraTemplate.java:537)
at org.springframework.data.cassandra.core.CassandraTemplate.update(CassandraTemplate.java:532)
Please find below the description of the table in cassandra.
CREATE TABLE user_info (
name text,
email text,
phone int
PRIMARY KEY ((email))
) WITH
bloom_filter_fp_chance=0.010000 AND
caching='KEYS_ONLY' AND
comment='' AND
dclocal_read_repair_chance=0.100000 AND
gc_grace_seconds=864000 AND
index_interval=128 AND
read_repair_chance=0.000000 AND
replicate_on_write='true' AND
populate_io_cache_on_flush='false' AND
default_time_to_live=0 AND
speculative_retry='99.0PERCENTILE' AND
memtable_flush_period_in_ms=0 AND
compaction={'class': 'SizeTieredCompactionStrategy'} AND
compression={'sstable_compression': 'LZ4Compressor'};
Quick update : I just tried saving another class Test.java. It was mapped to a table "test_info". I got the following error
org.springframework.cassandra.support.exception.CassandraInvalidQueryException: unconfigured columnfamily test; nested exception is com.datastax.driver.core.exceptions.InvalidQueryException: unconfigured columnfamily test
at org.springframework.cassandra.support.CassandraExceptionTranslator.translateExceptionIfPossible(CassandraExceptionTranslator.java:128)
at org.springframework.cassandra.core.CqlTemplate.potentiallyConvertRuntimeException(CqlTemplate.java:946)
at org.springframework.cassandra.core.CqlTemplate.translateExceptionIfPossible(CqlTemplate.java:930)
at org.springframework.cassandra.core.CqlTemplate.translateExceptionIfPossible(CqlTemplate.java:912)
at org.springframework.cassandra.core.CqlTemplate.doExecute(CqlTemplate.java:278)
at org.springframework.cassandra.core.CqlTemplate.doExecute(CqlTemplate.java:559)
at org.springframework.cassandra.core.CqlTemplate.execute(CqlTemplate.java:1323)
at org.springframework.data.cassandra.core.CassandraTemplate.doInsert(CassandraTemplate.java:708)
at org.springframework.data.cassandra.core.CassandraTemplate.insert(CassandraTemplate.java:290)
at org.springframework.data.cassandra.core.CassandraTemplate.insert(CassandraTemplate.java:285)
I am just wondering if my Java class name and the table name in cassandra should always be the same. Because its looking for the columnfamily "test" instead of "test_info" which I have specified in the #Table annotation.
Below is the description of my keyspace
CREATE KEYSPACE data_collection WITH replication = {
'class': 'SimpleStrategy',
'replication_factor': '3'
};
EDIT - SOLVED :
I found the solution based on the conversation with #pinkpanther.
I had imported com.datastax.driver.mapping.annotations.Table instead of org.springframework.data.cassandra.mapping.Table which is why it didn't honor the table name mapping. Thanks for your help.
The problem might be with the package import of Table, Spring Data Cassandra needs org.springframework.data.cassandra.mapping.Table. Replace the imported com.datastax.driver.mapping.annotations.Table with it.
I'm following the Cassandra java object mapper tutorial on Datastax website.
and while defining accessors
#Query("SELECT * FROM testks.user ")
Result<User> getAll();
running this query give me a
com.datastax.driver.core.exceptions.InvalidQueryException: Some partition key parts are missing: id
Looking around it seems that you cannot query in cassandra without providing a partition key. Is that the case? This seems like a strange requirement. If I want a select all query, how would I go about doing that?
the table is defined as
CREATE TABLE testks.user (
id text PRIMARY KEY,
name text
)
You didn't provide many details. if you follow these below steps you shouldn't get any error
You should define the user model like below
User.java
#Table(name = "user")
public class User {
#PartitionKey
private String id;
private String name;
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
#Override
public String toString() {
return "User{" + "id=" + id + ", name=" + name + '}';
}
}
And Repository like below
UserAccessor.java
#Accessor
public interface UserAccessor {
#Query("SELECT * FROM user")
Result<User> getAll();
}
Here is how you can use the repository
Main.Java
public static void main(String[] args) {
try (Cluster cluster = Cluster.builder().addContactPoints("127.0.0.1").withCredentials("cassandra", "cassandra").build(); Session session = cluster.connect("test")) {
MappingManager manager = new MappingManager(session);
UserAccessor userAccessor = manager.createAccessor(UserAccessor.class);
System.out.println(userAccessor.getAll().all());
}
}
Can you provide an example for mapping collections using datastax api annotations to use Map.
class pojo {
#PartitionKey(value = 0)
#Column(name = "user_id")
String userId;
#Column(name = "attributes")
// How to map it
Map<String, String> attributes;
}
error log :
2015-08-03 16:33:34,568 INFO com.jpma.jpmc.slot.persistance.DAOFactory main - Cassandra Cluster Details: ConnectionCfg [userName=test, password=test, port=9042, seeds=[Ljava.lang.String;#1a85bd75, keySpace=test]
java.lang.Class
2015-08-03 16:33:34,646 DEBUG com.datastax.driver.mapping.EntityMapper main - Preparing query INSERT INTO "test"."user_event_date"("user_id","entry_date","entry_time","app","attributes","user_ip","user_authschemes") VALUES (?,?,?,?,?,?,?);
com.datastax.driver.core.exceptions.InvalidQueryException: Unknown identifier attributes
Based on the error message you are seeing, I'm guessing that attributes is not defined in your table definition. Would you mind editing your post with that?
But when I build my CQL table like this (note the compound partition key of itemid and version):
CREATE TABLE products.itemmaster (
itemid text,
version int,
productid uuid,
supplierskumap map<uuid, text>,
PRIMARY KEY ((itemid,version), productid)
);
...insert this row:
INSERT INTO products.itemmaster (itemid,version,productid,supplierskumap)
VALUES ('item1',1,26893749-dcfc-42c7-892c-bee8c9cff630,
{1351f82f-5dc5-4328-82f4-962429c92a2b:'86CCG123'});
...and I build my POJO like this:
#Table(keyspace = "products", name = "itemmaster")
public class Product {
#PartitionKey(0)
private String itemid;
#PartitionKey(1)
private int version;
#ClusteringColumn
private UUID productid;
#Column(name="supplierskumap")
private Map<UUID,String> suppliersku;
public UUID getProductid() {
return productid;
}
public void setProductid(UUID _productid) {
this.productid = _productid;
}
public int getVersion() {
return this.version;
}
public void setVersion(int _version)
{
this.version = _version;
}
public String getItemid() {
return itemid;
}
public void setItemid(String _itemid) {
this.itemid = _itemid;
}
public Map<UUID, String> getSuppliersku() {
return suppliersku;
}
public void setSuppliersku(Map<UUID, String> _suppliersku) {
this.suppliersku = _suppliersku;
}
}
...with this constructor and getProd method on my data access object (dao):
public ProductsDAO()
{
session = connect(CASSANDRA_NODES, USERNAME, PASSWORD);
prodMapper = new MappingManager(session).mapper(Product.class);
}
public Product getProd(String itemid, int version, UUID productid) {
return prodMapper.get(itemid,version,sku);
}
...then this main class successfully queries my table and maps my Map:
private static void main(String[] args) {
ProductsDAO dao = new ProductsDAO();
Product prod = dao.getProd("item1", 1, UUID.fromString("26893749-dcfc-42c7-892c-bee8c9cff630"));
System.out.println(
prod.getProductid() + " - " +
prod.getItemid() + " - " +
prod.getSuppliersku().get(UUID.fromString("1351f82f-5dc5-4328-82f4-962429c92a2b")));
dao.closeCassandra();
}
...and produces this output:
26893749-dcfc-42c7-892c-bee8c9cff630 - item1 - 86CCG123
NOTE: Edit made to the above example to support a compound partition key.