We Keep Coding

reactive repository throws exception when saving a new object

I am using r2dbc, r2dbc-h2 and experimental spring-boot-starter-data-r2dbc
implementation 'org.springframework.boot.experimental:spring-boot-starter-data-r2dbc:0.1.0.M1'
implementation 'org.springframework.data:spring-data-r2dbc:1.0.0.RELEASE' // starter-data provides old version
implementation 'io.r2dbc:r2dbc-h2:0.8.0.RELEASE'
implementation 'io.r2dbc:r2dbc-pool:0.8.0.RELEASE'
I have created reactive repositories
public interface IJsonComparisonRepository extends ReactiveCrudRepository<JsonComparisonResult, String> {}
Also added a custom script that creates a table in H2 on startup
#SpringBootApplication
public class JsonComparisonApplication {
public static void main(String[] args) {
SpringApplication.run(JsonComparisonApplication.class, args);
}
#Bean
public CommandLineRunner startup(DatabaseClient client) {
return (args) -> client
.execute(() -> {
var resource = new ClassPathResource("ddl/script.sql");
try (var is = new InputStreamReader(resource.getInputStream())) {
return FileCopyUtils.copyToString(is);
} catch (IOException e) {
throw new RuntimeException(e);
} })
.then()
.block();
}
}
My r2dbc configuration looks like this
#Configuration
#EnableR2dbcRepositories
public class R2dbcConfiguration extends AbstractR2dbcConfiguration {
#Override
public ConnectionFactory connectionFactory() {
return new H2ConnectionFactory(
H2ConnectionConfiguration.builder()
.url("mem:testdb;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=FALSE")
.username("sa")
.build());
}
}
My service where I perform the logic looks like this
#Override
public Mono<JsonComparisonResult> updateOrCreateRightSide(String comparisonId, String json) {
return updateComparisonSide(comparisonId, storedComparisonResult -> {
storedComparisonResult.setRightSide(json);
return storedComparisonResult;
});
}
private Mono<JsonComparisonResult> updateComparisonSide(String comparisonId,
Function<JsonComparisonResult, JsonComparisonResult> updateSide) {
return repository.findById(comparisonId)
.defaultIfEmpty(createResult(comparisonId))
.filter(result -> ComparisonDecision.NONE == result.getDecision()) // if not NONE - it means it was found and completed
.switchIfEmpty(Mono.error(new NotUpdatableCompleteComparisonException(comparisonId)))
.map(updateSide)
.flatMap(repository::save);
}
private JsonComparisonResult createResult(String comparisonId) {
LOGGER.info("Creating new comparison result: {}.", comparisonId);
var newResult = new JsonComparisonResult();
newResult.setDecision(ComparisonDecision.NONE);
newResult.setComparisonId(comparisonId);
return newResult;
}
The domain looks like this
#Table("json_comparison")
public class JsonComparisonResult {
#Column("comparison_id")
#Id
private String comparisonId;
#Column("left")
private String leftSide;
#Column("right")
private String rightSide;
// #Enumerated(EnumType.STRING) - no support for now
#Column("decision")
private ComparisonDecision decision;
private String differences;
The problem is that when I try to add any object to the database it fails with the exception
org.springframework.dao.TransientDataAccessResourceException: Failed to update table [json_comparison]. Row with Id [4] does not exist.
at org.springframework.data.r2dbc.repository.support.SimpleR2dbcRepository.lambda$save$0(SimpleR2dbcRepository.java:91) ~[spring-data-r2dbc-1.0.0.RELEASE.jar:1.0.0.RELEASE]
at reactor.core.publisher.FluxHandle$HandleSubscriber.onNext(FluxHandle.java:96) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onNext(FluxOnErrorResume.java:73) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.MonoUsingWhen$MonoUsingWhenSubscriber.deferredComplete(MonoUsingWhen.java:276) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.FluxUsingWhen$CommitInner.onComplete(FluxUsingWhen.java:536) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.Operators$MultiSubscriptionSubscriber.onComplete(Operators.java:1858) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.Operators.complete(Operators.java:132) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.MonoEmpty.subscribe(MonoEmpty.java:45) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
at reactor.core.publisher.MonoDefer.subscribe(MonoDefer.java:52) ~[reactor-core-3.3.1.RELEASE.jar:3.3.1.RELEASE]
For some reason during save in SimpleR2dbcRepository library class it doesn't consider the objectToSave as new, but then it fails to update as it is in reality doesn't exist.
// SimpleR2dbcRepository#save
#Override
#Transactional
public <S extends T> Mono<S> save(S objectToSave) {
Assert.notNull(objectToSave, "Object to save must not be null!");
if (this.entity.isNew(objectToSave)) { // not new
....
}
}
Why it is happening and what is the problem?

TL;DR: How should Spring Data know if your object is new or whether it should exist?
Relational Spring Data Repositories (both, JDBC and R2DBC) must differentiate on [Reactive]CrudRepository.save(…) whether the given object is new or whether it exists in your database. Performing a save(…) operation results either in an INSERT or UPDATE statement. Issuing the wrong statement either causes a primary key violation or a no-op as standard SQL does not have a way to express an upsert.
Spring Data JDBC|R2DBC use by default the presence/absence of the #Id value. Generated primary keys are a widely used mechanism. If the primary key is provided, the entity is considered existing. If the id value is null, the entity is considered new.
Read more in the reference documentation about Entity State Detection Strategies.

You have to implement Persistable because you’ve provided the #Id. The library needs to figure out, whether the row is new or whether it should exist. If your entity implements Persistable, then save(…) will use the outcome of isNew() to determine whether to issue an INSERT or UPDATE.
For example:
public class Product implements Persistable<Integer> {
#Id
private Integer id;
private String description;
private Double price;
#Transient
private boolean newProduct;
#Override
#Transient
public boolean isNew() {
return this.newProduct || id == null;
}
public Product setAsNew() {
this.newProduct = true;
return this;
}
}

May be you should consider this:
Choose data type of your id/Primary Key as INT/LONG and set it to AUTO_INCREMENT (something like below):
CREATE TABLE PRODUCT(id INT PRIMARY KEY AUTO_INCREMENT NOT NULL, modelname VARCHAR(30) , year VARCHAR(4), owner VARCHAR(50));
In your post request body, do not include id field.

Removing #ID issued insert statement

Spring Data MongoDB - ignore empty objects

I'm using Spring Data with a MongoDB to save some documents. When saving documents, I would like that Mongo does not contain empty objects. (How) can this be achieved?
Say I have the following main class:
#Document(collection = "main_doc")
public class MainDoc {
#Id
private String id;
private String title;
private SubDoc subDoc;
}
that contains an object of the following class:
public class SubDoc {
private String title;
private String info;
}
Now if I would try to save the following object:
MainDoc main = new MainDoc();
main.setTitle("someTitle");
main.setSubDoc(new SubDoc());
Note: in reality I do not control the fact that the SubDoc is set like this. It can either be empty or filled in. What I want is that if an element's properties/fields are all NULL, it will not be stored in mongo at all.
This results in something like this in mongo:
{
"_id" : "5a328f9a-6118-403b-a3a0-a55ce52099f3",
"title": "someTitle",
"subDoc": {}
}
What I would like is that if an element contains only null properties, they aren't saved at all, so for the above example I would want the following result:
{
"_id" : "5a328f9a-6118-403b-a3a0-a55ce52099f3",
"title": "someTitle"
}
Saving of documents is done with the help of a repository as following:
#NoRepositoryBean
public interface MainRepo extends CrudRepository<MainDoc, String> {
// save inherited
}
Thanks in advance.

One thing you can do here is to write your custom converter for MainDoc:
public class MainDocConverter implements Converter<MainDoc, DBObject> {
#Override
public DBObject convert(final MainDoc source) {
final BasicDbObject dbObject = new BasicDBObject();
...
if(/* validate is subdoc is not null and not empty */) {
dbOject.put("subDoc", source.getSubDoc());
}
}
}
You can register it in #Configuration file for example:
#Configuration
#EnableMongoRepositories(basePackages = {"package"})
public class MongoConfig {
private final MongoDbFactory mongoDbFactory;
public MongoConfig(final MongoDbFactory mongoDbFactory) {
this.mongoDbFactory = mongoDbFactory;
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
final MongoTemplate mongoTemplate = new MongoTemplate(mongoDbFactory, getDefaultMongoConverter());
return mongoTemplate;
}
#Bean
public MappingMongoConverter getDefaultMongoConverter() throws Exception {
final MappingMongoConverter converter = new MappingMongoConverter(
new DefaultDbRefResolver(mongoDbFactory), new MongoMappingContext());
converter.setCustomConversions(new CustomConversions(Arrays.asList(new MainDocConverter())));
return converter;
}
}
If you don't want to write a custom converter for your object toy can use default one and and modify it a bit.
final Document document = (Document) getDefaultMongoConverter().convertToMongoType(mainDoc);
if(/* validate is null or is empty */) {
document .remove("subDoc");
}
mongoTemplate().save(document);
Actually it's not the best way. As guys wrote empty object should be stored as {}, but converter can help you with your case.

Java Mongodb connection failed

I have a java application which uses around 3000 reusable threads which is always running and processing items from a queue. I use MongoDB for my data storage and every time I run it it works perfectly for around 40 minutes, after that Mongo DB Object start returning Nullpointer Exception for queries. At first I suspected that it might be due to connections being lost but as you can see in the Google monitoring graph the connections are still open, but there is a significant decrease in number of Mongo queries. Is there anything Im missing here?
My MongoDB class is like this:
public class MongoDB {
private static MongoClient mongoClient;
private static MongoClient initMongoClient() {
ServerAddress server = new ServerAddress("X.X.X.X");
MongoClientOptions.Builder builder = new MongoClientOptions.Builder();
builder.threadsAllowedToBlockForConnectionMultiplier(50000);
builder.socketKeepAlive(true);
builder.connectionsPerHost(10000);
builder.minConnectionsPerHost(2500);
MongoClientOptions options = builder.build();
MongoClient mc = new MongoClient(server, options);
mongoClient = mc;
return mc;
}
public static MongoClient getMongoClient() {
if(mongoClient == null) {
mongoClient = initMongoClient();
}
return mongoClient;
}
public static DB getDb() {
DB db;
MongoClient mc;
try {
mc = getMongoClient();
mc.getDatabaseNames();
} catch(MongoException e) {
mc = initMongoClient();
}
db = mc.getDB("tina");
return db;
}
}

You should switch to Morphia!
https://github.com/mongodb/morphia
Use the factory pattern to produce a single Mongo instance and tie it to a Morphia Datastore object. You can then use the Datastore object to interface with your MongoDB.
public class DatastoreFactory {
private static Datastore ds;
public static Datastore getDatastore() {
//Lazy load the datastore
if(ds == null) {
try {
Morphia morphia = new Morphia();
ds = morphia.createDatastore(
new MongoClient("server", port, "database"));
//... Other datastore options
} catch(Exception e) {
// Handle it
}
}
return ds;
}
Then wherever you need your MongoDB instance you simple use the Datastore object and get it from the factory
Datastore ds = DatastoreFactory.getDatastore();
You could also use CDI to inject the datastore if that's what you're into
#Singleton
public class DatastoreFactory {
private Datastore ds;
#Produces
public Datastore getDatastore() {
//Lazy load the datastore
if(ds == null) {
try {
Morphia morphia = new Morphia();
ds = morphia.createDatastore(
new MongoClient("server", port, "database"));
//... Other datastore options
} catch(Exception e) {
// Handle it
}
}
return ds;
}
Then inject it like so
#Inject
Datastore ds;
BONUS
To further decouple your code from MongoDB, it would be proper design to create Data Access Objects (DAO) to access your database which will contain the Morphia Datastore object. Your DAO will have the methods you wish to use on the database (get, create, save, delete). This way if you decide to move away from MongoDB you will only have to change the DAO object and not all of your code!

Update and delete not working with Google Cloud Endpoints

I have a class, Student and the generated Endpoint class for it. ListStudents and insertStudents methods work without any problems, but update and remove don't cause any change in the datastore. The methods don't throw any errors and the call returns, but no changes are made.
My endpoints code is mostly the code generated by google plugin for eclipse:
#ApiMethod(name = "removeStudent", path="remove_student")
public void removeStudent(#Named("email") String email) {
EntityManager mgr = getEntityManager();
try {
Student student = getStudentByEmailName(email);
mgr.remove(student);
} finally {
mgr.close();
}
}
Entitiy manager getter method:
private static EntityManager getEntityManager() {
return EMF.get().createEntityManager();
}
#ApiMethod(name = "updateStudent")
public Student updateStudent(Student student) {
EntityManager mgr = getEntityManager();
try {
if (!containsStudent(student)) {
throw new EntityNotFoundException("Object does not exist");
}
mgr.persist(student);
} finally {
mgr.close();
}
return student;
}
And my EMF class:
public final class EMF {
private static final EntityManagerFactory emfInstance = Persistence
.createEntityManagerFactory("transactions-optional");
private EMF() {
}
public static EntityManagerFactory get() {
return emfInstance;
}
}
The client that uses this endpoint is Android. I have only tried testing on my local server.
Please tell me if I'm doing something wrong. Thank you

Do you have your student entities indexed by email?
This is a typical issue when you move to nosql and expect all queries to work without indexes.
Note that records inserted before defining index would not be in index.

The datastore is eventually consistent and your code should work. What is the return value that you get in the Student object from your updateStudent method.
As much as I don't want to, after you do a mgr.persist(...) , add mgr.flush() and see if that makes a difference.

How to run arbitrary sql with mybatis?

I've an application that use mybatis for object persistence. But there are chances I need to run arbitrary sql(from user). Can I do it with mybatis?
Update:
I choose to use dbutils (JDBC) to run user-defined sql, but I need a instance of DataSource to create QueryRunner. Is there any way I can get datasource from mybatis?

I use this utilitary class:
import java.util.List;
import org.apache.ibatis.annotations.SelectProvider;
public interface SqlMapper {
static class PureSqlProvider {
public String sql(String sql) {
return sql;
}
public String count(String from) {
return "SELECT count(*) FROM " + from;
}
}
#SelectProvider(type = PureSqlProvider.class, method = "sql")
public List<?> select(String sql);
#SelectProvider(type = PureSqlProvider.class, method = "count")
public Integer count(String from);
#SelectProvider(type = PureSqlProvider.class, method = "sql")
public Integer execute(String query);
}

Your question is similar to How to exequte query directly from java code using mybatis?
I have already given the answer to that question. But I hope this solution will help you.
Mybatis has already this function, but you must use the adapter as follows.
create an adapter class;
public class SQLAdapter {
String sql;
public SQLAdapter(String sql) {
this.sql = sql;
}
public String getSql() {
return sql;
}
public void setSql(String sql) {
this.sql = sql;
} }
create typeAlias of class SQLAdapter
<typeAlias alias="sqladapter" type="com.zj.xxx.xxx.SQLAdapter" />
put select tag in each object xml where you need to execute the sql directly.
<select id="findRecords" parameterType="SQLAdapter" resultMap="xxxxxResultMap">
${sql}
</select>
call this select method like
String _sql = "select * from table where... order by... limit...";
xxxxx.findRecords(new SQLAdapter(_sql));
Things have been all done. you can no longer write complex sql language in the xml file. Good Luck.

Based on the answers provided, they both are good. But both of them required an Adapter class to be used.
Using Mybatis version 3, I succeeded using a HashMap<String, String> to keep and pass the SQL.
See the codes below.
in Mapper class
final String sql = "${sql}";
#Select(sql)
void execute(HashMap<String, String> m);
when invoke the method:
String sql = "SELECT * FROM record limit 1";
HashMap<String, String> map = new HashMap<String, String>();
map.put("sql", sql);
mapper.execute(map);
HashMap provides a way that you don't have to define the Class properties, or fields in code, you can use a Map to define it redomly.
Thanks.

Reusable fragment of SQL can be used to create select part of query dynamically. In you mapper pass query as normal parameter:
#Param("sql")String sql
In your query just access the parameter using ${sql} instead of #{sql}.
Value in parameter sql can be a fully valid sql query or a fragment of sql query.

For testing I use
import org.apache.ibatis.jdbc.ScriptRunner;
import java.io.Reader;
import java.io.StringReader;
public class test {
private static final String conf = "mybatis.conf.xml";
private SqlSessionFactoryBuilder builder;
private SqlSessionFactory sessionFactory;
Reader reader;
private SqlSession session;
private ScriptRunner runner;
#Before
public void before() {
builder = new SqlSessionFactoryBuilder();
try {
reader = Resources.getResourceAsReader(conf);
} catch (IOException e) {
e.printStackTrace();
}
sessionFactory = builder.build(reader);
session = sessionFactory.openSession();
runner = new ScriptRunner(session.getConnection());
runner.setAutoCommit(true);
runner.setStopOnError(true);
}
#Test
public void testSelectChapelStatus() {
Reader populate = new StringReader("insert into person values (7553,0,'201002496','Wish','Jill','Rain',1,0,NULL,'xxx#LCU.EDU');\r\n"
+ "");
runner.runScript(populate);
}