I am following spring batch admin.
I want to use database for saving meta data values.
My meta data tables are created, but data does not go to those tables. It still uses in memory for storing metadata.
I know spring uses MapJobRepositoryFactoryBean as the implementation class for jobRepository bean to store data in memory, and we have to change it to
JobRepositoryFactoryBean if we want to store meta data in database.
However, even after changing it, I see no effect. (I have cleaned and recompiled, no issues there)
I have wasted some time searching it, but with no success. Can any one help?
My batch-oracle.properties file is-
batch.jdbc.driver=oracle.jdbc.driver.OracleDriver
batch.jdbc.url=jdbc:oracle:thin:#192.168.2.45:1521:devdb
batch.jdbc.user=hsdndad
batch.jdbc.password=isjdsn
batch.jdbc.testWhileIdle=false
batch.jdbc.validationQuery=
batch.drop.script=classpath:/org/springframework/batch/core/schema-drop-oracle10g.sql
batch.schema.script=classpath:/org/springframework/batch/core/schema-oracle10g.sql
batch.business.schema.script=classpath:oracle/initial-query.sql
batch.database.incrementer.class=org.springframework.jdbc.support.incrementer.OracleSequenceMaxValueIncrementer
batch.database.incrementer.parent=sequenceIncrementerParent
batch.lob.handler.class=org.springframework.jdbc.support.lob.OracleLobHandler
batch.grid.size=2
batch.jdbc.pool.size=6
batch.verify.cursor.position=true
batch.isolationlevel=ISOLATION_SERIALIZABLE
batch.table.prefix=BATCH_
After some digging I came to know about the particular convention of properties file(earlier I was giving it name batch-default.properties) . . so now I think it is trying to insert but throwing some SERIALIZABLE exceptions. – Nirbhay Mishra
Try changing the isolationLevelForCreate for the JobRepository to ISOLATION_READ_COMMITTED
Related
I'm migrating from MySQL to mongodb and wondering about couple of things:
1> when using JpaRepository interface and MySQL i need to provide configuration(in my case in the form of settings in application.properties) for it to work otherwise the application fails with a message saying configuration of DataSource failed... but with MongoRepository the application works without any sort of configuration and the data is saved to the default test db. I'd like to know how and where spring makes the initial configuration of the Driver , db name, user and password to use.
2> All objects saved through MongoRepository contain an additional _class field with FQN of the object,
in section 4.6.2 Type mapping the docs say that it needs to do that because
the values held inside that property have to be read in correctly when retrieving the object.
which i didnt fully understand and would love an elaboration on. i created a test with User collection and a few documents in it, some with _class and some without(by inserting the objects using the mongoshell) and the test application had no problem retrieving the User objects that didnt have the _class field and printing them to the console. I also changed the _class field for the collections that had it to "" and it still printed them correctly so how does Spring uses it?
I am very new to Springboot and Spring Data JPA and working on a use case where I am required to create users in different databases.
The application will receive 2 inputs from a queue - username and database name.
Using this I have to provision the given user in the given database.
I am unable to understand the project architecture.
Since the query I need to run will be of the format - create user ABC identified by password;
How should the project look like in terms of model class, repositories etc? Since I do not have an actual table against which the query will be run, do I need a model class since there will be no column mappings happening as such.
TLDR - Help in architecturing Springboot-Spring Data JPA application configured with multiple data sources to run queries of the format : create user identified by password
I have been using this GitHub repo for reference - https://github.com/jahe/spring-boot-multiple-datasources/blob/master/src/main/java/com/foobar
I'll be making some assumptions here:
your database of choice is Oracle, based on provided syntax: create user ABC identified by password
you want to create and list users
your databases are well-known and defined in JNDI
I can't just provide code unfortunately as setting it up would take me some work, but I can give you the gist of it.
Method 1: using JPA
first, create a User entity and a corresponding UserRepository. Bind the entity to the all_users table. The primary key will probably be either the USERNAME or the USER_ID column... but it doesn't really matter as you won't be doing any insert into that table.
to create and a user, add a dedicated method to your own UserRepository specifying the user creation query within a #NativeQuery annotation. It should work out-of-the-box.
to list users you shouldn't need to do anything, as your entity at this point is already bound to the correct table. Just call the appropriate (and already existing) method in your repository.
The above in theory covers the creation and listing of users in a given database using JPA.
If you have a limited number of databases (and therefore a limited number of well-known JNDI datasources) at this point you can proceed as shown in the GitHub example you referenced, by providing different #Configuration classes for each different DataSource, each with the related (identical) repository living in a separate package.
You will of course have to add some logic that will allow you to appropriately select the JpaRepository to use for the operations.
This will lead to some code duplication and works well only if the needs remain very simple over time. That is: it works if all your "microservice" will ever have to do is this create/list (and maybe delete) of users and the number of datasources remains small over time, as each new datasource will require you to add new classes, recompile and redeploy the microservice.
Alternatively, try with the approach proposed here:
https://www.endpoint.com/blog/2016/11/16/connect-multiple-jpa-repositories-using
Personally however I would throw JPA out of the window completely as it's anything but easy to dynamically configure arbitrary DataSource objects and reconfigure the repositories to work each time against a different DataSource and the above solution will force you to constant maintenance over such a simple application.
What I would do would be sticking with NamedParameterJdbcTemplate initialising it by using JndiTemplate. Example:
void createUser(String username, String password, String database) {
DataSource ds = (new JndiTemplate()).lookup(database);
NamedParameterJdbcTemplate npjt = new NamedParameterJdbcTemplate();
Map<String, Object> params = new HashMap<>();
params.put("USERNAME", username);
params.put("PASSWORD", password);
npjt.execute('create user :USERNAME identified by :PASSWORD', params);
}
List<Map<String, Object>> listUsers() {
DataSource ds = (new JndiTemplate()).lookup(database);
NamedParameterJdbcTemplate npjt = new NamedParameterJdbcTemplate();
return npjt.queryForList("select * from all_users", new HashMap<>());
}
Provided that your container has the JNDI datasources already defined, the above code should cover both the creation of a user and the listing of users. No need to define entities or repositories or anything else. You don't even have to define your datasources in a spring #Configuration. The above code (which you will have to test) is really all you need so you could wire it in a #Controller and be done with it.
If you don't use JNDI it's no problem either: you can use HikariCP to define your datasources, providing the additional arguments as parameters.
This solution will work no matter how many different datasources you have and won't need redeployment unless you really have to work on its features. Plus, it doesn't need the developer to know JPA and it doesn't need to spread the configuration all over the place.
Thanks for reading this in advance.
I use Spring data JPA to query data from DB:
UserProfile userProfile = userProfileRepository.findByCustomerID(id);
After the data returned I want to change some value to some field:
userProfile.setRemark("some remark");
However, I notice that Spring will automatically update the database of this record which I don't want, as this field is just a temp value for some further processing.
May I know how can I let Spring data JPA to kinda "ignore" or "drop" this entity from the managed pool? I found someone said to use:
entityManager.clear();
And tested it definitely will not update the database if this one is called. But I am not sure if this is the correct way to do so, does entityManager.clear(); will clear other managed entities as well?
Many thanks.
I am developing a web application using JSF2, JPA2, EJB3 via JBoss7.1.
I have an Entity(Forum) which contains a list of child entities(Topic).
When I tried to get the list of Topics by forumId for the first time the data is being loaded from DB.
List<Topic> topics = entityManager.find(Forum.class, 1).getTopics();
After that I am adding few more child entities(Topics) to Forum and then again I am trying to retrieve list of Topics by forumId. Nut I am getting the old cached results only. The newly inserted child records are not being loaded from DB.
I am able to load the child entities(Topics) by using following methods:
Method1: Calling entityManager.clear() before entityManager.find()
Method2: Using
em.createQuery("select t from Topic t where t.forum.forumId=?1", Topic.class);
or
em.createQuery("SELECT t FROM Topic t JOIN t.forum f WHERE f.forumId = ?1", Topic.class);
I am aware of setting the QueryHints on NamedQueries. But em.find() method is in a super CrudService which is being extended by all DAOs(Stateless EJBs). So setting QueryHints won't work for me.
So I want to know how can i make em.find() method to load data from DB instead of Cache?
PS: I am using Extended Persistence Context type.
#PersistenceContext(unitName="forum", type=PersistenceContextType.EXTENDED)
protected EntityManager em;
You can specify the behavior of individual find operations by setting additional properties that control the entity managers interaction with the second level cache.
Map<String, Object> props = new HashMap<String, Object>();
props.put("javax.persistence.cache.retrieveMode", CacheRetrieveMode.BYPASS);
entityMgr.find(Forum.class, 1, props).getTopics();
Is it possible that the relation between Forum and Topic was only added in one direction in your entity beans? If you set the forum id on the topic, you should also add this topic to the Forum object to have consistent data inside the first level cache. You should also make sure that you are not using two different entity managers for the update and find. The first level cache is only kept per entity manager, another em can still contain an older version of the entitiy.
Probably unrelated, but with JPA2 you also have a minimal api to evict entities from the second level cache, which could be used after an update:
em.getEntityManagerFactory().getCache().evict(Forum.class, forumId);
Put #Cacheable(false) within the Forum.class.
I have a Spring/Hibernate webapp that has some integration tests that run on an in-memory HSQL database. Hibernate takes this blank database and creates all of my test tables and constraints thanks to hbm2ddl=create. However, I have a new bean that checks for a particular config value from the database during its afterPropertiesSet() method, and so when this bean is initialized, such a row needs to exist in the database.
Is there any good way to set up a Java/Spring/Hibernate equivalent of Rail's test fixtures? I'm trying to find a way to tell Hibernate "whenever you create this table, insert these rows immediately afterwards". I couldn't find a callback or a hook I could add, but maybe there's another way.
I'm trying to find a way to tell Hibernate "whenever you create this table, insert these rows immediately afterwards"
Since Hibernate 3.1, you can include a file called import.sql in the runtime classpath of Hibernate and at the time of schema export, Hibernate will execute the SQL statements contained in that file after the schema has been exported.
This feature has been announced in the Rotterdam JBug and Hibernate's import.sql blog post:
import.sql: easily import data in your unit tests
Hibernate has a neat little feature
that is heavily under-documented and
unknown. You can execute an SQL script
during the SessionFactory creation
right after the database schema
generation to import data in a fresh
database. You just need to add a file
named import.sql in your classpath
root and set either create or
create-drop as your
hibernate.hbm2ddl.auto property.
I use it for Hibernate Search in
Action now that I have started the
query chapter. It initializes my
database with a fresh set of data for
my unit tests. JBoss Seam also uses it
a lot in the various examples.
import.sql is a very simple feature
but is quite useful at time. Remember
that the SQL might be dependent on
your database (ah portability!).
#import.sql file
delete from PRODUCTS
insert into PRODUCTS (PROD_ID, ASIN, TITLE, PRICE, IMAGE_URL, DESCRIPTION) values ('1', '630522577X', 'My Fair Lady', 19.98, '630522577X.jpg', 'My Fair blah blah...');
insert into PRODUCTS (PROD_ID, ASIN, TITLE, PRICE, IMAGE_URL, DESCRIPTION) values ('2', 'B00003CXCD', 'Roman Holiday ', 12.98, 'B00003CXCD.jpg', 'We could argue that blah blah');
For more information about this
feature, check Eyal's blog, he
wrote a nice little entry about it.
Remember if you want to add additional
database objects (indexes, tables and
so on), you can also use the auxiliary
database objects feature.
It is still not really documented.
In hibernate 3.6 the configuration that allows to run arbitrary sql commands is:
hibernate.hbm2ddl.import_files
See in http://docs.jboss.org/hibernate/core/3.6/reference/en-US/html_single/, noticing there is an error in the documentation: the property is import_files, with an s in the end.
If you're talking about JUnit tests and using AbstractTransactionalDataSourceSpringContextTests there's methods you can override like onSetupBeforeTransaction that provide a hook to pre-populate test table data etc.