Articles are scraped from the web, so all properties (like Medium, Source etc) initially have an id of NULL and the scraped data in their fields. Before saving an article, the *toEntities()-methods make sure that the data that already exists in the DB, is reused. If the data is new, it's created via cascading in Article.
It is possible that a duplicate article is scraped which throws a "not unique exception" on save, which is good, but after that the other articles should still be saved. I guess I'm struggling with handling this exception correctly. A workaround would be to check if the entity exists but this would result in unnecessary work for 95% of the time.
In the code you see my final attempt, where I tried to save each article in its own transaction but I get these exceptions:
[SqlExceptionHelper.java:129] Duplicate entry '5628BA1A0D4115E091B5EED94FEF5840' for key 'UK_ftcejiicdrly4nl40b7lbvx6e' (the duplicate is determined by the hash value).
org.hibernate.AssertionFailure: null id in
org.observer.media.model.Article entry (don't flush the Session after
an exception occurs)
Here is the code of ArticleService. If you spot other things that are wrong with this class I'd be glad to hear them too.
#Service
public class ArticleService {
#Autowired
private ArticleRepository articleRepository;
#Autowired
private CategoryRepository categoryRepository;
#Autowired
private SourcesRepository sourcesRepository;
#Autowired
private MediumRepository mediumRepository;
private final static Logger logger = LoggerFactory.getLogger("o.o.m.s.ArticleService");
#Transactional
public void save(Iterable<Article> articles) {
for (Article article : articles) {
save(article);
}
}
#Transactional(propagation = Propagation.REQUIRES_NEW)
public void save(Article article) {
dataToEntities(article);
try {
articleRepository.save(article);
} catch (Exception e) {
e.printStackTrace();
}
}
#Transactional
public void delete(Iterable<Article> articles) {
if (articles == null) {
return;
}
for (Article article : articles) {
if (article.getId() != 0L) {
delete(article);
}
}
}
#Transactional
public void delete(Article article) {
article.setMedium(null);
article.setCategories(null);
article.setSources(null);
articleRepository.delete(article);
}
private void dataToEntities(Article article) {
categoriesToEntities(article);
sourcesToEntities(article);
mediumToEntity(article);
}
private void mediumToEntity(Article article) {
if (article.getMedium() == null) {
return;
}
Medium medium = mediumRepository.findByName(article.getMedium().getName());
if (medium != null) {
article.setMedium(medium);
}
}
private void sourcesToEntities(Article article) {
Set<Source> sources = new HashSet<>();
if (article.getSources() == null) {
return;
}
for (Source source : article.getSources()) {
Source entity = sourcesRepository.findByName(source.getName());
if (entity != null) {
sources.add(entity);
} else {
sources.add(source);
}
}
article.setSources(sources);
}
private void categoriesToEntities(Article article) {
Set<Category> categories = new HashSet<>();
if (article.getCategories() == null) {
return;
}
for (Category category : article.getCategories()) {
category = categoryRepository.findByName(category.getName());
categories.add(category);
}
article.setCategories(categories);
}
}
EDIT:
I followed up on the possible duplicate question and tried
the NoRollBackFor-option with all possible exceptions like below on both methods:
#Transactional(propagation = Propagation.REQUIRES_NEW, noRollbackFor = {ConstraintViolationException.class, MySQLIntegrityConstraintViolationException.class, DataIntegrityViolationException.class, AssertionFailure.class, RollbackException.class})
This resulted in "TransactionSystemException: Could not commit JPA transaction; nested exception is javax.persistence.RollbackException: Transaction marked as rollbackOnly".
I discarded the StatelessSession approach because it deviates away from the JPA spec, and because cascading would not work according to the docs. Also, it exposes a variant of Session which feels even more like a hack than just exposing the normal hibernate Session.
Related
While implementing OData V4 using Olingo in Java, I am getting a NullPointerException.
Here is a detailed description --
Trying to implement OData V4 using olingo in Java in my Spring Boot application. I am following the official documentation https://olingo.apache.org/doc/odata4/tutorials/readep/tutorial_readep.html
Rather than feeding the data manually/static data, I am using a database that is providing the data.
As per the documentation, I have created a class Storage.java to simulate the data layer.
public class Storage {
#Autowired
CompanyService cservice;
private List<Entity> companyentityList;
public Storage() throws Exception {
companyentityList = new ArrayList<Entity>();
initSampleData();
}
// PUBLIC FACADE
public EntityCollection readEntitySetData(EdmEntitySet edmEntitySet) throws NullPointerException {
// actually, this is only required if we have more than one Entity Sets
if (edmEntitySet.getName().equals(DemoEdmProvider.ES_COMPANY_RECORDS)) {
return getCompaniesData();
}
return null;
}
public Entity readEntityData(EdmEntitySet edmEntitySet, List<UriParameter> keyParams) throws Exception {
EdmEntityType edmEntityType = edmEntitySet.getEntityType();
// actually, this is only required if we have more than one Entity Type
if (edmEntityType.getName().equals(DemoEdmProvider.ET_COMPANY)) {
return getCompany(edmEntityType, keyParams);
}
return null;
}
// INTERNAL
public EntityCollection getCompaniesData() throws NullPointerException {
EntityCollection retEntitySet = new EntityCollection();
for (Entity companyEntity : this.companyentityList) {
retEntitySet.getEntities().add(companyEntity);
}
return retEntitySet;
}
public Entity getCompany(EdmEntityType edmEntityType, List<UriParameter> keyParams) throws Exception {
// the list of entities at runtime
EntityCollection entitySet = getCompaniesData();
// generic approach to find the requested entity
Entity requestedEntity = Util.findEntity(edmEntityType, entitySet, keyParams);
if (requestedEntity == null) {
// this variable is null if our data doesn't contain an entity for the requested
// key
// Throw suitable exception
throw new ODataApplicationException("Entity for requested key doesn't exist",
HttpStatusCode.NOT_FOUND.getStatusCode(), Locale.ENGLISH);
}
return requestedEntity;
}
// Helper
public void initSampleData() {
try {
getData();
} catch (NullPointerException e) {
System.out.print("<<<<<<<---------------- Database unable to provide data ------------>>>>>>");
}
}
public List<Company> getAllcompanyList() {
Collection<Company> checkingdata = new ArrayList<>();
try {
checkingdata = cservice.getDetails();
} catch (NullPointerException e) {
System.out.print("<<<<<<<---------------- Database unable to provide data ------------>>>>>>");
}
return (List<Company>) checkingdata;
}
// final EntityCollection entitySet = new EntityCollection();
// loop over List<Company> converting each instance of Company into and Olingo Entity
public EntityCollection makeEntityCollection(List<Company> companyList) {
EntityCollection entitySet = new EntityCollection();
for (Company cmp : companyList) {
entitySet.getEntities().add(createEntity(cmp));
}
return entitySet;
}
// Convert instance of cmp object into an Olingo Entity
public Entity createEntity(Company cmp) {
final Entity tmpEntity = new Entity().addProperty(new Property(null, "ID", ValueType.PRIMITIVE, cmp.getId()))
.addProperty(new Property(null, "Name", ValueType.PRIMITIVE, cmp.getContent()));
companyentityList.add(tmpEntity);
return tmpEntity;
}
public void getData() throws NullPointerException {
// ... code to get Data from the DataBase in List and calling makeEntityCollection
List<Company> companyList = getAllcompanyList();
makeEntityCollection(companyList);
// System.out.println(companyList.size());
}
}
In the above mentioned code I #Autowired the CompanyService interface reference cservice.
Here is the implementation of CompanyService -
public interface CompanyService {
Collection<Company> getDetails() throws Exception;
}
CompanyService interface is implemented by CompanyServiceImplementation -
#Service
public class CompanyServiceImplementation implements CompanyService{
#Autowired
private CompanyDAOImplementation cDAOWrapper;
public Collection<Company> getDetails() throws Exception {
return cDAOWrapper.findAll();
}
}
In the above class, the findAll() method is returning the data from the database.
So the problem is that - The CompanyService reference cservice which is #Autowired in Storage.java class is null and it is not getting initialised and hence I am getting a NullPointerException while calling cservice.getDetails().
Please let me know what is wrong with my Code. Thanks in advance.
I am trying to add some custom functionality to a spring data repository.
Using this as my starting point http://docs.spring.io/spring-data/jpa/docs/current/reference/html/#repositories.single-repository-behaviour I have created the following code:
public interface TableLock<T> {
void checkout(T entity);
void checkin(T entity, boolean cmpltBatch);
}
public interface BatchTableLock extends TableLock<MyEntity> {
}
public class BatchTableLockImpl implements BatchTableLock {
private static final Logger logger = LoggerFactory.getLogger(BatchTableLockImpl.class);
#PersistenceContext(unitName = "mysql")
private EntityManager em;
#Override
public void checkout(MyEntity batch) {
Long id = batch.getMyEntityId();
try {
MyEntity p = em.find(MyEntity.class, id, LockModeType.PESSIMISTIC_WRITE);
if (p == null) {
logger.error("checkout : MyEntity id {} must be valid", id);
throw new PessimisticLockException();
}
if (myCondition is true) {
return;
}
} catch (LockTimeoutException | PessimisticLockException e) {
logger.error("checkout : Unable to get write lock on MyEntity id {}", id, e);
}
throw new PessimisticLockException();
}
#Override
public void checkin(MyEntity batch, boolean cmplt) {
Long id = batch.getMyEntityId();
try {
MyEntity p = em.find(MyEntity.class, id, LockModeType.PESSIMISTIC_WRITE);
if (p == null) {
logger.error("complete : MyEntity id {} must be valid", id);
return;
}
if (this is true) {
if (cmplt) {
yep;
} else {
nope;
}
} else if (cmplt) {
logger.error("complete: Unable to complete MyEntity {} with status.", id);
}
} catch (LockTimeoutException | PessimisticLockException e) {
logger.error("complete : Unable to get write lock on MyEntity id {}", id, e);
}
}
}
#Repository
public interface MyDao extends CrudRepository<MyEntity, BigInteger>, BatchTableLock {
... My queries ...
}
unfortunately I am getting the following error:
org.springframework.data.mapping.PropertyReferenceException: No property checkin found for type MyEntity!
This if I'm not mistaken means that spring is trying to generate a query based on the method 'checkin' and it can't find a field in MyEntity with the name 'checkin'. which is correct there is no such field. how do I make it stop doing this? based on the link above I don't think it should be trying to generate a query for this method, but it seems to be doing it anyway. I may be missing something, that is usually the case, but I don't see what it is.
As stated in the reference documentation section you linked to, you need a MyDaoImpl class that implements the custom methods. I guess the easiest way is to either rename BatchTableLockImpl to that or just create an empty MyDaoImpl extending that class.
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = {HibernateConfigTest.class})
#Transactional
#Sql(scripts = {"api_routes.sql",
"profile.sql",
"status.sql",
"user.sql",
"game_token.sql",
"game.sql",
"message.sql"},
config = #SqlConfig(transactionMode = ISOLATED),
executionPhase = ExecutionPhase.BEFORE_TEST_METHOD)
#Sql(scripts = "delete_data.sql",
executionPhase = ExecutionPhase.AFTER_TEST_METHOD)
public class GameDaoTest {
#Autowired
private GameDao gameDao;
#Test
public void getGetRecentGames() {
Game game = null;
for (int i = 0; i < 1000; i++) {
game = new Game(i + 1000);
game.setStartedAt(DateUtils.getCurrentUTCDate());
gameDao.save("game", game);
}
List<Game> recentGames = gameDao.getRecentGames(1000);
assertNotNull(recentGames);
assertEquals(1000, recentGames.size());
}
}
When I get to the line List<Game> recentGames = gameDao.getRecentGames(1000); hibernate prints out all of the insert statements. Unfortunately, when the games are retrieved, none of the ones I inserted are retrieved. Is there any way to retrieve those games? Maybe a better question would be how do I put those inserts in their own transaction so they are persisted for the subsequent methods?
Here is the AbstractDao that GameDao extends:
public class AbstractDao {
#PersistenceContext
protected EntityManager entityManager;
protected Session getSession() {
return entityManager.unwrap(Session.class);
}
public void save(Object entity) {
getSession().save(entity);
}
public void save(String entityName, Object entity) {
getSession().save(entityName, entity);
}
public void persist(Object entity) {
getSession().persist(entity);
}
}
The persist method throws an exception for some detached entity reason that I am not familiar with.
Try calling flush on your session before calling getRecentGames. If that method is executing a custom query, as opposed to using one of Hibernate's session methods, it's going directly against the database -- but since your save hasn't yet been synced with the underlying database, the data isn't there. You don't actually need to do a true commit as long as Hibernate can see the data.
for (int i = 0; i < 1000; i++) {
game = new Game(i + 1000);
game.setStartedAt(DateUtils.getCurrentUTCDate());
gameDao.save("game", game);
}
gameDao.flush(); // calls getSession().flush()
List<Game> recentGames = gameDao.getRecentGames(1000);
// call assert methods as needed
In my Spring web app I'm using a generic dao class:
public abstract class GenericDaoImpl<T> implements GenericDao<T> {
#Override
public T create(final T t) {
this.getEntityManager().persist(t);
return t;
}
#Override
public void delete(final Object id) {
this.getEntityManager().remove(
this.getEntityManager().getReference(getEntityType(), id));
}
#Override
public T find(final Object id) {
return (T) this.getEntityManager().find(getEntityType(), id);
}
#Override
public T update(final T t) {
return this.getEntityManager().merge(t);
}
}
I implement this class for every entity in my model and it works perfectly. For example:
#Repository
public class GruppoDaoImpl extends GenericDaoImpl<Gruppo> implements GruppoDao {
}
I use these dao classes in my service layer. I have a service layer for every entity in my model, but methods for most of these classes, are the same, so I tried to create a generic service class that I can extend in the same way I do for the generic dao:
public abstract class GenericAdminServiceImpl<ENTITY extends AbstractEntity, DTO extends AbstractDto>
implements GenericAdminService<ENTITY, DTO> {
private GenericDao<ENTITY> dao;
private Class<ENTITY> entityClass;
private Class<DTO> dtoClass;
#SuppressWarnings({ "unchecked", "rawtypes" })
protected GenericAdminServiceImpl(GenericDao<ENTITY> dao) {
this.dao = dao;
//
Type t = getClass().getGenericSuperclass();
ParameterizedType pt = (ParameterizedType) t;
this.entityClass = (Class) pt.getActualTypeArguments()[0];
this.dtoClass = (Class) pt.getActualTypeArguments()[1];
}
public DTO getById(Object id) {
DTO dto = null;
ENTITY entity = dao.find(id);
if (entity != null) {
try {
dto = dtoClass.newInstance();
initDto(entity, dto);
} catch (Exception e) {
}
}
return dto;
}
public void create(DTO dto) throws ServiceOperationException {
ENTITY entity;
try {
entity = entityClass.newInstance();
initEntity(dto, entity);
Date dt = new Date();
entity.setDataUltimoAggiornamento(dt);
entity.setUtenteUltimoAggiornamento(dto.getLoggedUser());
entity.setDataInserimento(dt);
entity.setUtenteInserimento(dto.getLoggedUser());
dao.create(entity);
} catch (Exception e) {
throw new ServiceOperationException("impossibile creare entity ["
+ entityClass.getSimpleName() + "]", e);
}
}
public void update(DTO dto) throws ServiceOperationException {
ENTITY entity = dao.find(dto.getId());
if (!entityExists(entity)) {
throw new ServiceOperationException("entity non esistente ["
+ entityClass.getSimpleName() + "#" + dto.getId() + "]");
}
initEntity(dto, entity);
Date dt = new Date();
entity.setDataUltimoAggiornamento(dt);
entity.setUtenteUltimoAggiornamento(dto.getLoggedUser());
dao.update(entity);
}
public void delete(Object id) throws ServiceOperationException {
try {
dao.delete((int) id);
} catch (Exception e) {
throw new ServiceOperationException(
"impossibile eliminare entity ["
+ entityClass.getSimpleName() + "#" + id + "]", e); // TODO
}
}
protected abstract void initDto(ENTITY entity, DTO outDto);
protected abstract void initEntity(DTO dto, ENTITY outEntity);
protected abstract boolean entityExists(ENTITY entity);
}
Extending this class I just have to implement specific parts for every entity, leaving all the common stuff in the abstract/generic class.
The problem is that using the generic service, merge, persist and delete don't work. Only select seems to work and I cannot understand why...
When I run debug mode in Eclipse all seems correct. A consistent entity is passed to merge/persist methods, so why they don't work? can you help me?
UPDATE #1
This is an example of implementation:
#Service
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class GruppoServiceImplG extends
GenericAdminServiceImpl<Gruppo, GruppoDto> implements GruppoServiceG {
#Autowired
protected GruppoServiceImplG(GruppoDao gruppoDao) {
super(gruppoDao);
}
#Override
protected void initDto(Gruppo entity, GruppoDto outDto) {
outDto.setId(entity.getId());
outDto.setNome(entity.getNome());
outDto.setDescrizione(entity.getDescrizione());
outDto.setDataInizioValidita(entity.getDataInizioValidita());
outDto.setDataFineValidita(entity.getDataFineValidita());
}
#Override
protected void initEntity(GruppoDto dto, Gruppo outEntity) {
outEntity.setId(dto.getId());
outEntity.setNome(dto.getNome());
outEntity.setDescrizione(dto.getDescrizione());
outEntity.setDataInizioValidita(dto.getDataInizioValidita());
outEntity.setDataFineValidita(dto.getDataFineValidita());
}
#Override
protected boolean entityExists(Gruppo entity) {
return entity != null && entity.getId() > 0;
}
}
UPDATE #2
Following Łukasz L. suggestion, I added to all my crud methods a flush(). Now I get this exception javax.persistence.TransactionRequiredException: no transaction is in progress. What's wrong with my transaction declaration? it works fine with non-generic serices...
If you read that question about Spring and Hibernate flush behaviour, it's not easy that commiting your transaction will make also the EntityManager to save all changes. Spring and JPA (Hibernate&CO) are designed to work quite nice (from the Spring side) but nevertheless, you must assert that your entity manager will write all queries to database before commiting transaction.
The problem: JPAs like to cache. It means, they tend to avoid issuing queries. If you do SELECT, they have no choice - they must fetch some data (as long as that data portion was not fetched - like when getting single entity by ID). By INSERTs and UPDATEs - well, they CAN cache. It means, that create, merge or remove will usually not issue a query to RDBMS until you call flush() on EntityManager.
If you leave transactional block without calling flush, and entity manager is delaying operations, you'll commit transactions, by which the modifying queries were not issued!
Just make sure to call EntityManager.flush() at least at the end of the transactional method. You can also call it after each DML operation, it's your choice (I prefer that way because it gives me full control in which order the DML queries are issued by JPA, if you heavily uses DB constraints/triggers, it can be essential).
#Transactional
public void myTransactionalMethod() {
getEntityManager().persist(a); // I've made some JPA change, that is not issued to DB yet
...
// I'm doing something more
...
getEntityManager().flush(); // the last moment to flush, after that instruction I leave transactional context
}
Following Łukasz L. suggestion I discovered the actual issue in my generic class.
Transaction declaration was wrong. I set #Transactional(propagation = Propagation.REQUIRES_NEW) only in concrete service class.
I solved this way:
#Transactional(propagation = Propagation.REQUIRES_NEW)
public abstract class GenericAdminServiceImpl<ENTITY extends AbstractEntity, DTO extends AbstractDto>
implements GenericAdminService<ENTITY, DTO> {
// ...
}
And (in concrete implementation):
#Service
#Transactional
public class GruppoServiceImplG extends
GenericAdminServiceImpl<Gruppo, GruppoDto> implements GruppoServiceG {
// ...
}
I'm usnig Google App Engine with Java, and using Datastore via JPA.
I want to save profile image for each user:
#Entity
public class User implements Serializable {
#Id
private String userId;
private String imageKey; // saves BlobKey.getKeyString()
// ... other fields and getter/setter methods ...
}
If user wants to change their profile image, I (should) update imageKey field AND delete old image associated with old imageKey.
But, the document of Blobstore for Python says:
Deleting a Blobstore value occurs separately from datastore transactions. If you call this method during a datastore transaction, it takes effect immediately, regardless of whether the transaction commits successfully or is retried.
This seems that I can't make updating imageKey and deleting old image as one atomic action, and it also would affect to Java.
This is my attempt to do this work:
public class Engine implements ServletContextListener {
private EntityManagerFactory emf;
private BlobstoreService blobstoreService;
// Servlet will call getUser, modify returned object, and call updateUser with that object
public User getUser(final String userId) throws EngineException {
return doTransaction(new EngineFunc<User>() { // Utility method for create Entity manager and start transaction
#Override
public User run(EntityManager em) throws EngineException {
return em.find(User.class, key);
}
});
}
public void updateUser(final User user) throws EngineException {
doTransaction(new EngineFunc<Void>() {
#Override
public Void run(EntityManager em) throws EngineException {
em.merge(user);
return null;
}
});
user.purgeOldImages(blobstoreService);
}
// ... Other methods ...
}
public class User {
#Transient
private transient Set<String> oldImageList = new CopyOnWriteArraySet<>();
public void setImageKey(String imageKey) {
if (this.imageKey != null) {
oldImageList.add(this.imageKey);
}
this.imageKey = imageKey;
if (imageKey != null) {
oldImageList.remove(imageKey);
}
}
public void purgeOldImages(BlobstoreService blobService) {
Set<BlobKey> toDelete = new HashSet<>();
for (String s : oldImageList) {
toDelete.add(new BlobKey(s));
oldImageList.remove(s);
}
blobService.delete(toDelete.toArray(new BlobKey[0]));
}
// ... Other methods ...
}
I think this is neither "beautiful" nor correct code.
Is there the right way to do this?