I have a local server provided by GlassFish and I have 2 clients: one is generated by server: JSF; and the other one is not on the server: Terminal client written in Java.
Everything was working fine till this moment. Every time when I added a new row into a table, my auto incrementing Primary Key (numeric id) was finely raised by 1 and a row was added.
Now, every time when I freshly start a client and I add a data row into a table via client (does not matter which one - both do the same mess), id is raised to the closest 51 (i.e. last id was 303, so after adding, the id of the new row in the table is 351). If my client is still running since the last addition and I add the next row, it works fine: the next id is 352. It continues without problems till I restart my client. When I add a new row after restarting client, it does the same stuff again: raising id to the closest 51 and then raising but 1.
Is this some kind of bug due to relation of Java and MySQL? This has never happened to me before when I used PHP.
EDIT: For manipulating with database I am using ORM technique via EclipseLink (QuestionDAO.java):
This is what I do in my data layer when I am adding a question:
public void add( Question q )
{
em . persist(q);
}
This is what I do in my business layer when I am calling a method from data layer (Facade.java):
public String addQuestion( String text, String subject )
{
Subject s = subjectDAO.find( subject );
if( s != null )
{
Question q = new Question( text, s );
questionDAO . add(q);
return "Question successfully added";
}
else
return "Subject does not exist";
}
Also here is a method from REST:
#POST
#Path("add")
public String add( #FormParam("text") String text, #FormParam("subject") String subject )
{
String s = facade.addQuestion(text, subject);
return s;
}
Annotations in my Entity class:
#Id
#GeneratedValue(strategy=GenerationType.AUTO)
#Basic(optional = false)
#Column(name = "question_number")
private Integer questionNumber;
I do not believe that a mistake is in my code yet.
Related
I have a query that would be fairly simple using SQL databases. However, in MongoDB, I'm not sure on how to do it. I am building a notification system. I have a collection that stores notifications, and another that creates a document when the user has seen the notification, as such:
notifications collection:
_id: 1
content: "some content"
targetGroup: "somegroup"
seen-notification collection
_id: 1
notificationId: 1
userName: "johndoe"
I'm building an endpoint that should return all notifications in a specific group, that the user has not already seen. In a SQL pseudo-code, I'm trying to do:
SELECT
*
FROM
notifications AS n
INNER JOIN
seen-notification AS sn
ON n._id = sn.notificationId
WHERE
sn.notificationId IS NULL
AND n.targetGroup = "somegroup"
So, basically, a right join with a where clause. I'm using spring data with a MongoRepository. Right now, i'm making two different selects and iterating through them to remove the already seen. I've looked into mongo's $lookup and several SO questions regarding it, but I just couldn't get it work with spring data and MongoRepository. I'm fairly new to mongo with spring data, so maybe there's a way to accomplish this with $lookup and I just didn't figure out how.
try
https://mongoplayground.net/p/-dZHmP9yj-c
expects collection names of notifications and seenNotifications
#Aggregation(pipeline = {
"{$match: { targetGroup: ?0 } }",
"{$lookup: { from: \"seenNotifications\", localField: \"_id\", foreignField: \"notificationId\", as: \"seenNotificationsDocs\"}}",
"{$match: { seenNotificationsDocs: { $size: 0 } } }",
"{$project: { seenNotificationsDocs: 0 } }",
})
List<Notifications> findUnseenNotificationByTargetGroup(String targetGroup);
#Document("notifications")
public class Notifications {
#Id
private Integer id;
private String content;
private String targetGroup;
//getter & setter
}
I have three tables Account, AccountStatus and AccountStatusCodes. AccountStatusCodes is a master table at it has fixed account status codes. Account table will have different account listed. AccountStatus table is a kind of history table, whenever user performs some action on account, the old acount status is updated with flag N and new account status will be inserted. So on each operation on Account the Account status will mantain history with timestamp, user id who updated the status and Y/N flag.
Relationship - Many to many relationship from Account table to AccountStatusCodes code is broken like this
1. One Account can have multiple AccountStatus - Account --> one to many --> AccountStatus
2. One AccountStatusCodes can have multiple AccountStatus - AccountStatusCodes --> one to many --> AccountStatus
JPA Entity code - The actual code is not allowed to share, hence sharing the modifiable code to explain the scenario.
Class: AccountEntity
#DynamicUpdate(value = true)
public class AccountEntity{
private Long accountKey;
//Skipped other variables and getter setters
}
Class: AccountStatusEntity
public class AccountStatusEntity{
#Id
#SequenceGenerator(name = "ACCOUNT_STATUSES_DOCUMENT_STATUSKEY_GENERATOR" , sequenceName = "AS_SEQ")
#GeneratedValue(generator = "ACCOUNT_STATUSES_DOCUMENT_STATUSKEY_GENERATOR")
#Column(name = "ACCOUNT_STATUS_KEY" , unique = true , nullable = false , precision = 10)
private Integer accountStatusKey;
#ManyToOne
#JoinColumn(name = "ACCOUNT_STATUS_CODE_KEY" , nullable = false)
private AccountStatusCodeEntity accountStatusCodeEntity;
#ManyToOne
#JoinColumn(name = "ACCOUNT_KEY" , nullable = false)
private AccountEntity accountEntity;
#Column(name = "CURRENT_STATUS_IND" , nullable = false , length = 1)
private String currentStatusInd;
//skipped other variables and getter setters
}
Class: AccountStatusCodeEntity
public class AccountStatusCodeEntity{
#Id
#Column(name = "ACCOUNT_STATUS_CODE_KEY")
#GeneratedValue(generator = "ASC_SEQ")
#SequenceGenerator(name = "ASC_SEQ", sequenceName = "ASC_SEQ")
private Integer accountStatusCodeKey;
#OneToMany(mappedBy = "accountStatusEntity")
private List<AccountStatusEntity> accountStatuseEntitiess;
}
In the application each user performs some operation on the account and each time the account status gets incremented to next AccountStatusCode and it maintains the history in AccountStatusEntity table by modifying existing status to flag N and inserting new status with timestamp, user id and flag Y.
So there will be two DB operation performed with the #Transaction first one is to update old status to N and insert new status with Y.
The method which does this has following code.
private void moveAccountStatus(final Long accountKey, final String loggedInUserID, final Integer currentStatus,
final Integer nextStatus) {
//Search existing account status with accountKey
// here I have skipped the code which will pull latest status entity from the history table based on date
final AccountStatusEntity accountStatusEntity =
accountDAO.findAccountStatusByAccountKey(accountKey);
AccountEntity accountEntity;
AccountStatusEntity newAccountStatusEntity;
if (accountStatusEntity != null) {
accountEntity = accountDAO.findAccountByAccountKey(accountKey);
accountStatusEntity.setCurrentStatusInd(Constants.NO);
accountStatusEntity.setModifiedBy(loggedInUserID);
accountStatusEntity.setModifiedTs(new Date());
accountStatusEntity.setAccountEntity(accountEntity);
//The update method here is calling the JPA merge() to update the records in the table.
accountDAO.update(accountStatusEntity);
//Create new object of AccountStatusEntity to insert new row with the flag Y
newAccountStatusEntity = new AccountStatusEntity();
//Set the next status
newAccountStatusEntity.setAccountStatusCodeEntity(
(AccountStatusCodeEntity) accountDAO.getById(AccountStatusCodeEntity.class, nextStatus));
newAccountStatusEntity.setCurrentStatusInd(Constants.YES);
newAccountStatusEntity.setCreatedBy(loggedInUserID);
newAccountStatusEntity.setCreatedTs(new Date());
newAccountStatusEntity.setAccountEntity(accountEntity);
//The create() method is also calling the JPA merge() method. The Id is null hence it will consider a insert statement and will insert a new record.
accountDAO.create(newAccountStatusEntity);
}
}
This process works 99% fine in the Production but some times this method is createing duplicate record in the table AccountStatusEntity with the same timestamp, userid and flag Y. The method did not update record with flag N or due to some issue the old record is also getting updated with flag Y.
Table: AccountStatus
___________________________________________________________________________________________________________________
accountStatusKey | accountKey | accountStatusCodeKey | currentStatusInd | Created_TS | Created_BY
___________________________________________________________________________________________________________________
| | | | |
1 | 5 | 3 | Y | A | 4/9/2018
2 | 5 | 3 | Y | A | 4/9/2018
___________________________________________________________________________________________________________________
The above table shows the records which is getting created after the method execution. The accountStatusKey 1 should have the accountStatusCodeKey 2 (the old status code is 2 and next status code is 3) and currentStatusInd N. But somehow the merge method is inserting two records here.
One solution I can make to create unique constraint on the column so that it will avoid this situation but I just wanted to know why the merge method of JPA is creating this issue.
Another solution which I have not tried is to use JPA's persist() method at the time of insert instead of merge().
This issue is hard to reproduce in Development environment as 99% time it works, Also users reports this issue very late so that could not trace the log files.
As per the logs on dev environment, when the JPA transaction gets executed the insert statement is getting logged first and then the update query is getting logged in the log file. I am not sure how the statement order follows in the Transaction case.
I know the question is too long but I just wanted to give the exact background for understanding this issue. TIA.
I have modified code and instead of updating child records directly I am creating and updating child entity and adding into the Parent entity list and asking JPA to Save or update. So based on the Id either it is a null or having valid id the Parent will decide whether to add or update child. I have move my code to Prod environment and from last 2-3 weeks I am not seeing any duplicate rows issue.
I will keep you posted if I see this issue again. Thanks!
This was the known issue in the older Hibernate versions. My application was using hibernate version 4.2.8.Final.
Reported issue link - https://hibernate.atlassian.net/browse/HHH-6776
This issue was fixed in 5.0.8.Final version. I have updated my maven dependencies and will keep testing.
Link: https://hibernate.atlassian.net/browse/HHH-5855
More information about this issue -
https://javacodinggeeks.blogspot.com/2015/05/hibernate-inserts-duplicate-child-on.html
Hibernate inserts duplicates into a #OneToMany collection
I am trying to insert a list of rows(questions) to a table.(lets say 'Question_Table').
The whole process is performed in a single transaction. (ie. either i have to insert all questions or none). I am using Spring's declarative transaction.
I have customized the ID generation for Question_Table.(Ref : Custom id generation)
It works for the first question. But it wont work for the second question as the first row is un-committed and the table will be empty. I am not able to inject the DAO class into Id generator as it is not a spring managed bean(so i can have a method in DAO class that reads un-committed records).
What is the best approach to use in this situation.
Generator class
public class IdGenerator implements IdentifierGenerator, Configurable {
private String prefix = "";
private String queryKey = "";
#Override
public Serializable generate(SessionImplementor sessionImpl, Object arg1) throws HibernateException {
long count = (long)sessionImpl.getNamedQuery(queryKey).list().get(0);
System.out.println("COUNT >>> "+count);
long id = count + 1;
if(id == 4) throw new NullPointerException();
String generatedId = prefix + id;
return generatedId;
}
#Override
public void configure(Type arg0, Properties arg1, ServiceRegistry arg2) throws MappingException {
prefix=arg1.getProperty("PREFIX");
queryKey=arg1.getProperty("QUERY_KEY");
}
}
Query : select count(*) from Question_Table
As i stated in the comment, you maybe can use this approach if you did not have problem using combination of string and sequence. But the downside is the value will always increase even after you delete all record in that table.
If you insist of using count, then the solution is to define your entity id on save manually like. .save(question, "QSTN_"+(row_count + i)); but you will need to be able pass that row_count which i think is not a problem since it must be on one request.
I have no answer to your specific question but i'd like to share some considerations.
If your id generation depends on the database state, then it must be done at the database level (implementation is up to you, autoincrement, custom function or sequences, etc, etc)...
Otherwise if you do it at the application level you will necessary encounter concurrent access problems and have to mitigate it using some lock or dedicated transaction which will have a significant impact on the application performance and may become inconsistent later (when adding horizontal scalability or sharding for example).
However if you want to generate your ids in an applicative layer (which can be a very good idea) then you must have an unique, distributed system dedicated for this task which is not part of your current unit of work.
#Transactional(isolation = Isolation.READ_COMMITTED)
public AccountDto saveAccount(AccountDto accountDto) {
Long accountTypeId = accountDto.getAccountTypeId();
AccountTypes accountTypes = accountTypesDao.getById( accountTypeId ).orElseThrow( NotFoundAppException::new );
account.setAccountName( newAccountName );
account.setAccountType( accountTypes );
...
accountDao.save( account );
accountDao.flush();
// new inserted account id is in the transaction now
return createAccountDtoFrom( account );
}
the question i am going to ask is very old and i think it asked on SO 5-10 times.
but mine has a different situation.
Read my Problem before making it duplicate by so many wise (over) SO users.
I am importing CSV sheet containing 10K records in my application.
My logic works in following manner,
(1) Validate & Import Sheet
(2) Save to the database if record does not exist
Step 2 is done for each and every record of the sheet.
in the step -2 i have to generate UUID to identify a particular record later ,
in my first solution
// this might be unique in some cases
String id = UUID.randomUUID().toString();
but i checked that it does generate unique id in each case , for example
if i import 10 sheet one by one with different records in it, all 10 times i am getting
duplicate key error from database for at least 4000 times in each import and save operation,
that means that out of 10,000 key generation it generates only 6000 unique ids.
so then i generate an alphanumeric code which length is 6 , some thing like
eSv3h7
and append it to previously generated id and hence get the following id
d545f2b2-63ab-4703-89b0-f2f8eca02154-eSv3h7
after testing still there is a problem of id duplication.
I also tried several combination mentioned here and on other sites but still there is a same problem of id duplication,
Now i am wondering that this occurs only for 10k records saving in loop , actually i need to import sheet which is having 8 million records in it
so how can i solve my problem of generating a Unique id in my particular case ?
Update 1 - based on all the comments
Try this thing at you end.
loop through 1 to 10,000
generate uuid in the loop
store it somewhere in simple text file
then make a simple program to find the duplicates from them , if you do not find any one duplicate in first attempt , repeat all above steps again and again and i am sure you will find duplicates.
in the past i am also strong believer of the same thing that UUID will never generates duplicates, share me your result of above test.
Update 2 - Code
This is the method which is called by each record of the sheet to be saved by caller method' loop.
#Override
public void preSynchronizedServiceExecution(ServiceData sData,
ValueObject valueObject) throws BlfException {
PropertyVO pVO = (PropertyVO) valueObject;
ArrayList<CountyAuctionPropertyVO> capList = pVO
.getCountyAuctionPropertyList();
for (CountyAuctionPropertyVO caVO : capList) {
TadFrameworkUtil.processValueObjectKeyProperty(caVO, true);
TadFrameworkUtil.processValueObjectKeyProperty(caVO
.getPropertyLastOwner(), true);
TadFrameworkUtil.processValueObjectKeyProperty(caVO
.getPropertyLastOwner().getAdd(), true);
}
ArrayList<PropertyAminitiesVO> amList = pVO.getPropertyAminitiesList();
for (PropertyAminitiesVO pamVO : amList) {
TadFrameworkUtil.processValueObjectKeyProperty(pamVO, true);
}
ArrayList<PropertyAttributesVO> atList = pVO
.getPropertyAttributesList();
for (PropertyAttributesVO patVO : atList) {
TadFrameworkUtil.processValueObjectKeyProperty(patVO, true);
}
TadFrameworkUtil.processValueObjectKeyProperty(pVO, true);
TadFrameworkUtil.processValueObjectKeyProperty(pVO.getSiteAdd(), true);
}
Following is id generation method
public static String generateUUID() throws BlfException {
// this might be unique in some cases
String id = UUID.randomUUID().toString();
// introduce custom random string in mixing of upper and lower
// alphabets,
// which is 6 character long
// and append it to generated GUID.
String rs = randomString(6);
id = id.concat("-").concat(rs);
return id;
}
Update 3 (Method added)
public static void processValueObjectKeyProperty(ValueObject valueObject,
boolean create) throws BlfException {
String key = (String) BlfConverter.getKey(valueObject);
if (!StringUtility.isStringNonEmpty(key)) {
throw new BlfException(valueObject.getObjectName()
+ "- key property does not exist.");
}
if (create) {
String id = generateUUID();
valueObject.setProperty(key, id);
} else {
String exisitingId = valueObject.getProperty(key);
if (!StringUtility.isStringNonEmpty(exisitingId)) {
String id = generateUUID();
valueObject.setProperty(key, id);
}
}
}
The random string method is just a simple methods of 2 lines which generates alpha numeric random string of length 6.
please ask me if you need anything more so i can post here.
Update 4 (Sample genearted UUID )
d545f2b2-63ab-4703-89b0-f2f8eca02154-eSv3h7
6f06fa28-6f36-4ed4-926b-9fef86d002b3-DZ2LaE
20142d05-f456-4d72-b845-b6819443b480-xzypQr
67b2a353-e7b4-4245-90a0-e9fca8644713-AgSQZm
8213b275-2cb1-4d37-aff0-316a47e5b780-vMIwv9
and i am getting accurate result from database if i need to fetch it from there.
Thanks
Thanks for all your user who seriously study my question and spent some time to help me in solving it.
I found that the error was in Database layer of business logic foundation.
One of the Object needs to Updated but it was created using previous
existing id so that i was getting the Duplicate Primary key Error.
I develop a Unit Test for id generation and tested UUID for more than
one billion key , it is guaranteed to be unique, it is true in all
circumstances.
Thanks again to everyone.
Ok, I have this code & it works fine in eclipse
public static long getPropertyMaxIDStronglyConsistent(String entityName, String propertyName, Key parentKey){
// Order alphabetically by property name:
//Key parentKey=KeyFactory.createKey(parentEntityName, parentName);
Query q = new Query(entityName, parentKey).setAncestor(parentKey)
.addSort(propertyName, SortDirection.DESCENDING);
//List<Entity> results = datastore.prepare(q).asList(FetchOptions.Builder.withDefaults());
List<Entity> results = datastore.prepare(q)
.asList(FetchOptions.Builder.withLimit(5));
if(results.size()>0){
Entity e=results.get(0);
long maxID=(Long)e.getProperty(propertyName);
return maxID;
}
else{
return 0;
}
}
However, after deployed to Google App Engine, it messed up everything.
Based on my research, it seems that Google return extra row when we get List<Entity> results = datastore.prepare(q).asList(FetchOptions.Builder.withLimit(5)); unlike MYSQL and JDBC Driver
For example, if the real table has 2 rows, it will return 3 rows & if we Entity e=results.get(0); it somehow does not show anything.