I am working on the backend of a game. Players first purchase their tickets, which are stored into the database, then they play and can win certain prizes. Each player can buy a max of 60 tickets.
A new requirement appeared, setting an upper bound for the total number of tickets per game - 10 000 tickets. The implementation would involve adding to the purchase endpoint a test checking that the total number of purchased tickets at that time + number of tickets required on current purchase is less or equal to the max number of tickets for that game, in order for a purchase to be accepted.
The problem is that, calculating the current number of purchased tickets by using count on the tickets table, the returned value may be out-of-date because in the meantime other players could have purchased tickets.
A possible solution is to lock the tickets table during the purchase, but this can have a negative impact on performance and user experience. Please let me know if you have a different idea.
If yours is a hard constraint then you must avoid multiple purchases being conducted concurrently. Probably an appropriate table lock in the DB is the best way to accomplish such serialization. There could be other alternatives (i.e. performing the serialization somewhere in the front end), but they are likely to be messier to implement, more prone to bugs, and worse-performing.
In fact, it may be difficult to make your game exhibit consistent behavior at all without appropriate locking in the DB. On the other hand, you probably don't need explicit locking. If you configure your database with an appropriate transaction isolation level then it should do all necessary locking for you.
Since there's no clear win to be had here, I recommend shelving the performance question until you can actually test your performance. If it turns out not to be good enough then you can determine from actual measurement what parts of the system can most benefit from tuning.
Related
I am new to java programming and I am using databases to create a simple Bank Management system. I need to create user accounts with an account number with 9 digits, that does not start with 0. also, all the account numbers must be different(one account number can have repeating digits). Every time I run the program, it should give me a 9-digit number that is not stored in the 'accounts' table under 'account_number' in the 'login_info' database.
The program runs like this,
User goes to the login page
If he does not have an account he can go to the create account page
create account page has 3 tabs, Personnel info, bank account info, and online account info
user must fill out personnel info got be able to go to the bank account info
when the user goes to bank account info, it shows the previously filled items (Personel info) and a couple of additional items - Bank account type and account number
The account number must be generated when the user goes to the bank account info tab. The user cannot change the account number. (The account number must be one that has not been saved in the database before)This is the step I am stuck
After selecting the account type, the user can go to the online account info page and select a unique username and password
After that, the user can hit create account button to create his bank account along with the online account or he can exit the program if he is having second thoughts
When the user hit create account button, a query will run and insert data into MySQL tables. This is the point where the account is created.
The program runs beautifully, but I can't seem to generate a unique bank account number. Other steps are already completed.
(This is not for any commercial products, just a project I work on to get myself acquainted with database handling with java.)
Any Idea how to do this?
Thank you.
Here is an approach you could take, elaborating on my comments posted above:
pre-calculate all valid account numbers – there are a total of 3,265,920 valid combinations
9 choices for first digit: 1-9,
9 choices for 2nd digit: 0 is available, so 10 possible digits (0-9) except that something was used for the 1st digit
8 choices for 3rd, etc.
so: 9*9*8*7*6*5*4*3*2 combinations
populate each of those ~3 million numbers into the database table
include a "status" field to represent things like: "unclaimed", "pending", "claimed"
all account numbers start out as "unclaimed"
In your Java app, when you want to present the user with a candidate account number, you can use the database to update a single row from "unclaimed" to "pending" and return that account number – so the database would enforce correctness even if multiple callers simultaneously try to set up a new account. Something like: "update accounts where status=unclaimed set status=pending limit 1" and return the updated row.
From there:
If the user accepts the account number, simply change the status from "pending" to "claimed"
If they decline, change it back to "unclaimed"
This approach would allow you to do numeric specifics only once, up front, when creating all initial unclaimed accounts. It does have a potential drawback that you have 3 million rows sitting there unused, but there is a nice simplicity overall – no further combinations to worry about, or generating a number and then involving the database to see if that number is claimed or not (and re-generating if number in use), or any kind of scanning through the database to use as input for the number selecting process, or whatever other ideas people might come up with. Just do the work up front.
If you want the number to be random, create a number from 0 to 899 999 999 and then add 100 000 000 (so the first will never be 0). If the numbers are stored in an Arraylist (in this case called nums), the following code should help:
int num;
Do{
num = (int)(Math.random()*800000000)+100000000;
}while (nums.contains(num));
nums.add(num);
I am using databases to create a simple Bank Management system. I need
to create user accounts with an account number with 9 digits, that
does not start with 0. also, all the account numbers must be
different. Every time I run the program, it should give me a 9-digit
number that is not stored in the 'accounts' table under
'account_number' in the 'login_info' database.
This actually turns out to be an unexpectedly complicated question to answer. The best answer depends upon other factors which you have not mentioned:
Does the account number have to be unique "each time the program is run" or "each time an account number is requested during the same run (but not necessarily from run to run)"?
Is this a single-threaded environment (i.e., how many simultaneously executing threads might be trying to create account numbers at the same time)?
Does the account number have the sense of identity; does it need to be able to uniquely identify a specific account at all times (or are there times, perhaps when the account is being created from a form, when an account doesn't need to have identity?
What manages the identity? Your program system? The database that you are using to store the information?
If the first bullet item answer is "each time the program is run", then you need to think about how the account number seed (the quantity or quantities that are used to generate your next available account number) is persisted from run to run (the seed otherwise is destroyed when the JVM exits).
If the second bullet item is "multiple threads", then you need to think about how to prevent the same account number from being inadvertently provided to threads requesting one at the same time.
The remaining bullet items deal with identity. Account numbers are generally used to identify specific accounts to the system (but not necessarily). So, you have to consider when the identity needs to become available to the system. You also have to think about what system is responsible for managing the identity. If the identity is needed as soon as an account entity is created, then it needs to be obtained during the instantiation of the account by Java. If it is not needed until the account entity is persisted to the database, then it may be possible for the database system to create the identity.
A hybrid solution would be to persist the account as soon as it was instantiated, and then read it right back again to obtain the account number (at the cost of increasing database traffic). Logic might be needed to delete the database record if the account did not need to be persisted after all (because, say, it had been canceled).
Depending on the answers to these questions, and upon other important factors such as availability, scalability, redundancy, security etc., the complete solution to the original problem could become arbitrarily complex.
But if we assume that account numbers need to be unique only during the same run of the Java system and that account accesses are only single threaded, then a solution could become very simple:
public static class AccountNumberManager {
private static long accountNumberSeed = getStartupSeedValue();
private AccountNumberManager() { }
public long getNextAccountNumber() {
return ++accountNumberSeed;
}
private static long getStartupSeedValue() {
return 100000000L;
}
}
Some brief notes:
A static class for account number management makes the initialization of the account number seed value thread safe. The initial value initialization occurs while the class is locked by the class loader during the loading process and is therefore thread safe (the class and its static fields can only be initialized once and once only). This is the static holder class initialization method. This would be handy if the system ever evolved into a concurrent environment.
This solution, of course, assumes that the account number is needed when the account entity is initialized. I think this is a good practice anyway. My bias is that database systems should not be used to create system-wide identitifiers when possible (only to persist them).
It is not a best practice to use account numbers as database primary keys. This is an opinion that is fairly widely, not but universally, held.
In my App I'm modelling an Invoice. In my country (Italy) every invoice must have a unique sequential number without holes, that every year have to restart from 1.
I thought long and hard about the best way to implement it but I have not found a good guide about this. For now I have a JpaRepository where I've my custom synchronized save() method in which I get the last id used:
SELECT MAX(numero) FROM Invoice WHERE YEAR(date) = :year
The problem of this approach is that is not very safe because the developer should know that the save should be done only with that particular service.
Instead I'd like more an approach that is hidden to the developer.
I thought to use a #Prepersist method in a #EntityListeners. This sounds good but do get entity manager inside this class is not so simple....so maybe is not the optimal place...
Finally I thought about Hibernate Interceptor....
Please give me some hints. The problem seems a quite generic problem; so maybe there is yet a good practice to follow.
Thanks
This problem can be broken down into the following requirements:
Sequentially unique: Generate numbers in a sequence, starting from a given value (say, 1000001) and then always incrementing by a fixed value (say, 1).
No gaps: There must not be any gaps between the numbers. So, if the first number generated is 1000001, the increment is 1 and 200 numbers have been generated so far, the latest number should be 1000201.
Concurrency: Multiple processes must be able to generate the numbers at the same time.
Generation at creation: The numbers must be generated at the time of creation of a record.
No exclusive locks: No exclusive locks should be required for generating the numbers.
Any solution can only comply with 4 out of these 5 requirements. For example, if you want to guarantee 1-4, each process will need to take locks so that no other process can generate and use the same number that it has generated. Therefore, imposing 1-4 as requirements will mean that 5 will have to be let gone of. Similarly, if you want to guarantee 1, 2, 4 and 5, you need to make sure that only one process (thread) generates a number at a time because uniqueness cannot be guaranteed in a concurrent environment without locking. Continue this logic and you will see why it is impossible to guarantee all of these requirements at the same time.
Now, the solution depends on which one out of 1-5 you are willing to sacrifice. If you are willing to sacrifice #4 but not #5, you can run a batch process during idle hours to generate the numbers. However, if you put this list in front of a business user (or a finance guy), they will ask you to comply with 1-4 as #5 is a purely technical issue (to them) and therefore they would not want to be bothered with it. If that is the case, a possible strategy is:
Perform all possible computation required to generate an invoice upfront, keeping the invoice number generation step as the very last step. This will ensure that any exceptions that can occur, happen before the number is generated and also to make sure that a lock is taken for a very short amount of time, thereby not affecting the concurrency or performance of the application too much.
Keep a separate table (for example, DOCUMENT_SEQUENCE) to keep a track of the last generated number.
Just before saving an invoice, take an exclusive row-level lock on the sequence table (say, isolation level SERIALIZABLE), find the required sequence value to use and save the invoice immediately. This should not take too much time because reading a row, incrementing its value and saving a record should be a short enough operation. If possible, make this short transaction a nested transaction to the main one.
Keep a decent-enough database timeout so that concurrent threads waiting for a SERIALIZABLE lock do not time out too fast.
Keep this whole operation in a retry loop, retrying at least 10 times before giving up completely. This will ensure that if the lock queue builds up too fast, the operations are still tried a few times before giving up totally. Many commercial packages have retry count as high as 40, 60 or 100.
In addition to this, if possible and allowed by your database design guidelines, put a unique constraint on the invoice number column so that duplicate values are not stored at any cost.
Spring gives you all the tools to implement this.
Transactions: Through the #Transactional annotation.
Serialization: Through the isolation attribute of the #Transactional annotation.
Database access: Through Spring JDBC, Spring ORM and Spring Data JPA.
Retries: Through Spring Retry.
I have a sample app that demonstrates using all these pieces together.
I am currently developing a game for education purposes.
The game is a small chat where each player has a figure. This figure has its position written in a MySQL database, and updates every time a players moves his figure.
Currently, every 60 frames, the game updates all the players positions and changes them in the gameclient. The performance of the database is not my only concern, as this causes the game framerate to drop to around 30-50 fps.
Obviously, this generates 1 query for position updates a second (Game runs at 60fps), per player. Thinking a little larger than me and a few friends as users, i could imagine this may be a problem.
Would a standard rented webserver be able to handle this? How could i improve it? (It can't update the positions less than once a second)
I hope you have some ideas :-)
This sort of data is not really saved to a database after every single change, instead it's kept in some kind of in-memory context so that it's much faster to access. What you store in a database is things that change less frequently, like on the order of once per minute.
What you might do is snapshot the current in-memory context to the database on a regular schedule. That is to say every minute or so you would "persist" the game state so that if the engine crashed it could restore from that point. If you create the proper architecture, this would be no more difficult than calling a method periodically.
Certain things may be persisted immediately to avoid people exploiting this lag between an action occurring and the record of that action being permanent. For example, any transactions with in-game currency would be recorded immediately and the in-memory cache would not be trusted for these.
For low numbers of players, like under a hundred, you should have no problem with using a commodity VPS. To host more players you would have to be very careful to be efficient.
You can use an in-memory Java database, which allows you to update the players more frequently. Changes are then stored in the MySQL database at a less frequent rate.
I built a social android application in which users can see other users around them by gps location. at the beginning thing went well as i had low number of users, But now that I have increasing number of users (about 1500 +100 every day) I revealed a major problem in my design.
In my Google App Engine servlet I have static HashMap that holding all the users profiles objects, currenty 1500 and this number will increase as more users register.
Why I'm doing it
Every user that requesting for the users around him compares his gps with other users and check if they are in his 10km radius, this happens every 5 min on average.
That is why I can't get the users from db every time because GAE read/write operation quota will tare me apart.
The problem with this desgin is
As the number of users increased the Hashmap turns to null every 4-6 hours, I thing that this time is getting shorten but I'm not sure.
I'm fixing this by reloading the users from the db every time I detect that it became null, But this causes DOS to my users for 30 sec, So I'm looking for better solution.
I'm guessing that it happens because the size of the hashmap, Am I right?
I would like to know how to manage ALL users profiles with max aviablity.
Thanks.
I would not store this data in the HashMap as it does not really scale if you run on multiple instances and furthermore you use a lot of memory.
Why do you not use some different storages like MongoDB which is also available 'in the cloud'? (e.g. www.mongohq.com).
If you would like to scale you need to separate the data from the processors. E.g. have x servers running your servlet (or let Google AppEngine scale this on themselves) and have the data at a different place (e.g. in a MongoDB or PostgreSQL).
You need to rethink your whole design. Storing all users in one huge HashMap won't scale (sooner or later you'll have to cluster your application). Also the complexity of your algorithm is quite high - you need to traverse the whole map for each user.
A much more scalable solution would be to use a spatial database. All major relation databases and some NoSQL products offer geospatial indexing. Basically the database query engine is optimized for queries like: give me all the records with near this given point.
If your application is really successful, even an in-memory map will be slower than enterprise-grade geospatial index.
Part of my project requires that we maintain stats of our customers products. More or less we want to show our customers how often their products has been viewed on the site
Therefore we want to create some form of Product Impressions Counter. I do not just mean a counter when we land on the specific product page, but when the product appears in search results and in our product directory lists.
I was thinking that after calling the DB I would extract the specific product ids and pass them to a service that will then inserted then into the stats tables. Or another is using some form of singleton buffer writer which writes to the DB after it reaches a certains size?
Has anyone ever encountered this in there projects and have any ideas that they would like to share?
And / or does anyone know of any framework or tools that could aid this development?
Any input would be really appreciated.
As long as you don't have performance problems, do not over-engineer your design. On the other hand, depending on how big the site is, it seem that you are going to have performance problems due to huge amount of writes.
I think real time updates will have a huge performance impact. Also it is very likely that you will update the same data multiple times in short period of time. Another thing is that, although interesting, storing this statistics is not mission-cricital and it shouldn't affect normal system work. Final thought: inconsistencies and minor inaccuracy is IMHO acceptable in this use case.
Taking all this into account I would temporarily hold the statistics in memory and flush them periodically as you've suggested. This has the additional benefit of merging events for the same product - if between two flushed some product was visited 10 times, you will only perform one update, not 10.
Technically, you can use properly synchronized singleton with background thread (a lot of handcrafting) or some intelligent cache with write-behind technology.