We Keep Coding

Interactive Broker Java API

Everytime before I place a new order to IB, I need to make a request to IB for next valid orderId and do Thread.Sleep(500) to sleep for 0.5 seconds and wait for IB API's callback function nextValidId to return the latest orderID. If I want to place multiple orders out, then I have to naively do thread.sleep multiple times, This is not a very good way to handle this, as the orderID could have been updated earlier and hence the new order could have been placed earlier. And what if the orderID takes longer time to update than thread sleep time, this would result in error.
Is there a more efficient and elegant way to do this ?
Ideally, I want the program to prevent running placeNewOrder until the latest available orderID is updated and notify the program to run placeNewOrder.
I do not know much about Java data synchronization but I reckon there might be a better solution using synchronized or wait-notify or locking or blocking.
my code:
// place first order
ib_client.reqIds(-1);
Thread.sleep(500);
int currentOrderId = ib_wrapper.getCurrentOrderId();
placeNewOrder(currentOrderId, orderDetails); // my order placement method
// place 2nd order
ib_client.reqIds(-1);
Thread.sleep(500);
int currentOrderId = ib_wrapper.getCurrentOrderId();
placeNewOrder(currentOrderId, orderDetails); // my order placement method
IB EWrapper:
public class EWrapperImpl implements EWrapper {
...
protected int currentOrderId = -1;
...
public int getCurrentOrderId() {
return currentOrderId;
}
public void nextValidId(int orderId) {
System.out.println("Next Valid Id: ["+orderId+"]");
currentOrderId = orderId;
}
...
}

You never need to ask for id's. Just increment by one for every order.
When you first connect, nextValidId is the first or second message to be received, just keep track of the id and keep incrementing.
The only rules for orderId is to use an integer and always increment by some amount. This is per clientId so if you connect with a new clientId then the last orderId is something else.
I always use max(1000, nextValidId) to make sure my id's start at 1000 or more since I use <1000 for data requests. It just helps with errors that have ids.
You can also reset the sequence somehow.
https://interactivebrokers.github.io/tws-api/order_submission.html
This means that if there is a single client application submitting
orders to an account, it does not have to obtain a new valid
identifier every time it needs to submit a new order. It is enough to
increase the last value received from the nextValidId method by one.

You should not mess around with order ID, it's automatically tracked and being set by the API. Otherwise you will get the annoying "Duplicate order id" error 103. From ApiController class:
public void placeOrModifyOrder(Contract contract, final Order order, final IOrderHandler handler) {
if (!checkConnection())
return;
// when placing new order, assign new order id
if (order.orderId() == 0) {
order.orderId( m_orderId++);
if (handler != null) {
m_orderHandlers.put( order.orderId(), handler);
}
}
m_client.placeOrder( contract, order);
sendEOM();
}

Hibernate custom id generation inside transaction

I am trying to insert a list of rows(questions) to a table.(lets say 'Question_Table').
The whole process is performed in a single transaction. (ie. either i have to insert all questions or none). I am using Spring's declarative transaction.
I have customized the ID generation for Question_Table.(Ref : Custom id generation)
It works for the first question. But it wont work for the second question as the first row is un-committed and the table will be empty. I am not able to inject the DAO class into Id generator as it is not a spring managed bean(so i can have a method in DAO class that reads un-committed records).
What is the best approach to use in this situation.
Generator class
public class IdGenerator implements IdentifierGenerator, Configurable {
private String prefix = "";
private String queryKey = "";
#Override
public Serializable generate(SessionImplementor sessionImpl, Object arg1) throws HibernateException {
long count = (long)sessionImpl.getNamedQuery(queryKey).list().get(0);
System.out.println("COUNT >>> "+count);
long id = count + 1;
if(id == 4) throw new NullPointerException();
String generatedId = prefix + id;
return generatedId;
}
#Override
public void configure(Type arg0, Properties arg1, ServiceRegistry arg2) throws MappingException {
prefix=arg1.getProperty("PREFIX");
queryKey=arg1.getProperty("QUERY_KEY");
}
}
Query : select count(*) from Question_Table

As i stated in the comment, you maybe can use this approach if you did not have problem using combination of string and sequence. But the downside is the value will always increase even after you delete all record in that table.
If you insist of using count, then the solution is to define your entity id on save manually like. .save(question, "QSTN_"+(row_count + i)); but you will need to be able pass that row_count which i think is not a problem since it must be on one request.

I have no answer to your specific question but i'd like to share some considerations.
If your id generation depends on the database state, then it must be done at the database level (implementation is up to you, autoincrement, custom function or sequences, etc, etc)...
Otherwise if you do it at the application level you will necessary encounter concurrent access problems and have to mitigate it using some lock or dedicated transaction which will have a significant impact on the application performance and may become inconsistent later (when adding horizontal scalability or sharding for example).
However if you want to generate your ids in an applicative layer (which can be a very good idea) then you must have an unique, distributed system dedicated for this task which is not part of your current unit of work.

#Transactional(isolation = Isolation.READ_COMMITTED)
public AccountDto saveAccount(AccountDto accountDto) {
Long accountTypeId = accountDto.getAccountTypeId();
AccountTypes accountTypes = accountTypesDao.getById( accountTypeId ).orElseThrow( NotFoundAppException::new );
account.setAccountName( newAccountName );
account.setAccountType( accountTypes );
...
accountDao.save( account );
accountDao.flush();
// new inserted account id is in the transaction now
return createAccountDtoFrom( account );
}

how to make sure int is assigned a unique id for that thread

I need to insert into database which has two columns-
ID PrimaryKey String
ACCOUNT String
So that means each thread should be using unique id always and I need to store the same ID in Account column also. So suppose if ID is 1 then in Database it should be stored as
ID Account
1 SomeString+1
2 SomeString+2
3 SomeString+3
....
..
100 SomeString+100
I am concatenating that userID with that String always in Account column.
Below is my multithreaded code which will spawn multiple threads- And each thread will be getting a new unique ID everytime as I am using AtomicInteger for that. And it will insert that ID to ID column and also append that ID to Account column
But somehow in my below program what I have seen in that database is-
ID Account
1 String+2
2 String+1
3 String+3
Which is not right. It should be something like this-
ID Account
1 String+1
2 String+2
3 String+3
Below is the code
public static void main(String[] args) {
final int noOfThreads = 4;
final int noOfTasks = 10;
final AtomicInteger id = new AtomicInteger(1);
ExecutorService service = Executors.newFixedThreadPool(noOfThreads);
for (int i = 0; i < noOfTasks * noOfThreads; i++) {
service.submit(new Task(id));
}
}
class Task implements Runnable {
private final AtomicInteger id;
private volatile int userId;
public Task(AtomicInteger id) {
this.id = id;
}
#Override
public void run() {
dbConnection = getDBConnection();
preparedStatement = dbConnection.prepareStatement(Constants.INSERT_ORACLE_SQL);
userId = id.getAndIncrement();
preparedStatement.setString(1, String.valueOf(userId));
preparedStatement.setString(2, Constants.getaAccount(userId));
preparedStatement.executeUpdate();
}
}
And below is my Constants class which I have made immutable.
public final class Constants {
public static String A_ACCOUNT;
public final static String INSERT_ORACLE_SQL = "INSERT INTO XMP_TEST"
+ "("
+ "ID, A_ACCOUNT) VALUES"
+ "(?, ?)";
public static String getaAccount(int userId) {
A_ACCOUNT = "{\"lv\":[{\"v\":{\"userId\":"+userId+"},\"cn\":1}]}";
return A_ACCOUNT;
}
}
Can anyone tell me what wrong I am doing here? I believe it's happening because of thread safety issue. Multiple threads modifying the userID integer I guess and that's why it is getting written wrongly to database.
How can I fix this problem?

The major problem I see is not with Task.userId, but rather with Constants.A_ACCOUNT: if two separate threads call getaAccount at the same time, then they'll both set Constants.A_ACCOUNT and both read it, so they can end up both having the same value, or each having the other's value, or whatnot. To fix this, you can use a local variable instead of a static field:
public static String getaAccount(int userId) {
final String ret = "{\"lv\":[{\"v\":{\"userId\":"+userId+"},\"cn\":1}]}";
return ret;
}
or just dispense with the variable:
public static String getaAccount(int userId) {
return "{\"lv\":[{\"v\":{\"userId\":"+userId+"},\"cn\":1}]}";
}
(You say that you've made Constants immutable, but that's not really true. Instances of Constants will be immutable, because they have no fields at all; but Constants itself has a publically modifiable field, so it's very mutable!)
More generally, you shouldn't be using fields for temporary values only needed within a specific method, and only during a single call to it. Even when it's not a synchronization problem, it's a maintenance problem. For example, Task does not need a volatile int userId; userId should just be a local variable inside its run method.
Also, I'd recommend wrapping your AtomicInteger in its own class, IncrementingCounter or something, that offers just one method, called (say) getNewId. Then getNewId will be the only class that has to deal with coordination between threads. All other classes can be made threadsafe by regular techniques (immutability, only existing within a single thread, etc.).

You're reading and modifying a static variable without any kind of synchronization from multiple threads: A_ACCOUNT. Just make it a local variable in getaacount() and everything should work as intended.

considering that the digits of a number is also a string, this problem was solved many years ago:
Add an auto-increment column to your parent table
Now inserting a parent record will give you a unique number
You don't say which database you're using, but every DB has a way of retrieving the value of the auto increment just inserted, so retrieve that and use it as your unique value.
If you can't change the data type if your key, also copy the auto increment value into your string column. It doesn't matter that it's numerical - it's still unique.

Java Server Client, shared variable between threads

I am working on a project to create a simple auction server that multiple clients connect to. The server class implements Runnable and so creates a new thread for each client that connects.
I am trying to have the current highest bid stored in a variable that can be seen by each client. I found answers saying to use AtomicInteger, but when I used it with methods such as atomicVariable.intValue() I got null pointer exception errors.
What ways can I manipulate the AtomicInteger without getting this error or is there an other way to have a shared variable that is relatively simple?
Any help would be appreciated, thanks.
Update
I have the AtomicInteger working. The problem is now that only the most recent client to connect to the server seems to be able to interact with it. The other client just sort of freeze.
Would I be correct in saying this is a problem with locking?

Well, most likely you forgot to initialize it:
private final AtomicInteger highestBid = new AtomicInteger();
However working with highestBid requires a great deal of knowledge to get it right without any locking. For example if you want to update it with new highest bid:
public boolean saveIfHighest(int bid) {
int currentBid = highestBid.get();
while (currentBid < bid) {
if (highestBid.compareAndSet(currentBid, bid)) {
return true;
}
currentBid = highestBid.get();
}
return false;
}
or in a more compact way:
for(int currentBid = highestBid.get(); currentBid < bid; currentBid = highestBid.get()) {
if (highestBid.compareAndSet(currentBid, bid)) {
return true;
}
}
return false;
You might wonder, why is it so hard? Image two threads (requests) biding at the same time. Current highest bid is 10. One is biding 11, another 12. Both threads compare current highestBid and realize they are bigger. Now the second thread happens to be first and update it to 12. Unfortunately the first request now steps in and revert it to 11 (because it already checked the condition).
This is a typical race condition that you can avoid either by explicit synchronization or by using atomic variables with implicit compare-and-set low-level support.
Seeing the complexity introduced by much more performant lock-free atomic integer you might want to restore to classic synchronization:
public synchronized boolean saveIfHighest(int bid) {
if (highestBid < bid) {
highestBid = bid;
return true;
} else {
return false;
}
}

I wouldn't look at the problem like that. I would simply store all the bids in a ConcurrentSkipListSet, which is a thread-safe SortedSet. With the correct implementation of compareTo(), which determines the ordering, the first element of the Set will automatically be the highest bid.
Here's some sample code:
public class Bid implements Comparable<Bid> {
String user;
int amountInCents;
Date created;
#Override
public int compareTo(Bid o) {
if (amountInCents == o.amountInCents) {
return created.compareTo(created); // earlier bids sort first
}
return o.amountInCents - amountInCents; // larger bids sort first
}
}
public class Auction {
private SortedSet<Bid> bids = new ConcurrentSkipListSet<Bid>();
public Bid getHighestBid() {
return bids.isEmpty() ? null : bids.first();
}
public void addBid(Bid bid) {
bids.add(bid);
}
}
Doing this has the following advantages:
Automatically provides a bidding history
Allows a simple way to save any other bid info you need
You could also consider this method:
/**
* #param bid
* #return true if the bid was successful
*/
public boolean makeBid(Bid bid) {
if (bids.isEmpty()) {
bids.add(bid);
return true;
}
if (bid.compareTo(bids.first()) <= 0) {
return false;
}
bids.add(bid);
return true;
}

Using an AtomicInteger is fine, provided you initialise it as Tomasz has suggested.
What you might like to think about, however, is whether all you will literally ever need to store is just the highest bid as an integer. Will you never need to store associated information, such as the bidding time, user ID of the bidder etc? Because if at a later stage you do, you'll have to start undoing your AtomicInteger code and replacing it.
I would be tempted from the outset to set things up to store arbitrary information associated with the bid. For example, you can define a "Bid" class with the relevant field(s). Then on each bid, use an AtomicReference to store an instance of "Bid" with the relevant information. To be thread-safe, make all the fields on your Bid class final.
You could also consider using an explicit Lock (e.g. see the ReentrantLock class) to control access to the highest bid. As Tomasz mentions, even with an AtomicInteger (or AtomicReference: the logic is essentially the same) you need to be a little careful about how you access it. The atomic classes are really designed for cases where they are very frequently accessed (as in thousands of times per second, not every few minutes as on a typical auction site). They won't really give you any performance benefit here, and an explicit Lock object might be more intuitive to program with.

Acquiring multiple locks atomically in java

I have the following code:
Note: I simplified the code as much as possible for readability.
If I forgot any critical pieces let me know.
public class User(){
private Relations relations;
public User(){
relations = new Relations(this);
}
public getRelations(){
return relations;
}
}
public class Relations(){
private User user;
public Relations(User user){
this.user = user;
}
public synchronized void setRelation(User user2){
Relations relations2 = user2.getRelations();
synchronized(relations2){
storeRelation(user2);
if(!relations2.hasRelation(user))
relations2.setRelation(user);
}
}
public synchronized boolean hasRelation(User user2){
... // Checks if this relation is present in some kind of collection
}
/*Store this relation, unless it is already present*/
private void storeRelation(User user2){
... // Stores this relation in some kind of collection
}
}
This implementation should make sure that for all Relations x, y with:
x.user = u_x
y.user = u_y
the following invariant holds:
x.hasRelation( u_y ) <=> y.hasRelation( u_x )
I believe that holds for the code stated above?
Note: It does of course not hold during the execution of setRelation(..),
but at that moment the locks for both relations involved are
held by the executing thread so no other thread can read the
hasRelation(..) of one of the relations involved.
Assuming that this holds i believe there is still a potential deadlock-risk.
Is that correct? And if it is, how can I solve it?
I think i would need to obtain both locks in setRelation(..) atomically somehow.

You are correct on both points: your invariant does hold (assuming that I understand correctly what your method-names mean and so on, and assuming that by if(!relations.hasRelation(user)) relations2.setRelation(user2); you meant to write if(!relations2.hasRelation(user)) relations2.setRelation(user);), but you do have the risk of a deadlock: if one thread needs to obtain a lock on x and then on y, and another thread needs to obtain a lock on y and then on x, then there's a risk that each thread will succeed in getting its first lock, and thereby prevent the other from getting its second lock.
One solution is to enforce a strict universal ordering for getting locks on Relations instances. What you do is, you add a constant integer field lockOrder:
private final int lockOrder;
and a static integer field currentLockOrder:
private static int currentLockOrder = 0;
and every time you create a Relations instance, you set its lockOrder to the current value of currentLockOrder, and increment said:
public Relations()
{
synchronized(Relations.class) // a lock on currentLockOrder
{
lockOrder = currentLockOrder;
++currentLockOrder;
}
}
such that every instance of Relations will have a distinct, immutable value for lockOrder. Your setRelation method would then obtain locks in the specified order:
public void setRelation(final User thatUser)
{
final Relations that = thatUser.getRelations();
synchronized(lockOrder < that.lockOrder ? this : that)
{
synchronized(lockOrder < that.lockOrder ? that : this)
{
storeRelation(thatUser);
if(! that.hasRelation(user))
that.storeRelation(user);
}
}
}
thereby ensuring that if two threads both need to get locks on both x and y, then either they'll both first get locks on x, or they'll both first get locks on y. Either way, no deadlock will occur.
Note, by the way, that I changed setRelation to storeRelation. setRelation would work, but why add that complexity?
Also, there's still one thing I don't get: how come x.setRelation(u_y) calls x.storeRelation(u_y) unconditionally, but calls y.setRelation(u_x) (or y.storeRelation(u_x)) only if y doesn't already have the relationship? It doesn't make sense. It seems like either both checks are needed, or neither check is. (Without seeing the implementation of Relations.storeRelation(...), I can't guess which of those is the case.)