We Keep Coding

Retrieve generated keys from multiple queries in single Spring JDBC Update

I am using a single Spring JDBC update to make an update to two tables in my Postgres database. My SQL query is as follows:
UPDATE accounts SET last_transaction_amount = :transaction_amount WHERE acct_num = :acct_num; INSERT INTO transactions (transaction_amout) VALUES (:transaction_amount);
Using NamedParameterJdbcTemplate#update, I have no issue executing this query and achieving the expected results.
The transactions table generates a sequential transaction identifier, and I want to return this to my application.
I've tried passing a GeneratedKeyHolder in the update call. This is returning the error "A result was returned when none was expected". Docs link.
I've tried passing a GeneratedKeyHolder and array of column names (new String[] {"transaction_id"}). This is returning the error that the column doesn't exist. Note this method call does work to return the transaction id when I only pass the INSERT query without the preceding UPDATE query. Docs link.
How can I retrieve the generated key? Thank you!

You seem to be looking for the RETURNING clause. Assuming that the serial number is called transaction_id:
INSERT INTO transactions (transaction_amout)
VALUES (:transaction_amount)
RETURNING transaction_id;

MongoWriteException when inserting into Mongodb with composite custom _id

I have a MongoDB remote server that I am using.
My KEY is a custom object that has other nested objects in it.
Simple inserting works fine, although if I try to run
collection.replaceOne(eq("_id", KEY), document, new UpdateOptions().upsert(true));
I get com.mongodb.MongoWriteException: After applying the update, the (immutable) field '_id' was found to have been altered to _id: .......
If I have only have primitives in the key it works fine. Of course the value of the KEY is not changed (traced all the way down).
Is this a Mongo Java Driver bug with the ReplaceOne function?

As it turns out for Mongo filters, the order of json properties matter. With debugging it is possible to see what the actual orders of the properties in the filters are and then you can set your model properties order with #JsonPropertyOrder("att1", att2") so they match in order.
It was confirmed by a member of Mongo team.
Mongo ticket-> https://jira.mongodb.org/browse/JAVA-3392

manage concurrency access mongoDB

i need to manage concurrency access for data updates in mongoDB.
Example: two users a and b connect to my application. The user a has updated a data and the user b wants to update the same data that the user a has already updated, so i want the user b cannot update this data because is has already updated by the user a.

if user A and user B only update one document and your can know the initial value and updated value try this code：
The code try to update the secret field，and we know the inital value is expertSecret
public void compareAndSet(String expertSecret, String targetSecret) {
// get a mongodb collection
MongoCollection<Document> collection = client.getDatabase(DATABASE).getCollection(COLLECTION);
BasicDBObject filter = new BasicDBObject();
filter.append("secret", expertSecret);
BasicDBObject update = new BasicDBObject();
update.append("secret", targetSecret);
collection.updateOne(filter, update);
}
If don't know initial value,how to do?
you can add a filed to representative operation,and before update check the filed.
If need to update more than one document,how to do?
Multi-document transactions need mongo server to support,get more information from here
However, for situations that require atomicity for updates to multiple documents or consistency between reads to multiple documents, MongoDB provides the ability to perform multi-document transactions against replica sets. Multi-document transactions can be used across multiple operations, collections, databases, and documents. Multi-document transactions provide an “all-or-nothing” proposition.

Spark join/groupby datasets take a lot time

I have 2 datasets(tables) with 35kk+ rows.
I try to join(or group by) this datasets by some id. (in common it will be one-to-one)
But this operation takes a lot time: 25+ h.
Filters only works fine: ~20 mins.
Env: emr-5.3.1
Hadoop distribution:Amazon
Applications:Ganglia 3.7.2, Spark 2.1.0, Zeppelin 0.6.2
Instance type: m3.xlarge
Code (groupBy):
Dataset<Row> dataset = ...
...
.groupBy("id")
.agg(functions.min("date"))
.withColumnRenamed("min(date)", "minDate")
Code (join):
...
.join(dataset2, dataset.col("id").equalTo(dataset2.col("id")))
Also I found this message in EMR logs:
HashAggregateExec: spark.sql.codegen.aggregate.map.twolevel.enable is set to true, but current version of codegened fast hashmap does not support this aggregate.

There Might be a possibility of Data getting Skewed. We faced this. Check your joining column. This happens mostly if your joining column has NULLS
Check Data Stored pattern with :
select joining_column, count(joining_col) from <tablename>
group by joining_col
This will give you an idea that whether the data in your joining column is Evenly distributed

MongoDB: insert documents with specific id instead of auto generated ObjectID

I need to insert documents on MongoDB (with specific id instead of auto generated ObjectID) using java..
To insert one document or update if exist, I tried use findOne to search for the id, if it doesn't exist then insert the id and then findAndModify. It works but I don't feel that it's efficient way, it's time consuming. Is there a better way to achieve that?
To insert multiple documents at once,I'm following this solution. but I don't know how I can insert my custom Id instead of objectID?
Any help will be appreciated

For your first problem MongoDB has upsert so
db.collection.update(
{query for id},
{document},
{upsert: true}
)
or in the Java driver
yourCollection.update(searchObject, modifiedObject, true, false);
If you want to set a custom ID you just set the _id key manually i.e.
yourBasicDBObject.put("_id",yourCustomId)
you just have to ensure it is unique for each document.
You will also need to set the _id in your modifiedObject otherwise a new one will be generated.
As for the bulk operations, just setting a custom ID for each document by giving the _id key should also work.

Try this #ebt_dev:
db.collection("collectionname").insertOne(data, { forceServerObjectId: false })