I have several databases ,and need exchange data between them. When I export from db A import into db B, Id confliction will happen. I think out two approach, no one satisfy me.
select max(id) then create new id to avoid confliction ,but one column store json structure and contains id too! (history reason). So I need create new Id (primary key) and modify all ids in that json column.
or I can add a batch info for each data import. When I import data, I find out every id in sql and add batch id before them. Such as:
The original db like:
ID COL_JSON
11 {id:11,name:xx ...}
I want to insert a new record :11 ,after insert I add a batch info "1000" before id
now db looks like
ID COL_JSON
11 {id:11,name:xx ...}
100011 {id:100011,name:xx ...}
the next batch will be 1001,1002 1003 ..., so if a new 11 record need to be insert the db looks like
ID COL_JSON
11 {id:11,name:xx ...}
100011 {id:100011,name:xx ...}
100111 {id:100111,name:xx ...}
Although the two approach can resolve conflict, I feel the two approach is stupid. Is there some graceful scheme?
For legacy system, there is no better approach except to align with it. We cannot change it more on legacy system, so your 2th approach seems good. Frankly, tt's not stupid and just the right way to go.
I don't understand exactly what your databases exchange.
If you need data from both databases in both of them you could use something similar to your batch but using characters - A11 and B11 ids.
This way you won't have conflicts even if your database grows a lot.
Edit: You could make also a primary key with two fields: the integer ID with autoincrement and a varchar for the original database name.
When I have a table that should be synchronized (not on the fly), I use this approach :
The main table that will be overwritten has a big autoincrement (ie : autoincrement = 100000)
The secondary table that will be merged to the main one has a normal autoincrement starting to 1.
The only requirement is that you ensure that the main table has a big enough autocrement set up, so the secondary table id will never reach the main table first ID
Related
Is it possible to have autoincrementing id among several tables? What I mean exactly - I have (let's say five) tables, one of them is a table containing information about sales (sale_id, sold_item_id) and another four contain info about different kind of sold stuff. I want these four to share one pool of ids. How do I do that?
Edit.
I decided to choose Juxhin solution and I created additional table. Everytime I create a record in one of these 4 tables, I autoincrement new id in that additional table and this id is in one of columns of that new row.
This sounds like the use case for a sequence and the link seems to indicate that javadb supports this.
So you create one common sequence for all tables:
CREATE SEQUENCE MASEQUENCE
and then use it when inserting into your tables:
INSERT INTO TAB1(ID,....) VALUES(NEXT VALUE FOR MYSEQUENCE,...)
Each NEXT VALUE will advance the sequence and so all ids will be unique across all tables.
If you want a new record to be inserted into all 5 tables when you insert something into one of them, then you can create a trigger for this.
It may also be helpful to create foreign keys on the id columns in the other tables (to keep the tables in sync).
I have a table named GROUPS which has a column GROUP_ID whose values are
GRP001, GRP002, GRP003
and so on. Now every time I insert a new row I have to insert it with a
(current) GROUP_ID= (highest)GROUP_ID+ 1 , for example if highest GROUP_ID= GRP003 I have to generate a new GROUP_ID GRP004 when I insert a new row.
How can I do this using java?
I am currently using Hibernate along with Struts 2 in my program
Is there any way to deal with this using hibernate? Or will I have to write additional code to lock the table, check the db for max Id (and then increment it) and finally release the lock?
I remember solving a problem similar to this once. What I did was I create a custom primary key generator as supported by hibernate.
This guy explains it clearly here: "Custom Hibernate Primary Key Generator"
Basically you just need to implement org.hibernate.id.IdentifierGenerator and all should be set.
Just be aware that the solution implemented in the example above is database dependent. But I think sometimes common sense should prevail over overengineering.
Here is the problem that i am currently stuck with for the past few days. And I am looking for guidance / approaches on how to handle.Hints & suggestions welcome.
so here is the problem.The database has a table "group" which has two columns : group_id on parent_group_id.group_id is the primary key for the table .All entries in this table represent groups/sub-groups.If one adds a sub-group from the front end ,then an entry gets inserted in to the group table with an auto-generated group_id which MySQL generates.the parent_group_id corresponds to the group_id of the group on which a sub-group was added.So in essence it's acting like a foreign key to the group_id column.My task cut out here is to generate an XML in java using the data from the group table. So this is where i am stuck.I know it's gonna be a recursive function which needs to be written but cant figure out a way how to dynamically create the nodes and fill the data from the Db at the same time.The end XML needs to be sent as json data to the front end.
A group can have n-sub groups and the hierarchy can go on.For ex- Say Vehicle is root node with group_id =1.It can have cars & bikes as sub-groups.so the parent_group_id will be 1 for car and bike and group id say will be 2& 3 respectively.
P.S: this is the first time i am posting here having had used this site for the past one year.Please let me know if any more info is needed or whether you are able to comprehend my problem.
If you split the task into two, it will be more manageable.
Here are some useful links on querying hierarchical data in relational databases and specifically in MySQL:
What are the options for storing hierarchical data in a relational database?
http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/
http://www.slideshare.net/billkarwin/models-for-hierarchical-data
http://en.wikipedia.org/wiki/Common_table_expressions#Common_table_expression
As long as you have the query result properly sorted, you will be able to traverse it recursively, building the XML tree step by step.
Was able to solve it by using recursive functions :). Loaded all the data using the entity class and then iterated over the data using recursive functions to build the tree like structure.I didn't try and take the sql way.
Hello and happy new year for everyone.
I need to insert a record at the end of a table (the table has not set autoincrement) using JPA.
I know I could get the last id (integer) and apply to the entity before insert, but how could that be done? Which way would be most effective?
There is no such thing as "the end of the table". Rows in a relational table are not sorted.
Simply insert your new row. If you need any particular order, you need to apply an ORDER BY when selecting the rows from the table.
If you are talking about generating a new ID, then use an Oracle sequence. It guarantees uniqueness.
I would not recommend using a "counter table".
That solution is either not scalable (if it's correctly implemented) or not safe (if it's scalable).
That's what sequences were created for. I don't know JPA, but if you can't get the ID from a sequence then I suggest you find a better ORM.
Well, while i do not know where the end of a table really is, JPA has a lot of options for plugging in ID generators.
One common option is to use a table of its own, having a counter for each entity you need an ID for (from http://download.oracle.com/docs/cd/B32110_01/web.1013/b28221/cmp30cfg001.htm).
#Id(generate=TABLE, generator="ADDRESS_TABLE_GENERATOR")
#TableGenerator(
name="ADDRESS_TABLE_GENERATOR",
tableName="EMPLOYEE_GENERATOR_TABLE",
pkColumnValue="ADDRESS_SEQ"
)
#Column(name="ADDRESS_ID")
public Integer getId() {
return id;
}
...other "Generator" strategies to be googled...
EDIT
I dare to reference #a_horse_with_no_name as he says he does not know about JPA. If you want to use native mechanisms like sequence (that are not available in every DB) you can declare such a generator in JPA, too.
I do not know what issues he encountered with the table approach - i know large installations running this successfully. But anyway, this depends on a lot of factors besides scalability, for example if you want this to be portable etc. Just lookup the different strategies and select the appropriate.
I have a webservice in java that receives a list of information to be inserted or updated in a database. I don't know which one is to insert or update.
Which one is the best approach to abtain better performance results:
Iterate over the list(a object list, with the table pk on it), try to insert the entry on Database. If the insert failed, run a update
Try to load the entry from database. if the results retrieved update, if not insert the entry.
another option? tell me about it :)
In first calls, i believe that most of the entries will be new bd entries, but there will be a saturation point that most of the entries will be to update.
I'm talking about a DB table that could reach over 100 million entries in a mature form.
What will be your approach? Performance is my most important goal.
If your database supports MERGE, I would have thought that was most efficient (and treats all the data as a single set).
See:
http://www.oracle.com/technology/products/oracle9i/daily/Aug24.html
https://web.archive.org/web/1/http://blogs.techrepublic%2ecom%2ecom/datacenter/?p=194
If performance is your goal then first get rid of the word iterate from your vocabulary! learn to do things in sets.
If you need to update or insert, always do the update first. Otherwise it is easy to find yourself updating the record you just inserted by accident. If you are doing this it helps to have an identifier you can look at to see if the record exists. If the identifier exists, then do the update otherwise do the insert.
The important thing is to understand the balance or ratio between the number of inserts versus the number of updates on the list you receive. IMHO you should implement an abstract strategy that says "persists this on database". Then create concrete strategies that (for example):
checks for primary key, if zero records are found does the insert, else updates
Does the update and, if fails, does the insert.
others
And then pull the strategy to use (the class fully qualified name for example) from a configuration file. This way you can switch from one strategy to another easily. If it is feasible, could be depending on your domain, you can put an heuristic that selects the best strategy based on the input entities on the set.
MySQL supports this:
INSERT INTO foo
SET bar='baz', howmanybars=1
ON DUPLICATE KEY UPDATE howmanybars=howmanybars+1
Option 2 is not going to be the most efficient. The database will already be making this check for you when you do the actual insert or update in order to enforce the primary key. By making this check yourself you are incurring the overhead of a table lookup twice as well as an extra round trip from your Java code. Choose which case is the most likely and code optimistically.
Expanding on option 1, you can use a stored procedure to handle the insert/update. This example with PostgreSQL syntax assumes the insert is the normal case.
CREATE FUNCTION insert_or_update(_id INTEGER, _col1 INTEGER) RETURNS void
AS $$
BEGIN
INSERT INTO
my_table (id, col1)
SELECT
_id, _col1;
EXCEPTION WHEN unique_violation THEN
UPDATE
my_table
SET
col1 = _col1
WHERE
id = _id;
END;
END;
$$
LANGUAGE plpgsql;
You could also make the update the normal case and then check the number of rows affected by the update statement to determine if the row is actually new and you need to do an insert.
As alluded to in some other answers, the most efficient way to handle this operation is in one batch:
Take all of the rows passed to the web service and bulk insert them into a temporary table
Update rows in the mater table from the temp table
Insert new rows in the master table from the temp table
Dispose of the temp table
The type of temporary table to use and most efficient way to manage it will depend on the database you are using.