We Keep Coding

Position Autoincrement in Talend

So i a bit lost and don t really know how to hang up this one...
Consider that i have a 2 DB table in Talend, let say firstly
A table invoices_only which has as fields, the invoiceNummer and the authors like this
Then, a table invoices_table with the field (invoiceNummer, article, quantity and price) and for one invoice, I can have many articles, for example
and through a tmap want to obtain a table invoice_table_result, with new columns, one for the article position, an one other for the total price. for the position i know that i can use something like the Numeric.sequence("s1",1,1) function, but don t know how to restart my counter when a new invoices nummer is found, and of course for the total price it is just a basic multiplication
so my result should be some thing like this
Here is a draft of my talend job, i m doing a lookup on the invoicenummer between the table invoice_only and invoices
Any Advices? thanks.

A trick I use is to do the sequence like this:
Numeric.sequence("s" + row.InvoiceNummer, 1, 1)
This way, the sequence gets incremented while you're still on the same InvoiceNummer, and a new one is started whenever a new InvoiceNummer is found.

There are two ways to achieve it,
tJavaFlex
Sql
tJavaFlex
You can compare current data with the previous data and reset the sequence value using below function,
if () {
Numeric.resetSequence(seqName, startValue);
}
Sql
Once data is loaded into the tables, create a post job and use an update query to update the records. You have to select the records and take the rank of the values. On top of the select you have to perform the update.
select invoicenumber, row_number() over(partition by invoicenumber, order by invoicenumber) from table name where -- conditions if any.
Update statements vary with respect to the database, please provide which database are you using, so that can provide the update query.
I would recommend you to achieve this through Sql

Get identity after Instead of insert trigger

I am using Hibernate with MSSQL server writing the software that integrates with an existing database. There is an instead of insert trigger on the table that I need to insert into and it messes up ##Identity, which means on Hibernate's save I can't get the id of inserted row. I can't control the trigger (can't modify it). I saw this question, but it involves procedures, which my trigger does not have, so I thought my question is different enough. I can't post the whole trigger, but hopefully I can post enough to get the point across:
CREATE TRIGGER TrigName ON TableName
INSTEAD OF INSERT
AS
SET XACT_ABORT ON
BEGIN TRANSACTION
-- several DECLARE, SET statements
-- a couple of inserts into other tables for business logic
-- plain T-SQL statements without procedures or functions
...
-- this is the actual insert that i need to perform
-- to be honest, I don't quite understand how INSERTED table
-- was filled with all necessary columns by this point, but for now
-- I accept it as is (I am no SQL pro...)
INSERT INTO ClientTable (<columns>)
SELECT <same columns> from INSERTED
-- a couple of UPDATE queries to unrelated tables
...
COMMIT TRANSACTION;
I was wondering if there is a reliable way to get the id of the row being inserted? One solution I thought of and tried to make is to install an on insert trigger on the same table that writes the newly inserted row into a new table I added to the db. I'd use that table as a queue. After transaction commit in Hibernate I could go into that table and run a select with the info I just inserted (I still have access to it from the same method scope), and I can get the id and finally remove that row. This is a bulky solution, but best I can come up with so far.
Would really appreciate some help. I can't modify existing triggers and procedures, but I can add something to the db if it absolutely does not affect existing logic (like that new table and a on insert trigger).
To sum up: I need to find a way to get the ID of the row I just inserted with Hibernate's save call. Because of that instead of insert trigger, hibernate always returns identity=0. I need to find a way to get that ID because I need to do the insert in a few other tables during one transaction.

I think I found an answer for my question. To reply to #SeanLange's comment: I can't actually edit insert code - it's done by another application and inquiry to change that will take too long (or won't happen - it's a legacy application). What I did is insert another trigger on insert on the same table. Since I know the order of operations in the existing instead of insert trigger I can see that the last insert operation will be in the table I want so that means my on insert trigger will fire right after that. In the scope of that trigger I have access to inserted table out of which I pull out the id.
CREATE TRIGGER Client_OnInsert ON myClientTable
FOR INSERT
AS
BEGIN
DECLARE #ID int;
SET #ID = (select ClientID from inserted);
INSERT INTO ModClient (modClientId)
OUTPUT #ID
VALUES (#ID);
END
GO
Then in Hibernate (since I can't use save() anymore), I use a NativeQuery to do this insert. I set parameters and run the list() method of NativeQuery, which returns a List where the first and only argument is the id I want.
This is a bulky way, I know. If there is anything that's really bad that will stand out to people - please let me know. I would really appreciate some feedback on this. However, I wanted to post this answer as a potential answer that worked so far, but it does not mean it's very good. For this solution to work I did have to create another small table ModClient, which I will have to use as a temp id storage for this exact purpose.

Cassandra Updates not working properly

BoundStatement UpdateTable = new BoundStatement(preparedStatement);
UpdateTable.bind(productId, productname, time);
session.execute(UpdateTable);
I am using the following commands to update cassandra tables.Sometimes it updates and sometimes it doesn't.
UPDATE product SET count = count + 1 where productId = ? AND productname = ? AND time = ?;
It never throws an error.
Why is this ?
EDIT
Table structure
CREATE TABLE IF NOT EXISTS product(productId int,productname text,time timestamp , count counter,PRIMARY KEY (productid,productname,time));

By looking at your (Java?) code, I can't really tell what kind of object insertUpdateTable is. But the bind method should return a BoundStatement object that can be executed. And while UpdateTable is indeed a BoundStatement, I don't see that you're actually binding your variables to it.
Based on the limited amount of code shown, I see two solutions here:
Call the bind method on UpdateTable inside your session.execute:
session.execute(UpdateTable.bind(productId, productname, time));
Wrap your insertUpdateTable.bind inside a session.execute:
session.execute(insertUpdateTable.bind(productId, productname, time));
Check out the DataStax documentation on Using Bound Statements with the Java driver for more information.
Sometimes it updates and sometimes it doesn't.
If you had posted your Cassandra table definition, it might shed some more light on this. But it is important to remember that Cassandra PRIMARY KEYs are unique, and that INSERTs and UPDATEs are essentially the same (an INSERT can "update" existing values and an UPDATE can "insert" new values). Sometimes an UPDATE may appear to not work, when it may be performing a write with the same key values. Just something to look out for.
Also important to note, is that UPDATE product SET count = count + 1 will only work under two conditions:
count is a counter column.
product is a counter table, consisting of only keys and counter columns (all non-counter columns must be a part of the PRIMARY KEY).
Worth noting is that counter columns underwent a big change/improvement with Cassandra 2.1. If you need to use counters and are still on Cassandra 2.0, it may be worth upgrading.

You said the update sometimes works, but note that if you ever delete row with a counter, you'll be unable to modify the row again without dropping the table and recreating it. The update will appear to fail silently. For more, see CASSANDRA-8491

I had a similar issue during high frequency writes & updates.
As the number of concurrent requests goes up, there is good chance that the latest bind may over write the previous bound params. So instead of using single boundStatement, used preparedStatement.bind() in the session.execute
Can you try the following.
Instead of using :
UpdateTable.bind(productId, productname, time);
session.execute(UpdateTable);
Use :
session.execute(preparedStatement.bind(productId, productname, time));

Table rows seem to be disapearing

I have a ton of raw html files that I'm parsing and inserting to a MySQL database via a connection in Java.
I'm using "REPLACE INTO" statements and this method:
public void migrate(SomeThread thread) throws Exception{
PreparedStatement threadStatement = SQL.prepareStatement(threadQuery);
thread.prepareThreadStatement(threadStatement);
threadStatement.executeUpdate();
threadStatement.close();
for(SomeThread.Post P : thread.threadPosts){
PreparedStatement postStatement = SQL.prepareStatement(postQuery);
P.preparePostStatement(postStatement);
postStatement.executeUpdate();
postStatement.close();
}
}
I am running 3 separate instances of my program each in its own command prompt, with their own separate directory of htmls to parse and commit.
I'm using HeidiSQL to monitor the database and a funny thing is happening where I'll see that I have 500,000 rows in a table at one point for example, then I'll close HeidiSQL and check back later to find that I now have 440,000 rows. The same thing occurs for the two tables that I'm using.
Both of my tables use a primary key called "id", each of their ID's have their own domain but it's possible their values overlap and are overwriting each other? I'm not sure if this could be an issue because I'd think SQL would differentiate between the table's "local" id values.
Otherwise I was thinking it could be that since I'm running 3 separate instances that each have their connection to the DB, some kind of magic is happening where right as one row is being committed, the execution swaps to another commit statement, displaces the table, then back to the first commit and then some more magic that causes the database to roll back the number of rows collected.
I'm pretty new to SQL so I'm not too sure where to start, if somebody has an idea about what the heck is going on and could point me in the right direction I'd really appreciate it.
Thanks

You might want to use INSERT INTO instead of REPLACE INTO.
Data doesn't disappear.
Here are some tips:
Do you have another thread running that actually deletes entries?
Do other people have access to the database?
Not sure what HeidiSQL may do. To exclude that possibility maybe use MySQL Workbench instead.

Yeah now that I run a COUNT(*) query against my tables I see that all my rows are in fact there.
Most likely the heidiSQL summary page is just a very rough estimate.
Thanks for the suggestion to use workbench pete, I will try it and see if it is better than Heidi as Heidi is freezing up on me on a regular basis.

how to suppress mysql error java

I'm trying to automate the addition of links from an RSS feed to a mySQL table. I want to take input from the feed every hour, but there are usually links that I have already grabbed still present on the RSS page when I reference it again.
I've got java code that works the first time, but when I try to add a duplicate file to the sql table I get an exception. I thought that mySQL would just ignore and pass over the duplicates, as I'm using the MUL key on the field name in question, but I get an exception instead.
Any ideas on how to get this rolling? I don't want duplicates, and I don't want duplicates to stop other new things from being added.
Thanks!

You can use REPLACE INTO instead of INSERT INTO:
REPLACE works exactly like INSERT, except that if an old row in the
table has the same value as a new row for a PRIMARY KEY or a UNIQUE
index, the old row is deleted before the new row is inserted.
http://dev.mysql.com/doc/refman/5.0/en/replace.html

You can also use INSERT IGNORE INTO ... if you want to keep the old value instead of replacing it.