We Keep Coding

Position Autoincrement in Talend

So i a bit lost and don t really know how to hang up this one...
Consider that i have a 2 DB table in Talend, let say firstly
A table invoices_only which has as fields, the invoiceNummer and the authors like this
Then, a table invoices_table with the field (invoiceNummer, article, quantity and price) and for one invoice, I can have many articles, for example
and through a tmap want to obtain a table invoice_table_result, with new columns, one for the article position, an one other for the total price. for the position i know that i can use something like the Numeric.sequence("s1",1,1) function, but don t know how to restart my counter when a new invoices nummer is found, and of course for the total price it is just a basic multiplication
so my result should be some thing like this
Here is a draft of my talend job, i m doing a lookup on the invoicenummer between the table invoice_only and invoices
Any Advices? thanks.

A trick I use is to do the sequence like this:
Numeric.sequence("s" + row.InvoiceNummer, 1, 1)
This way, the sequence gets incremented while you're still on the same InvoiceNummer, and a new one is started whenever a new InvoiceNummer is found.

There are two ways to achieve it,
tJavaFlex
Sql
tJavaFlex
You can compare current data with the previous data and reset the sequence value using below function,
if () {
Numeric.resetSequence(seqName, startValue);
}
Sql
Once data is loaded into the tables, create a post job and use an update query to update the records. You have to select the records and take the rank of the values. On top of the select you have to perform the update.
select invoicenumber, row_number() over(partition by invoicenumber, order by invoicenumber) from table name where -- conditions if any.
Update statements vary with respect to the database, please provide which database are you using, so that can provide the update query.
I would recommend you to achieve this through Sql

How to Fetch Last 10 database Transaction In IBM DB2..?

I would like to fetch last 10 database transaction in IBM DB2..
Means Which Last 10 transaction execute In DB2..

Depending on what you need that for, you will have to set up the DB2 audit facility or use an activity event monitor.

SQL tables have no implicit ordering, the order has to come from the
data. Perhaps you should add a field to your table (e.g. an int
counter) and re-import the data.
If you cannot do so, then here is one more idea which is coming to my mind, while writing this answer. Can we use rownum to get the last 10 records? Perhaps yes, here is what you can try, i am just throwing this idea and have not tested.
Get the MAX(rownum) from the table
Fetch the records from the table between the max(rownum) to max(rownum) -10
Aghh sounds ugly but see if it works for u.
Btw if you don't know about rowid then here is link to learn about that:
http://pic.dhe.ibm.com/infocenter/db2luw/v9r7/index.jsp?topic=%2Fcom.ibm.db2.luw.apdv.porting.doc%2Fdoc%2Fr0052875.html

If there is a column in your table that you can use to ascertain the correct order, such h as a transaction number or a value generated by a sequence reference, or some column(s) that you can use to ORDER BY, then simply add DESCENDING after each column in the ORDER BY clause, and FETCH FIRST 10 ROWS.

Hibernate column "user1_.user_id" must appear in the GROUP BY clause or be used in an aggregate function

I'm trying to write a simple query in hibernate where I just get a descending list of users with the most points. Basically I'm trying to write this query select user_id from user_points group by user_id order by sum(points) desc. This works on the Postgres db I'm working on.
I have the following HSQL query select c.user from UserPoints c group by c.user order by sum(c.points) desc. The Points table has a column called user_id which refers to the user table. In our mapping user_id is mapped to a User class.
However when I run this query I get column "user1_.user_id" must appear in the GROUP BY clause or be used in an aggregate function. I tried searching for similar issues but only really came up with this: JPA Query/Hibernate: Column must appear in the GROUP BY clause or be used in an aggregate function which is unanswered.
I'm kind of stuck with HQL because the other thing I need to do is join on a table that doesn't have a mapped association (but also has a user_id key). I'm also trying to keep this query as fast as possible. Any help would be really appreciated as I'm really confused as to why this isn't working.

So I think I may have found what the issue was. We actually have two mappings to the user_id column. One is a property where we have the user_id as just a regular long and the other many to one mapping where we map it to the user class. I set the logging level to debug so I can see the query that hibernate was creating and it seems like it was using turning the select c.user to all the columns in the user table (which means it was using the many to one mapping correctly), but for some reason in the group by clause it was converting c.user to the user_id property. Since I can't change the mapping and I'm on a tight deadline I've opted to just write a regular SQL query to grab the user ids and then run another criteria query to get all the users in that list of ids.

How to deal with pagination for a large, non materializable resultset in Oracle and Java

I'm dealing with some sort of problem here. A web application built on java stuff calls a stored procedure in oracle which has as out parameters some varchars and a parameter whose type is a ref cursor returning a record type (both explicitly defined).
The content of the ref cursor is gathered using a complex query which I guess runs O(n) depending on the number of records in a table.
The idea is to paginate the result in the server because getting all the data causes a long delay (500 records take about 40-50 seconds due to the calculation and the join resolution). I've already rebuilt the query using row_number()
open out_rcvar for
SELECT *
FROM ( select a, b, c,..., row_number() over (order by f, g) rn
from t1, t2,...
where some_conditions
) where rn between initial_row and final_row
order by rn;
in order to avoid the offset-limit approach (and its equivalence in oracle). But, here's the catch, the user wants a pagination menu like
[first || <<5previous || 1 2 3 4 5 || next5>> || last ]
and knowing the total rows implies counting (hence, "querying") the whole package and taking the whole 50secs. What approach could I use here?
Thanks in advance for your help.
EDIT: The long query should not be setted a s a materialized view because the data in the records is required to be updated as it is requested (the web app does some operations with the data and needs to know if the selected item is "available" or "sold")

You could do something like:
SELECT *
FROM ( select count(*),a, b, c,..., row_number() over (order by f, g) rn
from t1, t2,...
where some_conditions
) where rn between initial_row and final_row
order by rn;
This is probably inefficient given your description, but if you find some quicker way to calculate the total rows, you could stick it the inner select, and return it with every row. It's not great, but it works and it's a single select (as opposed to having one for the total row number and a second one for the actual rows).

What is the performance if you do not select any columns but just a count to determine the rows? Is that acceptable?
And use that as a guide to build the pagination.
Otherwise we have no option without knowing the count to build the number of pages as (1,2,3,45)
The other option is to not show the number of pages, but just show next and previous.
Just my thoughts.

Perhaps you might consider creating temporary table. You might store your results there and then use some paging mechanism. This way the computation will be done once. Then you will only select the data, which will be pretty fast.
There is one catch, in this approach. You have to ensure that you will not break session, since temporary tables are private and exists only for your session. Take a look at this link.

Simple question on database query

I have been asked in an interview to write a SQL query which fetches the first three records with highest value on some column from a table. I had written a query which fetched all the records with highest value, but didn't get how exactly i can get only first three records of those.
Could you help me in this.
Thanks.

SELECT TOP 3 * FROM Table ORDER BY FieldName DESC
From here, but might be a little out of date:
Postgresql:
SELECT * FROM Table ORDER BY FieldName DESC LIMIT 3
MS SQL Server:
SELECT TOP 3 * FROM Table ORDER BY FieldName DESC
mySQL:
SELECT * FROM Table ORDER BY FieldName DESC LIMIT 3

Select Top 3....

Depending on the database engine, either
select top 3 * from table order by column desc
or
select * from table order by column desc limit 3

The syntax for TOP 3 varies widely from database to database.
Unfortunately, you need to use those constructs for the best performance.
Libraries like Hibernate help here, because they can translate a common API into the various SQL dialects.
Since you are asking about Java, it is possible to just SELECT everything from the database (with an ORDER BY), but just fetch only the first three rows. Depending on how the query needs to be executed this might be good enough (especially if no sorting on the database has to happen thanks to appropriate indexes, for example when you sort by primary key fields).
But in general, you want to go with an SQL solution.

In oracle you can also use where rownum < 4...
Also on mysql there is a Limit keyword (i think)