getting data from result set is too slow - java

Fetching data from PostgreSQL database with Result Set is too slow.
Here is my code.
for (int i = 0; i < qry_list.size(); i++) {
try {
resultSet = statement.executeQuery(qry_list.get(i));
resultSet.setFetchSize(0);
while (resultSet.next()) {
totalfilecrated = totalfilecrated
+ resultSet.getInt(columnname);
}
} catch (SQLException e) {
e.printStackTrace();
}
}
Here I try to fetch data inside a for loop.is it good?
Here is my query.
For getting ID of individual Organisations(org_unit_id).
"select org_unit_id from emd.emd_org_unit where org_unit_id
in(select org_unit_id from emd.emd_org_unit_detail where entity_type_id=1 and is_active=true) and
is_active=true order by org_unit_name_en";
Then i want to get the count of files with each org_unit_id
select count(*) as totalfilecreatedelectronic from fl_file ff
left join vw_employee_details_with_department epd on epd.post_detail_id=ff.file_opened_by_post_fk
where ff.file_nature = 'E' and ((ff.migration_date>='2011-01-01' and ff.migration_date<='2015-01-01') or
(ff.opening_date >='2011-01-01' and ff.opening_date <='2015-01-01')) and
epd.departmentid=org_unit_id";

Seeing how your second query already contains a reference to a column that's an org_unit_id, you might think joining emd_org_unit table in directly:
select org.org_unit_id, count(*) as totalfilecreatedelectronic
from fl_file ff
left join vw_employee_details_with_department epd on epd.post_detail_id=ff.file_opened_by_post_fk
-- join to active entries in emd_org_unit
inner join from emd.emd_org_unit org ON epd.departmentid=org.org_unit_id
AND org.is_active=true
where ff.file_nature = 'E'
and (
(ff.migration_date>='2011-01-01' and ff.migration_date<='2015-01-01') or
(ff.opening_date >='2011-01-01' and ff.opening_date <='2015-01-01'))
-- and now group by org_unit_id to get the counts
group by org_unit_id
If you'd create a SQLFiddle for this, things would get much clearer I guess.

Related

How can I use join in SQL to delete record?

I hava a problem with my program I'm trying to delete a record from a table using a join with Java this is my code:
try{
String sql ="DELETE f FROM facture f INNER JOIN client c ON f.idClient=c.id WHERE c.nom= ? ORDER BY idFact DESC LIMIT 1";
PreparedStatement pr = conn.prepareStatement(sql);
pr.setString(1,nom);
pr.executeUpdate();
System.out.println("supprimer");
}catch (SQLException e){
e.printStackTrace();
}
and this is the error :
java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'ORDER BY idFact DESC LIMIT 1' at line 1.
In MySQL/MariaDB you have a choice:
You can use ORDER BY and LIMIT and the FROM can only refer to one table.
You can have a FROM that refers to multiple tables.
The solution? Rephrase the query:
DELETE FROM facture f
WHERE EXISTS (SELECT 1
FROM client c
WHERE f.idClient = c.id AND c.nom = ?
)
ORDER BY f.idFact DESC
LIMIT 1;
Or you can use a subquery to get the row to delete:
DELETE f
FROM facture f JOIN
(SELECT f.idFact
FROM facture f JOIN
client c
ON f.idClient = c.id AND c.nom = ?
ORDER BY f.idFact DESC
LIMIT 1
) ff
ON ff.idFact = f.idFact

Java SQL Resultset DataTypes

I’m a java novice and have been tinkering with an existing Proc. I’m attempting to perform an out.write on a value derived from a SQL resultset 'Grid_ProjectCode’, as shown here:
sStmt=null;rs=null;sql=null;
sStmt = conn.createStatement();
sql = "select Proj_Code, Placement_ACR, Category_ID, Alt_Aisle_Shelf, Placement_Start_Dt, Placement_End_Dt, Sales_Manager from DH_CURRENT_FUTURE_INVENTORY_VW where Placement_ACR in ('banFSI_ROI', 'banFSI','FSI')";
rs = sStmt.executeQuery(sql);
while ( rs.next() )
{
String Grid_ProjectCode = rs.getString("Proj_Code");
String Dup_Placement_ACR = rs.getString("Placement_ACR");
if (plCode.equalsIgnoreCase(Dup_Placement_ACR))
{
out.write("Dupe Found");
out.newLine();
out.write(Grid_ProjectCode);
out.newLine();
}
}
if(conn != null) { rs.close(); sStmt.close(); }
The Proc fails with the following error (runs as expected when the second out.write is removed/commented):
java.lang.ClassCastException: [Ljava.lang.Integer; incompatible with
[Ljava.lang.String;
The Proj_Code field referred to in the SQL query is an NVARCHAR2 defined in an Oracle Exadata schema.
The error suggests some sort of data type mismatch, but I’m not sure how this is can be fixed; would really appreciate some guidance.
Edited to include DDL for DH_CURRENT_FUTURE_INVENTORY_vw:
CREATE OR REPLACE FORCE VIEW "UNICA_F1"."DH_CURRENT_FUTURE_INVENTORY_VW" ("PROJ_CODE", "FLAG_PROJ_REQUEST", "OBJECT_ID", "UAP_GRID_ROW_ID", "PLACEMENT_IMPRESSIONS", "SUPPLIER_CATEGORY", "PLACEMENT_CLASH", "PLACEMENT_NAME", "PLACEMENT_AISLE_SHELF", "PLACEMENT_START_DT", "PLACEMENT_RATE", "PLACEMENT_SIZE", "PLACEMENT_END_DT", "PLACEMENT_DURATION", "PLACEMENT_ACR", "SALES_VALUE", "NOTES", "STATUS", "AD_SERVER", "UNIT_NR", "SORT_ORDER", "CATEGORY_ID", "AISLE_SHELF_ID", "PLACEMENT_COUNT", "RETAIL_ACR", "ALT_AISLE_SHELF", "PLACEMENT_SIZE2", "PLACEMENT_SIZE3", "PLACEMENT_SIZE4", "PLACEMENT_SIZE5", "CREATIVE_YN", "CPM", "CREATIVE_SIZE", "PRODUCT_ID", "SALES_AREA", "IMPRESSION_GOAL", "CLIENT_CATEGORY", "CLIENT_SUB_CAT", "BRAND_NM", "FORMAT_DESC", "IMPRESSION_CPM", "IMPRESSION_VALUE", "CREATIVE_COST", "SIZE_DESC", "DELIVERY_PROCESS", "PACKAGE_FLAG", "PACK_ID", "RETARGETING_FLAG", "BATCH_FLAG", "BASE_COST", "DISCOUNT", "CATEGORY", "SALES_MANAGER") AS
Select
PROJ_CODE,
FLAG_PROJ_REQUEST,
OBJECT_ID,
UAP_GRID_ROW_ID,
PLACEMENT_IMPRESSIONS,
SUPPLIER_CATEGORY,
PLACEMENT_CLASH,
PLACEMENT_NAME,
PLACEMENT_AISLE_SHELF,
PLACEMENT_START_DT,
PLACEMENT_RATE,
PLACEMENT_SIZE,
PLACEMENT_END_DT,
PLACEMENT_DURATION,
PLACEMENT_ACR,
SALES_VALUE,
NOTES,
STATUS,
AD_SERVER,
UNIT_NR,
SORT_ORDER,
CATEGORY_ID,
AISLE_SHELF_ID,
PLACEMENT_COUNT,
RETAIL_ACR,
ALT_AISLE_SHELF,
PLACEMENT_SIZE2,
PLACEMENT_SIZE3,
PLACEMENT_SIZE4,
PLACEMENT_SIZE5,
CREATIVE_YN,
CPM,
CREATIVE_SIZE,
PRODUCT_ID,
SALES_AREA,
IMPRESSION_GOAL,
CLIENT_CATEGORY,
CLIENT_SUB_CAT,
BRAND_NM,
FORMAT_DESC,
IMPRESSION_CPM,
IMPRESSION_VALUE,
CREATIVE_COST,
SIZE_DESC,
DELIVERY_PROCESS,
PACKAGE_FLAG,
PACK_ID,
RETARGETING_FLAG,
BATCH_FLAG,
BASE_COST,
DISCOUNT,
CATEGORY,
SALES_MANAGER
from
(
select b.Proj_Code, b.Flag_Proj_Request, a.*, c.Category, d.Sales_Manager
from dh_ddp_inventory a
inner join uap_projects b on a.Object_ID = b.Project_ID
inner join dh_lkp_taxo_categ_vw c on a.Category_ID = c.Category_ID
inner join dh_ddp_Request d on a.Object_ID = d.Object_ID
where b.Flag_Proj_Request = 'Y' and a.Placement_End_Dt >= Current_date
and b.Proj_Code not in (select Proj_Code from uap_projects where Flag_Proj_Request = 'N')
Union
select b.Proj_Code, b.Flag_Proj_Request, a.*, c.Category, d.Sales_Manager
from dh_ddp_inventory a
inner join uap_projects b on a.Object_ID = b.Project_ID
inner join dh_lkp_taxo_categ_vw c on a.Category_ID = c.Category_ID
inner join dh_ddp_Request d on a.Object_ID = d.Object_ID
where b.Flag_Proj_Request = 'N' and a.Placement_End_Dt >= Current_date
)
Order By Proj_Code asc, Object_ID desc, Placement_Name asc;
Somewhere you do in some form or the other:
String[] stringArray = ...;
Integer[] intArray = (Integer[]) stringArray;
As [Ljava.lang.Integer =
array ([)
of class java.lang.Integer (L)
To drill down to the error:
try {
... code ...
} catch (ClassCastException e) {
e.printStackTrace(); // To System.err.
e.printStackTrace(System.out); // To System.out.
logger.error("OMG", e); // To logger if there is one.
throw e; // Act as is the exception was not thrown
}
Look at the stack trace, it also lists the source causing the error, together with the line number.

Querying through java

I have two tables: harvested_record and harvested_record_simple_keys there are in relation one-to-one.
harvested_record
id|harvested_record_simple_keys_id|a lot of columns
harvested_record_simple_keys
id| a lot of columns
and I want to make a query in which I need to join there two tables. As a result I will have a table:
joined_table
id(harvested_record)| (harvested_record_simple_keys)|a lot of columns.
Unfortunately I get an exception: nested exception is java.sql.SQLSyntaxErrorException: Column name 'ID' matches more than one result column.
I've understood this is because after join I will have two columns 'id'. Does anyone can help me with solution?
P.S.
SQL statement(works in IDEA console):
SELECT * FROM ( SELECT TMP_ORDERED.*, ROW_NUMBER() OVER () AS ROW_NUMBER FROM (SELECT * FROM harvested_record hr JOIN harvested_record_simple_keys hrsk ON hrsk.id = hr.harvested_record_simple_keys_id WHERE import_conf_id = ? ) AS TMP_ORDERED) AS TMP_SUB WHERE TMP_SUB.ROW_NUMBER <= 2 ORDER BY import_conf_id ASC, record_id ASC;
Java code(suppose error is here):
JdbcPagingItemReader<HarvestedRecord> reader = new JdbcPagingItemReader<HarvestedRecord>();
SqlPagingQueryProviderFactoryBean pqpf = new SqlPagingQueryProviderFactoryBean();
pqpf.setDataSource(dataSource);
pqpf.setSelectClause("SELECT *");
pqpf.setFromClause("FROM harvested_record hr JOIN harvested_record_simple_keys hrsk ON hrsk.id = hr.harvested_record_simple_keys_id");
String whereClause = "WHERE import_conf_id = :configId";
if (from!=null) {
fromStamp = new Timestamp(from.getTime());
whereClause += " AND updated >= :from";
}
if (to!=null) {
toStamp = new Timestamp(to.getTime());
whereClause += " AND updated <= :to";
}
if (configId != null) {
pqpf.setWhereClause(whereClause);
}
pqpf.setSortKeys(ImmutableMap.of("import_conf_id",
Order.ASCENDING, "record_id", Order.ASCENDING));
reader.setRowMapper(harvestedRecordRowMapper);
reader.setPageSize(PAGE_SIZE);
reader.setQueryProvider(pqpf.getObject());
reader.setDataSource(dataSource);
if (configId != null) {
Map<String, Object> parameterValues = new HashMap<String, Object>();
parameterValues.put("configId", configId);
parameterValues.put("from", fromStamp );
parameterValues.put("to", toStamp);
reader.setParameterValues(parameterValues);
}
reader.afterPropertiesSet();
Thank you in advance.
If you name your columns in the select statement instead of using 'SELECT *', you can omit the ID from one of the tables since it is always equal to the id from the other table.

Merging three queries that use 1:* relationships into one query

Lets say i have four tables i want to read from:
customer
customer_id, customer_name
1 Joe Bolggs
customer_orders
customer_id, order_no
----------------------------
1 1
1 2
1 3
customer_addresses
customer_id address
----------------------------
1 11 waterfall road
1 23 The broadway
customer_tel_no
customer_id number
----------------------------
1 523423423432
1 234342342343
The customer information shown above (for the customer with id=1) is to be stored in a Java object as shown below
public class Customer{
String customer_id;
String customerName;
ArrayList<String> customerOrders;
ArrayList<String> customerAddress;
ArrayList<String> customerTelephoneNumbers;
}
The only way i can think of to get the above information is by using three queries. The reason is that there is a 1:* relationship between the customer table and each of the other tables. To get the data i am doing something like this:
Customer customer = new Customer()
String customerSQL = "Select * from customer where customer_id = ?";
statement = getConnection().prepareStatement(contactsQuery);
statement.setString(1,1);
resultSet = statement.executeQuery();
while (resultSet.next()){
customer.customer_id = resultSet.get(1); //No getters/setters in this example
customer.customerName = resultSet.get(2);
}
String customerOrdersSQL = "Select * from customer_orders where customer_id = ?";
statement = getConnection().prepareStatement(customerOrdersSQL);
statement.setString(1,1);
resultSet = statement.executeQuery();
customer.customerOrders = new ArrayList();
while (resultSet.next()){
customer.customerOrders.add(resultSet.getString(2); // all the order numbers
}
String customerAddressesSQL = "Select * from customer_addresses where customer_id = ?";
statement = getConnection().prepareStatement(customerAddressesSQL );
statement.setString(1,1);
resultSet = statement.executeQuery();
customer.customerAddresses = new ArrayList();
while (resultSet.next()){
customer.customerAddresses.add(resultSet.getString(2); // all the addresses
}
String customerTelSQL = "Select * from customer_tel_no where customer_id = ?";
statement = getConnection().prepareStatement(customerTelSQL);
statement.setString(1,1);
resultSet = statement.executeQuery();
customer.customerTelephoneNumbers = new ArrayList();
while (resultSet.next()){
customer.customerTelephoneNumbers.add(resultSet.getString(2); // all the order numbers
}
The problem with the above is that i am making three calls to the database. Is there a way i can merge the above into a single query please?
I cant see how a join would work because for example, a join between customer and customer_orders will return a customer row for each row in customer_orders. Is there anyway i can merge the above three queries into one?
I would think that something like this would work:
SELECT c.customer_id, c.customer_name, o.order_no, a.address, t.number
FROM customer c LEFT JOIN customer_orders o ON c.customer_id = o.customer_id
LEFT JOIN customer_addresses a ON c.customer_id = a.customer_id
LEFT JOIN customer_tel_no t ON c.customer_id = t.customer_id
WHERE c.customer_id = ?
Then, in your code, after you execute the query:
while (resultSet.next())
{
customer.customerOrders.add(resultSet.getString(3));
customer.customerAddresses.add(resultSet.getString(4));
customer.customerTelephoneNumbers.add(resultSet.getString(5));
}
Of course, this does not take into account the fact that you will have null values along the way, so I'd advise checking for nulls to make sure that you aren't adding a lot of junk to your array lists. Still, that's probably a lot less costly than 3 separate queries.
Nothing prevents you from iterating and processing the joined result into your customer object. If your application is complex enough, you could look into ORM frameworks which would do that for you under the covers. If you are working with JavaEE, have a look at JPA.
use this query and reduce the number of call. And, in while loop process on data.
select customer.customer_id,customer.customer_name,order_no,address,number
from customer,customer_orders,customer_addresses,customer_tel_no
where customer.customer_id = customer_orders.customer_id
AND
customer.customer_id = customer_addresses.customer_id
AND
customer.customer_id = customer_tel_no.customer_id

Is MERGE an atomic statement in SQL2008?

I am using a MERGE statement as an UPSERT to either add a new record or update the current one. I have multiple threads driving the database through multiple connections and multiple statements (one connection and statement per thread). I am batching the statements 50 at a time.
I was very surprised to get a duplicate key violation during my tests. I expected that to be impossible because the MERGE will be performed as a single transaction, or is it?
My Java code looks like:
private void addBatch(Columns columns) throws SQLException {
try {
// Set parameters.
for (int i = 0; i < columns.size(); i++) {
Column c = columns.get(i);
// Column type is an `enum` with a `set` method appropriate to its type, e.g. setLong, setString etc.
c.getColumnType().set(statement, i + 1, c.getValue());
}
// Add the insert as a batch.
statement.addBatch();
// Ready to execute?
if (++batched >= MaxBatched) {
statement.executeBatch();
batched = 0;
}
} catch (SQLException e) {
log.warning("addBatch failed " + sql + " thread " + Thread.currentThread().getName(), e);
throw e;
}
}
The query looks like this:
MERGE INTO CustomerSpend AS T
USING ( SELECT ? AS ID, ? AS NetValue, ? AS VoidValue ) AS V
ON T.ID = V.ID
WHEN MATCHED THEN
UPDATE SET T.ID = V.ID, T.NetValue = T.NetValue + V.NetValue, T.VoidValue = T.VoidValue + V.VoidValue
WHEN NOT MATCHED THEN
INSERT ( ID,NetValue,VoidValue ) VALUES ( V.ID, V.NetValue, V.VoidValue );
The error reads:
java.sql.BatchUpdateException: Violation of PRIMARY KEY constraint 'PK_CustomerSpend'. Cannot insert duplicate key in object 'dbo.CustomerSpend'. The duplicate key value is (498288 ).
at net.sourceforge.jtds.jdbc.JtdsStatement.executeBatch(JtdsStatement.java:944)
at x.db.Db$BatchedStatement.addBatch(Db.java:299)
...
The key on the table is a PRIMARY key on the ID field.
MERGE is atomic meaning that either all changes are committed or all changes are rolled back.
It does not prevent duplicate keys in case of high concurrency. Adding holdlock hint will take care of that.
MERGE INTO CustomerSpend WITH (HOLDLOCK) AS T
USING ( SELECT ? AS ID, ? AS NetValue, ? AS VoidValue ) AS V
ON T.ID = V.ID
WHEN MATCHED THEN
UPDATE SET T.ID = V.ID, T.NetValue = T.NetValue + V.NetValue, T.VoidValue = T.VoidValue + V.VoidValue
WHEN NOT MATCHED THEN
INSERT ( ID,NetValue,VoidValue ) VALUES ( V.ID, V.NetValue, V.VoidValue );

Categories