We Keep Coding

How to retrieve all the tables from database using Java

I am using JDBC and PostgreSQL as database, I was trying to create a logic in such a way that we can fetch all the data from a table, whatever table name user gives in the input it should get fetched, but the issue here is, I don't know how to do that.
Whenever we used to fetch table data from the database we are required to specify the the type of data we are getting on every index while we use ResultSet.
How to overcome from this hardcoded need of providing this metadata and make our code more general for any table with any number of columns and with any type
My code:
Statement sttm = con1.createStatement();
System.out.println("Enter table name (usertable)");
String name = sc.next();
String tableData="";
String qu = "select * from "+name;
ResultSet rs =sttm.executeQuery(qu);
while(rs.next()) {
// here we need to define the type by writing .getInt or getString
tableData = rs.getInt(1)+":"+rs.getString(2)+":"+rs.getInt(3);
System.out.println(tableData);
}
System.out.println("*********---------***********-----------**********");
sttm.close();
Anyone please suggest me some way to do it.

You can use ResultSet.getObject(int). getObject will automatically retrieve the data in the most appropriate Java type for the SQL datatype of the column.
To retrieve the number of columns, you can use ResultSet.getMetaData(), and then use ResultSetMetaData.getColumnCount() to retrieve the number of columns.
In short, to print all columns of all rows, you can do something like:
try (ResultSet rs = stmt.executeQuery(qu)) {
ResultSetMetaData rsmd = rs.getMetaData();
int columnCount = rsmd.getColumnCount();
while (rs.next()) {
StringBuilder tableData = new StringBuilder();
for (int colIdx = 1; colIdx <= columnCount; colIdx++) {
tableData.append(rs.getObject(colIdx));
if (colIdx != columnCount) {
tableData.append(':');
}
}
System.out.println(TableData);
}
}
You can also use ResultSetMetaData to get more information on the columns of the result set, for example if you need specific handling for certain types of columns. You can use getColumnType to get the java.sql.Types value of the column, or getColumnTypeName to get the type name in the database, or getColumnClassName to get the name of the class returned by ResultSet.getObject(int/String), etc.
However, as Sorin pointed out in the comments, accepting user input and concatenating it into a query string like you're currently doing, makes you vulnerable to SQL injection. Unfortunately, it is not possible to parameterize object names, but you can mitigate this risk somewhat by 1) checking the table against the database metadata (e.g. DatabaseMetaData.getTables), and 2) using Statement.enquoteIdentifier (though this won't necessarily protect you against all forms of injection).

If you want to print data of any table from a database then check my github project over CRUD java MySQL
https://github.com/gptshubham595/jdbc_mysql_CRUD-JAVA-
These are implemented

Why is reading a JDBC ResultSet by position faster than by name and how much faster?

Announcing Hibernate 6 the Hibernate team claims that by switching from
read-by-name to read-by-position in JDBC ResultSet they gain a performance benefit.
High-load performance testing showed that Hibernate’s approach of
reading values from ResultSet by name to be its most limiting factor
in scaling through-put.
Does that mean they are changing calls from getString(String columnLabel) to getString(int columnIndex)?
Why is this faster?
As ResultSet is an interface doesn't performance gain depend on the JDBC driver implementing it?
How big are the gains?

Speaking as a JDBC driver maintainer (and, I admit, making some sweeping generalizations which not necessarily apply to all JDBC driver), row values will usually be stored in an array or list because that most naturally matches the way the data is received from the database server.
As a result, retrieving values by index will be the simplest. It might be as simple as something like (ignoring some of the nastier details of implementing a JDBC driver):
public Object getObject(int index) throws SQLException {
checkValidRow();
checkValidIndex(index);
return currentRow[index - 1];
}
This is about as fast as it gets.
On the other hand, looking up by column name is more work. Column names need to be treated case-insensitive, which has additional cost whether you normalize using lower or uppercase, or use a case-insensitive lookup using a TreeMap.
A simple implementation might be something like:
public Object getObject(String columnLabel) throws SQLException {
return getObject(getIndexByLabel(columnLabel));
}
private int getIndexByLabel(String columnLabel) {
Map<String, Integer> indexMap = createOrGetIndexMap();
Integer columnIndex = indexMap.get(columnLabel.toLowerCase());
if (columnIndex == null) {
throw new SQLException("Column label " + columnLabel + " does not exist in the result set");
}
return columnIndex;
}
private Map<String, Integer> createOrGetIndexMap() throws SQLException {
if (this.indexMap != null) {
return this.indexMap;
}
ResultSetMetaData rsmd = getMetaData();
Map<String, Integer> map = new HashMap<>(rsmd.getColumnCount());
// reverse loop to ensure first occurrence of a column label is retained
for (int idx = rsmd.getColumnCount(); idx > 0; idx--) {
String label = rsmd.getColumnLabel(idx).toLowerCase();
map.put(label, idx);
}
return this.indexMap = map;
}
Depending on the API of the database and available statement metadata, it may require additional processing to determine the actual column labels of a query. Depending on the cost, this will likely only be determined when it is actually needed (when accessing column labels by name, or when retrieving result set metadata). In other words, the cost of createOrGetIndexMap() might be pretty high.
But even if that cost is negligible (eg the statement prepare metadata from the database server includes the column labels), the overhead of mapping the column label to index and then retrieving by index is obviously higher than directly retrieving by index.
Drivers could even just loop over the result set metadata each time and use the first whose label matches; this might be cheaper than building and accessing the hash map for result sets with a small number of columns, but the cost is still higher than direct access by index.
As I said, this is a sweeping generalization, but I would be surprised if this (lookup index by name, then retrieve by index) isn't how it works in the majority of JDBC drivers, which means that I expect that lookup by index will generally be quicker.
Taking a quick look at a number of drivers, this is the case for:
Firebird (Jaybird, disclosure: I maintain this driver)
MySQL (MySQL Connector/J)
PostgreSQL
Oracle
HSQLDB
SQL Server (Microsoft JDBC Driver for SQL Server)
I'm not aware of JDBC drivers where retrieval by column name would be equivalent in cost or even cheaper.

In the very early days of making jOOQ, I had considered both options, of accessing JDBC ResultSet values by index or by name. I chose accessing things by index for these reasons:
RDBMS support
Not all JDBC drivers actually support accessing columns by name. I forgot which ones didn't, and if they still don't, because I never touched that part of JDBC's API again in 13 years. But some didn't and that was already a show stopper for me.
Semantics of the name
Furthermore, among those that do support column names, there are different semantics to a column name, mainly two, what JDBC calls:
The column name as in ResultSetMetaData::getColumnName
The column label as in ResultSetMetaData::getColumnLabel
There is a lot of ambiguity with respect to implementations of the above two, although I think the intent is quite clear:
The column name is supposed to produce the name of the column irrespective of aliasing, e.g. TITLE if the projected expression is BOOK.TITLE AS X
The column label is supposed to produce the label (or alias) of the column, or the name if no alias is available, e.g. X if the projected expression is BOOK.TITLE AS X
So, this ambiguity of what a name/label is is already very confusing and concerning. It doesn't seem something an ORM should rely on in general, although, in Hibernate's case, one can argue that Hibernate is in control of most SQL being generated, at least the SQL that is produced to fetch entities. But if a user writes an HQL or native SQL query, I would be reluctant to rely on the name/label - at least without looking things up in ResultSetMetaData, first.
Ambiguities
In SQL, it's perfectly fine to have ambiguous column names at the top level, e.g.:
SELECT id, id, not_the_id AS id
FROM book
This is perfectly valid SQL. You can't nest this query as a derived table, where ambiguities aren't allowed, but in top level SELECT you can. Now, what are you going to do with those duplicate ID labels at the top level? You can't know for sure which one you'll get when accessing things by name. The first two may be identical, but the third one is very different.
The only way to clearly distinguish between the columns is by index, which is unique: 1, 2, 3.
Performance
I had also tried performance at the time. I don't have the benchmark results anymore, but it's easy to write another benchmark quickly. In the below benchmark, I'm running a simple query on an H2 in-memory instance, and consume the ResultSet accessing things:
By index
By name
The results are staggering:
Benchmark Mode Cnt Score Error Units
JDBCResultSetBenchmark.indexAccess thrpt 7 1130734.076 ± 9035.404 ops/s
JDBCResultSetBenchmark.nameAccess thrpt 7 600540.553 ± 13217.954 ops/s
Despite the benchmark running an entire query on each invocation, the access by index is almost twice as fast! You can look at H2's code, it's open source. It does this (version 2.1.212):
private int getColumnIndex(String columnLabel) {
checkClosed();
if (columnLabel == null) {
throw DbException.getInvalidValueException("columnLabel", null);
}
if (columnCount >= 3) {
// use a hash table if more than 2 columns
if (columnLabelMap == null) {
HashMap<String, Integer> map = new HashMap<>();
// [ ... ]
columnLabelMap = map;
if (preparedStatement != null) {
preparedStatement.setCachedColumnLabelMap(columnLabelMap);
}
}
Integer index = columnLabelMap.get(StringUtils.toUpperEnglish(columnLabel));
if (index == null) {
throw DbException.get(ErrorCode.COLUMN_NOT_FOUND_1, columnLabel);
}
return index + 1;
}
// [ ... ]
So. there's a hashmap with upper casing, and each lookup also performs upper casing. At least, it caches the map in the prepared statement, so:
You can reuse it on every row
You can reuse it on multiple executions of the statement (at least that's how I interpret the code)
So, for very large result sets, it might not matter as much anymore, but for small ones, it definitely does.
Conclusion for ORMs
An ORM like Hibernate or jOOQ is in control of a lot of SQL and the result set. It knows exactly what column is at what position, this work has already been done when generating the SQL query. So, there's absolutely no reason to rely on the column name any further when the result set comes back from the database server. Every value will be at the expected position.
Using column names must have been some historic thing in Hibernate. It's probably also why they used to generate these not so readable column aliases, to make sure that each alias is non-ambiguous.
It seems like an obvious improvement, irrespective of the actual gains in a real world (non-benchmark) query. Even if the improvement had been only 2%, it would have been worth it, because it affects every query execution by every Hibernate based application.
Benchmark code below, for reproduction
package org.jooq.test.benchmarks.local;
import java.io.*;
import java.sql.*;
import java.util.Properties;
import org.openjdk.jmh.annotations.*;
import org.openjdk.jmh.infra.*;
#Fork(value = 1)
#Warmup(iterations = 3, time = 3)
#Measurement(iterations = 7, time = 3)
public class JDBCResultSetBenchmark {
#State(Scope.Benchmark)
public static class BenchmarkState {
Connection connection;
#Setup(Level.Trial)
public void setup() throws Exception {
try (InputStream is = BenchmarkState.class.getResourceAsStream("/config.properties")) {
Properties p = new Properties();
p.load(is);
connection = DriverManager.getConnection(
p.getProperty("db.url"),
p.getProperty("db.username"),
p.getProperty("db.password")
);
}
}
#TearDown(Level.Trial)
public void teardown() throws Exception {
connection.close();
}
}
#FunctionalInterface
interface ThrowingConsumer<T> {
void accept(T t) throws SQLException;
}
private void run(BenchmarkState state, ThrowingConsumer<ResultSet> c) throws SQLException {
try (Statement s = state.connection.createStatement();
ResultSet rs = s.executeQuery("select c as c1, c as c2, c as c3, c as c4 from system_range(1, 10) as t(c);")) {
c.accept(rs);
}
}
#Benchmark
public void indexAccess(Blackhole blackhole, BenchmarkState state) throws SQLException {
run(state, rs -> {
while (rs.next()) {
blackhole.consume(rs.getInt(1));
blackhole.consume(rs.getInt(2));
blackhole.consume(rs.getInt(3));
blackhole.consume(rs.getInt(4));
}
});
}
#Benchmark
public void nameAccess(Blackhole blackhole, BenchmarkState state) throws SQLException {
run(state, rs -> {
while (rs.next()) {
blackhole.consume(rs.getInt("C1"));
blackhole.consume(rs.getInt("C2"));
blackhole.consume(rs.getInt("C3"));
blackhole.consume(rs.getInt("C4"));
}
});
}
}

How to build a query based on a number of different URIs

What I currently have done: Since I am using aws lambda and aws API Gateway, I have learned to capture specific parts of the URI such as the Path and the queryParameters, and I am building different queries to my database based on the path and parameter given.
Example:
https://ApiURI/user?id=id_1
Path=User
Parameter=id:id_1
I put both pieces of data in a Map<String, Map<String, String>object in which the first string is the key (either a path or a parameter) and the inner map holds the actual value. I did it this way because I felt it would scale better, since I do not know beforehand how many paths or parameters are going to come with the URI (the idea is to be able to filter a given URI/concept/entity with 0 up to N filters, applying 0 or even all at the same time, and some URIs currently can have 8 filters), the path also could have several "slashes" such as /user/profile or skill/profile?date=today but as of right now it is not the case.
Example: /course can have the following filters (also called queryParameters since that's the name I saw AWS giving it at times):
skill
cost
online
duration
level
certified
language
I lack the experience to see how this can be made with scaling in mind. Currently what I am doing is something like the following example (queries are simplified for the sake of brevity). In the example, I catch whether or not a certain key appears in the map, and if present, I build the appropriate query. The biggest problem right now to apply several filters at once is that they can require a Join with different tables (suppose that for the list before, when ?skill appears I build a query with an inner join with the table skill, and when ?cost or level appears, I have to make a join with another table called course, for example) :
if(pathParameters.containsValue("profiles")) {
if(queryParameters == null || queryParameters.isEmpty()) {
try {
preparedstatement = con.prepareStatement("SELECT * from profiles");
} catch (SQLException e) {
e.printStackTrace();
}
}
else if(queryParameters.containsKey("zone")) {
try {
preparedstatement = con.prepareStatement("SELECT * from profiles where id = ? GROUP BY id;");
preparedstatement.setString(1, queryParameters.get("zone"));
} catch (SQLException e) {
e.printStackTrace();
}
}
else if(queryParameters.containsKey("skill")) {
try {
preparedstatement = con.prepareStatement("SELECT * from skills where id = ?");
preparedstatement.setString(1, queryParameters.get("skill"));
} catch (SQLException e) {
e.printStackTrace();
}
}
else if(queryParameters.containsKey("experiencia")) {
try {
[....]
}
}
}
So summing it up, I have a number of branches of logic that I cannot quite know how to simplify since each branch can have a different query associated, that needs to join a certain table. To avoid having 7! number of possibilities with IF cases, I would need to build the query more dynamically (it's the only idea I have come up with so far). For that, I would need to join two different queries with different logic, and I do not know if that is possible.

I have not yet implemented this options, but so far this looks close to what I am looking for, so I post it as an answer in case someone else may use them in the future, but so far it is a partial answer:
DbExtensions although it's for .Net and not java unfortunately.
http://www.mybatis.org/mybatis-3/statement-builders.html
Particularly code like this is what I am aiming for:
void DynamicSql(int? categoryId, int? supplierId) {
var query = SQL
.SELECT("ID, Name")
.FROM("Products")
.WHERE()
._If(categoryId.HasValue, "CategoryID = {0}", categoryId)
._If(supplierId.HasValue, "SupplierID = {0}", supplierId)
.ORDER_BY("Name DESC");
Console.WriteLine(query);
}

How to process ResultSet you know has only one record in it

I'm struggling with a homework assignment and am getting hung up on some SQL queries.
My query is interrogating an inventory database for the quantity of some item. The query requests the column with the name quantity_in_stock from the table, given the primary key.
I have initialized some prepared statements. This is the one I'm using here:
stmtFindColumn = Database.getConnection().prepareStatement(String.format("select ? from %s where %s = ?",
INVENTORY_TABLE_NAME, SKU) );
Now a separate method is called. I pass it a static const QTY_IN_STOCK, which is defined as "quantity_in_stock" and the item's SKU number, which is the primary key in the table.
private int getIntegerFromTable(String column, String key) {
int toReturn = 0;
try {
// Complete the prepared statement
stmtFindColumn.setString(1, column);
stmtFindColumn.setString(2, key);
ResultSet result = stmtFindColumn.executeQuery();
toReturn = result.getInt(column);
} catch (SQLException e) {
LOG.error(e.getMessage());
e.printStackTrace();
}
return toReturn;
}
When I run the query I get an sql exception that tells me: Invalid column name quantity_in_stock.
I have tried using a while loop processing result.next() and get the same error. I can't find any examples of how to properly get the results when you know only a single record is being returned.
Help!
EDIT: OK, I've found that part of my problem is I'm not getting a result set, where I should expect one. Any ideas?
UPDATE: I've tested my code, using a garden variety statement and a plain string query instead and it works just fine. So the problem is in my use of the prepared statement. Can someone check if I'm using the ? wildcards correctly? Thanks!

as far as i know, the column name may not be a parameter ...

DarkSquirrel42 is right -- you can't replace the column list of the select using a ? parameter marker. Instead, you can String.format that into place too, for example.
bad:
*select ? from INVENTORY_TABLE_NAME where SKU = ?
good:
select QUANTITY_IN_STOCK from INVENTORY_TABLE_NAME where SKU = ?

Insert or update table using JDBC

I have some records to import. It's ok the first time when they are being inserted. If I try to import the same data again I receive a org.postgresql.util.PSQLException: ERROR: duplicate key value violates unique constraint. How can I update the records in the database if the data is the same/or changed and insert if it's new data using JDBC?
public void store(Object entity) throws Exception {
try {
if (this.updateEntity((XEntity) entity) == 0) {
this.insertEntity((XEntity) entity);
}
...
} catch (SQLException sqlEx) {
...
}
}
private int updateEntity(XEntity entity) throws SQLException {
PreparedStatement prepStmt = this.getUpdatePreparedStmt();
...
return prepStmt.executeUpdate();
}
private void insertEntity(XEntity entity) throws SQLException {
...
this.getInsertPreparedStmt().executeUpdate();
}
The problem is fixed now. I've provided an answer below.

You can try using postgres SQL 'MERGE' or 'REPLACE'

You can pass the UPDATE command as a string through JDBC.
According to this SO post, you will have to write 2 statements.

If you want to use the same method to insert and update your data, you'll need to check if the data exists first. The SQL command used to insert a new object is INSERT, whereas the one used to update an element is UPDATE. So, what you could do is do a SELECT to check if your data is already here, and then do an INSERT or UPDATE based on the results.
However, this is a workaround. You would really need to clarify your implementation, and make different methods whether you are adding or updating data. Business-side, these are clearly two very different functions, so one method for both seems to me like a design problem.

This test logic works.
if (this.updateEntity((XEntity) entity) == 0) {
this.insertEntity((XEntity) entity);
}
The problem was in the method that updated the record. The WHERE clause in the update prepared statement was using different data(data containing spaces) so updateEntity would always return 0. That was the reason why only inserts were made, instead of updates. Thank you very much for your help.