We Keep Coding

What is the best way Loading XML data to mySql using Java?

I am already working on a project to optimize interactions with dataBases using JAVA.
First Step , I began with loading XML data to mysql.
I found many articles working on this issue , and they parse Data before inserting it , like this article :
https://dzone.com/articles/load-xml-into-mysql-using-java
But I tried to do things simpler : so
I write this code that load data using LOAD local XML infile .. ( an sql Query ) and it works well .
package my.project;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
public class App {
static final String dbUrl = "jdbc:mysql://localhost/dbOptimization";
static final String password = "azerty";
static final String user = "root";
public Connection conn;
/*
* Load jdbc Driver
*/
static {
try {
Class.forName("com.mysql.jdbc.Driver");
System.out.println("Dirver loaded");
} catch (ClassNotFoundException ex) {
System.err.println("Cannot load driver " + ex);
}
}
/*
* Connect to DB
*/
public void connect() {
try {
conn = DriverManager.getConnection(dbUrl, user, password);
System.out.println("Database connected!");
} catch (SQLException e) {
System.err.println("Cannot connect the database!");
}
}
/*
* Create Table and Load Data
*/
public void createTable() {
try {
conn.createStatement().execute("create Table badges(Id INTEGER,UserId VARCHAR(20),Name varchar(20),Date DATE ,Class INTEGER ,TagBased VARCHAR(20))");
System.out.println("table created");
conn.createStatement().execute("Load xml local infile '/home/lenovo/Bureau/Project/3dprinting/Badges.xml'into Table badges(Id,UserId,Name,Date,Class,TagBased)");
System.out.println("data parsed");
} catch (SQLException e) {
// TODO Auto-generated catch block
System.out.println("connot create Table" + e);
}
}
public static void main(String[] args) {
System.out.println("Hello World!");
App app =new App();
app.connect();
app.createTable();
}
}
So please , Is there any problem with my code .?!
What are the pros and cons of each method ?
which one has a better performence ?
Thanks

The answer depends on whether or not you're going to use the XML as a whole document or not.
If you use XPath to search inside the document parsing and loading might make sense.

MySQL insert query not working in aws rds abruptly

I have a java based web application in which am inserting a row in aws MySQL database.
The problem is that, after 1-2 hours, the code stops inserting the rows in the database and am not getting any sort of error in my log files.
The structure of the table is as below:
Now when am calling the servlet, am using this piece of code.
JSONObject result=t_s.ro(jc.getT_conn(), t,true);
t is the json and true/false ia a boolean value according to my case.
Now inside jc.getT_conn() am using this code:
public static Connection getT_conn() throws ClassNotFoundException, JSONException {
Connection c=null;
if(t_conn==null)
{
c=rds_conn();
}
else
{
c=t_conn;
}
return c;
}
Here t_conn is a global variable for that java file and rds_conn() returns me a new connection after creating it.
Now from t_s.ro class am calling a function which inserts the row into the database based on a condition, if that's satisfied.
Here is the code:
public static boolean dPOI(Connection conn,String d,String u,ArrayList<String> l,ArrayList<String> li) throws SQLException
{
long startTime=System.currentTimeMillis();
System.out.println("Time for sql start is : "+System.currentTimeMillis());
PreparedStatement stmt = null;
boolean action=false;
try {
String sql="INSERT INTO `ce`.`cse`(`twsD`,`twsID`,`twsi`)VALUES(?,?,?)";
stmt = conn.prepareStatement(sql);
stmt.setString(1, u);
stmt.setString(2, d);
stmt.setString(3, l.toString()+"~"+li.toString());
System.out.println(stmt.toString());
action = stmt.execute();
//conn.close();
} catch (SQLException e) {
// handle sql exception
System.out.println("SQL Exception");
e.printStackTrace();
}catch (Exception e) {
// TODO: handle exception for class.forName
System.out.println("Exception");
e.printStackTrace();
}
stmt.close();
long endTime=System.currentTimeMillis();
System.out.println("Time taken inside sql Query is : "+(endTime-startTime));
return action;
}
Below is the log file which am getting.
Time for sql start is : 1486393105661
com.mysql.jdbc.JDBC42PreparedStatement#59037dda: INSERT INTO `ce`.`cse`(`twsD`,`twsID`,`twsi`)VALUES('Bana','2fdb0c926765','[\'FOM\', \'MONEY CENTER KOLA - BAORE\']~[83.80, 272.20]')
Time taken inside sql Query is : 1
Now if you can see, I am not getting any SQL exception or any other kind of exception. And moreover, the time taken is always 1 (when it stops inserting) otherwise it's somewhere between 20-25.
Moreover, thee auto increment ID always gets used up, by that what I mean is if the last row was inserted at ID 1, the subsequent query which I insert through MySQL workbench has an ID somewhere around 40 i.e if we assume that 39 of the remaining rows didn't get inserted.

Taking Mark B's comment as a starting point, I decided to create a connection pool which will be providing the connections.
Below is the code which I used:
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import org.apache.commons.dbcp2.BasicDataSource;
public final class Database {
private static final String SQL_EXIST = "show tables;";
public static void main(String[] args) throws SQLException {
// TODO Auto-generated method stub
boolean exist = false;
try (
Connection connection = Database.getConnection();
PreparedStatement statement = connection.prepareStatement(SQL_EXIST);
)
{
try (ResultSet resultSet = statement.executeQuery()) {
exist = resultSet.next();
}
}
System.out.println("Value is : "+ exist);
}
private static final BasicDataSource dataSource = new BasicDataSource();
static {
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("CONNECTION_STRING");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
dataSource.setMaxTotal(100);
}
private Database() {
//
}
public static Connection getConnection() throws SQLException {
return dataSource.getConnection();
}
}
FIrst function was just for testing purpose.
Now after creating this Database class, just call Database.getConnection() whenever you need to get the connection. The connection pool will take care of providing you with a valid connection.
Correct me if am wrong.

Java and SQL Server

I am new to java, and I am trying to create a method that will retrieve information from the database based on the query that will pass to it.
I thought that I could create by method by creating an object of type:
private Connection controlTableConnection = null;
and then
Statement statement = controlTableConnection.createStatement();
but when I do that piece of code, I get a highlight error:
Unhandled exception
Any help, would be appreciated.
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.Statement;
public class ConnectMSSQLServer {
private static final String db_connect_string = "jdbc:sqlserver://Cdsx\\SQxxs";
private static final String db_userid = "aa";
private static final String db_password = "bb";
private Connection controlTableConnection = null;
public void dbConnect() {
try {
Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver");
Connection controlTableConnection = DriverManager.getConnection(db_connect_string, db_userid, db_password);
} catch (Exception e) {
e.printStackTrace();
}
}
public void dbDisconnect() {
try {
if (controlTableConnection != null && !controlTableConnection.isClosed()) {
controlTableConnection.close();
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void createstatement() {
Statement statement = controlTableConnection.createStatement();
}
}

You have to wrap the createStatement line like below, as you have to handle the SQLException.
try {
Statement statement = controlTableConnection.createStatement();
} catch (SQLException e) {
e.printStackTrace();
}

isn't the Connection null? Do you have a driver on the classpath? is the default port correct? Is the sql server live? What kind of exception do you get exactly?
You need to post at least the stack trace or logs

Connecting to database at a servlet or at a managed bean

I am new to javaEE and trying to make database connection. I can do it but i think my way of doing it is inefficient. Here is what i do:
static String dbUrl="jdbc:mysql://localhost:3306/Bank";
static String username="root";
static String password="";
static Connection con=null;
public static void connect ()
{
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
con=(Connection) DriverManager.getConnection(dbUrl,username,password);
System.out.println("Connected!");
}
catch (Exception e) {
e.printStackTrace();
System.out.println("not connected");
}
}
public void someFunctiontoConnectToDB{
try {
connect();
DO_THE_QUERY.....etc
}
}
Here is my problem, this method works fine, but i have to duplicate the same code whenever i try to connect to DB in another servlet or managed bean. I tried to create a class for connection and pass the Connection c as the parameter, but this time when it returns from the connection class, Connection object becomes null. Is there any other way i can make connection easier, and without code duplication?
Thanks

If it is possible try to get into JPA, it makes life much easier.
But if you are stuck with JDBC, here is a good approach for abstracting and encapsulating the Database Layer Core J2EE Patterns - Data Access Object
In a summary for your case I would do something like this:
The DAOFactory:
class MySqlDAOFactory {
static String dbUrl="jdbc:mysql://localhost:3306/Bank";
static String username="root";
static String password="";
static private Connection con;
public static void createConnection() {
try {
Class.forName("com.mysql.jdbc.Driver").newInstance();
con=(Connection) DriverManager.getConnection(dbUrl,username,password);
System.out.println("Connected!");
}
catch (Exception e) {
e.printStackTrace();
System.out.println("not connected");
}
}
public static BankDAO getBankDAO() {
return new BankDAO(con);
}
}
And the BankDAO:
class BankDAO {
Connection con;
public BankDAO(Connection con) {
this.con = con;
}
public Account getAccountFor(String name) {
//JDBC Operations
return acc;
}
}
In your Managed Bean or Servlet:
public void someFunction() {
MySqlDAOFactory.createConnection();
BankDAO dao = MySqlDAOFactory.getBankDAO();
//get other DAOs
Account acc = dao.getAccountFor("bob");
}

You can create a class that will connect to database and make a method return the connection.
When you want to use connection in other classes, you simply call that method.
Or else what you can do is
Create a ServletContextListener
Add database connection code in the listener.
Add a method to return connection
Use it whenever required.

Java newbie needs help in database connection

I'm new to Java and even newer to java database connections. I've managed to create a database connection and query a table when I put it in the Main class. Now that I've moved it into a new class called Connection I am getting errors:
package lokate;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.sql.ResultSet;
public class Connection {
private static Statement stmt = null;
private static ResultSet rs = null;
private static Connection con = null;
public Connection() throws SQLException {
try {
Class.forName("com.mysql.jdbc.Driver");
String connectionUrl = "jdbc:mysql://localhost:3306/Lokate?" +
"user=root&password=";
con = DriverManager.getConnection(connectionUrl);
stmt = con.createStatement();
retriveData("SELECT * FROM Users");
int rowsEffected = 0;
} catch (SQLException sqlEx) {
System.out.println("SQL Exception: "+ sqlEx.toString());
} catch (ClassNotFoundException classEx) {
System.out.println("Class Not Found Exception: "+ classEx.toString());
} catch (Exception Ex) {
System.out.println("Exception: "+ Ex.toString());
}
}
public static void retriveData(String SQL) throws Exception {
rs = stmt.executeQuery(SQL);
while (rs.next())
{
System.out.println(rs.getString("fname") + " : " + rs.getString("lname"));
}
}
}
I'm getting an error saying cannot find symbol. Symbol:method createStatement() and incomparable types for con = DriveManager.....
Can anyone help?
Also, is it best practice to put the connection in the class like this then call a new object every time I want to do something with the db?
Regards,
Billy

I'd say your code is an example of many worst practices. Let me count the ways:
Your Connection class is a poor abstraction that offers nothing over and above that of java.sql.Connection.
If you use your class, you'll never get to take advantage of connection pooling.
You hard wire your driver class, your connection URL, etc. You can't change it without editing and recompiling. A better solution would be to externalize such things.
Printing an error message in the catch blocks is far less information than supplying the entire stack trace.
Your code hurts my eyes. It doesn't follow the Sun Java coding standards.
Your retrieveData method is utterly worthless. What will you do with all those printed statements? Wouldn't it be better to load them into a data structure or object so the rest of your code might use that information?
It's rowsAffected - "affect" is the verb, "effect" is the noun. Another variable that's not doing any good.
You're on the wrong track. Rethink it.
I think you'll find this code more helpful.
package persistence;
import java.sql.*;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
public class DatabaseUtils
{
public static Connection createConnection(String driver, String url, String username, String password) throws ClassNotFoundException, SQLException
{
Class.forName(driver);
if ((username == null) || (password == null) || (username.trim().length() == 0) || (password.trim().length() == 0))
{
return DriverManager.getConnection(url);
}
else
{
return DriverManager.getConnection(url, username, password);
}
}
public static void close(Connection connection)
{
try
{
if (connection != null)
{
connection.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
public static void close(Statement st)
{
try
{
if (st != null)
{
st.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
public static void close(ResultSet rs)
{
try
{
if (rs != null)
{
rs.close();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
public static void rollback(Connection connection)
{
try
{
if (connection != null)
{
connection.rollback();
}
}
catch (SQLException e)
{
e.printStackTrace();
}
}
public static List<Map<String, Object>> map(ResultSet rs) throws SQLException
{
List<Map<String, Object>> results = new ArrayList<Map<String, Object>>();
try
{
if (rs != null)
{
ResultSetMetaData meta = rs.getMetaData();
int numColumns = meta.getColumnCount();
while (rs.next())
{
Map<String, Object> row = new HashMap<String, Object>();
for (int i = 1; i <= numColumns; ++i)
{
String name = meta.getColumnName(i);
Object value = rs.getObject(i);
row.put(name, value);
}
results.add(row);
}
}
}
finally
{
close(rs);
}
return results;
}
}

Your problem is that DriverManager.getConnection returns a java.sql.Connection. Since your class is also called Connection you are getting a name clash between lokate.Connection and java.sql.Connection. You Will need to specify the fully qualified class name where ever you want to use java.sql.Connection, otherwise lokate.Connection is assumed.
Specify fully qualified class name like so:
java.sql.Connection con = null;
// ....
con = DriverManager.getConnection(connectionUrl);
Alternatively, rename your Connection class to something else and you will not get this naming conflict.

Connection is an existing type in the java.sql package, which is what DriverManager.getConnection returns. You have named your class as Connection too, so this is causing the confusion. The simplest way out would be to rename your class to something else and add an import java.sql.Connection; at the top.

Also, is it best practice to put the connection in the class like this then call a new object every time I want to do something with the db?
I think the best practice would be to use an existing solution to this problem, so that you can avoid re-inventing the wheel and focus on what makes your problem unique.
If you are writing a server application (it isn't clear whether you are), then I would also consider using a database connection pool. Creating new database connections on the fly is not efficient and doesn't scale well. You can read about database connection issues in this article I wrote a while back.