I am already working on a project to optimize interactions with dataBases using JAVA.
First Step , I began with loading XML data to mysql.
I found many articles working on this issue , and they parse Data before inserting it , like this article :
https://dzone.com/articles/load-xml-into-mysql-using-java
But I tried to do things simpler : so
I write this code that load data using LOAD local XML infile .. ( an sql Query ) and it works well .
package my.project;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
public class App {
static final String dbUrl = "jdbc:mysql://localhost/dbOptimization";
static final String password = "azerty";
static final String user = "root";
public Connection conn;
/*
* Load jdbc Driver
*/
static {
try {
Class.forName("com.mysql.jdbc.Driver");
System.out.println("Dirver loaded");
} catch (ClassNotFoundException ex) {
System.err.println("Cannot load driver " + ex);
}
}
/*
* Connect to DB
*/
public void connect() {
try {
conn = DriverManager.getConnection(dbUrl, user, password);
System.out.println("Database connected!");
} catch (SQLException e) {
System.err.println("Cannot connect the database!");
}
}
/*
* Create Table and Load Data
*/
public void createTable() {
try {
conn.createStatement().execute("create Table badges(Id INTEGER,UserId VARCHAR(20),Name varchar(20),Date DATE ,Class INTEGER ,TagBased VARCHAR(20))");
System.out.println("table created");
conn.createStatement().execute("Load xml local infile '/home/lenovo/Bureau/Project/3dprinting/Badges.xml'into Table badges(Id,UserId,Name,Date,Class,TagBased)");
System.out.println("data parsed");
} catch (SQLException e) {
// TODO Auto-generated catch block
System.out.println("connot create Table" + e);
}
}
public static void main(String[] args) {
System.out.println("Hello World!");
App app =new App();
app.connect();
app.createTable();
}
}
So please , Is there any problem with my code .?!
What are the pros and cons of each method ?
which one has a better performence ?
Thanks
The answer depends on whether or not you're going to use the XML as a whole document or not.
If you use XPath to search inside the document parsing and loading might make sense.
Related
I'm wondering if anyone can shed some light on this topic, as I have been racking my brain for days and can't quite understand why this does not work. I have three classes
main, RetrieveDBVersion,GetOracleConnection I've been doing some testing with oracle JDBC, UCP and Java 1.7.
According to the Oracle documentation, If I use connection pooling the connection will be returned to the pool as soon as I close the connection, Invalidate it and set it to null See Here. So I decided to give it a whirl and see if it would perform just like the documentation says it should. In my Main application I have a simple loop which makes a connection 200 times by calling RetrieveDBVersion. RetrieveDBVersion is simply performing a query and returning the driver version. My loop works fine until I hit the magic number of 68 and then I receive an error which states
java.sql.SQLException: Exception occurred while getting connection:
oracle.ucp.UniversalConnectionPoolException:
Cannot get Connection from Datasource: java.sql.SQLException:
Listener refused the connection with the following error:
ORA-12516, TNS:listener could not find available handler with matching protocol stack
These are the detail of the 3 methods. These methods are not in a server environment. They are simply calling a local oracle express database and I'm running them from my desktop. Why would I keep getting this error? If I'm returning the connections back to the pool?
Main
import com.jam.DB.JDBCVersion;
import static java.lang.System.out;
public class MainApp {
public static void main(String[] args) {
String myMainJDBCVar;
try{
for(int i=1; i<200; i++ )
{
myMainJDBCVar= JDBCVersion.RetrieveDBVersion();
out.println(myMainJDBCVar + " " + i);
}
out.println("this is Done!");
}
catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
RetrieveDBVersion
import java.sql.*;
import oracle.ucp.jdbc.ValidConnection;
public class JDBCVersion {
public static String DBVersion;
public static String RetrieveDBVersion()throws SQLException {
Connection conn = JDBCConnection.GetOracleConnection("test");
try {
DatabaseMetaData meta = conn.getMetaData();
//get driver info
System.out.println("JDBC driver version is " + meta.getDriverMajorVersion());
DBVersion = meta.getDriverVersion();
} catch (SQLException e) {
e.printStackTrace();
DBVersion = e.getMessage();
}
finally {
System.out.println("hit the finally clause");
((ValidConnection) conn).setInvalid();
conn.close();
conn=null;
}
return DBVersion;
}
GetOracleConnection
import oracle.ucp.jdbc.PoolDataSource;
import oracle.ucp.jdbc.PoolDataSourceFactory;
import java.sql.*;
public class JDBCConnection {
public static Connection GetOracleConnection(String Enviroment) throws SQLException{
PoolDataSource pds = PoolDataSourceFactory.getPoolDataSource();
Connection conn = null; //ora.defaultConnection();
try {
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL("jdbc:oracle:thin:#//localhost:1521/xe");
pds.setUser("system");
//pds.setInitialPoolSize(5);
pds.setPassword("xxx");
pds.setMaxStatements(10);
conn = pds.getConnection();
return conn;
}
catch(Exception e){
e.printStackTrace();
}
return conn;
}
So after careful though and getting a little extra help from the Oracle forum. I finally understand why the above referenced code is giving the error message that I'm receiving. See Here For Response
Because I'm setting the data source everytime the loop goes around, I'm essentially creating more than one pool. The way to do this, is create one pool and than pull connections from that pool.
New code to replace the GetOracleConnection I created a singleton class for datasource and in code I simply retrieve the connection from the data source like such
Connection conn = Database.getInstance().GetPoolSource().getConnection();
package com.jam.DB;
import oracle.ucp.jdbc.PoolDataSource;
import oracle.ucp.jdbc.PoolDataSourceFactory;
public class Database {
private static Database dbIsntance;
private static PoolDataSource pds;
private Database() {
// private constructor //
}
public static Database getInstance() {
if (dbIsntance == null) {
dbIsntance = new Database();
}
return dbIsntance;
}
public PoolDataSource GetPoolSource() {
if (pds == null) {
pds = PoolDataSourceFactory.getPoolDataSource();
try {
pds.setConnectionFactoryClassName("oracle.jdbc.pool.OracleDataSource");
pds.setURL("jdbc:oracle:thin:#//localhost:1521/xe");
pds.setUser("system");
pds.setPassword("xxxx");
pds.setMaxStatements(15);
return pds;
} catch (Exception e) {
}
return pds;
}
return pds;
}
}
I have a java based web application in which am inserting a row in aws MySQL database.
The problem is that, after 1-2 hours, the code stops inserting the rows in the database and am not getting any sort of error in my log files.
The structure of the table is as below:
Now when am calling the servlet, am using this piece of code.
JSONObject result=t_s.ro(jc.getT_conn(), t,true);
t is the json and true/false ia a boolean value according to my case.
Now inside jc.getT_conn() am using this code:
public static Connection getT_conn() throws ClassNotFoundException, JSONException {
Connection c=null;
if(t_conn==null)
{
c=rds_conn();
}
else
{
c=t_conn;
}
return c;
}
Here t_conn is a global variable for that java file and rds_conn() returns me a new connection after creating it.
Now from t_s.ro class am calling a function which inserts the row into the database based on a condition, if that's satisfied.
Here is the code:
public static boolean dPOI(Connection conn,String d,String u,ArrayList<String> l,ArrayList<String> li) throws SQLException
{
long startTime=System.currentTimeMillis();
System.out.println("Time for sql start is : "+System.currentTimeMillis());
PreparedStatement stmt = null;
boolean action=false;
try {
String sql="INSERT INTO `ce`.`cse`(`twsD`,`twsID`,`twsi`)VALUES(?,?,?)";
stmt = conn.prepareStatement(sql);
stmt.setString(1, u);
stmt.setString(2, d);
stmt.setString(3, l.toString()+"~"+li.toString());
System.out.println(stmt.toString());
action = stmt.execute();
//conn.close();
} catch (SQLException e) {
// handle sql exception
System.out.println("SQL Exception");
e.printStackTrace();
}catch (Exception e) {
// TODO: handle exception for class.forName
System.out.println("Exception");
e.printStackTrace();
}
stmt.close();
long endTime=System.currentTimeMillis();
System.out.println("Time taken inside sql Query is : "+(endTime-startTime));
return action;
}
Below is the log file which am getting.
Time for sql start is : 1486393105661
com.mysql.jdbc.JDBC42PreparedStatement#59037dda: INSERT INTO `ce`.`cse`(`twsD`,`twsID`,`twsi`)VALUES('Bana','2fdb0c926765','[\'FOM\', \'MONEY CENTER KOLA - BAORE\']~[83.80, 272.20]')
Time taken inside sql Query is : 1
Now if you can see, I am not getting any SQL exception or any other kind of exception. And moreover, the time taken is always 1 (when it stops inserting) otherwise it's somewhere between 20-25.
Moreover, thee auto increment ID always gets used up, by that what I mean is if the last row was inserted at ID 1, the subsequent query which I insert through MySQL workbench has an ID somewhere around 40 i.e if we assume that 39 of the remaining rows didn't get inserted.
Taking Mark B's comment as a starting point, I decided to create a connection pool which will be providing the connections.
Below is the code which I used:
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.ResultSet;
import java.sql.SQLException;
import org.apache.commons.dbcp2.BasicDataSource;
public final class Database {
private static final String SQL_EXIST = "show tables;";
public static void main(String[] args) throws SQLException {
// TODO Auto-generated method stub
boolean exist = false;
try (
Connection connection = Database.getConnection();
PreparedStatement statement = connection.prepareStatement(SQL_EXIST);
)
{
try (ResultSet resultSet = statement.executeQuery()) {
exist = resultSet.next();
}
}
System.out.println("Value is : "+ exist);
}
private static final BasicDataSource dataSource = new BasicDataSource();
static {
dataSource.setDriverClassName("com.mysql.jdbc.Driver");
dataSource.setUrl("CONNECTION_STRING");
dataSource.setUsername("USERNAME");
dataSource.setPassword("PASSWORD");
dataSource.setMaxTotal(100);
}
private Database() {
//
}
public static Connection getConnection() throws SQLException {
return dataSource.getConnection();
}
}
FIrst function was just for testing purpose.
Now after creating this Database class, just call Database.getConnection() whenever you need to get the connection. The connection pool will take care of providing you with a valid connection.
Correct me if am wrong.
I spend like 5 hour re-writing all methods cause thought i make mistake while inserting data into DB BUT i noticed everything is fine when i got database outside src folder.
This is my code
public class DataBaseModel {
public static final String DRIVER = "org.sqlite.JDBC";
// public static final String DB_URL = "jdbc:sqlite::resource:sqlite/databse.db"; // insider src/sqlite
public static final String DB_URL = "jdbc:sqlite:database.db";
private Connection conn=null;
private Statement stat=null;
public DataBaseModel() {
try {
Class.forName(DataBaseModel.DRIVER);
} catch (ClassNotFoundException e) {
System.err.println("NO JDBC DRIVER");
e.printStackTrace();
}
try {
conn = DriverManager.getConnection(DB_URL);
stat = conn.createStatement();
} catch (SQLException e) {
System.err.println("Connection problem");
e.printStackTrace();
}
createTables();
}
When i got my project setup like that:
public static final String DB_URL = "jdbc:sqlite:database.db";
everythin work fine:
creating records
display records form db
creating tables
But when i change to
public static final String DB_URL = "jdbc:sqlite::resource:sqlite/databse.db";
there is an error
Connection problem java.sql.SQLException: resource sqlite/database.db not found: java.net.MalformedURLException: no protocol: sqlite/database.db
I get rid of error unconsciously by copping database to src/sqlite thats why i was unable to see any error for all that time ...
Any idea what cause problem when database.db is inside src/sqlite?
I explained as much as i can.
Thx for help
I'm trying to create a table, insert into the table and print the contents of the table using Derby (as shown below).
TestProject class:
package com.user.DerbyTest;
public class TestProject {
public static void main(String[] args) {
DBConnection db = new DBConnection();
db.createTable();
db.insertIntoTable("todd", 23, 'M');
db.insertIntoTable("wayne", 54, 'M');
db.printAll();
}
}
DBConnection class:
package com.user.DerbyTest;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
public class DBConnection {
private static final String DRIVER = "org.apache.derby.jdbc.*";
private static final String JDBC_URL = "jdbc:derby:derbytest;create=true";
Connection conn;
public DBConnection(){
try {
this.conn = DriverManager.getConnection(JDBC_URL);
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (this.conn != null){
System.out.println("Connected to database.");
}
}
public void createTable(){
try {
conn.createStatement().execute("Create TABLE MyDerbytable(Name varchar(50), Age INT, Gender char(1))");
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void insertIntoTable(String name, int age, char gender){
try {
conn.createStatement().execute("INSERT INTO MyDerbytable Values ("+name+","+age+","+gender+")");
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void printAll(){
try {
Statement statement = this.conn.createStatement();
ResultSet res = statement.executeQuery("Select * FROM MyDerbytable");
while(res.next()){
System.out.println(res.getString("Name") + res.getString("Age") + res.getString("Gender"));
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
I'm getting a whole plethra of errors when trying to to run this:
Any ideas?
EDIT: Changing to DROP errors:
Your table already exits. Try DROP TABLE myderbytable;
Your first INSERT statement will end up being:
INSERT INTO MyDerbytable Values (todd,23,M)
You have no quotes around the todd and M values so there are treated as references to column names giving you the error you see. You must enclose literal values like this in single quotes:
INSERT INTO MyDerbytable Values ('todd',23,'M')
So your insert code might be:
"INSERT INTO MyDerbytable Values ('"+name+"','"+age+"','"+gender+"')"
(Note extra ' characters).
But concatenating strings like you are doing leaves you wide open to SQL Injection Attacks. It will also give you problems if any of your input contains a quote character. Learn about using PreparedStatement.
i have java class in which i have done database connection for SQL server and trying to retrieve data from table its working fine. but now i created web service in java for that class then its showing me ClassNotFoundException :net.sourceforge.jtds.jdbc.Driver i have import external jar as well.
Actually i need to show data from database to Android layout that's why i created web service which will helpful to retrieve data from database.
But when i am trying to retrieve data from database then Drivers are not loading.its showing above error.
i also tried like following :
//DatabaseConnetivityClass.java
public class DatabaseConnetivityClass
{
public static void main(String Args[])
{
new DatabaseConnetivityClass().getData();
}
public String getData()
{
String s = null;
try {
s = new MainConnection().getData();
} catch (SQLException e) {
e.printStackTrace();
}
return s;
}
}
MainConnection.java
public class MainConnection {
Connection con;
public MainConnection() {
try{
Class.forName("net.sourceforge.jtds.jdbc.Driver");
con=DriverManager.getConnection("jdbc:jtds:sqlserver://localhost:1433/databasename","username","password");
}
catch (Exception e) {
System.out.println("exp:"+e);
}
}
public String getData() throws SQLException {
StringBuffer sb = new StringBuffer();
Statement select = con.createStatement();
ResultSet result = select.executeQuery("SELECT * FROM Personal_Info");
while (result.next()) {
// process results one row at a time
String val = result.getString(1);
sb.append(val);
System.out.println("val = " + val);
}
select.close();
return sb.toString();
}}
i thought it will work but its giving the same error.
So please help me if anybody knows the solution