I have an JPS Project.
If I have different computers using the system, they use the same MySQL connection.
When the system is running any query and a client tries to make any mysql command, it puts everyone in a queue, the system is very slow.
I want each client has a different connection with mysql.
Sorry if I was not clear enough.
package br.com.scope.model;
import java.io.IOException;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import org.springframework.context.annotation.Scope;
import org.springframework.context.annotation.ScopedProxyMode;
import br.com.scope.log.LoggerIntegrador;
public class ConexaoMysql {
private static Connection connection;
private final static ConexaoMysql conexaoMysql = new ConexaoMysql();
private static Properties prop;
private LoggerIntegrador logger;
private ConexaoMysql() {
super();
prop = new Properties();
Class<? extends ConexaoMysql> cls = this.getClass();
InputStream is = cls.getResourceAsStream("db.properties");
try {
prop.load(is);
} catch (IOException e) {
logger = LoggerIntegrador.getInstance();
logger.error(e.getMessage(), e);
}
}
public static ConexaoMysql getConnection() throws SQLException, ClassNotFoundException, IOException {
if (connection == null || connection.isClosed()){
conexaoMysql.abreConexao();
}
return conexaoMysql;
}
public static void beginTransaction() throws SQLException, ClassNotFoundException, IOException {
getConnection();
connection.setAutoCommit(false);
}
public static void commit() throws SQLException {
connection.commit();
connection.setAutoCommit(true);
}
public static String getDriver() {
return prop.getProperty("driver");
}
public static String getConnectionString() {
return prop.getProperty("connectionstring");
}
public static String getUser() {
return prop.getProperty("user");
}
public static String getPassword() {
return prop.getProperty("password");
}
private void abreConexao() throws ClassNotFoundException, SQLException, IOException{
Class.forName(getDriver());
connection = DriverManager.getConnection(
getConnectionString(),
getUser(),
getPassword());
}
public static void fechaConexao() throws SQLException {
if (!connection.isClosed()) {
connection.close();
}
}
public PreparedStatement getPreparedStatement(String sql) throws SQLException {
return connection.prepareStatement(sql, Statement.RETURN_GENERATED_KEYS);
}
public static int getId() {
return conexaoMysql.hashCode();
}
}
What you're looking for is a connection pool. It is a bounded pool of connections that the clients can take a connection from when they need it, and put it back to when done. This saves you from the overhead or creating and destroying connections all the time. It also makes sure the number of connections grows predictably.
Most containers provide a facility like this e.g. here's the documentation for configuring a connection pool in Tomcat. Find the one for your container.
If for some reason you can not use that or do not want the container to be in charge, you can use a library that helps you manage a connection pool yourself, in your application. c3p0 is a popular one with many examples on the web.
EDIT:
Hikari is the new cool choice for connection pooling.
Related
I was trying to create a Datasource in tomcat with Oracle UCP and my requirement was not add the password at server.xml , I need to add that at server runtime, I have tried in many ways but didn't work.
Here is my code samples
server.xml in tomcat
<Resource
name="testds"
connectionPoolName="testds"
auth="Container"
factory="com.test.tomcat.datasorceEncrypt.CustomizeOracleUCPDataSource"
type="oracle.ucp.jdbc.PoolDataSource"
connectionFactoryClassName="com.test.tomcat.datasorceEncrypt.CustomizeOracleUCPDataSource2"
jmxEnabled="true"
initialPoolSize="10"
minPoolSize="10"
maxPoolSize="300"
fastConnectionFailoverEnabled="true"
url="jdbc:oracle:thin:#(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)
(HOST=localhost)(PORT=3203))(CONNECT_DATA=
(SERVICE_NAME=employee)))"
sqlForValidateConnection="select 1 from DUAL" />
CustomizeOracleUCPDataSource .java
package com.test.tomcat.datasorceEncrypt;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.Properties;
import oracle.ucp.jdbc.PoolDataSourceImpl;
public class CustomizeOracleUCPDataSource extends PoolDataSourceImpl {
/* public CustomizeOracleUCPDataSource() {
Properties dbProperties = null;
try {
this.setPassword("pass");
this.setUser("user");
System.out.println(this.getUser() + "-------" + this.getPassword());
} catch (Exception e) {
}
}*/
#Override
public Connection getConnection() throws SQLException {
try {
this.setPassword("pass");
this.setUser("user");
System.out.println(this.getUser() + "-------" + this.getPassword());
} catch (Exception e) {
}
return super.getConnection(username, password);
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
// TODO Auto-generated method stub
return super.getConnection(username, password);
}
#Override
protected void createPoolWithDefaultProperties() throws SQLException {
try {
this.setPassword("pass");
this.setUser("user");
System.out.println(this.getUser() + "-------" + this.getPassword());
} catch (Exception e) {
}
super.createPoolWithDefaultProperties();
}
}
CustomizeOracleUCPDataSource2.java
package com.test.tomcat.datasorceEncrypt;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.Properties;
import oracle.jdbc.pool.OracleDataSource;
public class CustomizeOracleUCPDataSource2 extends OracleDataSource {
public CustomizeOracleUCPDataSource2() throws SQLException {
super();
this.setPassword("pass");
this.setUser("user");
}
#Override
public Connection getConnection() throws SQLException {
this.setPassword("pass");
this.setUser("user");
return super.getConnection(username, password);
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return super.getConnection(username, password);
}
}
Nothing is working out my user name and passwords were printed in console, but couldn't able to set to the datasource.
I am getting below log info while datasource was getting created.
19-Dec-2019 19:34:35.763 FINE [http-nio-8080-exec-1] oracle.ucp.logging.ClioSupport._log oracle.ucp.jdbc.PoolDataSourceImpl:createPoolWithDefaultProperties:oracle.ucp.jdbc.PoolDataSourceImpl#6b938ce5:Connection pool instance is
19-Dec-2019 19:34:35.803 FINE [http-nio-8080-exec-1] oracle.ucp.logging.ClioSupport._log oracle.ucp.jdbc.PoolDataSourceImpl:createPool:oracle.ucp.jdbc.PoolDataSourceImpl#6b938ce5:Connection pool instance is created
With the details aI am not able to create a datasource I am getting below Error :
Error occurred while establishing connection to the Test database
Any help could be greatly appreciated.
I am using Ojdbc8.jar,ucp.jar
Finally I achieved encryption and decryption of password in tomcat , this solution will work for all your password problems please go to this link for more details.
Tomcat support for password encryption and deryption
Loading class com.mysql.jdbc.Driver'. This is deprecated. The new driver class is com.mysql.cj.jdbc.Driver.
I am updating mysql server and workbench and now I lost connection in all applications in my workspace.
SQL Server is version 8.0.15.
I put it connector8.0.15.jar in all applications but it does not work.
import java.sql.Connection;
import java.sql.DriverManager;
public class DBConnectionManager {
public DBConnectionManager () {}
private static Connection connection;
public static String driverClass = "com.mysql.jdbc.Driver";
public static String connURL = "jdbc:mysql://localhost:3306/mydb";
private static String user = "root";
private static String password = "root";
public static Connection getConnection() {
try {
if(connection == null) {
Class.forName(driverClass);
connection = DriverManager.getConnection(connURL, user, password);
}
} catch (Exception ex) {
ex.printStackTrace();
// TODO: handle exception
}
return connection;
} // this is my connectionManager class, i have connector8.0.15.jar
Your connection statement should look like this：
public static String connURL = "jdbc:mysql://127.0.0.1:3306/work?useUnicode=true&characterEncoding=utf8&serverTimezone=GMT%2B8&useSSL=false";
Class.forName("com.mysql.cj.jdbc.Driver");
Connection conn =(Connection)DriverManager.getConnection(connURL ,"root", "123");
The content behind the connURL may vary from region to region.
I am using neo4-jdbc with pool lib BasicDataSource.
I had huge latency problems so we profiled the app and we found out that opening connection is the cause. I didnt understand why open-connection takes so long we using pool. this is screenshot from our profiles:
This is how the Neo4jDatasourceRemote looks like:
package com.comp.wm.common.repo;
import com.comp.wm.common.utils.Constants;
import org.apache.commons.dbcp.BasicDataSource;
import org.apache.log4j.Logger;
import org.springframework.beans.factory.annotation.Value;
import javax.annotation.PostConstruct;
import java.sql.Connection;
import java.sql.SQLException;
private final Logger logger = Logger.getLogger(Neo4jDataSourceRemote.class);
private BasicDataSource ds;
#Value("${neo4j.host:localhost}")
private String NEO4J_HOST;
#Value("${neo4j.port:7474}")
private String NEO4J_PORT;
#Value("${neo4j.username:nouser}")
private String NEO4J_USERNAME;
#Value("${neo4j.password:nopass}")
private String NEO4J_PASSWORD;
#Value("${neo4j.pool.size:200}")
private int NEO4J_POOL_SIZE;
private String GetUrl() {
return String.format(Constants.NEO4J_JDBC_CONNECTIVITY_STRING, NEO4J_HOST, NEO4J_PORT);
}
#PostConstruct
public void init(){
ds = new BasicDataSource();
ds.setInitialSize(300);
ds.setDriverClassName("org.neo4j.jdbc.Driver");
ds.setUrl(GetUrl());
ds.setUsername(NEO4J_USERNAME);
ds.setPassword(NEO4J_PASSWORD);
}
#Override
public Connection openConnection() throws SQLException {
return this.ds.getConnection();
}
#Override
public void closeConnection(Connection conn) {
try {
if (conn != null)
conn.close();
} catch (SQLException ex) {
logger.info("error closing connection", ex);
}
}
}
and this is a sample of how I execute query against the graph:
public List<NearbyItem> executeQuery(..) {
conn = neo4jDataSource.openConnection();
String getUsersStatement = "some query..";
try (PreparedStatement ps = conn.prepareStatement(getUsersStatement)) {
..
ResultSet rs = ps.executeQuery();
while (rs.next()) {
...
}
} catch (Exception e) {
throw new RuntimeException("Error returning userId=" + userIdInput, e);
} finally {
neo4jDataSource.closeConnection(conn);
}
return distItemDatas;
}
any ideas?
Based on comments above, I'll add this as a reply.
By default Neo4j runs in the http interface 10 threads for core. You can tweak the total number of threads in neo4j-server.properties
org.neo4j.server.webserver.maxthreads=200
However the more threads you have the more you're suffering from context switches and lock contention. If you increase the number of threads I won't expect a large increase of throughput, you just shift the point where you have to wait. From initialization (openCOnnection) to processing the query.
As part of my project I am trying to connect it with database. I searched in google for the code and I got the following code. In that I don't understand 2 things - "import com.mysql.jdbc.Driver;" and "new Driver". What do these 2 mean ?
package javasql;
import com.mysql.jdbc.Driver;
import java.sql.*;
public class Connect {
public Connect() throws SQLException{
makeConnection();
}
private Connection koneksi;
public Connection makeConnection() throws SQLException {
if (koneksi == null) {
new Driver();
// buat koneksi
koneksi = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/mysql","root","virus");
}
return koneksi;
}
public static void main(String args[]) {
try {
Connect c = new Connect();
System.out.println("Connection established");
}
catch (SQLException e) {
e.printStackTrace();
System.err.println("Connection Failure");
}
}
}
package javasql;
import java.sql.*;
public class SqlStatement {
private Statement statement;
public SqlStatement() throws SQLException{
makeStatement();
}
public Statement makeStatement() throws SQLException{
Connect c = new Connect();
Connection conn = c.makeConnection();
statement = conn.createStatement();
return statement;
}
public void insert(String name,int npm)throws SQLException{
statement.execute("insert into Student values(\""+name+"\","+npm+");");
}
public static void main(String arg[]){
try {
SqlStatement s = new SqlStatement();
s.insert("Ferdi2",3);
s.insert("Anca2",3);
System.out.println("Success");
}
catch(SQLException e){
System.out.println("Failed");
e.printStackTrace();
}
}
}
I use NetBeans IDE to develop my project. When I used these codes I made it as a new project. Then it worked fine. But whenever I tried to include these codes in another projects errors are showing at "import com.mysql.jdbc.Driver;". Why is it so ? Can I use these 2 codes in another projects ?
The driver serves as an interface between your application and the database.
Are you using MySQL? If so, you can find the MySQl Java drivers here.
All you need is
// This will load the MySQL driver, each DB has its own driver
Class.forName("com.mysql.jdbc.Driver")
This acts like class loader and load your driver class for you. For that you need to add the corresponding jar file.
Using import com.mysql.jdbc.Driver; in JDBC code is not a good practice
and you need to import only java.sql.* and javax.sql.*. The reason is to detach the code from the specific driver implementation.
See here for more information how to make JDBC connections. And DriverManager.getConnection(...) is enough to get connection.
I have written a simple Java code, which attempts to store files in an hsqldb database. All it does is reads the files from a certain directory and puts them in the DB. It is single threaded, but I am using pooled connections from the apache commons.dbcp in order to be able to cope with the multithreaded access later.
The problem is that the code blocks after reading a few files.
Please, find below the entire source code.
Program.java
import java.io.File;
import java.io.IOException;
import java.sql.SQLException;
import javax.sql.DataSource;
import org.apache.commons.dbcp.ConnectionFactory;
import org.apache.commons.dbcp.DriverManagerConnectionFactory;
import org.apache.commons.dbcp.PoolableConnectionFactory;
import org.apache.commons.dbcp.PoolingDataSource;
import org.apache.commons.pool.KeyedObjectPoolFactory;
import org.apache.commons.pool.impl.GenericKeyedObjectPoolFactory;
import org.apache.commons.pool.impl.GenericObjectPool;
public class Program {
public static DataSource getPoolingDataSource(String driverClass, String url, String user, String password) throws ClassNotFoundException {
Class.forName(driverClass);
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(url, user, password);
GenericObjectPool connectionPool = new GenericObjectPool();
KeyedObjectPoolFactory stmtPool = new GenericKeyedObjectPoolFactory(null);
new PoolableConnectionFactory(connectionFactory, connectionPool, stmtPool, null, false, true);
return new PoolingDataSource(connectionPool);
}
public static void main(String[] args) throws ClassNotFoundException, SQLException, IOException, InterruptedException {
String root = args.length == 0 ? "c:/Work/java/ntxdb" : args[0];
Runtime run = Runtime.getRuntime();
Process pr = run.exec("cmd /c del /s/q c:\\tmp\\file.db*");
pr.waitFor();
DataSource ds = getPoolingDataSource("org.hsqldb.jdbcDriver", "jdbc:hsqldb:file:c:/tmp/file.db", "sa", "");
HsqldbFileStorage fs = new HsqldbFileStorage(ds);
putFiles(fs, new File(root));
}
private static void putFiles(HsqldbFileStorage fs, File parent) throws IOException, SQLException {
for (File child : parent.listFiles()) {
if (child.isDirectory()) {
putFiles(fs, child);
} else {
System.out.println(child.getCanonicalPath());
fs.put(child);
}
}
}
}
HsqldbFileStorage.java
import java.io.BufferedInputStream;
import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.sql.Connection;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import javax.sql.DataSource;
public class HsqldbFileStorage {
private static final String SET_SQL = "MERGE INTO test" +
" USING (VALUES ?, CAST(? AS BLOB)) I (name, data)" +
" ON (test.name=I.name)" +
" WHEN MATCHED THEN UPDATE SET test.data = I.data" +
" WHEN NOT MATCHED THEN INSERT (name, data) VALUES (I.name, I.data)";
private DataSource m_dataSource;
public HsqldbFileStorage(DataSource dataSource) throws SQLException {
super();
m_dataSource = dataSource;
Connection c = dataSource.getConnection();
c.createStatement().execute("Create Cached Table IF NOT EXISTS test (name VARCHAR(256), data BLOB(10M));");
}
public void put(File file) throws IOException, SQLException {
put(file.getCanonicalPath(), file);
}
public void put(String name, File file) throws IOException, SQLException {
InputStream is = new BufferedInputStream(new FileInputStream(file));
try {
put(name, is);
} finally {
is.close();
}
}
public void put(String name, InputStream data) throws SQLException, IOException {
PreparedStatement set = m_dataSource.getConnection().prepareStatement(SET_SQL);
try {
set.setString(1, name);
set.setBinaryStream(2, data);
set.executeUpdate();
} finally {
set.close();
}
}
}
The code depends on commons-dbcp 1.4, commons-pool 1.6 and hsqldb 2.2.9
Running it on the project directory itself should put 62 files in the DB (I have much more files there than the two aforementioned source files), printing a line for each file.
Unfortunately, it blocks on the eighth file with the following stack trace:
at java.lang.Object.wait(Object.java:-1)
at java.lang.Object.wait(Object.java:485)
at org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1118)
at org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106)
at HsqldbFileStorage.put(HsqldbFileStorage.java:41)
at HsqldbFileStorage.put(HsqldbFileStorage.java:34)
at HsqldbFileStorage.put(HsqldbFileStorage.java:28)
at Program.putFiles(Program.java:42)
at Program.putFiles(Program.java:39)
at Program.putFiles(Program.java:39)
at Program.main(Program.java:33)
What am I doing wrong?
You are calling:
m_dataSource.getConnection()
in your put(String, InputStream) method. This creates a new connection, which you never close.
So when you are hitting putting 62 files in at some stage it will hit the maximum number of connections in your pool, after which the pool will wait for a connection to be returned to the pool.
If you modify your method as such:
public void put(String name, InputStream data) throws SQLException, IOException {
Connection con = null;
PreparedStatement set = null;
try {
con = m_dataSource.getConnection();
set.prepareStatement(SET_SQL);
set.setString(1, name);
set.setBinaryStream(2, data);
set.executeUpdate();
} finally {
if (set != null) {
set.close();
}
if (con != null) {
con.close();
}
}
}
Note, when you access connections through a pool, you still need to call close. This won't actually close the connection but return it to the pool.