I'm writing Java application with Apache Camel and use JDBC DataSource(Apache commons DBCP).
I can create many connections but all connections are handled in one Oracle client process, what I may see in sessions table select * from v$session.
The question is how to connect from one Java application to different Oracle processes to improve performance? If you know a way to do it using other Java technologies, not used in my example, it is also very interesting.
public static void main(String... args) throws Exception {
Main main = new Main();
String url = "some url";
String user = "user";
String password = "password";
DataSource dataSource = setupDataSource(url, user, password);
// bind dataSource into the registery
main.bind("oraDataSource", dataSource);
main.enableHangupSupport();
main.addRouteBuilder(new MyRouteBuilder());
main.run(args);
}
private static DataSource setupDataSource(String connectURI, String user, String password) {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setMaxActive(20);
LOG.info("max active conn: " + ds.getMaxActive());
ds.setUsername(user);
ds.setPassword(password);
ds.setUrl(connectURI);
return ds;
}
public class MyRouteBuilder extends RouteBuilder {
Processor logProcessor = new LogProcessor();
Processor createAnswer = new CreateAnswerProc();
Processor dbPaymentProcessor = new DbPaymentQueryProcessor();
/**
* Let's configure the Camel routing rules using Java code...
*/
public void configure() {
from("rabbitmq://localhost:5672/ps.vin_test_send?exchangeType=topic&autoDelete=false&queue=ps.test_send_queue&concurrentConsumers=20&threadPoolSize=20")
.unmarshal().json(JsonLibrary.Jackson, Payment[].class)
.process(dbPaymentProcessor)
.to("jdbc:oraDataSource")
.process(logProcessor)
.process(createAnswer)
.to("rabbitmq://localhost:5672/ps.vin_test?username=test&password=test&exchangeType=topic&autoDelete=false&routingKey=ps.vin_test_key&queue=vin_test_queue&concurrentConsumers=20");
}
So, the way for oracle is to change db propeties from dedicated to shared. And it will share processes between users. but it is not recomended and not give any performance.
Creation of multiple datasources only reduces performance.
Related
I have a MongoDB with two redundant MongoS router hosts. When using org.springframework.data.mongo to create a MongoTemplate and MongoClient, I can only add a single host. In the event that the host in use falls over, there is no failover to the alternate router host.
I initially referenced https://dzone.com/articles/multiple-mongodb-connectors-with-spring-boot , but the use case there is for two entirely different repositories, where as my case is a single database with dual routers.
In the code below, we would like to add a redundant second host in case the first host fails during runtime.
public class MongoConfiguration extends AbstractMongoConfiguration {
#Value("${mongo.database}")
private String databaseName;
#Value("${mongo.host}")
private String host;
#Value("${mongo.readFromSecondary}")
private String readFromSecondary;
#Value("${mongo.port}")
private int port;
#VaultKey("vault.mongo_username")
private String username;
#VaultKey("vault.mongo_password")
private String password;
#Override
protected String getDatabaseName() {
return databaseName;
}
#Override
#Primary
public MongoClient mongoClient() {
final ServerAddress serverAddress = new ServerAddress(host, port);
final MongoCredential credential = MongoCredential.createCredential(username,
getDatabaseName(), password.toCharArray());
return new MongoClient(serverAddress, credential,
MongoClientOptions.builder().build());
}
#Override
#Primary
#Bean(name = "mongoTemplate")
public MongoTemplate mongoTemplate() throws Exception {
final MongoTemplate template = super.mongoTemplate();
if (this.readFromSecondary != null && Boolean.valueOf(this.readFromSecondary)) {
template.setReadPreference(ReadPreference.secondary());
}
return template;
}
}
Currently at startup a connection to the host in the config file will be loaded with out error, we would like to rotate in a backup host.
You can achieve this by two ways:
1. Multiple Mongo Client or multiple Server Address(hosts):
A MongoDB client with internal connection pooling. For most applications, you should have one MongoClient instance for the entire JVM.
The following are equivalent, and all connect to the local database running on the default port:
MongoClient mongoClient1 = new MongoClient();
MongoClient mongoClient1 = new MongoClient("localhost");
MongoClient mongoClient2 = new MongoClient("localhost", 27017);
MongoClient mongoClient4 = new MongoClient(new ServerAddress("localhost"));
MongoClient mongoClient5 = new MongoClient(new ServerAddress("localhost"),
new MongoClientOptions.Builder().build());
You can connect to a replica set using the Java driver by passing a ServerAddress list to the MongoClient constructor. For example:
MongoClient mongoClient = new MongoClient(Arrays.asList(
new ServerAddress("localhost", 27017),
new ServerAddress("localhost", 27018),
new ServerAddress("localhost", 27019)));
You can connect to a sharded cluster using the same constructor. MongoClient will auto-detect whether the servers are a list of replica set members or a list of mongos servers.
By default, all read and write operations will be made on the primary, but it's possible to read from secondaries by changing the read preference:
mongoClient.setReadPreference(ReadPreference.secondaryPreferred());
By default, all write operations will wait for acknowledgment by the server, as the default write concern is WriteConcern.ACKNOWLEDGED
2. Using Multiple Mongo Connectors and Multiple Mongo Templates:
First of all create the following #ConfigurationProperties class.
#ConfigurationProperties(prefix = "mongodb")
public class MultipleMongoProperties {
private MongoProperties primary = new MongoProperties();
private MongoProperties secondary = new MongoProperties();
}
And then add the following properties in the application.yml
mongodb:
primary:
host: localhost
port: 27017
database: second
secondary:
host: localhost
port: 27017
database: second
Now it’s necessary to create the MongoTemplates to bind the given configuration in the previous step.
#EnableConfigurationProperties(MultipleMongoProperties.class)
public class MultipleMongoConfig {
private final MultipleMongoProperties mongoProperties;
#Primary
#Bean(name = "primaryMongoTemplate")
public MongoTemplate primaryMongoTemplate() throws Exception {
return new MongoTemplate(primaryFactory(this.mongoProperties.getPrimary()));
}
#Bean(name = "secondaryMongoTemplate")
public MongoTemplate secondaryMongoTemplate() throws Exception {
return new MongoTemplate(secondaryFactory(this.mongoProperties.getSecondary()));
}
#Bean
#Primary
public MongoDbFactory primaryFactory(final MongoProperties mongo) throws Exception {
return new SimpleMongoDbFactory(new MongoClient(mongo.getHost(), mongo.getPort()),
mongo.getDatabase());
}
#Bean
public MongoDbFactory secondaryFactory(final MongoProperties mongo) throws Exception {
return new SimpleMongoDbFactory(new MongoClient(mongo.getHost(), mongo.getPort()),
mongo.getDatabase());
}
}
With the configuration above you’ll be able to have two different MongoTemplates based in the custom configuration properties that we provided previously in this guide.
In the previous step we created two MongoTemplates, primaryMongoTemplate and secondaryMongoTemplate
More details: https://blog.marcosbarbero.com/multiple-mongodb-connectors-in-spring-boot/
I'm using the DataSourceFactory of dropwizard and H2 to create an in-memory db for testing. Here is what I have
private static final String DBNAME = String.format("JDBITest-%d", System.currentTimeMillis());
protected final DataSourceFactory config = new DataSourceFactory();
{
final String url = String.format("jdbc:h2:mem:%s;", DBNAME) +
"MODE=MySQL;" +
"TRACE_LEVEL_FILE=3;" +
"DB_CLOSE_DELAY=-1;" +
"IGNORECASE=TRUE";
System.out.println("Creating in memory H2 using " + url);
BootstrapLogging.bootstrap();
config.setUrl(url);
config.setUser("sa");
config.setDriverClass("org.h2.Driver");
config.setValidationQuery("SELECT 1");
}
#Before
public void setUp() throws Exception {
Server server = Server.createTcpServer().start(); // (4)
System.out.println("Server started and connection is open.");
System.out.println("URL: jdbc:h2:" + server.getURL() + "/mem:" + DBNAME);
}
When this runs I see
Creating in memory H2 using jdbc:h2:mem:JDBITest-1541641621470;MODE=MySQL;TRACE_LEVEL_FILE=3;DB_CLOSE_DELAY=-1;IGNORECASE=TRUE
Server started and connection is open.
URL: jdbc:h2:tcp://0.0.17.56:9092/mem:JDBITest-1541641621470
Why is the TCP 0.0.17.56? I cannot access this nor can I use the H2.jar to access the shell.
There are several wrong things here.
H2 has different connections modes. In your example you configure DataSourceFactory for in memory connection, but then in #Before method you create new instance of H2 with tcp based connection. More about connection modes here.
So now you have basically 2 instances of H2, in memory and tcp and they are completely unrelated. So you probably need to have only 1 connection type configured for you tests.
If you want to connect to your H2 db outside of JVM (from browser for example) - then you need to have tcp-based connection.
To be able to connect to db from browser you also need to run console application. Which should be inside h2.jar with command like that java -jar h2*.jar. More about this here.
And finally, this peace of code should suitable for you (with in-memory connection):
private static final String DBNAME = String.format("JDBITest-%d", System.currentTimeMillis());
private ManagedDataSource dataSource;
#Before
public void setUp() {
System.out.println("Server started and connection is open.");
final String url = String.format("jdbc:h2:mem:%s;", DBNAME) +
"MODE=MySQL;" +
"TRACE_LEVEL_FILE=3;" +
"DB_CLOSE_DELAY=-1;" +
"IGNORECASE=TRUE";
System.out.println("Creating in memory H2 using " + url);
DataSourceFactory config = new DataSourceFactory();
BootstrapLogging.bootstrap();
config.setUrl(url);
config.setUser("sa");
config.setDriverClass("org.h2.Driver");
config.setValidationQuery("SELECT 1");
dataSource = config.build(null, "test");
}
#Test
public void test() throws SQLException {
Connection connection = dataSource.getConnection();
connection.createStatement().executeUpdate("CREATE TABLE TEST (`id` INT)");
connection.createStatement().executeUpdate("INSERT INTO TEST (`id`) VALUES (1)");
ResultSet resultSet1 = connection.createStatement().executeQuery("SELECT * FROM TEST WHERE `id` = 1");
resultSet1.next();
resultSet1.getInt(1);
System.out.println("Found ID: " + resultSet1.getInt(1));
}
Out:
Server started and connection is open.
Creating in memory H2 using jdbc:h2:mem:JDBITest-1541649996267;MODE=MySQL;TRACE_LEVEL_FILE=3;DB_CLOSE_DELAY=-1;IGNORECASE=TRUE
Found ID: 1
I'm trying to write a simple Java sample application which uses Flyway to migrate many databases on one IP. When I run this application it completes without error but nothing seems to happen, the schema_version table is not created.
public static void main(String[] args) {
ApplicationContext ctx = new ClassPathXmlApplicationContext("databaseConfig.xml");
ClientDAO clientDao = ctx.getBean("clientDao", ClientDAO.class);
List<String> clients = clientDao.getClients();
Flyway flyway = new Flyway();
for (String client : clients) {
flyway.setDataSource("jdbc:mysql://" + STAGE_DB_IP +"/" + client, DB_USER, DB_PASSWORD);
flyway.baseline();
flyway.setBaselineOnMigrate(true);
flyway.migrate();
}
System.out.println("Done");
}
The databases already exist with tables.
I am developing a simple CRUD application, using JDBC to establish connection and perform basic CRUD operations. In that process, created a DatabaseListener to create a connection object at startup and storing it in the context attribute for reuse.
Below is the code.
import javax.servlet.ServletContext;
import javax.servlet.ServletContextEvent;
import javax.servlet.ServletContextListener;
import org.apache.log4j.Logger;
public class DatabaseInitListner implements ServletContextListener {
private static final Logger LOG = Logger.getLogger(DatabaseInitListner.class);
private DBUtil databaseUtil = null;
#Override
public void contextDestroyed(ServletContextEvent event) {
databaseUtil.closeConnection();
}
#Override
public void contextInitialized(ServletContextEvent contextinitEvent) {
ServletContext servletContext = contextinitEvent.getServletContext();
String database = servletContext.getInitParameter("db_name");
String url = servletContext.getInitParameter("db_url")
+ database;
String username = servletContext.getInitParameter("db_user");
String password = servletContext.getInitParameter("db_password");
String driverName = servletContext.getInitParameter("db_driver");
databaseUtil = new DBUtil(url, username, password,
driverName);
servletContext.setAttribute("databaseSingleConnectionObject",
databaseUtil.getConnection());
}
}
public class DBUtil {
private Connection connection = null;
private static final Logger LOG = Logger.getLogger(DatabaseUtil.class);
public DatabaseUtil(String url, String username, String password,
String driver) {
try {
Class.forName(driver);
this.connection = DriverManager.getConnection(url, username,
password);
LOG.debug("Connection Established... ");
} catch (ClassNotFoundException | SQLException e) {
LOG.error("Could not create connection... ", e);
}
}
public Connection getConnection() {
return connection;
}
public void closeConnection() {
if (connection != null) {
try {
connection.close();
} catch (SQLException e) {
LOG.error("Unable to close connection... ", e);
}
}
}
}
I am accessing the connection in servlets like this
Connection jdbcConnection = (Connection) getServletContext().getAttribute("databaseSingleConnectionObject");
I am not sure if this is right approach. What are the effects of single database connection?
When you use a single database connection like this you make your application slow and brittle.
Slow: because the connection implementation is synchronized, each user has to wait until the connection is free. If one user's query takes a while to come back that directly increases the time any other concurrent users spend waiting. If there were multiple connections available from a pool then the time spent by one user would not impact other users nearly as greatly (unless a query's results take all the JVM's memory or a big query bogs down the database server).
Brittle: The connection is a network connection, they tend to go down. Without a provision to create new connections any kind of timeout, network hiccup, or period of database non-availability (such as taking the database offline for maintenance) is going to require an application restart. Using a connection pool will mean your application will be able to survive these episodes and recover without outside intervention.
This will not be threadsafe, and if it were, performance would be really poor.
Look into using a Connection Pool, like DBCP or C3PO
You should let your application server manage database connection. Add a JNDI datasource in its configuration file and make a lookup from your application to get a connection when needed (for instance when you instantiate a class that must access your database).
You may configure the datasource to manage a connection pool so that each user session will get its own.
Depending on the AS you use run a search with keywords 'JNDI' and 'datasource' and you will get further details about the AS configuration and how to implement it in your application.
I have a server that submits a query to a remote host to access account information from a database to log into a gameserver. The problem is the connections time out randomly however, I am using DataSource which should automatically re-establish any lost connections. Anyone have a clue how to resolve the issue?
public class Database {
private final PoolingDataSource source;
public Database(String driver, String url, String user, String password)
throws ClassNotFoundException {
Class.forName(driver);
source = createSource(url, user, password);
}
#SuppressWarnings({ "rawtypes", "unchecked", "unused" })
private PoolingDataSource createSource(String connectURI, String user,
String password) {
GenericObjectPool.Config config = new GenericObjectPool.Config();
config.maxActive = 150;
config.maxIdle = 100;
config.minIdle = 30;
config.maxWait = 1000;
config.testWhileIdle = true;
ObjectPool connectionPool = new GenericObjectPool(null, config);
ConnectionFactory connectionFactory = new DriverManagerConnectionFactory(
connectURI, user, password);
PoolableConnectionFactory poolableConnectionFactory = new PoolableConnectionFactory(
connectionFactory, connectionPool, null, null, false, true);
PoolingDataSource poolingDataSource = new PoolingDataSource(
connectionPool);
return poolingDataSource;
}
public Connection getConnection() throws SQLException {
return getSource().getConnection();
}
public PoolingDataSource getSource() {
return source;
}
}
"Back in the day...", in 2003, I was working with Tomcat 4.1 and was unpleasantly surprised to find that its implementation of DataSource required you to give it a validationQuery or else it would initialize the Connection only once and never verify that database server side would still respect the connection. Our Oracle server would simply eliminate connections that hadn't been used for, I think, 60 minutes. The effect of this was that as my server went to low volume, some number of my pooled connections would get killed on the database server side, but the application server didn't know, until it tried to use them at higher volumes. At this point they just stacktraced and that was the end of it. Adding the validationQuery had the effect of the DataSource itself keeping the connection alive and everything just worked. The gory details of this discovery process is here https://groups.yahoo.com/neo/groups/seajug/conversations/messages/4902 , if you are interested.
I recommend that you check if your GenericObjectPool implementation has a concept of validation query or heartbeat or something and figure out how to leverage it to keep your connections "fresh".