We Keep Coding

In memory SQLITE in a Spring Boot application not working

I am trying to setup SQLITE as an in-memory database in my spring-boot application. But when i try to query the database then it gives me an error "No such table"
Can someone please recommend what am i doing wrong? I need to have SQLITE as a in memory only and we only use jdbc in our project.
Here is my code:
application.properties
spring.datasource.url=jdbc:sqlite:memory
spring.datasource.username=
spring.datasource.password=
spring.datasource.platform=sqlite
spring.datasource.driver-class-name=org.sqlite.JDBC
MyRepo.java
#Repository
public class MyRepo{
#Autowired
private NamedParameterJdbcTemplate namedJdbc;
public String getUserName() throws Exception{
String userName = null;
String sql = "SELECT username FROM emp WHERE username=:name";
MapSqlParameterSource paramSource = new MapSqlParameterSource();
paramSource.addValue("name", "tuser");
userName = this.namedJdbc.query(sql, paramSource, (rs) -> {
String name = null;
while (rs.next()) {
name = rs.getString("username").trim();
return name;
}
return null;
});
return userName;
}
}
UserDaoTest.java
#SpringBootTest
public class UserDaoTest {
#Autowired
private MyRepo rep;
#Test
public void testFindByname() throws Exception{
rep.getUserName();
}
}
I also have schema.sql and data.sql files under src/main/resources
schema.sql
DROP TABLE IF EXISTS emp;CREATE TABLE IF NOT EXISTS emp(username VARCHAR(20), empId BIGINT, PRIMARY KEY(empId) )
data.sql
INSERT INTO emp(username,empId) VALUES ('tuser',1001);
Exception that i am getting:
PreparedStatementCallback; uncategorized SQLException for SQL [SELECT username FROM Chats WHERE username=?]; SQL state [null]; error code [1]; [SQLITE_ERROR] SQL error or missing database (no such table: Chats)

well, I am shooting in the dark but looks like you need to add the schema for 'Chats' table as well to your schema.sql

https://sqlite.org/inmemorydb.html
The database ceases to exist as soon as the database connection is closed. Every :memory: database is distinct from every other. So, opening two database connections each with the filename ":memory:" will create two independent in-memory databases.
Your issue might be with spring boot opening multiple connections due to its connection pool configuration. If you're using hikari connection pool (default in newer spring boot versions), try adding these properties
spring.datasource.hikari.maximum-pool-size=1
spring.datasource.hikari.max-lifetime=0

H2 in-memory creating a server to access via Shell

I'm using the DataSourceFactory of dropwizard and H2 to create an in-memory db for testing. Here is what I have
private static final String DBNAME = String.format("JDBITest-%d", System.currentTimeMillis());
protected final DataSourceFactory config = new DataSourceFactory();
{
final String url = String.format("jdbc:h2:mem:%s;", DBNAME) +
"MODE=MySQL;" +
"TRACE_LEVEL_FILE=3;" +
"DB_CLOSE_DELAY=-1;" +
"IGNORECASE=TRUE";
System.out.println("Creating in memory H2 using " + url);
BootstrapLogging.bootstrap();
config.setUrl(url);
config.setUser("sa");
config.setDriverClass("org.h2.Driver");
config.setValidationQuery("SELECT 1");
}
#Before
public void setUp() throws Exception {
Server server = Server.createTcpServer().start(); // (4)
System.out.println("Server started and connection is open.");
System.out.println("URL: jdbc:h2:" + server.getURL() + "/mem:" + DBNAME);
}
When this runs I see
Creating in memory H2 using jdbc:h2:mem:JDBITest-1541641621470;MODE=MySQL;TRACE_LEVEL_FILE=3;DB_CLOSE_DELAY=-1;IGNORECASE=TRUE
Server started and connection is open.
URL: jdbc:h2:tcp://0.0.17.56:9092/mem:JDBITest-1541641621470
Why is the TCP 0.0.17.56? I cannot access this nor can I use the H2.jar to access the shell.

There are several wrong things here.
H2 has different connections modes. In your example you configure DataSourceFactory for in memory connection, but then in #Before method you create new instance of H2 with tcp based connection. More about connection modes here.
So now you have basically 2 instances of H2, in memory and tcp and they are completely unrelated. So you probably need to have only 1 connection type configured for you tests.
If you want to connect to your H2 db outside of JVM (from browser for example) - then you need to have tcp-based connection.
To be able to connect to db from browser you also need to run console application. Which should be inside h2.jar with command like that java -jar h2*.jar. More about this here.
And finally, this peace of code should suitable for you (with in-memory connection):
private static final String DBNAME = String.format("JDBITest-%d", System.currentTimeMillis());
private ManagedDataSource dataSource;
#Before
public void setUp() {
System.out.println("Server started and connection is open.");
final String url = String.format("jdbc:h2:mem:%s;", DBNAME) +
"MODE=MySQL;" +
"TRACE_LEVEL_FILE=3;" +
"DB_CLOSE_DELAY=-1;" +
"IGNORECASE=TRUE";
System.out.println("Creating in memory H2 using " + url);
DataSourceFactory config = new DataSourceFactory();
BootstrapLogging.bootstrap();
config.setUrl(url);
config.setUser("sa");
config.setDriverClass("org.h2.Driver");
config.setValidationQuery("SELECT 1");
dataSource = config.build(null, "test");
}
#Test
public void test() throws SQLException {
Connection connection = dataSource.getConnection();
connection.createStatement().executeUpdate("CREATE TABLE TEST (`id` INT)");
connection.createStatement().executeUpdate("INSERT INTO TEST (`id`) VALUES (1)");
ResultSet resultSet1 = connection.createStatement().executeQuery("SELECT * FROM TEST WHERE `id` = 1");
resultSet1.next();
resultSet1.getInt(1);
System.out.println("Found ID: " + resultSet1.getInt(1));
}
Out:
Server started and connection is open.
Creating in memory H2 using jdbc:h2:mem:JDBITest-1541649996267;MODE=MySQL;TRACE_LEVEL_FILE=3;DB_CLOSE_DELAY=-1;IGNORECASE=TRUE
Found ID: 1

Check if h2 database is corrupted and create new if corrupted

I'm a C# developer and need to maintain an existing Java Service application developed using spring boot framework. The responsible developer left the company some time ago so I have no possibility to get some help...
So far I have no experience with Java and the used spring boot framework.
What I need to achive:
Check if used h2 Database is corrupted
If corrupted: Delete the database and create a new empty one
I guess I need to implement the check and recreation in the main entry point
public static void main(String[] args) {
SpringApplication.run(MessageServiceApplication.class, args);
}
As I know spring-boot and Hibernate creates the db automatically on startup if the database does not exists. So far so good. Now I need to check if the database is corrupted. I thought about executing a query on the database and if I get an exception I recreate the database.
The Database is a h2 file database.
Hopefully I can get some assistance.
Edit #1
I thought about implementing a utils class which gets called on startup:
public class H2DbUtils {
public boolean IsH2FileDatabaseCorrupted()
{
boolean isCorrupted = false;
// Implement Logic to determine if db is corrupted
return isCorrupted;
}
public boolean ReCreateH2DatabaseFile()
{
boolean reCreated = false;
// Implement Logic to recreate db
return reCreated;
}
}
Calling this class on startup
public static void main(String[] args) {
H2DbUtils h2DbUtils = new H2DbUtils();
if(h2DbUtils.IsH2FileDatabaseCorrupted()) {
h2DbUtils.ReCreateH2DatabaseFile();
}
SpringApplication.run(MessageServiceApplication.class, args);
}
Update 2018-03-20
Currently found the following solution to achive this:
#Configuration
#Component
public class DataSourceBean {
#Autowired
private Environment currentEnvironment;
private final Logger logInstance = LoggerFactory.getLogger(this.getClass());
#Bean
#Primary
public DataSource dataSource()
{
DataSource dataSource = null;
try
{
// We try to get the Meta Data out of the database.
// If this fails the database is corrupted or has an other problem
// All in all this means we need to delete the current database file
// to avoid further problems.
dataSource = this.getDataSource();
dataSource.getConnection().getMetaData();
return dataSource;
}
catch (Exception ex)
{
logInstance.error("The h2 database file '{}' seems to be corrupted! Error: {}",
currentEnvironment.getProperty("dataBaseFile"),
ex.getMessage());
// dataBaseFile=./db/mydatabase.db
String databaseFilePath = String.format("%s.%s", currentEnvironment.getProperty("dataBaseFile"), "h2.db");
databaseFilePath = databaseFilePath.replace("/", "\\");
File databaseFile = new File(databaseFilePath);
if (databaseFile.exists()) {
File parentDirectory = new File(databaseFile.getParent());
if (parentDirectory.isDirectory()) {
try {
FileUtils.deleteDirectory(parentDirectory);
} catch (Exception fex) {
logInstance.error("Error occurred deleting the folder {}. Error: {}",
parentDirectory.getAbsolutePath(),
fex.getMessage());
}
}
}
dataSource = this.getDataSource();
}
finally {
return dataSource;
}
}
#ConfigurationProperties(prefix = "spring.datasource")
private DataSource getDataSource() {
return DataSourceBuilder.create()
.url(currentEnvironment.getProperty("spring.datasource.url"))
.driverClassName(currentEnvironment.getProperty("spring.datasource.driverClassName"))
.username(currentEnvironment.getProperty("spring.datasource.username"))
.password(currentEnvironment.getProperty("spring.datasource.password"))
.build();
}

It is possible to overwrite the DataSource bean and check database files
#Bean
#Primary // this will override the datasource autoconfiguration and use your own everywhere
public DataSource dataSource() {
// Open Connection
// Check Database
// Close Connection
// IF File corrupted delete files
// create regular data source
}

I've tried to add several listeners to the spring boot application, for example:
SpringApplication springApplication = new SpringApplication(testApplication.class);
springApplication.addListeners(new FailedEvent(testApplication.class));
SpringApplication.run(testApplication.class, args);
But I never get to one of this listeners in the startup of the spring application. As SpringApplication.run seems to initzialise the whole spring context it is also not possible to inject or get the configuration environment to get the connection string as the application stops within SpringApplication.run as the db is corrupted.
I assume that spring tries to initzialise hibernate and so on and fails to create a database connection as the db is corrupted
org.h2.jdbc.JdbcSQLException: Allgemeiner Fehler: "java.lang.RuntimeException: rowcount remaining=2 SYS"
General error: "java.lang.RuntimeException: rowcount remaining=2 SYS" [50000-196]
at org.h2.message.DbException.getJdbcSQLException(DbException.java:345) ~[h2-1.4.196.jar:1.4.196]
at org.h2.message.DbException.get(DbException.java:168) ~[h2-1.4.196.jar:1.4.196]
at org.h2.message.DbException.convert(DbException.java:295) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Database.openDatabase(Database.java:307) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Database.<init>(Database.java:270) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.openSession(Engine.java:64) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.openSession(Engine.java:176) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.createSessionAndValidate(Engine.java:154) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.createSession(Engine.java:137) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.Engine.createSession(Engine.java:27) ~[h2-1.4.196.jar:1.4.196]
at org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:354) ~[h2-1.4.196.jar:1.4.196]
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:116) ~[h2-1.4.196.jar:1.4.196]
at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:100) ~[h2-1.4.196.jar:1.4.196]
at org.h2.Driver.connect(Driver.java:69) ~[h2-1.4.196.jar:1.4.196]
This happens within the SpringApplicatio.run context. Before I found no chance to check if the db is corrupted and if so delete the database.

DDL command not created by liquibase when trying to run it from Java code

Update:
I think the problem is this line because it returns an empty list when I called changelog.getChangeSets(), so what is wrong this code?
static liquibase.changelog.DatabaseChangeLog changelog = new DatabaseChangeLog(
"src/main/resources/changelog.xml");
Original post:
My goal is to run database DDL commands with existing changelog in Java class.
Currently my set up is like this:
public class LiquibaseWithJavaCode {
//find the location of the existing changelog.xml file
static liquibase.changelog.DatabaseChangeLog changelog = new DatabaseChangeLog(
"src/main/resources/changelog.xml");
static void executeUpdate1() throws SQLException, LiquibaseException {
java.sql.Connection connection = getConnection();
Database database = DatabaseFactory.getInstance()
.findCorrectDatabaseImplementation(
new JdbcConnection(connection));
Liquibase liquibase = new liquibase.Liquibase(changelog,
new ClassLoaderResourceAccessor(), database);
liquibase.update(new Contexts(), new LabelExpression());
}
public static Connection getConnection() throws SQLException {
Connection conn = null;
Properties connectionProps = new Properties();
connectionProps.put("user", "liquibase");
connectionProps.put("password", "password");
conn = DriverManager
.getConnection(
"jdbc:sqlserver://111.11.1.1;databaseName=liquibase;SelectMethod=cursor",
connectionProps);
return conn;
}
}
public static void main(String[] args) throws SQLException,
LiquibaseException {
executeUpdate1();
}
When I run it here is the output:
I created the changelog.xml and it contains a lot of changeset that handles a lot of DDL command. But it seems the changeLogFile does not contain any changeset, it is empty. I double checked with the database, and it was indeed no actually table created, only two tables DATABASECHANGELOG and DATABASECHANGELOGLOCK being created.
Did I do anything wrong? How do I correct it? To be honest I am not familiar about it so please advice.

new DatabaseChangeLog just creates an new empty changelog file. To parse an existing file, use:
ChangeLogParser parser = ChangeLogParserFactory.getInstance().getParser("src/main/resources/changelog.xml", resourceAccessor);
DatabaseChangeLog databaseChangeLog = parser.parse("src/main/resources/changelog.xml", new ChangeLogParameters(), resourceAccessor);
For a resourceAccessor, you probably want new ClassLoaderResourceAccessor()

How to connect to different DB process from JDBC DataSource

I'm writing Java application with Apache Camel and use JDBC DataSource(Apache commons DBCP).
I can create many connections but all connections are handled in one Oracle client process, what I may see in sessions table select * from v$session.
The question is how to connect from one Java application to different Oracle processes to improve performance? If you know a way to do it using other Java technologies, not used in my example, it is also very interesting.
public static void main(String... args) throws Exception {
Main main = new Main();
String url = "some url";
String user = "user";
String password = "password";
DataSource dataSource = setupDataSource(url, user, password);
// bind dataSource into the registery
main.bind("oraDataSource", dataSource);
main.enableHangupSupport();
main.addRouteBuilder(new MyRouteBuilder());
main.run(args);
}
private static DataSource setupDataSource(String connectURI, String user, String password) {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName("oracle.jdbc.driver.OracleDriver");
ds.setMaxActive(20);
LOG.info("max active conn: " + ds.getMaxActive());
ds.setUsername(user);
ds.setPassword(password);
ds.setUrl(connectURI);
return ds;
}
public class MyRouteBuilder extends RouteBuilder {
Processor logProcessor = new LogProcessor();
Processor createAnswer = new CreateAnswerProc();
Processor dbPaymentProcessor = new DbPaymentQueryProcessor();
/**
* Let's configure the Camel routing rules using Java code...
*/
public void configure() {
from("rabbitmq://localhost:5672/ps.vin_test_send?exchangeType=topic&autoDelete=false&queue=ps.test_send_queue&concurrentConsumers=20&threadPoolSize=20")
.unmarshal().json(JsonLibrary.Jackson, Payment[].class)
.process(dbPaymentProcessor)
.to("jdbc:oraDataSource")
.process(logProcessor)
.process(createAnswer)
.to("rabbitmq://localhost:5672/ps.vin_test?username=test&password=test&exchangeType=topic&autoDelete=false&routingKey=ps.vin_test_key&queue=vin_test_queue&concurrentConsumers=20");
}

So, the way for oracle is to change db propeties from dedicated to shared. And it will share processes between users. but it is not recomended and not give any performance.
Creation of multiple datasources only reduces performance.