We Keep Coding

How to store files in H2 database - Spring Boot

Question
How do I store entire files in my H2 database and retrieve them using JDBC?
Some Background
I have some text files that I have as templates for various documents that will be generated in my Spring Boot app. Currently, I have my text files stored in my local file system on my PC, but that is not a long term solution. I need to somehow store them in the database and provide the necessary code for the JDBC for the retrieval of the files.
Are there any technologies/libraries out there that would help me with this? If so, please link me to them and provide an example of how to do it in Spring Boot.
Note: It is a new requirement given to me that the text files should be stored in the database, and not the file system.

You have to use a BLOB column in your database table.
CREATE TABLE my_table(ID INT PRIMARY KEY, document BLOB);
BLOB stands for Binary Large Object.
http://www.h2database.com/html/datatypes.html#blob_type
To store it with JdbcTemplate you have to create a ByteArrayInputStream
ByteArrayInputStream inputStream = new ByteArrayInputStream(document);
preparedStatement.setBlob(3, inputStream);
Please find more examples here:
https://www.logicbig.com/tutorials/spring-framework/spring-data-access-with-jdbc/jdbc-template-with-clob-blob.html

Ingest data from JDBC connections to Hive : Handling binary columns

Following diagram depicts the simplified ingestion flow we are building to ingest data from different RDBS to Hive.
Step 1: Using JDBC connection to the data-source, source data is streamed and saved in a CSV file on HDFS using HDFS java API.
Basically, execute a 'SELECT * ' query and each row is saved in CSV until the ResultSet is exhausted.
Step 2: Using LOAD DATA INPATH command, Hive table is populated using the CSV file created in Step 1.
We use JDBC ResultSet.getString() to get column data.
This works fine for non-binary data.
But for BLOC,CLOB type columns, we cannot write column data into a text/CSV file.
My question is it possible to use OCR or AVRO format to handle binary columns? Does these formats support write row-by-row?
(Update: We are aware of Sqoop/Nifi..etc technologies, the reason for implementing our custom ingestion-flow is beyond the scope of this question)

Reading file content to Mysql stored procedure

i have a scenario where i have a file of the form,
id,class,type
1,234,gg
2,235,kk
3,236,hth
4,237,rgg
5,238,rgr
I also have a table in my database of the form PROPS,
id,class,property
1,7735,abc
2,3454,efg
3,235,hij
4,238,klm
5,24343,xyx
Now the first file and the db table are joined based on class so that final output will be of the form:
id,class,type,property
1,235,kk,hij
2,238,rgr,klm
Now, i can search the db table for each class record of the first file and so forth.
But this will take too much time.
Is there any way to do this same thing through a MySQL STORED PROCEDURE?
My question is whether there is a way to read the first file content line by line(WITHOUT MAKING USE OF A TEMPORARY TABLE), check the class with the class in the db table and insert the result into an output file and return the output file using MYSQL STORED PROCEDURE?

Read multiple csv file with CsvJdbc

I need to bind a group of csv file in the format "YYYY-MM-DD hh:mm:ss.csv" that are present in the same folder with a unique table that contains all the data present in all the files.
I need to read the data from a Java EE application thus I would like to create a connection pool inside the application server. I found the CsvJdbc driver that allows the reading of multiple files as a single entity. A good starting point was this page in the section with this paragraph:
To read several files (for example, daily log files) as a single table, set the database connection property indexedFiles. The following example demonstrates how to do this.
The example could be fine for me but the problem is that I do not have a header word in the filename string. So the corresponding table becames an empty string that makes obviously impossible to query the table.
How can I tell the driver to map the pattern to a table that hasn't a header part?
P.S. I already tried to use hsqldb as a frontend to the csv files but it does not support multiple files.

Setup CsvJdbc to read several files as described in http://csvjdbc.sourceforge.net/doc.html and then use an empty table name in the SQL query because your CSV filenames do not have any header before the fileTailPattern regular expression. For example:
props.put("fileTailPattern", "(\\d+)-(\\d+)-(\\d+) (\\d+):(\\d+):(\\d+)");
props.put("fileTailParts", "Year,Month,Day,Hour,Minutes,Seconds");
...
ResultSet results = stmt.executeQuery("SELECT * FROM \"\" AS T1");

Change Table names in derby database using entitymanager

I am using an APACHE DERBY database, and basing my database interactions on EntityManager, and I don't want to use JDBC class to build a query to change my tables' names (i just need to put a prefix to each new user to the application, but have the same structure of tables), such as:
//em stands for EntityManager object
Query tableNamesQuery= em.createNamedQuery("RENAME TABLE SCHEMA.EMP_ACT TO EMPLOYEE_ACT");
em.executeUpdate();
// ... rest of the function's work
// The command works from the database command prompt but i don't know how to use it in a program
//Or as i know you can't change system tables data, but here's the code
Query tableNamesQuery= em.createNamedQuery("UPDATE SYS.SYSTABLES SET TABLENAME='NEW_TABLE_NAME' WHERE TABLETYPE='T'");
em.executeUpdate();
// ... rest of the function's work
My questions are :
This syntax is correct?
Will it work?
Is there any other alternative?
Should I just use the SYS.SYSTABLES and find all the tables that has 'T' as tabletype and alter their name their, will it change the access name ?

I think you're looking for the RENAME TABLE statement: http://db.apache.org/derby/docs/10.10/ref/rrefsqljrenametablestatement.html
Don't just issue update statements against the system catalogs, you will corrupt your database.