I've found numerous posts about reading CSV with Java and the APIs they were pointing at all had a line-oriented approach when it came to reading a CSV file. Something like "while you get a line, get the values of every column".
I'd appreciate a higher-level API, like in Perl where DBI allows you to use SQL on CSV like if it where a DB table. Otherwise I'll have to implement lots of access logic by myself.
Is there such an API? Am I missing something? There are some references about JDBC drivers but most are projects that haven't been updated the last 5 years.
You can use HSQL in order to do it, see the following links from the docs and a blog post describing exactly that.
You could give H2Database a go - it is rather heavy weight, but at least it is maintained.
Are you trying to avoid using a regular database for accessing CSV? MySQL supports CSV as one of it's table types if you are open to using a database system.
yes, JDBC with a CSV driver. You can try implement yourself or just try something like HXTT
Related
New to Oracle here but I have now read about the various bulk insert options for Oracle. In essence, true bulk loading is done using Direct Path loading mechanism via SQL*Loader. There's also APPEND hint options that use serial or parallel Direct Path loading. But each of these have the following limitations -
SQL*Loader works off of a Control File, which contains the path of the data file. In my case, there is no file.
APPEND hint option for INSERT can only use the syntax - insert into select from. In my case, the source data is not in any table.
Source of my data is actually a Spark dataframe. I am looking for options to push this data in chunks to Oracle tables, but using Direct Path loading option. For example, in Postgres, the PGConnection interface provides getCopyAPI.copyIn functionality and you can create a huge serialized blob than can be sent over as one big chunk using COPY tableName FROM STDIN yourBlob command. I am unable to find anything similar Java API for Oracle that works on in-memory records and is able to push data directly (without any insert statements).
Any ideas on how to achieve this? Anyone done this before?
In general, how do folks using Oracle and Spark push data to Oracle from a dataframe in an optimized way?
Thanks in advance!
I'd like to save persistent objects to the file system using Hibernate without the need for a SQL database.
Is this possible?
Hibernate works on top of JDBC, so all you need is a JDBC driver and a matching Hibernate dialect.
However, JDBC is basically an abstraction of SQL, so whatever you use is going to look, walk and quack like an SQL database - you might as well use one and spare yourself a lot of headaches. Besides, any such solution is going to be comparable in size and complexity to lighweight Java DBs like Derby.
Of course if you don't insist absolutely on using Hibernate, there are many other options.
It appears that it might technically be possible if you use a JDBC plaintext driver; however I haven't seen any opensource ones which provide write access; the one I found on sourceforge is read-only.
You already have an entity model, I suppose you do not want to lose this nor the relationships contained within it. An entity model is directed to be translated to a relational database.
Hibernate and any other JPA provider (EclipseLink) translate this entity model to SQL. They use a JDBC driver to provide a connection to an SQL database. This, you need to keep as well.
The correct question to ask is: does anybody know an embedded Java SQL database, one that you can start from within Java? There are plenty of those, mentioned in this topic:
HyperSQL: stores the result in an SQL clear-text file, readily imported into any other database
H2: uses binary files, low JAR file size
Derby: uses binary files
Ashpool: stores data in an XML-structured file
I have used HyperSQL on one project for small data, and Apache Derby for a project with huge databases (2Gb and more). Apache Derby performs better on these huge databases.
I don't know exactaly your need, but maybe it's one of below:
1 - If your need is just run away from SQL, you can use a NoSQL database.
Hibernate suports it through Hibernate OGM ( http://www.hibernate.org/subprojects/ogm ).
There are some DBs like Cassandra, MongoDB, CouchDB, Hadoop... You have some suggestions Here
.
2 - Now, if you want not to use a database server (with a service process running always), you can use Apache Derby. It's a DB just like any other SQL, but no need of a server. It uses a singular file to keep data. You can easily transport all database with your program.
Take a look: http://db.apache.org/derby/
3 - If you really want some text plain file, you can do like Michael Borgwardt said. But I don't know if Hibernate would be a good idea in this case.
Both H2 and HyperSQL support embedded mode (running inside your JVM instead of in a separate server) and saving to local file(s); these are still SQL databases, but with Hibernate there's not many other options.
Well, since the question is still opened and the OP said he's opened to new approaches/suggestions, here's mine (a little late but ok).
Do you know Prevayler? It's a Java Prevalence implementation which keep all of your business objects in RAM and mantain Snapshots/Changelogs in the File System, this way it's extremely fast and reliable, since if there's any crash, it'll restore it's last state and reapply every change to it.
Also, it's really easy to setup and run in your app.
Ofcourse this is possible, You can simply use file io features of Java, following steps are required:-
Create a File Object
2.Create an object of FileInputStream (though there are ways which use other Classes)
Wrap this object in a Buffer object or simply inside a java.util.Scanner.
use specific write functions of the object created in previous step.
Note that your object must implement Serializable interface. See following link,
I have stored some data in MySQL database through a java program. Now I want to export that data from the same program in .csv file.
I know One method of doing this is to get all the fields of one row in variables and then store them in a file by seperating them with a comma(,) and by repeating the same for every row in the database.
But I want to know that Can I export the same data through any other way through java in .csv format.
I used this once it worked for me.
http://opencsv.sourceforge.net/
Honestly, unless you want to do something fancy, what you are doing should work
You can Google for a third party CSV library, do the Java querying yourself with sqljdbc.jar or whatever is appropriate to your database, and feed the data to the CSV library.
You might even find a third-party library that can give you a CSV from a SQL query and a JDBC connection.
But if you're going to programmatically implement CSV, please implement it completely, including escaping spaces, quotes, etc. Wikipedia has details.
If you are using MySQL take a look at mysqldump
I extracted data from excel using poi api. Now I want to store the data in access please clarify
To the point, you just want to "convert" Excel to MSAccess using Java code? Here are the steps:
1) Extract data from Excel into Java objects (List, String, Number, Javabean, etc).
2) Insert the data in flavor of those Java objects into MSAccess.
That's basically all. For 1) you can use under each the Apache POI as you already found out. For 2) you can use the JDBC. I think your problem is more that you don't understand what JDBC is and how to work with it. In that case you may find the Sun's JDBC tutorial useful. Good luck.
You need something like JetProxy -- a JDBC driver interfacing to Jet (the internal name of Microsoft Access's own quasi-relational sort-of-SQL DB engine). There are other products much in the same vein, but I have no personal experience to help suggest which product is best for your purposes -- just try a few!-)
You can either use the commercial JDBC drivers by HXTT or the JDBC-ODBC bridge by Sun
I am using Hibernate in a Java application to access my Database and it works pretty well with MS-SQL and MySQL. But some of the data I have to show on some forms has to come from Text files, and by Text files I mean Human-Readable files, they can be CSV, Tab-Delimited, or even a key, value pair, per line since my data is as simple as this, but my preference of course is XML files.
My question is: Can I use hibernate to read those files using HQL, Query , EntityManager and all those resources Hibernate provides me to access files. Which file format should I use and How I configure My persistence.xml file to recognize files as Tables?
Hibernate is written against the JDBC API. So, you need a JDBC driver that works with the file format you are interested in. Obviously, even for read-only access, this isn't going to perform well, but it might still be useful if that's not a high priority. On a Windows system, you can set up ODBC datasources for delimited text files, Excel files, etc. Then you can set up the JdbcOdbcDriver in your Java application to use this data source.
For most of the applications I work on, I would not consider this approach; I would use an import/export mechanism to convert from a real database (even if it's an in-process database like Berkeley DB or Derby) to the text files. Yes, it's an extra step, but it could be automated, and the performance isn't likely to be much worse than trying to use the text files directly (it will likely be much better, overall), and it will be more robust and easy to develop.
A quick google came up with
JDBC driver for csv files
JDBC driver for XML files
Hope this might provide some inspiration?
Like erickson said, your only hope is in finding a JDBC driver for that task. There is maybe xlsql (CSV, XML and Excel driver) which could fit the task. After that, you just have to either find or write the most simple Hibernate Dialect which fits your driver.