I have an sql script which creates temp tables valid for only that session. Now after running the script, I am trying to read data from the table through spark and then process it. Below is the code I have code for spark read.
sparkSession.read().format("jdbc").option("url",
jdbcURL).option("dbtable", tableOrQuery).option("user",
userName).option("password", password)
.option("driver", driverName).load();
Now I need to pass the jdbc connection I created so that spark can read data in the same session. Is this possible ?
No, you cannot pass jdbc connection to spark. It will manage JDBC connection by itself.
JdbcRelationProvider Create Connection
JdbcUtils connect
Related
I use H2 database (file) in my Java app and due to some data appearance problem in IntelliJ, I use file option instead of memory.
Here is my url setting in application.properties:
spring.datasource.url=jdbc:h2:file:~/test-db;
DB_CLOSE_DELAY=-1;DATABASE_TO_UPPER=false;AUTO_SERVER=TRUE
I can connect and see table data via IntelliJ and H2 console by using the url parameter as shown below:
jdbc:h2:file:~/test-db
However, even I connect to the database successfully, the table data is not seen in DBeaver as shown below:
I think I should use tcp option to connect H2 daabase, but I cannot by using the following settings:
spring.datasource.url=jdbc:h2:tcp://localhost/~/test-db
or
jdbc:h2:tcp://localhost/C:/test-db
and get "Unable to open JDBC Connection for DDL execution" error (I use Windows). Any idea?
You're using a file-associated connection to that in-memory DB that can only be used by 1 tool at once.
So when IntelliJ is connected it generates a lock-file to protect it against multi-access.
I would suggest a TCP connection that allows multiple connections - see redundant question here:
H2 database error: Database may be already in use: "Locked by another process"
I want to create a blank Oracle 12c database programmaticaltlly with Java. But if I use getConnection() method in JDBC driver, it does not allow me to run a CREATE DATABASE statement with since I have to mention a spwcific SID/service name for the connection.
Is there a way that I can connect to Oracle server without specifying a service name/SID in the connection string? and create a blank database from within a Java application?
I'm trying to find a URL I could connect to in building a demo app for teaching purposes.
Anyone aware of one I could use with JDBC?
You could probably set up a database in a local file and connect to that via JDBC.
Two examples of database drivers that are freely available are:
SQLite Java, to connect to SQLite databases.
UCanAccess, to connect to MS Access databases.
Then you can connect to it via JDBC like this:
Connection conn = DriverManager.getConnection("jdbc:sqlite://<db file path>");
// or
Connection conn = DriverManager.getConnection("jdbc:ucanaccess://<mdb or accdb file path>");
The respective links contain more in-depth tutorials.
You can connect to a file and then set up the database via Java by executing SQL statements. The result is persisted in the file, which you could then use and redistribute for teaching.
I have two independent Java apps which I would like to communicate with each other through an in-memory H2-DB. In theory very straight forward, but I cannot get the connection to work.
What I am trying to do:
I create an in-memory DB executing jdbc:h2:mem:test.
With the client(s), I try connecting to it. I tried jdbc:h2:tcp://localhost/~/test and similar connection strings, but all without success.
Is it possible to connect to an in-memory DB? What should the connection strings look like to make this work? Thanks a bunch.
After a ton of reading and trial-and-error with H2 options (http://h2database.com/html/features.html and http://h2database.com/html/advanced.html), I found that it is possible to access an in-memory database from multiple processes on the same machine (or remotely) using TCP/IP or SSL/TLS. The connection string for an in-memory database test is jdbc:h2:tcp://localhost/mem:test.
H2 database can be shared but not in memory, Your may please refer to official documentation:
http://h2database.com/html/features.html#auto_mixed_mode
First application with open it in embedded mode and other application will use server mode.
I am trying to use the Hive JDBC driver to connect to the hive server running on a particular host port. I can establish the connection and create tables (the tables also show up on the namenode web UI) but I can't access existing tables. Note: The existing tables were running the hive console and (for some odd reason) can only be accessed when I run the hive console in that particular directory.
How can I get my jdbc client to access the hive tables using the JDBC connection?