We are trying to load JSON files using Java from a file location to Snowflake Named Stage. Currently, the PUT command only works for ODBC and not JDBC. So is there any way to execute PUT command using Java code?
Thanks
The Snowflake JDBC Driver does support use of PUT statements for local file uploads. The following java statement is considered a valid query and the file is uploaded:
statement.executeQuery("PUT file:///tmp/foo.json #JSONSTAGE/ overwrite=true");
Running it under a logger produces logs such as the following (logs here are from JDBC driver version v3.12.2):
n.s.c.jdbc.SnowflakeFileTransferAgent$1 FINE call:778 - filePath: /tmp/foo.json
n.s.c.jdbc.SnowflakeFileTransferAgent FINE uploadFiles:1751 - Done with uploading
The JDBC driver also supports a more efficient way of uploading a stream directly, documented here.
Related
I am trying to connect teradata server through PySpark.
My CLI code is as below,
from pyspark.sql import SparkSession
spark=SparkSession.builder
.appName("Teradata connect")
.getOrCreate()
df = sqlContext.read
.format("jdbc")
.options(url="jdbc:teradata://xy/",
driver="com.teradata.jdbc.TeraDriver",
dbtable="dbname.tablename",
user="user1",password="***")
.load()
Which is giving error,
py4j.protocol.Py4JJavaError: An error occurred while calling o159.load.
: java.lang.ClassNotFoundException: com.teradata.jdbc.TeraDriver
To resolve this I think, I need to add jar terajdbc4.jar and `tdgssconfig.jar.
In Scala, to add jar we can use
sc.addJar("<path>/jar-name.jar")
If I use the same for PySpark, I am having error,
AttributeError: 'SparkContext' object has no attribute 'addJar'.
or
AttributeError: 'SparkSession' object has no attribute 'addJar'
How can I add jar terajdbc4.jar and tdgssconfig.jar?
Try following this post which explains how to add jdbc drivers to pyspark.
How to add jdbc drivers to classpath when using PySpark?
The above example is for postgres and docker, but the answer should work for your scenario.
Note, you are correct about the driver files. Most JDBC drivers are in a single file, but Teradata splits it out into two parts. I think one is the actual driver and the other (tdgss) has security stuff in it. Both files must be added to the classpath for it to work.
Alternatively, simply google "how to add jdbc drivers to pyspark".
I am trying to utilize Jasperstarter with MSSQL on Windows with a JDBC driver. It seems never to find or connect with this driver for some reason despite using the same Driver and URL used by iReport which does work. I can execute reports that do not access the DB, but once I add the database parameters the call fails. I do not get any info back from the call, not sure where the debug stuff goes. Are there any special path settings that need to be made to get this to work? The exec call output just returns a 1 without any failure info.
PHP exec command stream:
"C:\\Inetpub\\wwwroot\\TekEnterpriseServer\\protected\\JasperPHP/JasperStarter/bin/jasperstarter pr ./reports/maintenance.jasper -f pdf -t generic --db-driver net.sourceforge.jtds.jdbc.Driver --db-url jdbc:jtds:sqlserver://localhost/tekenterprise"
The jdbc driver must be put into the jdbc directory of the JasperStarter installation or the path to the driver jar must be specified with --jdbc-dir.
You have to specify username and password as part of the --db-url, see
http://jtds.sourceforge.net/faq.html#urlFormat
You can get verbose output with:
jasperstarter -v pr ./reports/maintenance.jasper ...
I have a wowza server (built with Java) and need it to save logs to a SQL Server 2005 Database. I downloaded the sqljdbc4.jar jar file from Microsoft and placed it in C:\Program Files\Java\jre6\lib\ext. I also added the classpath to a windows variable, but am getting this error:
java.sql.SQLException: No suitable driver found for jdbc:sqlserver
These are the wowza configuration settings for SQL server:
log4j.appender.SQ=org.apache.log4j.jdbc.JDBCAppender
log4j.appender.SQ.Driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
log4j.appender.SQ.URL=jdbc:sqlserver:\\Myserver;databaseName=WowzaLog
log4j.appender.SQ.user=sa
log4j.appender.SQ.password=123
log4j.appender.SQ.layout=com.wowza.wms.logging.ECLFPatternLayout
log4j.appender.SQ.layout.OutputHeader=false
log4j.appender.SQ.sql=my insert SQL
It's not a wowza problem. It's something related to JAVA but I'm not a Java expert.
Thanks
Does Wowza have any lib folder? This would typically have lots of jar files in it that wowza would need to use.
Normally you put the JDBC driver jar file in there not in the Java runtime ext environment.
Also the Database path seems wrong. It needs to have forward slashes not backslashes. It needs to be like this:
jdbc:sqlserver://server:port;DatabaseName=dbname
I have downloaded MySQL JDBC driver from http://www.mysql.com/downloads/connector/j/. How do I configure it?
To the point, you just need to put it in the program's runtime classpath and then load it as follows:
try {
Class.forName("com.mysql.jdbc.Driver");
} catch (ClassNotFoundException e) {
throw new RuntimeException("Cannot find the driver in the classpath!", e);
}
If you're using an IDE, then you need to add the library to the project's Build Path. If you're running it using java.exe, then you need to specify the full path to the JDBC driver JAR file in the -cp argument, e.g. (Windows-targeted):
java -cp .;/path/to/mysql-connector-java-5.1.12.jar com.example.YourClass
For more information and hints check this small MySQL+JDBC kickoff tutorial.
Update: As per the comments, you're using Flex and you apparently want to interact with the DB at the server using Flex. Well, Flex runs at the client machine and Java + the DB runs at the server machine. Both are connected by network with HTTP as communication protocol. You need to write Java code on the server side accordingly (Servlet? Webservice?) which interacts with the DB based on the parameters/pathinfo given with the HTTP request and returns the desired results. Then you can just invoke HTTP requests from inside Flex and process the HTTP response.
You can follow the guidelines given at: https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-installing.html.
Also check the comments for more ideas and info. I.e: putting it in [PATH_TO_JAVA]/jre/lib/ext/ (on Win Mashine) or /Library/Java/Extensions (Mac OS X) etc.
Refer < http://www.developer.com/java/data/jdbc-and-mysql-installation-and-preparation-of-mysql.html> or
The JDBC .jar file needs to be added to the library, this can be done by adding it manually to '...jre\lib\ext' folder of your Java installation. It will be automatically included in the default library available to every project you create.
I need to connect to Oracle9,10 and 11 in my java application.The client will always be present where the app will be run and I want the app to just work with username,password and instance (specified in tnsnames.ora).Hence I would like oci drivers with a connection string of type : jdbc:oracle:oci:#testora .Im using the driver: oracle.jdbc.driver.OracleDriver.
I have a 10g client and am using jdk1.5. When I use the ojdbc14 jar from client lib path the app runs.
But if I use the ojdbc5 driver then the app fails with
Exception in thread "main" java.lang.UnsatisfiedLinkError: no ocijdbc11 in java.
library.path
exception.
What im looking for is a way to package a single jar(ojdbc5/ojdbc14) and a single driver which will use oci drivers to connect to the client that is present on local machine(9/10/11).This is the way I would prefer it.
If this is not possible can I search the unix box for oracle version,pick up the correct jar and then use it in classpath when invoking the app which is in jar format?
Thanks,
Fell
java.lang.UnsatisfiedLinkError: no ocijdbc11 in java.library.path
This means that you are missing a DLL (ocijdbc11.dll) in the Java library path. Make sure you have that DLL and start your program like this:
java -Djava.library.path=C:\mydirwiththedll com.mypackage.MyProgram
ojdbc5.jar is intended to work with jdk1.5.x either you are connecting to Oracle 10 or 11g shouldn't matter, ojdbc14.jar was intended for jdk1.4.x