DSE sqoop can't import date SQL Server to Cassandra table - java

Currently I have downloaded datastax enterprise edition 4.0 and used dse sqoop migrate SQL Server to Cassandra I am getting the following error:
ERROR:
sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.microsoft.sqlserver.jdbc.SQLServerDriver
java.lang.RuntimeException: Could not load db driver class: com.microsoft.sqlserver.jdbc.SQLServerDriver
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:653)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:52)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:542)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:565)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:194)
at org.apache.sqoop.manager.SqlManager.getColumnTypes(SqlManager.java:178)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:262)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1235)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1060)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:82)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:390)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:476)
at org.apache.sqoop.Sqoop.run(Sqoop.java:145)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:181)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:220)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:229)
at org.apache.sqoop.Sqoop.main(Sqoop.java:238)
at com.cloudera.sqoop.Sqoop.main(Sqoop.java:57)
I have put the sqljdbc4.jar file in the below paths
/usr/share/dse/sqoop/lib
/usr/share/dse/hadoop/lib
Still I am getting above error.

Have you tried placing the jar in /usr/share/dse/sqoop?

Related

Flink Oracle JDBC sink connector not loading the driver

I am trying to create Flink JBDC sink to an oracle database. When run locally (from a junit test and minicluster) it works but when deployed in k8s it throws an exception saying it cannot find a suitable Driver. The Classpath is:
Classpath: /flink/lib/flink-cep-scala_2.12-1.13.5-stream1.jar:/flink/lib/flink-connector-jdbc_2.12-1.13.5.jar:/flink/lib/flink-csv-1.13.5-stream1.jar:/flink/lib/flink-json-1.13.5-stream1.jar:/flink/lib/flink-queryable-state-runtime_2.12-1.13.5-stream1.jar:/flink/lib/flink-shaded-netty-tcnative-dynamic-2.0.30.Final-13.0-stream1.jar:/flink/lib/flink-shaded-zookeeper-3.4.14.jar:/flink/lib/flink-table-blink_2.12-1.13.5-stream1.jar:/flink/lib/flink-table_2.12-1.13.5-stream1.jar:/flink/lib/log4j-1.2-api-2.16.0.jar:/flink/lib/log4j-api-2.16.0.jar:/flink/lib/log4j-core-2.16.0.jar:/flink/lib/log4j-slf4j-impl-2.16.0.jar:/flink/lib/ojdbc8-21.5.0.0.jar:/flink/lib/vvp-flink-ha-kubernetes-flink113-1.4-20211013.091138-2.jar:/flink/lib/flink-dist_2.12-1.13.5-stream1.jar:::
I tried multiple things:
Included the driver in the flink/lib directory and the flink-connector-jdbc connector was packaged within the the jar and .withDriverName("oracle.jdbc.OracleDriver") /.withDriverName("oracle.jdbc.driver.OracleDriver")
Included both the driver and the connector into the flink/lib directory and .withDriverName("oracle.jdbc.OracleDriver") / .withDriverName("oracle.jdbc.driver.OracleDriver")
I also tried to change the classloading configuration to classloader.parent-first-patterns.additional: oracle.jdbc.
but nothing seems to be working for me. The exception is:
failure cause: java.io.IOException: unable to open JDBC writer
at org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.open(AbstractJdbcOutputFormat.java:56)
at org.apache.flink.connector.jdbc.internal.JdbcBatchingOutputFormat.open(JdbcBatchingOutputFormat.java:115)
at org.apache.flink.connector.jdbc.internal.GenericJdbcSinkFunction.open(GenericJdbcSinkFunction.java:49)
at org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:34)
at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
at org.apache.flink.streaming.api.operators.StreamSink.open(StreamSink.java:46)
at org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:442)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restoreGates(StreamTask.java:585)
at org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.call(StreamTaskActionExecutor.java:100)
at org.apache.flink.streaming.runtime.tasks.StreamTask.executeRestore(StreamTask.java:565)
at org.apache.flink.streaming.runtime.tasks.StreamTask.runWithCleanUpOnFail(StreamTask.java:650)
at org.apache.flink.streaming.runtime.tasks.StreamTask.restore(StreamTask.java:540)
at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:759)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:566)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: No suitable driver found for "jdbc:oracle:thin:#//SOMECONNECTION"
at org.apache.flink.connector.jdbc.internal.connection.SimpleJdbcConnectionProvider.getOrEstablishConnection(SimpleJdbcConnectionProvider.java:126)
at org.apache.flink.connector.jdbc.internal.AbstractJdbcOutputFormat.open(AbstractJdbcOutputFormat.java:54)
... 14 more
What am I missing?
There is no support in Flink 1.13 for Oracle via JDBC, that was only added in Flink 1.15

Error connecting to 19c Database "No Matching protocol"

We had upgraded from Oracle 11g to 19c recently and one of the Java program which connects to Oracle DB via OJDBC started throwing below error:
java.sql.SQLException: ORA-28040: No matching authentication protocol
at oracle.jdbc.driver.SQLStateMapping.newSQLException(SQLStateMapping.java:70)
at oracle.jdbc.driver.DatabaseError.newSQLException(DatabaseError.java:133)
at oracle.jdbc.driver.DatabaseError.throwSqlException(DatabaseError.java:206)
We have tried the below options :
Added the below lines in sqlnet.ora
SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8
Replaced ojdbc14.jar with ojdbc6.jar.
Other JARs that I tried are ojdbc7.jar , ojdbc8-full.jar.
None of the above seems to work. When I tried replacing with ojdbc8-full.jar, I get to see a different error:
java.sql.SQLException: No suitable driver found for
DB_URL=jdbc:oracle:thin:#(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)...
Oracle Version: 19c
Java Version: 1.7

Error connecting MATLAB to SQLServer

First, I have downloaded microsoft jdbc driver(sqljdbc42.jar) for Sqlserver and added it to path. And I can use it to connect to my local MySQL database using the following codes:
database('**','**','**','com.mysql.jdbc.Driver','jdbc:mysql://localhost:3306/')
But I cannot connect to a remote SQLServer with error message: Unable to find JDBC driver. The codes are:
database('**','**','**','com.microsoft.sqlserver.jdbc.SQLServerDriver','jdbc:sqlserver://**')

Which is the sybase driver(version) to be used in Weblogic 12c ? Where should I add the downloaded driver?

I am migrating my application from Weblogic 9 to 12c.
The connection pool - cvSybasepool created for my application is working fine on Web logic 9.
I had used com.sybase.jdbc.SybDriver to connect to Sybase database on Weblogic 9.
After replicating the connection pool(cvSybasepool) on Weblogic 12c, I get the following error:
<Jun 21, 2016 4:40:25 AM EDT> <Error> <Deployer> <BEA-149205> <Failed to initialize the application "cvSybasepool" due to error weblogic.application.ModuleException: weblogic.com
mon.resourcepool.ResourceSystemException: Cannot load driver class com.sybase.jdbc.SybDriver for datasource 'cvSybasepool'.
weblogic.application.ModuleException: weblogic.common.resourcepool.ResourceSystemException: Cannot load driver class com.sybase.jdbc.SybDriver for datasource 'cvSybasepool'.
at weblogic.jdbc.module.JDBCModule.prepare(JDBCModule.java:350)
at weblogic.application.internal.flow.ModuleListenerInvoker.prepare(ModuleListenerInvoker.java:100)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:175)
at weblogic.application.internal.flow.ModuleStateDriver$1.next(ModuleStateDriver.java:170)
at weblogic.application.utils.StateMachineDriver.nextState(StateMachineDriver.java:42)
Truncated. see log file for complete stacktrace
Caused By: weblogic.common.resourcepool.ResourceSystemException: Cannot load driver class com.sybase.jdbc.SybDriver for datasource 'cvSybasepool'.
at weblogic.jdbc.common.internal.JDBCUtil.parseException(JDBCUtil.java:322)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.loadDriver(ConnectionEnvFactory.java:82)
at weblogic.jdbc.common.internal.ConnectionEnvFactory.<init>(ConnectionEnvFactory.java:141)
at weblogic.jdbc.common.internal.ConnectionPool.initPooledResourceFactory(ConnectionPool.java:1110)
at weblogic.common.resourcepool.ResourcePoolImpl.start(ResourcePoolImpl.java:226)
Truncated. see log file for complete stacktrace
>
I suspect the Sybase JDBC driver jar is missing on Web logic 12c.
Also, When the Sybase driver provided by Web logic server is used, I get the following error
FILogger.log(232) - java.sql.SQLException: [FMWGEN][Sybase JDBC Driver][Sybase]A USE DATABASE statement is not allowed in a procedure or trigger. at weblogic.jdbc.sybasebase.ddcn.b(Unknown Source) at weblogic.jdbc.sybasebase.ddcn.a(Unknown Source) at weblogic.jdbc.sybasebase.ddcm.b(Unknown Source) at weblogic.jdbc.sybasebase.ddcm.a(Unknown Source)
Which driver needs to be used to connect to Sybase DB on Weblogic
12c? Jconn2.jar/Jconn3.jar?
Which jar needs to be added to Weblogic server?
Where do I need to add the jar and what configuration has be done?
I finally resolved the issue.
I could not use the sybase driver(weblogic.jdbc.sybase.SybaseDriver) bundled with weblogic as it was not complaint with my SQL code
Below are steps I followed.
Downloaded & copied JDBC driver JAR - jConnect.jar to
${WL_HOME}/server/lib/
Updated the setDomainEnv.cmd script in my domain's bin directory,
and prepended the JAR file to the PRE_CLASSPATH environment
variable(If the newly added JDBC driver JAR is required in other
domains, then edit the commEnv.cmd/sh script in WL_HOME/common/bin
and prepend your JAR file to the WEBLOGIC_CLASSPATH environment
variable.)
Restarted the server.
Update or re-create your datasource to use the sybase driver available with weblogic server 12c (weblogic.jdbc.sybase.SybaseDriver)

Connection with Sybase

I am new to Sybase ,I'm developing a web application with sybase and jboss, my app connect with the Sybase database. I have put the jconn4.jar in same directory with war file in deployments directory . My config in standalone.xml like this
<connection-url>jdbc:jtds:sybase://<myserver>:<myport>/<mydb></connection-url>
<driver-class>com.sybase.jdbc4.jdbc.SybDriver</driver-class>
<!--<driver-class>net.sourceforge.jtds.jdbc.Driver</driver-class> -->
<driver>jconn4.jar</driver>
and when i use com.sybase.jdbc4.jdbc.SybDriver got the error message
javax.resource.ResourceException: Wrong driver class [class com.sybase.jdbc4.jdbc.SybDriver] for this connection URL [jdbc:jtds:sybase://<myhost>:<myport>/<mydb>
...
when i use net.sourceforge.jtds.jdbc.Driver i got the message
java.lang.ClassNotFoundException: net.sourceforge.jtds.jdbc.Driver from [Module "deployment.jconn4.jar:main" from Service Module Loader]
I don't know what driver-class suitable for driver class jconn4.jar
You are using the driver url of the open source jTDS driver, but you are trying to use the Sybase driver. Use the right JDBC url for the Sybase driver: Check the documentation included with the Sybase driver for details (I can't be more specific because the Sybase documentation is usually hard to find online without access to the support portal). Or switch to jTDS, in which case you need to deploy it as a module to JBoss and reference the right driver module in the config.

Categories