MySQL to PostgreSQL migration: mysql connector - java

I am trying to migrate from MySQL to PostgreSQL and I have a Java-related problem that I am not able to fix. Full disclosure: I know little or nothing about Java, but the migration uses a Java-based script, so for me it becomes a configuration problem.
Short version of the problem:
The migration tool throws this exception:
java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
mysql-connector-java-5.0.8-bin.jar is already in the "JAVA_HOME\jre\lib\ext" directory, and I don't know how to solve this depencency problem.
Long version of the problem:
I was trying to migrate from MySQL to PostgreSQL. I checked the official postgresql documentation and I chose the free tool from entreprisedb (that can be downloaded here) to start the migration.
From the installation readme, they tell you that the mysql connector is not installed by default, but they also tell you the steps to solve this problem:
To enable MySQL connectivity, download MySQL's freely available JDBC driver from:
http://www.enterprisedb.com/downloads/third-party-jdbc-drivers
Place the mysql-connector-java-5.0.8-bin.jar file in the "JAVA_HOME\jre\lib\ext" directory (in my case: "C:\Program Files\Java\jre1.8.0_60\lib\ext\mysql-connector-java-5.0.8-bin.jar").
After configuring the tool properly and executing the .bat, this is the error I get:
Connecting with source MySQL database server...
MTK-11009: Error Connecting Database "MySQL Server"
DB-null: java.sql.SQLException: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
Stack Trace:
com.edb.MTKException: MTK-11009: Error Connecting Database "MySQL Server"
at com.edb.dbhandler.mysql.MySQLConnection.<init>(MySQLConnection.java:48)
at com.edb.common.MTKFactory.createMTKConnection(MTKFactory.java:250)
at com.edb.MigrationToolkit.createNewSourceConnection(MigrationToolkit.java:5982)
at com.edb.MigrationToolkit.initToolkit(MigrationToolkit.java:3346)
at com.edb.MigrationToolkit.main(MigrationToolkit.java:1700)
Caused by: java.sql.SQLException: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
at com.edb.Utility.processException(Utility.java:327)
at com.edb.dbhandler.mysql.MySQLConnection.<init>(MySQLConnection.java:47)
... 4 more
...which, to my understanding, probably means that mysql-connector-java-5.0.8-bin.jar is not found.
All the links I've found online regarding the error are specific for Eclipse or other IDEs, so I have not yet been able to solve this dependency problem.

SOLUTION
With the help of a friend that masters Java, this is the solution he achieved:
To start looking for the problem, we opened the runMTK.bat. The execution line reads:
cscript //nologo "..\etc\sysconfig\runJavaApplication.vbs" "..\etc\sysconfig\edbmtk-49.config" "-Dprop=..\etc\toolkit.properties -classpath -jar edb-migrationtoolkit.jar %*"
So then we opened this runJavaApplication.vbs, and in order to know the JAVA_EXECUTABLE_PATH that the program was using, we add this line to the script:
Wscript.Echo "JAVA_EXECUTABLE_PATH = " & JAVA_EXECUTABLE_PATH
With that info, we discover that the script is using the Java folder under C:\Program Files (x86), instead of the one under C:\Program Files (where I dropped the mysql jar). So we copy the mysql-connector-java-5.0.8-bin.jar in the \ext folder of the x86, and now the script works.
Word of advice: the script is throwing errors in half of the exported tables, so all the hassle may not be worth it. BUT if anyone is interested in making this migration script work from A to Z (which has been quite a challenge), here are the details:
HOW TO
Free tool (from entreprisedb):
http://www.enterprisedb.com/downloads/postgres-postgresql-downloads
Extract the files from the zip and fun the installer (ppasmeta-9.5.0.5-windows-x64.exe) as administrator.
To enable MySQL connectivity, download MySQL's freely available JDBC driver from:
http://www.enterprisedb.com/downloads/third-party-jdbc-drivers
Place the mysql-connector-java-5.0.8-bin.jar file in the "JAVA_HOME\jre\lib\ext" directory (in my case: "C:\Program Files\Java\jre1.8.0_60\lib\ext\mysql-connector-java-5.0.8-bin.jar").
The Migration Toolkit documentation can be found:
here (online doc): https://www.enterprisedb.com/docs/en/9.4/migrate/toc.html
or here (pdf doc): http://get.enterprisedb.com/docs/Postgres_Plus_Migration_Guide_v9.5.pdf
First: modify C:\Program Files\PostgresPlus\edbmtk\etc\toolkit.properties (Info here):
SRC_DB_URL=jdbc:mysql://SOURCE-HOST-NAME/SOURCE-DB-NAME
SRC_DB_USER=********
SRC_DB_PASSWORD=********
TARGET_DB_URL=jdbc:edb://localhost:5444/DESTINATION-DB-NAME
TARGET_DB_USER=enterprisedb
TARGET_DB_PASSWORD=********
Then: execute C:\Program Files\PostgresPlus\edbmtk\bin\runMTK.bat (Info here).
runMTK.bat -sourcedbtype mysql -targetdbtype enterprisedb -allTables YOUR_DB_SCHEMA
// ...or with a limited subset of tables:
runMTK.bat -sourcedbtype mysql -targetdbtype enterprisedb -tables TABLE1,TABLE2,TABLE3 YOUR_DB_SCHEMA
In order to get this subset of tables from MySQL:
SELECT
GROUP_CONCAT(TABLE_NAME)
FROM
information_schema.tables
WHERE
TABLE_SCHEMA = 'your_db_name'

Related

How to fix Derby issue for Apache Netbeans IDE 15

Unable to start server for Derby in IDE 15.
Tried to use Derby version 14, didn't work. Tried to change the location of the folder, didn't work.
When I press start server, it just hangs there, does no further processing.
Unresolved NetBeans Bug NETBEANS-3424 Unable to use Derby DB version 10.15.1.3 in Netbeans 11.2 looks relevant to your problem, though you are using different software releases of Derby, Java and NetBeans and your issue is with starting the server rather than establishing a connection.
I can reproduce the connection failure issue on NetBeans 15 using JDK 19 and Derby 10.16.1.1. After selecting the sample database connection jdbc:derby://localhost:1527/sample [app on APP] and selecting Connect... from the context menu this error is displayed:
Although that linked bug report is open and unresolved, a helpful user posted a workaround in the comments so that a suitable driver can be found:
In the Services panel in NetBeans select the entry Databases > Drivers > Java DB (Embedded), right click and select Customize... from the context menu.
The Customize JDBC Driver window will be displayed.
Click the Add... button three times to add the jars derbytools.jar, derbyshared.jar and derbyoptionaltools.jar from the lib directory of your Derby installation (which is named C:\Apache\db-derby-10.16.1.1-bin in my case, but yours is almost certainly something else).
After doing that your Customize JDBC Driver window for your Java DB (Embedded) driver should look similar to this:
Similarly, for the Databases > Drivers > Java DB (Network) entry add the jars derbytools.jar and derbyshared.jar so that your Customize JDBC Driver window looks like this:
After adding those jars to the two drivers you should immediately be able to connect to the APP database using the connection jdbc:derby://localhost:1527/sample [app on APP], and view its tables:
Notes:
You are using Java 8 which is unsupported in NetBeans 15. Update your IDE to use JDK 11 or greater.
You don't need to explicitly start the server by selecting Databases > Java DB > Start Server. Instead, just select Connect... for jdbc:derby://localhost:1527/sample [app on APP]

Oracle SQL Developer won't connect to local database, OCI library won't load

I opened Oracle SQL Developer after using it for a while a few days ago and then I run into this mysterious and infuriating issue where I cannot connect to my locally installed database. First I got a report of the JDBC driver being outdated when testing the connection, so I replaced it with the updated version and then tried again, but then got a different error saying the OCI library won't load. The path to my Oracle Databse Express Edition 18 installation that works as ORACLE_HOME is C:\app\nicho\product\18.0.0\dbhomeXE. I don't understand why the OCI library won't load, as the oci.dll and associated oci files are all in the directory the error message specifies: C:\app\nicho\product\18.0.0\dbhomeXE\bin. I tried using the Instant Client option in the Tools->Preferences->Database->Advanced section by downloading the Oracle Instant Client and changing the path, but I got the same OCI library load failed message. This problem is very significant because I need to use this software for my college project and cannot do my work. I don't understand why this error is happening.
Testing the Oracle Home located at C:\app\nicho\product\18.0.0\dbhomeXE
Testing client directory ... OK
Testing loading Oracle JDBC driver ... OK
Testing checking Oracle JDBC driver version ... OK
Driver version: 21.1.0.0.0
Testing testing native OCI library load ... Failed:
Error loading the native OCI library
The native OCI driver could not be loaded. The system propertyjava.library.path contains the entries from the environment variable PATH. Check it to verify that
the expected native library directory C:\app\nicho\product\18.0.0\dbhomeXE\bin is present and precedes any other client installations.
java.library.path = C:\Users\nicho\OneDrive\Documents\IS 410\sqldeveloper\sqldeveloper\bin;C:\Windows\Sun\Java\bin;C:\Windows\system32;C:\Windows;C:\Windows\system32;C:\Windows;C:\Windows\System32\Wbem;C:\Windows\System32\WindowsPowerShell\v1.0\;C:\Windows\System32\OpenSSH\;C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;C:\Program Files\NVIDIA Corporation\NVIDIA NvDLISR;C:\app\nicho\product\18.0.0\dbhomeXE\bin;C:\Users\nicho\AppData\Local\Microsoft\WindowsApps;;.
As you can see, the expected native library directory is in the path, but the OCI driver will still not load. My database and installation were working fine before today. Please help. :(

Running tests on travis using mysql

I've been trying for the last week or so to make integration tests work on travis for a school project. I've debugged a fair bit of the project but now I'm blocked and need external help.
To give a bit of context, so far, I've debugged the java project so that the tests can be launched from eclipse or from maven in command line. I've worked on the travis file so that a database is created, the database scripts run and the java tests launch. However, the tests fail on travis because of a "table missing" in the database.
This is a link to our repo.
This is the travis.yml file's code:
language : java
jdk:
- oraclejdk8
service:
- mysql
before_script:
- mysql -e 'DROP DATABASE IF EXISTS koalatest'
- mysql -e 'CREATE DATABASE IF NOT EXISTS koalatest;'
- mysql -u root --default-character-set=utf8 koalatest < backend/koalacal-backend/koalacal.sql
script: cd backend && cd koalacal-backend && mvn test -X
after_success:
- bash <(curl -s https://codecov.io/bash)
The java project that is being built and run by maven is located under rootfolder -> backend -> koalacal-backend.
Here is a link to the error log maven produces on travis.
This line seems to be the source of the error:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'koalatest.Calendar' doesn't exist
I have two hypothesis:
1- The sql script that creates all the tables is not being run properly by travis.
To test this hypothesis, I changed the name of the script called by travis. As expected, I got an error saying that travis can't find the file. So at least, I know that this line of code causes travis to look up at an sql file.
- mysql -u root --default-character-set=utf8 koalatest < backend/koalacal-backend/koalacal.sql
That being said, I have no idea if the file is run properly on the database.
For the sake of putting all relevant informations in this post, here is a link to the database script.
2- The tests can't connect properly to the database.
Here is the config file that contain the info regarding which database to connect to:
TestInstance=true
user=root
password=
serverName=localhost
databaseName=koalacal
portNumber=3306
testUser=root
testPassword=
testServerName=127.0.0.1
testDatabaseName=koalatest
testPortNumber=3306</code>
If the parameter TestInstance is set to true, the tests take the informations testUser, testPassword, testServerName, testDatabaseName and testPortNumber to connect to the relevant database.
I believe the connection informations currently contained in the config file match how the travis documentation says we need to connect to a mysql database. I tried to change the testUser to something invalid (like root3) and got error messages as expected.
Maybe somehow the tests can't connect to the database and don't produce a related error message, but I doubt it.
Can someone look at my problem and see if I've missed something obvious (or not)? I don't know what else to try and I don't want to be blocked one more week on a technical issue.
For anyone who may google travis mysql and has a similar error to the one I had, I solved my problem.
The error was caused by a case sensitivity issue. The java code tried to connect to tables like 'Calendar' and 'Event' while the sql script created the tables 'calendar' and 'event'.
It took a long time to troubleshoot this because the case sensitivity didn't pose any problem on my machine. Maven can run its tests properly without any issue. It's only on the travis servers that case sensitivity of the tables started to matter.

connecting MATLAB to Apache Derby database via JDBC

I've been asked to connect an Apache Derby database to our existing MATLAB data-mining tools. I am running MATLAB R2014a (no Database toolbox)
After some research and trial and error, here's what I did:
went to http://www.java2s.com/Code/Jar/d/Downloadderbyclientjar.htm and downloaded derbyclient.jar and saved it, and unzipped it to a shared folder.
edited the text file C:\Users\tyler.davis\AppData\Roaming\MathWorks\MATLAB\R2014a\javaclasspath.txt and added full path to derbyclient.jar, saved it, restarted MATLAB.
At MATLAB command line
javaclasspath
shows "S:\SHARED...\derbyclientjar\derbyclient.jar\derbyclient.jar" at the end of the static path list. So far so good.
Next, try
driverClassName = 'org.apache.derby.jdbc.ClientDriver';
java.lang.Class.forName(driverClassName);
reports error:
Java exception occurred:
java.lang.ClassNotFoundException: org/apache/derby/jdbc/ClientDriver
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Unknown Source)
Tried an alternative:
classLoader = com.mathworks.jmi.ClassLoaderManager.getClassLoaderManager;
driverClass = classLoader.loadClass(driverClassName);
That seemed to work, created objects "driverClass, type 1x1 java.lang.Class" and "classLoader, value 1x1 com.mathworks.jmi.ClassLoaderManager"
Then I tried
DriverManager.registerDriver(driverClass.newInstance);
which reports error
Undefined variable "DriverManager" or class "DriverManager.registerDriver".
and if I try to create a sample database using
cxnStr = 'jdbc:derby:sampleDB:create=true';
cxn = java.sql.DriverManager.getConnection(cxnStr);
I get
Java exception occurred:
java.sql.SQLException: No suitable driver found for jdbc:derby:sampleDB:create=true
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
I really don't know what I'm doing here; just copying other's code from around the web. Any suggestions on what to try next?
Since you're just getting started with Derby, can I recommend that you take an hour or two and run through the Derby tutorial: http://db.apache.org/derby/docs/10.11/getstart/cgstutorialintro.html
Also, can I recommend that you get in the habit of downloading Derby from the Apache website rather than other Internet sites? The copies of Derby on other sites are probably fine, but it seems most reliable and safest to get Derby from the Apache website: http://db.apache.org/derby/derby_downloads.html
Most of what you did in your description seems OK to me; I suspect the place where you went astray is when you unzipped derbyclient.jar itself.
Although a '*.jar' file is in fact a valid ZIP archive, you are not expected to unzip those packaged jar files. Instead, you should leave the jar as 'derbyclient.jar' and then set your CLASSPATH to include 'derbyclient.jar' as one of the entries in the CLASSPATH list.
Lastly, note that the JDBC Connection URL
jdbc:derby:sampleDB:create=true
is a correct JDBC Connection URL for the embedded configuration of Derby, not for the client-server configuration of Derby.
So even if you get a good copy of derbyclient.jar into your CLASSPATH correctly, you won't be able to use
jdbc:derby:sampleDB:create=true
Instead, you'll either have to use a client-server style JDBC Connection URL, such as:
jdbc:derby://my.server.name:1527/sampleDB
(and learn how to operate a Derby Network Server), or you'll need to switch to using the Embedded Driver (org.apache.derby.jdbc.EmbeddedDriver).
For more information about this notion of configurations, see: http://db.apache.org/derby/docs/10.11/getstart/cgsquck70629.html
And for more information on the JDBC Connection URL and the two JDBC driver classes, see: http://db.apache.org/derby/docs/10.11/getstart/cgsquck19524.html
Once you've had a chance to work through the Derby Getting Started guide, I recommend asking some follow-on questions based on what you learn during that experience.

Error: java.sql.SQLException: No suitable driver found for jdbc:sqlserver

I have a wowza server (built with Java) and need it to save logs to a SQL Server 2005 Database. I downloaded the sqljdbc4.jar jar file from Microsoft and placed it in C:\Program Files\Java\jre6\lib\ext. I also added the classpath to a windows variable, but am getting this error:
java.sql.SQLException: No suitable driver found for jdbc:sqlserver
These are the wowza configuration settings for SQL server:
log4j.appender.SQ=org.apache.log4j.jdbc.JDBCAppender
log4j.appender.SQ.Driver=com.microsoft.sqlserver.jdbc.SQLServerDriver
log4j.appender.SQ.URL=jdbc:sqlserver:\\Myserver;databaseName=WowzaLog
log4j.appender.SQ.user=sa
log4j.appender.SQ.password=123
log4j.appender.SQ.layout=com.wowza.wms.logging.ECLFPatternLayout
log4j.appender.SQ.layout.OutputHeader=false
log4j.appender.SQ.sql=my insert SQL
It's not a wowza problem. It's something related to JAVA but I'm not a Java expert.
Thanks
Does Wowza have any lib folder? This would typically have lots of jar files in it that wowza would need to use.
Normally you put the JDBC driver jar file in there not in the Java runtime ext environment.
Also the Database path seems wrong. It needs to have forward slashes not backslashes. It needs to be like this:
jdbc:sqlserver://server:port;DatabaseName=dbname

Categories