I've been trying for the last week or so to make integration tests work on travis for a school project. I've debugged a fair bit of the project but now I'm blocked and need external help.
To give a bit of context, so far, I've debugged the java project so that the tests can be launched from eclipse or from maven in command line. I've worked on the travis file so that a database is created, the database scripts run and the java tests launch. However, the tests fail on travis because of a "table missing" in the database.
This is a link to our repo.
This is the travis.yml file's code:
language : java
jdk:
- oraclejdk8
service:
- mysql
before_script:
- mysql -e 'DROP DATABASE IF EXISTS koalatest'
- mysql -e 'CREATE DATABASE IF NOT EXISTS koalatest;'
- mysql -u root --default-character-set=utf8 koalatest < backend/koalacal-backend/koalacal.sql
script: cd backend && cd koalacal-backend && mvn test -X
after_success:
- bash <(curl -s https://codecov.io/bash)
The java project that is being built and run by maven is located under rootfolder -> backend -> koalacal-backend.
Here is a link to the error log maven produces on travis.
This line seems to be the source of the error:
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table 'koalatest.Calendar' doesn't exist
I have two hypothesis:
1- The sql script that creates all the tables is not being run properly by travis.
To test this hypothesis, I changed the name of the script called by travis. As expected, I got an error saying that travis can't find the file. So at least, I know that this line of code causes travis to look up at an sql file.
- mysql -u root --default-character-set=utf8 koalatest < backend/koalacal-backend/koalacal.sql
That being said, I have no idea if the file is run properly on the database.
For the sake of putting all relevant informations in this post, here is a link to the database script.
2- The tests can't connect properly to the database.
Here is the config file that contain the info regarding which database to connect to:
TestInstance=true
user=root
password=
serverName=localhost
databaseName=koalacal
portNumber=3306
testUser=root
testPassword=
testServerName=127.0.0.1
testDatabaseName=koalatest
testPortNumber=3306</code>
If the parameter TestInstance is set to true, the tests take the informations testUser, testPassword, testServerName, testDatabaseName and testPortNumber to connect to the relevant database.
I believe the connection informations currently contained in the config file match how the travis documentation says we need to connect to a mysql database. I tried to change the testUser to something invalid (like root3) and got error messages as expected.
Maybe somehow the tests can't connect to the database and don't produce a related error message, but I doubt it.
Can someone look at my problem and see if I've missed something obvious (or not)? I don't know what else to try and I don't want to be blocked one more week on a technical issue.
For anyone who may google travis mysql and has a similar error to the one I had, I solved my problem.
The error was caused by a case sensitivity issue. The java code tried to connect to tables like 'Calendar' and 'Event' while the sql script created the tables 'calendar' and 'event'.
It took a long time to troubleshoot this because the case sensitivity didn't pose any problem on my machine. Maven can run its tests properly without any issue. It's only on the travis servers that case sensitivity of the tables started to matter.
Related
I recently switched over from using a PC to a Mac and now for whatever reason one of my Impala drivers that worked fine is no longer found when run in Python. I keep receiving this error every time I run the script : "java.lang.RuntimeException: Class com.cloudera.impala.jdbc41.Driver not found". Please see code snippet for my connection below.
c = jaydebeapi.connect
(jclassname='com.cloudera.impala.jdbc41.Driver',
url='jdbc:impala://cloudera-impala-proxy.live.bi.xxx/;AuthMech=3;ssl=1;',
driver_args=['xxx', self.dwh_password], jars='/Users/xxx/Desktop/ImpalaJDBC41 2.jar')
Any help or suggestions are appreciated, I feel like I'm going crazy trying to get this to work.
Did you check do you have the ImpalaJDBC***.jar in your new machine.
Please check properly weather it's available at classpath/build path or not.
Edit:
You can use hive jdbc jar as well to connect with impala , just use the port of impala rather hive in jdbc url.
Looking at this error means your jar is corrupt.
First check your impalaJDBC jar
java -jar ImpalaJDBC<version>.jar
If it gives you error that means your jar is corrupt.
Download the correct jar from cloudera
I have an experimental project on my Github used for the practising the CI service integrations. I struggle with using Sonarcloud.
I have followed both Tavis CI + Sonarcloud tutorial and Maven example. Mz first confusion starts with the token and properties file whereas the first source suggests creating sonar-project.properties file and the Maven example source does not have any. Here where the documentation is very unclear.
I ignored the properties file and I have done the following steps:
Generated token on sonarcloud.io for my project: e53.....239
Encrypted token on travis-encrypt.github.io since it's the most comfortable way for a Windows user. The 2 following inputs resulted in g3s.....+Q=:
NicharNET/Gistintex
SONAR_TOKEN="e53...239" - I tried both to wrap between quotation marks and without.
Added to Settings -> Environment variable key SONAR_TOKEN with the value of the generated Sonarcloud token e53...239.
Completed travis.yml with the secure token:
language: java
sudo: false
jdk: oraclejdk8
addons:
sonarcloud:
organization: "nicharnet-github"
token:
secure: "g3sLTdS597mklh1I9HgXr71NRCiOk2n9I41PL2wklnAidwibfocyEcWvSQxjmbhNTjdrcW0cAfzh73Ago8hwfdloraryDY9Ng9L1runxayShtPOqZMPI6nz8zUwLaDfqJVwXWv6A3xICOMFl0UhvF351GPErHhrUO7YIa8el3kESeotYy7kkTzeQ7BA239y7EVFw8H4OCW2gjP2G/InxKnlXfNjgQA3GsnZdJ3uWO3No5kwt5ybfuCCV42cp/UczLIzNbo0RW9Z9jBl5IFMzJaUQUfWV3Pp5QEPmpB7Anr/4W1EtjJPjpSIC/+jBmbrkvC/CSzzTPjaH9LzEQ5m4F5JpRw01blqgKH/153qfs1jRgZK6WVRuhe7yAATHCO/IM0DM+kC0mUJTcXKyv9pRm93E4wp+KzdHYy0LdA83wFgMmvlB8pcm20ImAe7B2SBFx5TGtETa6ea6k2BS+DfBXe6sLBnrl1zZB8nxCDnYiwT1BJFxizgv3EJ6Krjd3cSSxr6PGnXDD/yTeFNJarpMbA+wR6AQetJ89LiDBXYiw6GPgWk/A4vUDIsIaXGqCpydeZHkr2ufS1Ap4yYS0Um2GeDeQXTsBTSCg7BViFgduUI1NPYT1avKg7b64nsdpdjBVofOvzJsgiT66WlaJmt23BUCJsnmXwwEcW1387b4G+Q="
script:
- mvn clean install org.jacoco:jacoco-maven-plugin:prepare-agent sonar:sonar
Committed and pushed changes into the repository and Travis CI started to work and resulted in:
Failed to execute goal org.sonarsource.scanner.maven:sonar-maven-plugin:3.4.1.1168:sonar (default-cli) on project gistintex: Not authorized. Please check the properties sonar.login and sonar.password.
The error could be found on my Travis CI build. What do I do wrong?
I have skimmed through Error using Travis CI with Sonarcloud: Not authorized. Please check the properties sonar.login and sonar.password SO question and Travis CI Sonarqube analysis article, which unfortunately didn't help me though. My GitHub project source.
I have tried to add these plugins to pom.xml and run the analysis locally:
org.codehaus.mojo: sonar-maven-plugin: 5.1
org.sonarsource.scanner.maven: sonar-maven-plugin: 3.4.1.1168
Running this on my machine resulted in the correct Sonarcloud analysis:
mvn sonar:sonar \
-Dsonar.organization=nicharnet-github \
-Dsonar.host.url=https://sonarcloud.io \
-Dsonar.login=e53.....239
However, my goal is to run the analysis after every committed version using Travis CI. I am sure I have done wrong the key encryption but I can't figure what exactly.
The key encryption problem usually happens when you execute the generation without the login command against travis.com instead of travis.org
In order to run on every commit you could connect your repo with SonarCloud Scan, generate a secret in your sonarcloud.io project page and there you will find instructions to modify your .travis.yml
if you use travis.org
travis encrypt <YOUR_SONAR_SECRET>
if you use travis.com
travis login --pro
then generate your token like this:
travis encrypt --pro <YOUR_SONAR_SECRET>
finally you need to add your pipeline instruction to your .travis.yml file
Check in your Travis CI repo settings when it should be triggered
here some related links:
Error using Travis CI with Sonarcloud: Not authorized. Please check the properties sonar.login and sonar.password
https://docs.travis-ci.com/user/encryption-keys/#usage
https://github.com/marketplace/actions/sonarcloud-scan
https://sonarcloud.io/documentation/integrations/github/
I am trying to migrate from MySQL to PostgreSQL and I have a Java-related problem that I am not able to fix. Full disclosure: I know little or nothing about Java, but the migration uses a Java-based script, so for me it becomes a configuration problem.
Short version of the problem:
The migration tool throws this exception:
java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
mysql-connector-java-5.0.8-bin.jar is already in the "JAVA_HOME\jre\lib\ext" directory, and I don't know how to solve this depencency problem.
Long version of the problem:
I was trying to migrate from MySQL to PostgreSQL. I checked the official postgresql documentation and I chose the free tool from entreprisedb (that can be downloaded here) to start the migration.
From the installation readme, they tell you that the mysql connector is not installed by default, but they also tell you the steps to solve this problem:
To enable MySQL connectivity, download MySQL's freely available JDBC driver from:
http://www.enterprisedb.com/downloads/third-party-jdbc-drivers
Place the mysql-connector-java-5.0.8-bin.jar file in the "JAVA_HOME\jre\lib\ext" directory (in my case: "C:\Program Files\Java\jre1.8.0_60\lib\ext\mysql-connector-java-5.0.8-bin.jar").
After configuring the tool properly and executing the .bat, this is the error I get:
Connecting with source MySQL database server...
MTK-11009: Error Connecting Database "MySQL Server"
DB-null: java.sql.SQLException: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
Stack Trace:
com.edb.MTKException: MTK-11009: Error Connecting Database "MySQL Server"
at com.edb.dbhandler.mysql.MySQLConnection.<init>(MySQLConnection.java:48)
at com.edb.common.MTKFactory.createMTKConnection(MTKFactory.java:250)
at com.edb.MigrationToolkit.createNewSourceConnection(MigrationToolkit.java:5982)
at com.edb.MigrationToolkit.initToolkit(MigrationToolkit.java:3346)
at com.edb.MigrationToolkit.main(MigrationToolkit.java:1700)
Caused by: java.sql.SQLException: java.lang.ClassNotFoundException: com.mysql.jdbc.Driver
at com.edb.Utility.processException(Utility.java:327)
at com.edb.dbhandler.mysql.MySQLConnection.<init>(MySQLConnection.java:47)
... 4 more
...which, to my understanding, probably means that mysql-connector-java-5.0.8-bin.jar is not found.
All the links I've found online regarding the error are specific for Eclipse or other IDEs, so I have not yet been able to solve this dependency problem.
SOLUTION
With the help of a friend that masters Java, this is the solution he achieved:
To start looking for the problem, we opened the runMTK.bat. The execution line reads:
cscript //nologo "..\etc\sysconfig\runJavaApplication.vbs" "..\etc\sysconfig\edbmtk-49.config" "-Dprop=..\etc\toolkit.properties -classpath -jar edb-migrationtoolkit.jar %*"
So then we opened this runJavaApplication.vbs, and in order to know the JAVA_EXECUTABLE_PATH that the program was using, we add this line to the script:
Wscript.Echo "JAVA_EXECUTABLE_PATH = " & JAVA_EXECUTABLE_PATH
With that info, we discover that the script is using the Java folder under C:\Program Files (x86), instead of the one under C:\Program Files (where I dropped the mysql jar). So we copy the mysql-connector-java-5.0.8-bin.jar in the \ext folder of the x86, and now the script works.
Word of advice: the script is throwing errors in half of the exported tables, so all the hassle may not be worth it. BUT if anyone is interested in making this migration script work from A to Z (which has been quite a challenge), here are the details:
HOW TO
Free tool (from entreprisedb):
http://www.enterprisedb.com/downloads/postgres-postgresql-downloads
Extract the files from the zip and fun the installer (ppasmeta-9.5.0.5-windows-x64.exe) as administrator.
To enable MySQL connectivity, download MySQL's freely available JDBC driver from:
http://www.enterprisedb.com/downloads/third-party-jdbc-drivers
Place the mysql-connector-java-5.0.8-bin.jar file in the "JAVA_HOME\jre\lib\ext" directory (in my case: "C:\Program Files\Java\jre1.8.0_60\lib\ext\mysql-connector-java-5.0.8-bin.jar").
The Migration Toolkit documentation can be found:
here (online doc): https://www.enterprisedb.com/docs/en/9.4/migrate/toc.html
or here (pdf doc): http://get.enterprisedb.com/docs/Postgres_Plus_Migration_Guide_v9.5.pdf
First: modify C:\Program Files\PostgresPlus\edbmtk\etc\toolkit.properties (Info here):
SRC_DB_URL=jdbc:mysql://SOURCE-HOST-NAME/SOURCE-DB-NAME
SRC_DB_USER=********
SRC_DB_PASSWORD=********
TARGET_DB_URL=jdbc:edb://localhost:5444/DESTINATION-DB-NAME
TARGET_DB_USER=enterprisedb
TARGET_DB_PASSWORD=********
Then: execute C:\Program Files\PostgresPlus\edbmtk\bin\runMTK.bat (Info here).
runMTK.bat -sourcedbtype mysql -targetdbtype enterprisedb -allTables YOUR_DB_SCHEMA
// ...or with a limited subset of tables:
runMTK.bat -sourcedbtype mysql -targetdbtype enterprisedb -tables TABLE1,TABLE2,TABLE3 YOUR_DB_SCHEMA
In order to get this subset of tables from MySQL:
SELECT
GROUP_CONCAT(TABLE_NAME)
FROM
information_schema.tables
WHERE
TABLE_SCHEMA = 'your_db_name'
when I run sample IBM Bluemix Liberty for Java application https://github.com/ibmjstart/bluemix-java-postgresql-uploader.git following error:
-----> Downloaded app package (1.9M)
-----> Downloaded app buildpack cache (4.0K)
OK
/var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:101:in build_pack': Unable to detect a supported application type (RuntimeError) from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:74:inblock in compile_with_timeout'
from /usr/lib/ruby/1.9.1/timeout.rb:68:in timeout' from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:73:incompile_with_timeout'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:54:in block in stage_application' from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:50:inchdir'
from /var/vcap/packages/dea_next/buildpacks/lib/buildpack.rb:50:in stage_application' from /var/vcap/packages/dea_next/buildpacks/bin/run:10:in'
FAILED
Server error, status code: 400, error code: 170001, message: Staging error: cannot get instances since staging failed
TIP: use 'cf logs jpu-henryhan --recent' for more information
The top error looks like you left off the -p <path_to_war> parameter when doing a push. If you just push a directory containing a WAR file, it will not be detected by the Java buildpack.
The tip provided in the output of your cf push request is relevant.
TIP: use 'cf logs jpu-henryhan --recent' for more information
Running that command will tail the log files produced during the staging process and let you see what error may have been raised. Often, it can be a missing dependency or a transient failure of some sort.
I just successfully deployed the sample using the "deploy to Bluemix" button and manually via the cf command line tool. Unless you changed the code, it is most likely that this error is a transient failure.
Run following command:
$ cf push jpu- -b https://github.com/cloudfoundry/java-buildpack --no-manifest --no-start -p PostgreSQLUpload.war
add the parameter to set the buildpack "-b https://github.com/cloudfoundry/java-buildpack"
I developped my application locally with the default in memory h2 database. Everything works perfectly.
Now I tried to deploy my app by pushing to Heroku.
Before that I added a Procfile to my app's root folder with this line:
web: target/start -Dhttp.port=${PORT} -DapplyEvolutions.default=true -Ddb.default.url=${DATABASE_URL} -Ddb.default.driver=org.postgresql.Driver
My application.conf looks like:
db.default.driver=org.h2.Driver
db.default.url="jdbc:h2:mem:play;MODE=PostgreSQL"
So I pushed that and then, when I try to access my app i get:
Salieri:addressManager administrator$ heroku ps
=== web (1X): `target/start -Dhttp.port=${PORT} -DapplyEvolutions.default=true -Ddb.default.url=${DATABASE_URL} -Ddb.default.driver=org.postgresql.Driver`
web.1: crashed 2013/06/27 22:04:16 (~ 4m ago)
and in the heroku logs:
2013-06-27T20:04:14.511136+00:00 app[web.1]: [error] play - ERROR: syntax error at or near "user"
2013-06-27T20:04:14.515578+00:00 app[web.1]: at play.api.db.evolutions.Evolutions$.databaseEvolutions(Evolutions.scala:334)
2013-06-27T20:04:14.515153+00:00 app[web.1]: #6emdb57b1: Database 'default' is in inconsistent state!
I tried to delete the whole db with
heroku pg:reset HEROKU_POSTGRESQL_ROSE_URL
and then i pushed again, but no success.
I have to mention that I did a push before, and at that time my app was running on Heroku. The only problem was, that i couldn't log in with the credentials provided in my fixtures (Global.java file with a job on app start).
Any suggestions?
From this error:
ERROR: syntax error at or near "user"
I guess your Evolution scripts are wrong. User is a reserved word in SQL. PLease try to run the Evolution scripts locally in a dev PostgreSQL database. Once working, you can try to deploy again.
I'd also suggest disabling evolutions for production apps, it's a risky system sometimes.
PostgreSQL interprets user as a keyword. If you have an app/models/User class in your app, play will try to create a table named user. The easiest way to avoid this is to rename your User class to something like MyUser. (The same might happen if you use an attribute named user in one of your classes. Here, as well, you should rename it).
Then you should run your application locally and open it in the browser in order to get the evolution script file updated. You can then find the evolution script at conf/evolutions/default/1.sql. According to this example, it should create a table named my_user now.