Java switch from MS SQL to MySQL - Character sets - java

I am developing an application connecting to a database in Java. The customer has SQL Server and I tried the SQLExpress version von Microsoft as long as we don't want to buy a licence for the developement time. Sadly, the SQL Server Express does not allow network access so we can't work over network on the same database and have to install the SQL Server Express server on every developer client.
Today morning I decided to switch to MySQL during the developement process. I created a MySQL database with UTF8 charset and exported the data into CSV files which I also converted to UTF8 and imported them.
I connect with Java with the JDBC driver and now got weird behavior during execution. The results completetly differ from the client connecting to SQL Server. I have written the SQL to Java glue code myselft and am not using a framework like Hibernate or JPA.
I guess that the problem has to do with character encoding. The source code files are - dont't hit me - encoded with CP1252, because I just started developing in Eclipse on Windows and trusted the default settings. I query the database getting a ResultSet object and then read out the data with the getter methods provided by the ResultSet. I don't do any character conversion during the data fetching.
The problem is now that I don't get cryptic not well encoded output but instead NullPointerExceptions and weird data handling. For example: I have written a method which compares Jobs (an object representing a Job with a name, id, cargo and all that stuff). The results during the runtime differ. Some Jobs are equal on SQL Server and some on MySQL but the SQL Server result is the correct one.
I just viewed the database properties and saw that the character encoding of the SQL Server is in ISO-Latin-1.
Thanks for your help and regards from Cologne,
Marco

I know little about MS SQL, but if the MS SQL data is latin1 encoded, importing it into a UTF-8 database on the mySQL end must fail and result in broken data.
I would try to get the data fixed first. Can you retry the process without converting the dump file, and a latin1 database in mySQL?
Plus, there seem to be notable differences in data types between mySQL and MS SQL:
Be careful when planning this: you want to match data types by capacity and not necessarily by name. For example: a MySQL VARCHAR can hold up to 255 characters, whereas a SQL Server VARCHAR can hold up to 4000 characters. In this case you would need to use a MySQL TEXT column type instead of VARCHAR.

Related

Using the same database at the same time in different programs

I'm using the H2 Database Engine for java to have access to a database in my java programs. I developed many java programs which use the same database. The problem is that whenever I start such a program while another is already running it can't access the database because it is opened by the other program.
Is there a way to let both programs have a connection to the database? Whenever one program has to query the database the database should execute the query. In case it is executing the query of the other program the query should be executed directly after the query of the other program. Since my queries don't take long time the user wouldn't recognize that his program has to wait for a moment and everything would be fine.
H2 server mode is what you want.
You need to have at least started the server this way for example:
org.h2.tools.Server.createTcpServer().start();
Then replace all the jdbc url with jdbc:h2://yourhost/yourdb, keeping in mind yourdb.h2.db will be located where the server was started. I strongly advise not to use absolute path in your jdbc url, as it will give away the database path in case of hacking.
Last but not least: using the server mode for all has a performance penalty. You might want to use mixed mode so that the 1st client will have almost embedded performances.
To do this, just replace for this client the url with jdbc:h2:yourdb;AUTO_SERVER=TRUE. You could decide to use the same url for all clients as well: the 1st one to connect will be using embedded mode, the other clients will be using tcp performance.
Note that if you're using H2 > 1.4.*, you need to give absolute or relative path like this: jdbc:h2:./yourdb;AUTO_SERVER=TRUE. Note the ./

How to add japanese characters into mysql database through jdbc?

Whenever I try to add any japanese characters to the mysql database through jdbc, the characters are converted to question marks. I want to add those actual japanese characters. How can I do so?
There are similar questions on stackoverflow but none addresses the same issue as this.
PS. The mysql database is an AWS RDS database.
This has more to do with the encoding of your database than your actual SQL client. Your database should be configured to use an encoding that will allow Japanese characters. UTF-8 is recommended.
Specifying the encoding of your database is usually done during DB set-up time, not with your (Java) code. As you mentioned that you are using Amazon RDS I'm not sure what specific steps you should take. However, I usually do this per table on my MySQL set-ups. See here for the particular MySQL documentation.
Additionally, it may be that the Japanese characters are getting inserted fine but your viewing client (terminal, browser, etc.) is not configured to the proper encoding.

JAVA - SQL online database

I'm trying to write a code where i need to keep some data in database tables.
I have learned how to use SQL. My problem is that I want this program to run on different computers while having access to the same database.
Any suggestions ?
Thanks
You'll have to run an SQL server on a computer (ie: with mySQL, SQL Server, etc.). You will run queries against the tables that are stored on this database. There are already a lot of resources online with making a database connection to a server.
Just keep database system running in a machine, and instead of pointing to localhost in your JDBC URL, point to the machine's IP Address.
I would personally recommend setting up a script on a host machine and letting it communicate with the database, and have your program communicate with the script. This way, you won't have to keep your database login credentials in your Java program, which could be dangerous.

copy table from teradata to mysql

i have searched for hours with no real solution.
I want to setup an ongoing task (everynight). I have a table in a Teradata database on server 1. Everynight i need to copy an entire table from this teradata instance to my development server (server 2) that has MySQL 5.6.
How do i copy an entire table form server 1 to server 2?
Things i have tried:
1)Select all data from table x into ResultSet from teradata server 1. Insert into mysql via preparedStatement. But this is crazy slow. Also i am not sure how to Drop the table and recreate it each night with the schema from the teradata server.
Any help please?
There are a few ways you can do this.
Note: these may be older methods just trying to get you to thinking about how you can do this in your current environment. Plus I am not familiar with your data sensitivity and permissions, etc.
One would be teradata to MySQL via CSV file see the examples and links below. (these could be older posts but the basic ideas are what you need).
Export from teradata:
CREATE EXTERNAL TABLE
database_name.table_name (to be created) SAMEAS database_name.table_name (already existing, whose data is to be exported)
USING (DATAOBJECT ('C:\Data\file_name.csv')
DELIMITER '|' REMOTESOURCE 'ODBC');
Export From Teradata Table to CSV
Edit: If CREATE EXTERNAL TABLE doesn't fly then you may have to use java to extract first and then organize the data...Mimic the current method (however it works) at getting the data. Google-fu with this handy link https://www.google.com/search?q=external+csv+file+teradata&oq=external+csv+file+teradata&
(dnoeth) below recommends this: TPT Export in DELIMITED format (which looks like a hassle...but could be the only way) So here is a link that discusses it: http://developer.teradata.com/tools/articles/external-data-formats-supported-by-the-tpt-dataconnector-operator
Import to mysql (don't drop the table.just delete from table):
mysqlimport --ignore-lines=1 \
--fields-terminated-by=, \
--local -u root \
-p Database \
TableName.csv
http://chriseiffel.com/everything-linux/how-to-import-a-large-csv-file-to-mysql/
You would need to schedule this in both the environments and that could be a huge hassle.
Now I see you use Java and in Java you could create a simple scheduled task via (whatever you have available for scheduling tasks). It is possible that you will have to do trial and error runs and your data could be an issue depending on what it is. How it is delimited, if it has headers, etc.
Then you would call variants of the examples above. Through Java
here is a java example:
http://www.java-tips.org/other-api-tips/jdbc/import-data-from-txt-or-csv-files-into-mysql-database-t-3.html
and another:
How to uploading multiple csv file into mysql database using jsp and servlet?
another:
http://www.csvreader.com/java_csv_samples.php
So the basic idea is exporting your csv from teradata which should be simple.
Using Java to traverse the file server to get your file (you may need to FTP somewhere) you may need to consider this....
Consume your CSV file using Java and either a UDF or some package that can iterate over your CSV (stuff) and import into MySQL (write one yourself or find one on the "internet of things" as they now call it).
Edit: Here is info on scheduling tasks with java and links to review.
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ScheduledExecutorService.html
and check this SO convo...
How to schedule a periodic task in Java?

Java + Ms SQL Server 2008 R2 after insert trigger notification

I need to write web application using java webgui Vaadin Framework which will be real-time updated with new records inserted to MsSQL database by MsWin2k8R2 NPS.
Since NPS stores logs to files and/or DB, I prefer DB, but I'd like to know how my java program can be notified by SQL server when NPS server inserts data to database and then re-fetch or extract from SQL server notification and display it in browser.
I already have Java<->Ms SQL connection established.
What I already know:
SqlDependency class could be used but it is only for .NET and I'm working in Java
SQL Server Service Broker - this is some kind of a feature in SQL Server which can be used for this but I currently don't understand correctly how to do it and what it really is
SQL Extended Events - same as above
SQL Notifications - same as above
SQL CRL Triggers - trigger written in C++, C# or maybe in Java which is executed on specified situation but this is done synchronously and will slow down whole operation
Now. I've found this post:
https://stackoverflow.com/a/534831/1235977
I would suggest having a trigger on the table that calls the SQL Server Service Broker, that then (asynchronously) executes a CLR stored procedure that does all your work in a different thread.
But I don't know how to do this.
So what I need is:
Simple table in SQL Server which when inserted data into will notify my Java program somehow. It can be using REST web services, JMS or anything what is language independent.
Please provide step by step examples/solutions.
You can use trigger functionality of SQL.
There is a solution:
SQL Triggrer for table that calls Stored procedure
The stored procedure calls some c# code that will REST service by http.
Your back-end will process http requestes from stored proc.

Categories