My current task is to migrate a remote database to a localhost database. Everything seems fine up to now. The problem is when I'm checking whether the data are the same, data in my localhost weren't copied correctly. Japanese, Chinese and Arabic Characters were all question marks with boxes. I've searched the net and I've come to understand that it has something to do with the encoding scheme. I've checked the syntax, but none of them seem to work for me.
Can you provide me how to do it using JAVA?
NOTE: I seem to have a problem altering the database, (i think i don't have the permission ) so i would like to know if setting cssid in the tablespace is applicable.
When I once had trouble with encodings in DBs it was of a missing param in the commandline of my app:
java ... -Dfile.encoding=UTF-8 ...
The problem could come from the database codeset which permits to define the type of characters you are entering. However, it cannot be canged once the database was created.
Also, a normal column could not support graphic characters. Probably, you need to use a column that support graphic characters like: Graphic, vargraphic or dbclob
Related
As part of a project, we needed to move from Ubuntu 14.04 to Ubuntu 16.04. However, since the upgrade was completed, full functionality has not been working correctly. The encoding of the characters is being jumbled when stored in the database. The same debian version of the software produces different results, implying an ISO issue with a different library or some differences in Java behaviour.
The upgraded server is experiencing no problems and it persists only on newer installs, which implies an issue at the ISO level, but there is no obvious sign as to which library or similar may have failed to install.
Logging was added to print the bytes received, and Java still reads this as it would be expected. However, when it stores them in the database, they are completely different. This is done via a JPA connection setup earlier. This is already using the 'useUnicode=true&characterEncoding=UTF-8' field. When Java reads this data again, it still thinks it is using the correct bytes when it is not. Likewise, if you add something directly to the DB, Java's debugging logs do not show the correct bytes, yet the information is still shown correctly when displayed via the interface which could only have passed through here. This implies the issue is with storing the data rather than handling of it, but the same version of the debian install affects both versions. The working version reads the bytes correctly when it gets them out of the database.
شلاؤ, in Arabic for example is supposed to be encoded as (by using hex function in mysql/mariadb), comes out, in the correct version as "D8B4D984D8A7D8A4" BUT in the incorrect version, displays as "C398C2B4C399C284C398C2A7C398C2A4". This may provide more information as to why the encoding is failing to work correctly. With Java reading the incorrect bytes as if they are correct, this is more likely to be an issue with Java, but the confusion remains due to the inconsistency between systems.
D8B4D984D8A7D8A4 is the correct utf8 (or utf8mb4) encoding for شلاؤ. C398C2B4C399C284C398C2A7C398C2A4 is the "double-encoded" version. This implies that something is still specifying "latin1" as the character set. Perhaps you dumped and reloaded the data, and that is where it happened?
For more on such, see Trouble with UTF-8 characters; what I see is not what I stored and perhaps http://mysql.rjweb.org/doc.php/charcoll
For anyone who may be experiencing something similar, the result turned out the be that Java was running without defaulting to utf8. OpenEJB/JPA was configured correctly, as was the database, but one aspect of the server was defaulting to a different charset, so the startup arguments for the affected area resolved the problem!
So here is the problem. I am scraping some data with java and eventually i place that java into postgres database. When i run Java program, i get error ERROR: relation "table name" does not exist but when i personally write that same query in PGAdmin III, it works fine. I googled it and it's not about caps letters that most people have problems with. Here is a screenshot:
My first thought was that you were using double quotes for values, but then I looked again and realized you were assembling a query using string concatenation.
DON'T DO THAT. In addition to making these problems impossible to debug you open yourself up to sql injection.
In debugging something like this, you should first port to use placeholder syntax (which PostgreSQL's JDBC driver supports) and then, if that doesn't work, then post the server logs.
So I have an application written in JavaFx 2.2 that has been packaged for linux, mac, and windows. I am getting a strange issue with some of the text fields though. The application will read a file and populate some labels based on whats found in the file. When run on ubuntu or mac we get a result like as you can see we have that special accent character over the c and it looks just fine. However in Windows it shows up like this . Any idea as to why this is happening? I was a bit confused as it is the same exact application on all three. Thanks.
Make sure to specify character encoding when reading the file, in order to avoid using the platform's default encoding, which varies between operating systems. Just by coincidence, the default on Linux and Mac happens to match the file encoding and produces correct output, but you should not rely on it.
Good evening,
I am thoroughly baffled by a behaviour that happens with a Java application of mine.
The application is to be used to connect to a web API and download a table of data from there to be further used in academic research.
I was trying to set the export to correct encoding, to save time, I disabled iteration in calls to web API by disabling some methods and removing a variable from a URL string construction - placing an integer 42 in its place.
Upon this, I was surprised to see the correct encoding.
A bit down the road, and I can confirm - if I use this URL:
new URL("http://www.foo.bar/api/unit?id="+scraper.ujList[i]);
where ujList[] is an array of integers,
then the output has typical wrong encoded characters like this ÄŤasĹĄ RuĹľinov
meanwhile, if I use
new URL("http://www.foo.bar/api/unit?id="+42);
then the output is proper, like Ružinov.
I have also tried Integer.toString() as well as String.valueOf() on ujList member, but it didn't help.
FWIW it does not matter if I run a jar file, a class file, or from within Eclipse.
I have no idea how to continue.
I am facing a weird problem. I am getting exception when I try to update or delete row in updatable resultset which contains non-english utf-characters. However insert goes fine.
java.sql.SQLException: refreshRow() called on row that has been deleted or had primary key changed.
The weirdest things are:
This error happens only when compiled jar is run in windows
However same jar run in Linux runs fine for same data without problem.
Same project run from within IDE runs also fine in Windows.
Other information in case that will be helpful
OS: Windows XP (English with non-english language support installed)
DB: MySQL, encoding utf8, collation - utf8_general_ci
IDE: Netbeans 6.9.1
JDK: 6 update 23
Connector/J 5.1.15 (Just switch to check if this works but same problem with version 14 too)
Connection string includes: "useUnicode=true" and "characterEncoding=utf8"
Initially thought that IDE has something to do so posted this message in netbeans forum
http://forums.netbeans.org/topic36558.html
Also cross posted in mysql JDBC forums hoping to find some answer
http://forums.mysql.com/read.php?39,408795,408795
but couldn't get any help there.
So far, the problem seems to be Windows. May be this is just minor issue but can't think of any work around.
Need some suggestion
Thanks and regards
Deepak
It seems like your IDE is override the default encoding that you get when you run your application from the command line. If you check the actual JVM arguments the IDE uses (normally available in the output window of your IDE), you will probably see the inclusion of a file-encoding argument, like this:
-Dfile.encoding="UTF-8"
Try to start your application with this JVM argument and see if it makes any difference, and if not - compare the actual encoding used when run from the IDE and on the command line like this:
System.out.println(System.getProperty("file.encoding"));
I had the same problem and solved it. I don't understand why this is happenning but this is caused when your primary key of mysql table is combined. In my database there are many tables that have combined primary key and others that have auto-increment. Likely i noticed that this problem didn't occur in tables with auto-increment primary key.