How to load Arabic csv to MYSQL - java

i want to load csv arabic file into mysql
I tried the following
set session collation_database=utf8_general_ci;
set session character_set_database=utf8 ;
#SET NAMES utf8;
#SET character_set_database=utf8;
LOAD DATA local INFILE "D:\\trade20120314.csv"
INTO TABLE trade
CHARACTER SET utf8
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n' ;
but the arabic columns still appears as
"?????? ????????????????".
I am sure that the csv file is encoded to utf8 and the default character set of server and client is as utf8 and also the columns character set and collation as utf8 .
I don't know where is the problem , any suggestions .
Thanks .

Try adding to the JDBC URI ?useUnicode=true&characterEncoding=UTF-8 (or &...). The driver class also has to know the transfer protocol. (Of course the file should be in UTF-8.)

Related

LOAD DATA INFILE LINES TERMINATD doesn't work with \n

I have a csv file, that I want to import to MySQL, but LINES TERMINATED doesn't work with '\n'. I tried to replace \n with '\r' or '\r\n', but it still doesn't work.
If I open my file in HEX editor it is obvious, that my Java App (that writes this file) works fine ('\n' terminators are highlighted).
But when I run
LOAD DATA LOCAL INFILE '/home/vsafonov/testDir/test.csv'
INTO TABLE express.objs
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES;
I get
If I replace '\n' for new lines with some symbol (for example ';'), LOAD DATA INFILE works fine. But I have no ideas, why it is impossible to load file with '\n' line terminator. Some thoughts?
Table's CREATE TABLE
CREATE TABLE IF NOT EXISTS `test_objs` (
`id` bigint(20) NOT NULL,
`object_id` bigint(20) DEFAULT NULL,
`next_id` bigint(20) DEFAULT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET = utf8 COLLATE = utf8_unicode_ci;
My file example:
id,object_id,next_id
"1866227","98363301","156715750"
"293","171","454"
"1890275","171","177646470"
For me with query worked
LOAD DATA LOCAL INFILE '/home/vsafonov/testDir/test.csv'
INTO TABLE express.objs
FIELDS TERMINATED BY ',' ENCLOSED BY '"'
IGNORE 1 LINES;
Default terminator worked fine with my '/n'.
Terminator in my query was accepted literally (didn't understand why).

Unknown Os Character Set 'cp720' , switching to the default character set 'latin1'

When I run mysql -root -p db2 <mySuperMarketDB.sql and enter password then I get the following error:
Unknown Os Character Set 'cp720' , switching to the default character set 'latin1'
How can I fix that?
Change encoding to 1252
c:\chcp 1252
You can change code page permanantly as follows:
Start -> Run -> regedit
Go to [HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor]
Add new String Value named: Autorun
Change the value to 'chcp 1252'
If you are trying to run some sql text file in mysql and get an error like this:
C:\wamp\bin\mysql\mysql5.6.12\bin\mysql.exe: Unknown OS character set 'cp862'.
C:\wamp\bin\mysql\mysql5.6.12\bin\mysql.exe: Switching to the default character set 'latin1'.
You get the error because the engine is use the default. To set the default to UTF-8 in "my.ini" file you have to add this lines:
[client]
default-character-set=utf8
[mysql]
default-character-set=utf8
[mysqld]
collation-server = utf8_unicode_ci
init-connect='SET NAMES utf8'
character-set-server = utf8
save the file and restart the service.
credits goes here : https://shlomovitz.blogspot.com/2013/11/unknown-os-character-set-cp862-and.html?fbclid=IwAR15ADTZd5E3cKeQX0w2vR6b5Ef8GBW86ptpd-M2CpyORp3bNkLN2QiXLek

Croatian character in Java standard output

I have a database with some cratian characters in it like Đ , in the database the character is stored correctly, when using a datatable in primefaces it also shows the character in the webpage just fine.
The problem is that when I send it to the out.println() the character Đ in the name is missing.
for (People p : people) {
System.out.println("p.getName());
}
I tried using String name2 = p.getName().getBytes("ISO-8859-2"); but it still not working
I assume you are using UTF-8 as default encoding on the Database and for Primefaces
Have also a look to this:
Display special characters using System.out.println

Save string path in a database

Java MySQL Database
I'm doing a project on saving a string which is a path name like, "C:\Desktop\" into the database. I had create a entity class to update this path name into database, in java eclipse when i run my program it display the path is store in the database in this format, "C:\Desktop\" but in the database column for this path it only store "C: Desktop", without the '\'
You need to escape the \ with \\. Use this to store
C:\\Desktop\\
instead of
C:\Desktop\
Learn more about escape sequence in java : http://docs.oracle.com/javase/tutorial/java/data/characters.html
Simplest solution is use / instead of \ in path . Or escapes the characters in a String using Java String rules
A simple solution is to replace the the "\" before you store it in the database. Try:
string.replace("\","#");
Then your slashes are the # symbols. When you read the value again, you can do it the other way.
you may try storing it with forward slash i.e. "C:/Desktop/"

MYSQL special chars issue

I´ve been having this problem for a long time, I´ve searched the internet many times for the solution, tried lots of them but not found an adequate solution.
I really don´t know what to do so if you could please help me I´d be very thankful.
(Sorry for my poor english).
Question: How can I solve the charset incompatibility between the input archive and a MYSql table?
Problem: When importing the archive from on my computer the information appears in my database, but some chars as ('ã', 'ç', 'á', etc..) are shown as ?.
Aditional information
I'm using MYSql, my version and variable status are:
MySQL VERSION : 5.5.10
HOST : localhost
USER : root
PORT : 3306
SERVER DEFAULT CHARSET : utf8
character_set_client : utf8
character_set_connection : utf8
character_set_database : utf8
character_set_filesystem : BINARY
character_set_results : utf8
character_set_server : utf8
character_set_system : utf8
collation_connection : utf8_general_ci
collation_database : utf8_general_ci
collation_server : utf8_general_ci
completion_type : NO_CHAIN
concurrent_insert : AUTO
The query that´s being used is:
LOAD DATA LOCAL INFILE 'xxxxx/file.txt'
INTO TABLE xxxxTable
FIELDS TERMINATED BY ';'
LINES TERMINATED BY ' '
IGNORE 1 LINES
( status_ordenar,numero,newstatus,rede,data_emissao,inicio,termino,tempo_indisp
, cli_afet,qtd_cli_afet,cod_encerr,uf_ofensor,localidades,clientes_afetados
, especificacao,equipamentos,area_ofens,descricao_encerr,criticidade,cod_erro
, observacao,id_falha_perc,id_falha_conf,nba,solucao,falhapercebida,falhaconfirmada
, resp_i,resp_f,resp_ue,pre_handover,falha_identificada,report_netcool,tipo_falha
, num_notificacao,equip_afetados,descricao)
About the file being imported:
I´ve opened the file with open office whith 3 charsets:
UTF8 - Gave me strange chars in place of the 'ç', 'ã', etc...
ISO-8859-1 - OK.
WIN-1252 - OK.
ASCII/US - OK.
Already tested: I´ve tested some charsets in my database: latin1, utf-8, ascii, but all of them gave me the same result (? instead of 'á', 'ç' etc).
Extra: I'm using Java with Java JDBC to generate and send the query.
file.txt is saved in ISO-8859-1 or Windows-1252 (these two are very similar), and being interpreted as UTF-8 by MySQL. These are incompatible.
How can I tell?
See point 3.: the file displays correctly when interpreted as ISO-8859-1 or Windows-1252.
See point 1.: character_set_database : utf8
Solution: either convert the file to UTF-8, or tell MySQL to interpret it as ISO-8859-1 or Windows-1252.
Background: the characters you provide (ã etc.) are single-byte values in windows-1252, and these bytes are illegal values in UTF-8, thus yielding the '?'s (unicode replacement characters).
Snippet from MySQL docs:
LOAD DATA INFILE Syntax
The character set indicated by the character_set_database system variable is used to interpret the information in the file.
Saved your characters with standard Windows Notepad as UTF-8 file (Notepad++ is also OK).
Exact file content:
'ã', 'ç', 'á'
MySQL version: 5.5.22
Database charset: utf8
Database collation: utf8_general_ci
CREATE TABLE `abc` (
`qwe` text
) ENGINE=InnoDB DEFAULT CHARSET=utf8
Imported data with command
LOAD DATA LOCAL INFILE 'C:/test/utf8.txt'
INTO TABLE abc
FIELDS TERMINATED BY ';'
LINES TERMINATED BY ' '
IGNORE 1 LINES
( qwe)
Result (displayed in SQLyog):
So, first - you should check original file with reliable editor (notepad, notepad++). If file corrupted, then you should take another file.
Second - if file is OK, add you Java code for sending data to MySql into question.

Categories