I am trying to upload data from CSV file into a mysql database table but i am getting error "java.sql.SQLException: Invalid utf8 character string: '' " when using INSERT/REPLACE option in load data query. but the same query works fine without INSERT/REPLACE option.
Query:
Statement stmt = conn.createStatement(
ResultSet.TYPE_SCROLL_SENSITIVE,
ResultSet.CONCUR_UPDATABLE);
String query = "LOAD DATA INFILE 'D:\\"+flnm+"' REPLACE INTO TABLE prfl_hntr "
+ "FIELDS TERMINATED by ',' LINES TERMINATED BY '\n' IGNORE 1 LINES"
+ " (candidate, phone, mailid, skill, texp, rexp, pctc,np);";
stmt.executeUpdate(query);
I have verified the syntax in mysql documentation- https://dev.mysql.com/doc/refman/8.0/en/load-data.html.
Please note that i need to use REPLACE/IGNORE option in the query to eliminate duplicate entries.
You are right the issue is not with the syntax. The issue might be with difference in the characterset between Mysql table and your CSV file. I had a similar problem and resolved it by mentioning the characterset in the query. Try the query given below it should work and make sure your CSV file has data supported by Mysql table.
Query:
String query = "LOAD DATA INFILE 'D:\\"+flnm+"' REPLACE INTO TABLE prfl_hntr "
+"character set latin1 "
+ "FIELDS TERMINATED by ',' LINES TERMINATED BY '\n' IGNORE 1 LINES"
+ " (candidate, phone, mailid, skill, texp, rexp, pctc,np);";
stmt.executeUpdate(query);
Further you can follow the answer by #RolandoMySQLDBA for this question-: Trying to do LOAD DATA INFILE with REPLACE and AUTO_INCREMENT
It worked wonders for me.
LOAD DATA LOCAL INFILE '/home/xxxxx/conf.csv' INTO TABLE configuration FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n' IGNORE 1 ROWS;
try this once.
Related
I'm trying to run an insert or update on a table - the string generated from below works fine when copy pasted into HeidiSQL but throws SQLSyntaxErrorExceptions when run from Java:
Statement statement = con.createStatement();
String escapedXML = EscapeString(billboard.getXml());
String sql = String.format(
"DELIMITER $ \r\nBEGIN NOT ATOMIC\r\n" +
"IF EXISTS(SELECT * FROM billboards where Name='%s') THEN UPDATE billboards SET XML='%s' where Name='%s';\r\n" +
"ELSE insert into billboards(Name, XML, CreatorName) values('%s', '%s', '%s');\r\n" +
"END IF;\r\n" +
"END $\r\n" +
"DELIMITER ;", billboard.getName(), escapedXML, billboard.getName(), billboard.getName(), escapedXML, billboard.getCreatorName());
// Insert or update billboard
statement.execute(sql);
I can't figure out why.
I would recommend using the insert ... ok duplicate key syntax here rather than a code block. This is more efficient, and implements the lockout a single statement, which should avoid the problem you meet when running the query from your php code.
insert into billboards(Name, XML, CreatorName)
values(?, ?, ?)
on duplicate key update set XML = values(XML)
For this to work, you need a unique (or primary key) constraint on column Name.
Also, consider using a parameterized query rather than concatenating variables in your query stringW Escaping is inefficient and does not really make your code safer.
You should have tried NamedParameterStatement with your query to facilitate setting of string parameters and avoid their duplication (using refactored query suggested in GMB's earlier answer):
String sql = "INSERT INTO billboards (Name, XML, CreatorName) VALUES (:name, :xml, :creator) "
+ "ON DUPLICATE KEY UPDATE SET XML = :xml";
NamedParameterStatement statement = new NamedParameterStatement(con, sql);
statement.setString("name", billboard.getName());
statement.setString("xml", EscapeString(billboard.getXml()));
statement.setString("creator", billboard.getCreatorName());
// Insert or update billboard
statement.execute(sql);
The reason that you are getting a syntax error is that DELIMITER is a MySQL client command and not an SQL statement. MySQL commands may not be used in with JDBC.
For more information:
Delimiters in MySQL
I am trying to insert data into MySQL, and while doing so, was faced with the error where the input has single quotation, therefore breaking my sql insert string.
This is my line of code
String visitorSql = "INSERT INTO visitor" + "(encryptedEmail)" + "VALUES ('" + encryptedEmail.replace("'", "\\'")+"')";
The syntax error is,
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''w+8L8%]/ò“VZ(R?cêíµ–ö\')' at line 1
I've encrypted the email for the sake of the privacy. Any other idea how should I code it to escape the quotation?
You have to replace all the single quotes(') with ''(2 single quotes) to escape them.
String visitorSql = "INSERT INTO visitor(COLUMN_NM) VALUES ('" + encryptedEmail.replace("'", "''")+"')";
I use the following command to import data from a .csv file into a MySQL database table like so:
String loadQuery = "LOAD DATA LOCAL INFILE '" + file + "' INTO TABLE source_data_android_cell FIELDS TERMINATED BY ','" + "ENCLOSED BY '\"'"
+ " LINES TERMINATED BY '\n' " + "IGNORE 1 LINES(.....)" +"SET test_date = STR_TO_DATE(#var1, '%d/%m/%Y %k:%i')";
However, as one of the columns in the sourcefile contains a really screwy data which is: viva Y31L.RastaMod䋢_Version the program refuses to import the data into MySQL and keeps throwing this error:
java.sql.SQLException: Invalid utf8 character string: 'viva
Y31L.RastaMod'
I searched up on this but cant really understand what exactly the error was, other than that the INPUT format of this string "viva Y31L.RastaMod䋢_Version" was wrong and didn't fit the utf8 format used in the MySQL database?
However, I already did the following which is SET NAMES UTF8MB4 in my MySQL db, since it was suggested in other questions that UTF8MB4 was more flexible in accepting weird characters.
I explored this further by manually inserting that weird data into MySQL database table in the Command Prompt, which worked fine. In fact, the table displayed almost the full entry: viva Y31L.RastaMod?ã¢_Version. But if I ran my program from the IDE the file gets rejected.
Would appreciate any explanations.
Second minor question related to the import process of csv file into mySQL:
I noticed that I couldn't import a copy of the same file into the MySQL database. Errors thrown included that the data was a duplicate. Is that because MySQL rejects duplicate column data? But when I changed all the data of one column leaving the rest the same in that copied file, it gets imported correctly. Why is that so?
I don't think this immediate error has to do with the destination of the data not being able to cope with UTF-8 characters, but rather the way you are using LOAD DATA. You can try specifying the character set which should be used when loading the data. Consider the following LOAD DATA command, which is what you had originally but slightly modified:
LOAD DATA LOCAL INFILE path/to/file INTO TABLE source_data_android_cell
CHARACTER SET utf8
FIELDS TERMINATED BY ','
ENCLOSED BY '"'
LINES TERMINATED BY '\n'
IGNORE 1 LINES(.....)
SET test_date = STR_TO_DATE(#var1, '%d/%m/%Y %k:%i')
This being said, you should also make sure that the target table uses a character set which supports the data you are trying to load into it.
I am using eclipse and mySQL for coding, while inserting the values I received the syntax error.
if(!(nameOfConvo.equals(visitorName))){
staffConvo = StringUtils.substringAfter(convo, ": ");
System.out.println("Staff - " + staffConvo);
String staffSql = "INSERT INTO webchatdata" + "(staffConvo)" + "VALUES ('"+ staffConvo+ "')";
myStat.executeUpdate(staffSql);
}
else {
visitorConvo = StringUtils.substringAfter(convo, ": ");
System.out.println("Visitor - " + visitorConvo);
String visitorSql = "INSERT INTO webchatdata" + "(visitorConvo)" + "VALUES ('" +visitorConvo+"')";
myStat.executeUpdate(visitorSql);
}
while in mySQL it is printing out some values, it'll only print halfway and display :
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 's in a course that he has interest in, it is likely that he will excel in it. I' at line 1
Inserting other variables into the database were fine like ex.
String timeStampSql = "INSERT INTO conversation" + "(timestamp)" + "VALUES ('" +timeStamp+"')";
myStat.executeUpdate(timeStampSql);
The problem may be with the content you're putting in to SQL.
Assuming the full string might be something like this.. (You have not provided what the actual input is in this case, so I can only assume)
Bob's in a course that he has interest in, it is likely that he will excel in it. I'm writing
Notice that the first and last character are single quotation marks.
This is breaking your sql insert string, because it will close the string when it reads ' in the text.
When you save the string, you need to escape the quotations so the string is not finished incorrectly. Note the backslashes added.
Bob\'s in a course that he has interest in, it is likely that he will excel in it. I\'m writing
I have a html page that allows users to enter specific search terms to query the database I've created. The problem I'm having is that when I pass the string to execute as a sql query, it is not wrapping the query in single quotes which is needed to search for a string match in a sql database. Here is the code I have currently:
//Create a statement for the sql queries
java.sql.Statement stmt = conn.createStatement();
//Get results from queries entered
String result_event_name = request.getParameter("event_name");
//Create query string
String sqlQuery_event = "SELECT event_name FROM event where event_name = " + "\'" + result_event_name + "\'";
//execute query
java.sql.ResultSet rs_event = stmt.executeQuery(sqlQuery_event);`
This is the error I get:
SQLException: ERROR: syntax error at end of input Position: 49
I tried using prepare statement -- returns same error
I tried the query without escaping -- returns same error
I tried with no single quotes -- returns same error
The single quote ' only needs escaping once LIKE '%\'%'
But to query backslash \ you need to double escape to LIKE '%\\\\%'
If you wanted to query backslash+singlequote \' then LIKE '%\\\\\'%'
(with 5 backslashes)
Explanation Source excerpt:
Because MySQL uses C escape syntax in strings (for example, “\n” to
represent a newline character), you must double any “\” that you use
in LIKE strings. For example, to search for “\n”, specify it as “\n”.
To search for “\”, specify it as “\”; this is because the backslashes
are stripped once by the parser and again when the pattern match is
made, leaving a single backslash to be matched against.
Credit goes to xchiltonx
Resource Link:
mysql - How to handle query search with special characters /(forward slash) and \(backslash)