I am trying to learn the best way to write queries. I also understand the importance of being consistent. Until now, I have randomly used single quotes, double quotes, and backticks without any real thought.
Example:
$query = 'INSERT INTO table (id, col1, col2) VALUES (NULL, val1, val2)';
Also, in the above example, consider that table, col1, val1, etc. may be variables.
What is the standard for this? What do you do?
I've been reading answers to similar questions on here for about 20 minutes, but it seems like there is no definitive answer to this question.
Backticks are to be used for table and column identifiers, but are only necessary when the identifier is a MySQL reserved keyword, or when the identifier contains whitespace characters or characters beyond a limited set (see below) It is often recommended to avoid using reserved keywords as column or table identifiers when possible, avoiding the quoting issue.
Single quotes should be used for string values like in the VALUES() list. Double quotes are supported by MySQL for string values as well, but single quotes are more widely accepted by other RDBMS, so it is a good habit to use single quotes instead of double.
MySQL also expects DATE and DATETIME literal values to be single-quoted as strings like '2001-01-01 00:00:00'. Consult the Date and Time Literals documentation for more details, in particular alternatives to using the hyphen - as a segment delimiter in date strings.
So using your example, I would double-quote the PHP string and use single quotes on the values 'val1', 'val2'. NULL is a MySQL keyword, and a special (non)-value, and is therefore unquoted.
None of these table or column identifiers are reserved words or make use of characters requiring quoting, but I've quoted them anyway with backticks (more on this later...).
Functions native to the RDBMS (for example, NOW() in MySQL) should not be quoted, although their arguments are subject to the same string or identifier quoting rules already mentioned.
Backtick (`)
table & column ───────┬─────┬──┬──┬──┬────┬──┬────┬──┬────┬──┬───────┐
↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓ ↓
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`, `updated`)
VALUES (NULL, 'val1', 'val2', '2001-01-01', NOW())";
↑↑↑↑ ↑ ↑ ↑ ↑ ↑ ↑ ↑↑↑↑↑
Unquoted keyword ─────┴┴┴┘ │ │ │ │ │ │ │││││
Single-quoted (') strings ───────────┴────┴──┴────┘ │ │ │││││
Single-quoted (') DATE ───────────────────────────┴──────────┘ │││││
Unquoted function ─────────────────────────────────────────┴┴┴┴┘
Variable interpolation
The quoting patterns for variables do not change, although if you intend to interpolate the variables directly in a string, it must be double-quoted in PHP. Just make sure that you have properly escaped the variables for use in SQL. (It is recommended to use an API supporting prepared statements instead, as protection against SQL injection).
// Same thing with some variable replacements
// Here, a variable table name $table is backtick-quoted, and variables
// in the VALUES list are single-quoted
$query = "INSERT INTO `$table` (`id`, `col1`, `col2`, `date`) VALUES (NULL, '$val1', '$val2', '$date')";
Prepared statements
When working with prepared statements, consult the documentation to determine whether or not the statement's placeholders must be quoted. The most popular APIs available in PHP, PDO and MySQLi, expect unquoted placeholders, as do most prepared statement APIs in other languages:
// PDO example with named parameters, unquoted
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`) VALUES (:id, :col1, :col2, :date)";
// MySQLi example with ? parameters, unquoted
$query = "INSERT INTO `table` (`id`, `col1`, `col2`, `date`) VALUES (?, ?, ?, ?)";
Characters requring backtick quoting in identifiers:
According to MySQL documentation, you do not need to quote (backtick) identifiers using the following character set:
ASCII: [0-9,a-z,A-Z$_] (basic Latin letters, digits 0-9, dollar, underscore)
You can use characters beyond that set as table or column identifiers, including whitespace for example, but then you must quote (backtick) them.
Also, although numbers are valid characters for identifiers, identifiers cannot consist solely of numbers. If they do they must be wrapped in backticks.
There are two types of quotes in MySQL:
' for enclosing string literals
` for enclosing identifiers such as table and column names
And then there is " which is a special case. It could be used for one of above-mentioned purposes at a time depending on MySQL server's sql_mode:
By default the " character can be used to enclose string literals just like '
In ANSI_QUOTES mode the " character can be used to enclose identifiers just like `
The following query will produce different results (or errors) depending on SQL mode:
SELECT "column" FROM table WHERE foo = "bar"
ANSI_QUOTES disabled
The query will select the string literal "column" where column foo is equal to string "bar"
ANSI_QUOTES enabled
The query will select the column column where column foo is equal to column bar
When to use what
I suggest that you avoid using " so that your code becomes independent of SQL modes
Always quote identifiers since it is a good practice (quite a few questions on SO discuss this)
(There are good answers above regarding the SQL nature of your question, but this may also be relevant if you are new to PHP.)
Perhaps it is important to mention that PHP handles single and double quoted strings differently...
Single-quoted strings are 'literals' and are pretty much WYSIWYG strings. Double-quoted strings are interpreted by PHP for possible variable-substitution (backticks in PHP are not exactly strings; they execute a command in the shell and return the result).
Examples:
$foo = "bar";
echo 'there is a $foo'; // There is a $foo
echo "there is a $foo"; // There is a bar
echo `ls -l`; // ... a directory list
Backticks are generally used to indicate an identifier and as well be safe from accidentally using the Reserved Keywords.
For example:
Use `database`;
Here the backticks will help the server to understand that the database is in fact the name of the database, not the database identifier.
Same can be done for the table names and field names. This is a very good habit if you wrap your database identifier with backticks.
Check this answer to understand more about backticks.
Now about Double quotes & Single Quotes (Michael has already mentioned that).
But, to define a value you have to use either single or double quotes. Lets see another example.
INSERT INTO `tablename` (`id, `title`) VALUES ( NULL, title1);
Here I have deliberately forgotten to wrap the title1 with quotes. Now the server will take the title1 as a column name (i.e. an identifier). So, to indicate that it's a value you have to use either double or single quotes.
INSERT INTO `tablename` (`id, `title`) VALUES ( NULL, 'title1');
Now, in combination with PHP, double quotes and single quotes make your query writing time much easier. Let's see a modified version of the query in your question.
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
Now, using double quotes in the PHP, you will make the variables $val1, and $val2 to use their values thus creating a perfectly valid query. Like
$val1 = "my value 1";
$val2 = "my value 2";
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
will make
INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, 'my value 1', 'my value 2')
In MySQL, these symbols are used to delimit a query ` ," ,' and () .
" or ' are used for enclosing string-like values "26-01-2014 00:00:00" or '26-01-2014 00:00:00' . These symbols are only for strings, not aggregate functions like now, sum, or max.
` is used for enclosing table or column names, e.g. select `column_name` from `table_name` where id='2'
( and ) simply enclose parts of a query e.g. select `column_name` from `table_name` where (id='2' and gender='male') or name='rakesh' .
There has been many helpful answers here, generally culminating into two points.
BACKTICKS(`) are used around identifier names.
SINGLE QUOTES(') are used around values.
AND as #MichaelBerkowski said
Backticks are to be used for table and column identifiers, but are
only necessary when the identifier is a MySQL reserved keyword, or
when the identifier contains whitespace characters or characters
beyond a limited set (see below) It is often recommended to avoid
using reserved keywords as column or table identifiers when possible,
avoiding the quoting issue.
There is a case though where an identifier can neither be a reserved keyword or contain whitespace or characters beyond limited set but necessarily require backticks around them.
EXAMPLE
123E10 is a valid identifier name but also a valid INTEGER literal.
[Without going into detail how you would get such an identifier name], Suppose I want to create a temporary table named 123456e6.
No ERROR on backticks.
DB [XXX]> create temporary table `123456e6` (`id` char (8));
Query OK, 0 rows affected (0.03 sec)
ERROR when not using backticks.
DB [XXX]> create temporary table 123451e6 (`id` char (8));
ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '123451e6 (`id` char (8))' at line 1
However, 123451a6 is a perfectly fine identifier name (without back ticks).
DB [XXX]> create temporary table 123451a6 (`id` char (8));
Query OK, 0 rows affected (0.03 sec)
This is completely because 1234156e6 is also an exponential number.
The string literals in MySQL and PHP are the same.
A string is a sequence of bytes or characters, enclosed within either
single quote (“'”) or double quote (“"”) characters.
So if your string contains single quotes, then you could use double quotes to quote the string, or if it contains double quotes, then you could use single quotes to quote the string. But if your string contains both single quotes and double quotes, you need to escape the one that used to quote the string.
Mostly, we use single quotes for an SQL string value, so we need to use double quotes for a PHP string.
$query = "INSERT INTO table (id, col1, col2) VALUES (NULL, 'val1', 'val2')";
And you could use a variable in PHP's double-quoted string:
$query = "INSERT INTO table (id, col1, col2) VALUES (NULL, '$val1', '$val2')";
But if $val1 or $val2 contains single quotes, that will make your SQL be wrong. So you need to escape it before it is used in sql; that is what mysql_real_escape_string is for. (Although a prepared statement is better.)
In combination of PHP and MySQL, double quotes and single quotes make your query-writing time so much easier.
$query = "INSERT INTO `table` (`id`, `col1`, `col2`) VALUES (NULL, '$val1', '$val2')";
Now, suppose you are using a direct post variable into the MySQL query then, use it this way:
$query = "INSERT INTO `table` (`id`, `name`, `email`) VALUES (' ".$_POST['id']." ', ' ".$_POST['name']." ', ' ".$_POST['email']." ')";
This is the best practice for using PHP variables into MySQL.
If table cols and values are variables then there are two ways:
With double quotes "" the complete query:
$query = "INSERT INTO $table_name (id, $col1, $col2)
VALUES (NULL, '$val1', '$val2')";
Or
$query = "INSERT INTO ".$table_name." (id, ".$col1.", ".$col2.")
VALUES (NULL, '".$val1."', '".$val2."')";
With single quotes '':
$query = 'INSERT INTO '.$table_name.' (id, '.$col1.', '.$col2.')
VALUES (NULL, '.$val1.', '.$val2.')';
Use back ticks `` when a column/value name is similar to a MySQL reserved keyword.
Note: If you are denoting a column name with a table name then use back ticks like this:
`table_name`. `column_name` <-- Note: exclude . from back ticks.
Single quotes should be used for string values like in the VALUES() list.
Backticks are generally used to indicate an identifier and as well be safe from accidentally using the reserved keywords.
In combination of PHP and MySQL, double quotes and single quotes make your query writing time so much easier.
Besides all of the (well-explained) answers, there hasn't been the following mentioned and I visit this Q&A quite often.
In a nutshell; MySQL thinks you want to do math on its own table/column and interprets hyphens such as "e-mail" as e minus mail.
Disclaimer: So I thought I would add this as an "FYI" type of answer for those who are completely new to working with databases and who may not understand the technical terms described already.
SQL servers and MySQL, PostgreySQL, Oracle don't understand double quotes("). Thus your query should be free from double quotes(") and should only use single quotes(').
Back-trip(`) is optional to use in SQL and is used for table name, db name and column names.
If you are trying to write query in your back-end to call MySQL then you can use double quote(") or single quotes(') to assign query to a variable like:
let query = "select id, name from accounts";
//Or
let query = 'select id, name from accounts';
If ther's a where statement in your query and/or trying to insert a value and/or an update of value which is string use single quote(') for these values like:
let querySelect = "select id, name from accounts where name = 'John'";
let queryUpdate = "update accounts set name = 'John' where id = 8";
let queryInsert = "insert into accounts(name) values('John')";
//Please not that double quotes are only to be used in assigning string to our variable not in the query
//All these below will generate error
let querySelect = 'select id, name from accounts where name = "John"';
let queryUpdate = 'update accounts set name = "John" where id = 8';
let queryInsert = 'insert into accounts(name) values("John")';
//As MySQL or any SQL doesn't understand double quotes("), these all will generate error.
If you want to stay out of this confusion when to use double quotes(") and single quotes('), would recommend to stick with single quotes(') this will include backslash() like:
let query = 'select is, name from accounts where name = \'John\'';
Problem with double(") or single(') quotes arise when we had to assign some value dynamic and perform some string concatenation like:
let query = "select id, name from accounts where name = " + fName + " " + lName;
//This will generate error as it must be like name = 'John Smith' for SQL
//However our statement made it like name = John Smith
//In order to resolve such errors use
let query = "select id, name from accounts where name = '" + fName + " " + lName + "'";
//Or using backslash(\)
let query = 'select id, name from accounts where name = \'' + fName + ' ' + lName + '\'';
If need further clearance do follow quotes in JavaScript
It is sometimes useful to not use quotes... because this can highlight issues in the code generating the query... For example:
Where x and y are should always be integers...
SELECT * FROM table WHERE x= AND y=0
Is a SQL syntax error... a little lazy but can be useful...
I would like to create a SQL query containing ASCII or UNICODE character codes in it. For example, ASCII character code for single quote (') is 39 and unicode code is U+0027. In Java, I would like to write a query by replacing the single codes with their character codes:
ASCII:
connection.createStatement().executeQuery("select * from users where name =39test39")
Unicode:
connection.createStatement().executeQuery("select * from users where name =U+0027testU+0027")
All of these queries should be equivalent to "select * from users where name ='test'"
When I run the codes above, DBMS (I tried with Mysql and SQLite) does not recognize the ascii and unicode codes as a single quote.
In summary, I know parametrized queries are the ideal. But, here in this case what I wanted to do is, when the sql code is parsed by the DBMS, then the DBMS should recognize the unicode character. For example, if I use \u0027, the JVM would recognize this as a single quote, but I want JVM to not recognize and DMBS to recognize the character encoding.
Is there any way use char codes instead of the character itself?
No, you don't want to do that. You should be doing
PreparedStatement ps = conn.prepareStatement("select * from users where name = ?");
ps.setString(1, "test");
ResultSet rs = ps.executeQuery();
Remember that all strings in Java are Unicode strings, so what you are proposing is to start sending string values as byte streams to the JDBC driver, which would be messy and error-prone (if it is even possible).
When you put the ascii/unicode numbers within double quotes they aren't resolved to characters instead try something like:
"select * from users where name =" + Character.toString(Character.toChar(yourIntHere)) + ...
And then that should build the string you are looking for
You query should look like this :
"select * from users where name =" + Character.toString((char)39) + "test" + Character.toString((char)39) + "\""
I'm working with mySQL. It can not handle if ' is in the String that is being added to the database.
I tried:
replaceAll("'","\\'")
and
replaceAll("'","\'")
Any ideas how I would go about replacing ' with \'?
Don't use String replacements to handle this. Instead, use a prepared statement and thus let the JDBC driver escape the parameters for you:
String sql = "select a.foo from a where a.bar = ?";
PreparedStatement stmt = connection.prepareStatement(sql);
stmt.setString(1, aStringWhichMightContainQuotes);
ResultSet rs = stmt.executeQuery();
This is the proper way to have database-independent, robust code, that is not vulnerable to SQL injection attacks. And it also make it more efficient if you execute the same query several times with different parameters.
See the JDBC tutorial for more information.
You need to escape the backslash twice, once for the string processing engine and once for the regex engine:
replaceAll("'","\\\\'")
Caveat: While this answers the question about how to insert a backslash into a string, it certainly should not be used in an attempt to thwart SQL injection attacks.
To clarify: Imagine someone submits a string where the apostrophe is already escaped. This regex would then lead to the apostrophe being unescaped (because now the backslash would become escaped). So actually you'd need this regex to escape an apostrophe only if preceded by an even number of backslashes. This means
replaceAll("(?<!\\\\)((?:\\\\\\\\)*)'", "$1\\\\'")
This is rapidly becoming as unmaintainable as it looks, and it still doesn't cover all cases.
Is there a built-in method to escape a string for SQL? I would use setString, but it happens I am using setString multiple times in the same combined SQL statement and it would be better performance (I think) if the escape happened only once instead of each time I say setString. If I had the escaped string in a variable, I could re-use it.
Is there no way to do this in Java?
Current method, multi-source search. In reality they are three entirely different where statements including joins, but for this example I will just show the same where for each table.
String q = '%' + request.getParameter("search") + '%';
PreparedStatement s = s("SELECT a,b,c FROM table1 where a = ? UNION select a,b,c from table2 where a = ? UNION select a,b,c FROM table3 where a = ?");
s.setString(1, q);
s.setString(2, q);
s.setString(3, q);
ResultSet r = s.executeQuery();
I know this is not a big deal, but I like to make things efficient and also there are situations where it is more readable to use " + quote(s) + " instead of ? and then somewhere down the line you find setString.
If you use setString for a parameter (e.g. PreparedStatement.setString), there may well be no actual escaping required - it's likely that the data will be passed separately from the SQL itself, in a way that doesn't require escaping.
Do you have any concrete indication that this really is a performance bottleneck? It seems very unlikely that within a database query, the expensive part is setting the parameters locally...
Short answer: I wouldn't bother. It's best to do escaping at the last popssible moment. When you try to escape a string early and keep it around, it becomes much more difficult to verify that all strings have been escaped exactly once. (Escaping a string twice is almost as bad as not escaping it at all!) I've seen plenty of programs that try to escape strings early and then run into trouble because they need to update the string and then the programmer forgets to re-do the escape, or they update the escaped version of the string, or they have four strings and they escape three of them, etc. (I was just working on a bug where a programmer did HTML escapes on a string early, then decided he had to truncate the string to fit on a form, and ended up trying to output a string that ended with "&am". That is, he truncated his escape sequence so it was no longer valid.)
The CPU time to escape a string should be trivial. Unless you have a very large number of records or very big strings that are re-used, I doubt the savings would be worth worrying about. You'd probably be better off spending your time optimizing queries: saving a read of one record would probably be worth far more than eliminating 1000 trips through the string escape logic.
Longer answer: There's no built-in function. You could write one easily enough: Most flavors of SQL just need you to double any single quotes. You may need to also double backslashes or one or two other special characters. The fact that this can be different between SQL engines is one of the big arguments for using PreparedStatements and letting JDBC worry about it. (Personally I think there should be a JDbC function to do escaping that could then know any requirements specific to the DB engine. But there isn't so that's how it is.)
In any case, it's not clear how it would work with a PreparedStatement. There'd have to be some way to tell the PreparedStatement not to escape this string because it's already been escaped. And who really knows what's happening under the table in the conversation between JDBC and the DB engine: Maybe it never really escapes it at all, but passes it separately from the query. I suppose there could be an extra parameter on the setString that says "this string was pre-escaped", but that would add complexity and potential errors for very little gain.
Do not use org.apache.commons.lang.StringEscapeUtils.escapeSql(yourUnscapedSQL);
It does not escape characters like \
You can use StringEscapeUtils from Apache commons:
org.apache.commons.lang.StringEscapeUtils.escapeSql(yourUnscapedSQL);