I'm working with third party user data that may or may not fit into our database. The data needs to be truncated if it is too long.
We are using IBatis with Connector/J. If the data is too long a SQL exception is thrown. I have had two choices: either truncate the strings in Java or truncate the strings in sql using substring.
I don't like truncating the strings in sql, because I am writing table structure in our Ibatis XML, but SQL on the other hand knows about our database collation (which isn't consistent and would be expensive to make consistent) and can truncate string in a multibyte safe manner.
Is there a way to have the Connector/J just straight insert this SQL and if not which route would people recommend?
According to the MySQL documentation it's possible that inserting data that exceeds the length could be treated as a warning:
Inserting a string into a string
column (CHAR, VARCHAR, TEXT, or BLOB)
that exceeds the column's maximum
length. The value is truncated to the
column's maximum length.
One of the Connector/J properties is jdbcCompliantTruncation. This is its description:
This sets whether Connector/J should
throw java.sql.DataTruncation
exceptions when data is truncated.
This is required by the JDBC
specification when connected to a
server that supports warnings (MySQL
4.1.0 and newer). This property has no effect if the server sql-mode includes
STRICT_TRANS_TABLES. Note that if
STRICT_TRANS_TABLES is not set, it
will be set as a result of using this
connection string option.
If I understand correctly then setting this property to false doesn't throw the exception but inserts the truncated data. This solution doesn't require you to truncate the data in program code or SQL statements, but delegates it to the database.
Related
I am actually trying to insert the data in the H2 database. While starting up the application server, was getting the SQL syntax error exception. I am really not sure if H2 database supports the insertion of array in the column? Is there any issue with the below sql statement? Does H2 database support any of the array datatypes float[], String[]...?
INSERT INTO weather (id,date,temperature) values ('1','2019-09-11','{"37.3","36.8","36.4"}');
CREATE TABLE WEATHER(
id INT AUTO_INCREMENT PRIMARY KEY,
date DATE,
temperature text[]
);
You can't use PostgreSQL-style text[] as a data type in H2 and in other databases. H2 has the ARRAY data type for arrays:
https://h2database.com/html/datatypes.html#array_type
H2 1.4.201 will also support standard-compliant array data type with a component type:
componentDataType ARRAY[maximumCardinality]
You can build H2 from its current sources if you really need that functionality right now, but I think you don't really need it, non-standard plain ARRAY will work too.
'{"37.3","36.8","36.4"}' is a character string literal. H2 uses the standard array literals:
ARRAY[element, …]
https://h2database.com/html/grammar.html#array
If you use some outdated version of H2 you need to use non-standard (element, …) literal instead (but don't use that variant in recent versions, it will be parsed by them as a row value as required by the Standard).
It's not related with your question, but you really should use 1 instead of '1' as integer literal and DATE '2019-09-11' instead of '2019-09-11' as a date literal to avoid conversions from character strings to other data types.
Given this PostgreSQL table with a fixed length array column :
CREATE TABLE test (
id integer,
values integer[4],
);
Will JOOQ code generation create a java constant or method that provides the max number of elements that can be stored in the values column (i.e. 4)?
After reading through JOOQ documentation on code generation and support for SQL arrays, I couldn't find anything specific about fixed length arrays. Also, nothing jumps out at me in the generated code that provides this information.
No, version 3.9 of jOOQ doesn't know any fixed size or size limit of a database array (neither with PostgreSQL array types nor with Oracle VARRAY types).
I have registered feature request #5932 for this.
I asked this question in part because I was worried about array overruns in the PostgreSQL database. After researching a way to use straight SQL to determine the size constraint, I noticed the PostgreSQL ARRAY documentation makes the statement :
As before, however, PostgreSQL does not enforce the size restriction in any case.
Based on that statement, it would appear using the array constraint to enforce size is not necessary since all array columns appear to be treated as variable length. So even if one could retrieve the PostgreSQL array size constraint through JOOQ, Straight SQL, or any other means, why bother?
My database contains a field ARR_ID with data type as NUMBER(20),
the value is = 100013085001
Some java application is fetching this value and they are using Integer as the datatype to fetch the value.
In the output of their application the value is displayed as 1228837193.
How is this value getting converted in Java I do not know?
What happens when the data is too large to be contained in the datatype?
Shouldn't the application throw an error in that case?
It depends on the SQL driver you are using. I saw the circumstances you are describing once when I was using the SQLite driver. I was also a bit concerned and checked the sources. The datatype is determinded by the value in the column. If it doesn't fit in an Integer a Long is used.
I don't remember how this is working for a collection. You may want to check the sources of the driver which you are using (if it's also open source).
If you use JDBC then with a 20-digit integer (e.g. 12345678901234567890), the program throws com.mysql.jdbc.exceptions.jdbc4.MySQLDataException: '1.2345678901234567E19' in column '1' is outside valid range for the datatype INTEGER. It seems you use Oracle which I don't have, this is for mysql, you may try for yourself.
I'm using hibernate and db is sqlserver.
SQL Server differentiates it's data types that support Unicode from the ones that just support ASCII. For example, the character data types that support Unicode are nchar, nvarchar, longnvarchar where as their ASCII counter parts are char, varchar and longvarchar respectively. By default, all Microsoft’s JDBC drivers send the strings in Unicode format to the SQL Server, irrespective of whether the datatype of the corresponding column defined in the SQL Server supports Unicode or not. In the case where the data types of the columns support Unicode, everything is smooth. But, in cases where the data types of the columns do not support Unicode, serious performance issues arise especially during data fetches. SQL Server tries to convert non-unicode datatypes in the table to unicode datatypes before doing the comparison. Moreover, if an index exists on the non-unicode column, it will be ignored. This would ultimately lead to a whole table scan during data fetch, thereby slowing down the search queries drastically.
The solution we used is ,we figured that there is a property called sendStringParametersAsUnicode which helps in getting rid of this unicode conversion. This property defaults to ‘true’ which makes the JDBC driver send every string in Unicode format to the database by default. We switched off this property.
My question is now we cannot send data in unicode conversion. in future if db column of varchar is changed to nvarchar (only one column not all varchar columns), now we should sent the string in unicode format.
Please suggest me how to handle the scenario.
Thanks.
You need to specify property: sendStringParametersAsUnicode=false in connection string url.
jdbc:sqlserver://localhost:1433;databaseName=mydb;sendStringParametersAsUnicode=false
Unicode is the native string representation for communication with SQL Server, if you are converting to MBCS (Multibyte character sets), then you are doing 2 converts for every string. I suggest that if you are concerned with performance, use all Unicode instead of all MBCS
ref: http://social.msdn.microsoft.com/Forums/en/sqldataaccess/thread/249c629f-b8f2-4a8a-91e8-aad0d83919ca
I'm new to using JDBC + MySQL.
I have several 1/0 values which I want to stick into a database with a PreparedStatement. The destination column is a BIT(M!=1). I'm unclear on which of the setXXX methods to use. I can find the references for what data comes out as easily enough, but how it goes in is eluding me.
The values effectively live as an ordered collection of booleans in the objects used by the application. Also, I'll occasionally be importing data from flat text files with 1/0 characters.
To set a BIT(M) column in MySQL
For M==1
setBoolean(int parameterIndex, boolean x)
From the javadoc
Sets the designated parameter to the
given Java boolean value. The driver
converts this to an SQL BIT value when
it sends it to the database.
For M>1
The support for BIT(M) where M!=1 is problematic with JDBC as BIT(M) is only required with "full" SQL-92 and only few DBs support that.
Check here Mapping SQL and Java Types: 8.3.3 BIT
The following works for me with MySQL (at least with MySQL 5.0.45, Java 1.6 and MySQL Connector/J 5.0.8)
...
PreparedStatement insert = con.prepareStatement(
"INSERT INTO bittable (bitcolumn) values (b?)"
);
insert.setString(1,"111000");
...
This uses the special b'110101010' syntax of MySQL to set the value for BIT columns.
You can use get/setObject with a byte array (byte[]). 8 bits are packed into each byte with the least significant bit being in the last array element.