I use embedded Apache derby for my application. I have a SQL script called createdb.sql that creates all tables in a database and populates it with initial data, e.g.:
SET SCHEMA APP;
CREATE TABLE study (
study_id bigint not null GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
name varchar(50) not null,
note varchar(1000) DEFAULT '',
created TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
deleted boolean DEFAULT false,
UNIQUE(name),
CONSTRAINT primary_key PRIMARY KEY (study_id)
);
INSERT INTO "APP"."STUDY" (NAME) VALUES ('default');
CREATE TABLE img (
img_id bigint not null GENERATED ALWAYS AS IDENTITY (START WITH 1, INCREMENT BY 1),
filename varchar(200) not null,
path varchar(300) not null,
flipped boolean DEFAULT false,
type smallint not null,
note varchar(1000) DEFAULT '',
created TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
PRIMARY KEY (img_id)
);
ALTER TABLE img ADD COLUMN dpix integer DEFAULT -1;
ALTER TABLE img ADD COLUMN dpiy integer DEFAULT -1;
The question is how do I load this file and execute all the statements using java? I'm trying different function but they all don't work. For example,
Statement s = conn.createStatement();
s.execute(sqlStr);
or
Statement s = conn.createStatement();
s.executeUpdate(sqlStr);
where sqlStr is a String variable containing the contents of the createdb.sql file. How do I execute all the SQL commands contained in the script so that I can create all the tables and initialize them? Btw, the SQL script works, as I use it in SQuirreL SQL Client to manualy create and initialize the database. Now I would like to do it from within my application.
The below tutorial give how to run a mysql script(.sql file) . What you have to do is that Change the mysql db connection to derby db and run. It will work.
http://www.mkyong.com/jdbc/how-to-run-a-mysql-script-using-java/
Here is an alternative way to run a MySQL script without using any third party library.
http://coreyhulen.wordpress.com/2010/04/07/run-a-sql-script-for-mysql-using-java/
With Derby, you generally use the 'ij' tool to do this:
http://db.apache.org/derby/docs/10.9/tools/ttoolsij98878.html
If you want to do this from a Java program of your own, rather than from the command line, you'll want to study the 'runscript' feature of ij; see this related question:
How to run sql scripts in order to update a Derby schema from java code?
Related
I have a Spring Boot service with a MySQL database (AWS RDS).
There is a specific table, that contains around 2 millions of rows, and some queries on it make the CPU go up to the database instance.
I noticed that there isn't an index on the used column so I would like to try to add this index.
The questions are:
Can I add (without any problems) this index on this table that
already contains a lot of rows? I'm using Flyway to manage the db
migrations.
The specific column contains strings, are there some
other index configurations that It's better to use for this scenario?
Some additional infos:
MySQL version is 5.7.33;
The table, at the moment, does not contain any other relationships;
The table is very simple and it's reported below:
CREATE TABLE IF NOT EXISTS info(
field_1 varchar(36) NOT NULL,
field_2 text DEFAULT NULL,
my_key varchar(36) DEFAULT NULL,
field_3 varchar( 255) DEFAULT NULL,
field_4 varchar(10) DEFAULT NULL,
field_4 varchar(10) DEFAULT NULL,
field_6 varchar(10) DEFAULT NULL,
field_7 varchar(36)NOT NULL,
creation_date datetime DEFAULT NULL,
modification_date datetime DEFAULT NULL,
PRIMARY KEY (field_1)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
The table contains now around 2 millions of rows;
The query is something like:
SELECT * FROM info WHERE my_key = "xxxx"
and it will be executed a lot of times
The idea is to create this index:
CREATE INDEX my_key ON info (my_key);
With the more recent versions of MySql you can create an index without locking the table:
The table remains available for read and write operations while the index is being created. The CREATE INDEX statement only finishes after all transactions that are accessing the table are completed, so that the initial state of the index reflects the most recent contents of the table.
Obviously creating an index is an extra effort for the database so if your database is in suffering state try to update the index when you have a decrease of the activities performed on the db.
I have a csv file which I need to write to a Sql Server table using SQLServerBulkCopy. I am using SQLServerBulkCSVFileRecord to load data from the file.
The target table has the following structure:
create table TEST
(
ID int identity,
FIELD_1 int,
FIELD_2 varchar(20)
)
The csv file has the following structure:
4279895;AA00000002D
4279895;AA00000002D
4279895;AA00000002D
4279896;AA00000003E
4279896;AA00000003E
4279896;AA00000003E
As you can see the ID (identity) column is not present in the csv, I need the database to automatically add the identity value on insert.
My problem is that the bulk insert does not work as long as the table has the identity column, I got the following error:
com.microsoft.sqlserver.jdbc.SQLServerException: Source and destination schemas do not match.
at com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.validateColumnMappings(SQLServerBulkCopy.java:1749)
at com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(SQLServerBulkCopy.java:1579)
at com.microsoft.sqlserver.jdbc.SQLServerBulkCopy.writeToServer(SQLServerBulkCopy.java:606)
This is the relevant code:
try (
Connection targetConnection = DriverManager.getConnection(Configuration.TARGET_CONNECTION_URL);
SQLServerBulkCopy bulkCopy = new SQLServerBulkCopy(targetConnection);
SQLServerBulkCSVFileRecord fileRecord = new SQLServerBulkCSVFileRecord(csvPath, Charsets.UTF_8.toString(), ";", false);
) {
SQLServerBulkCopyOptions copyOptions = new SQLServerBulkCopyOptions();
copyOptions.setKeepIdentity(false);
bulkCopy.setBulkCopyOptions(copyOptions);
fileRecord.addColumnMetadata(1, null, java.sql.Types.INTEGER, 0, 0); // FIELD_1 int
fileRecord.addColumnMetadata(2, null, java.sql.Types.VARCHAR, 20, 0); // FIELD_2 varchar(20)
bulkCopy.setDestinationTableName("TEST");
bulkCopy.writeToServer(fileRecord);
}
catch (Exception e) {
// [...]
}
The bulk insert ends succesfully if I remove the identity column from the table. Which is the correct to perform an identity bulk-insert using java jdbc for Sql Server?
I think you don't need to set this option copyOptions.setKeepIdentity(false);
Try after removing this line. You can refer to this post as well SqlBulkCopy Insert with Identity Column
If you have a leading column with blank values Helen the identity will be generated on insert. Depending on the settings it might generate new identity even if the first column is not blank.
So either add an extra column or use another (staging) table.
BTW, if you have a really big table the command-line bcp utility is the fastest. From experience up to 5 times faster compared to Jdbc batch insert.
i am using SQuirrel SQL Client Version 3.7 to view my derby database
the username for this connection is umar and password is umar ..
when i run an insert query inside this connection i get the following error
i am using netbeans and it doesn't work there as well my code gives the same error .. what do i do to fix this
here is the statement i use for creating ADMINISTRATORS TABLE
CREATE TABLE ADMINSTRATORS(ADMIN_ID BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY,username VARCHAR(30) NOT NULL,password VARCHAR(16) NOT NULL,CONSTRAINT ADMIN_PK PRIMARY KEY (ADMIN_ID) )
it was a spelling mistake in my sql query
CREATE TABLE ADMINISTRATORS(ADMIN_ID BIGINT NOT NULL GENERATED ALWAYS AS IDENTITY,username VARCHAR(30) NOT NULL,password VARCHAR(16) NOT NULL,CONSTRAINT ADMIN_PK PRIMARY KEY (ADMIN_ID) )
please someone close this question
Maybe you're not pointing to the database you think you are, try issuing this before your insert:
use UMAR
Is your JDBC url having your database name ?
Ex.
private static String jdbcURL= "jdbc:derby://localhost:1527/UMAR";
I'm trying to embed h2 to test my mysql-application (integration-test)
I added com.h2database:h2:1.3.170 via maven and run the following code:
public class InMemoryTest
{
#Test
public void test() throws Exception {
Class.forName("org.h2.Driver");
Connection conn = DriverManager.
getConnection("jdbc:h2:mem:test;MODE=MySQL;IGNORECASE=TRUE;INIT=RUNSCRIPT FROM 'src/test/resources/test.sql'");
}
}
which gives me the following Exception:
Syntax error in SQL statement "
CREATE TABLE IF NOT EXISTS ""usr_avatar"" (
""usr_avatar_id"" INT(11) NOT NULL AUTO_INCREMENT,
""usr_avatar_user_id"" INT(11) NOT NULL,
""usr_avatar_img"" BLOB NOT NULL,
PRIMARY KEY (""usr_avatar_id""),
UNIQUE KEY ""usr_avatar_id_UNIQUE"" (""usr_avatar_id""),
UNIQUE KEY ""usr_avatar_user_id_UNIQUE"" (""usr_avatar_user_id""),
KEY ""usr_user_id"" (""usr_avatar_user_id""),
KEY ""fk_user_id"" (""usr_avatar_user_id"")
) AUTO_INCREMENT[*]=1 ";
Apparently, the "AUTO_INCREMENT" causes this?
Since this is valid MySQL (I exported the dump from my real database using MySQL Workbench), I'm a bit confused since h2 claims to support MySQL?
Here are a few lines from the .sql:
DROP TABLE IF EXISTS `usr_avatar`;
CREATE TABLE IF NOT EXISTS "usr_avatar" (
"usr_avatar_id" int(11) NOT NULL AUTO_INCREMENT,
"usr_avatar_user_id" int(11) NOT NULL,
"usr_avatar_img" blob NOT NULL,
PRIMARY KEY ("usr_avatar_id"),
UNIQUE KEY "usr_avatar_id_UNIQUE" ("usr_avatar_id"),
UNIQUE KEY "usr_avatar_user_id_UNIQUE" ("usr_avatar_user_id"),
KEY "usr_user_id" ("usr_avatar_user_id"),
KEY "fk_user_id" ("usr_avatar_user_id")
) AUTO_INCREMENT=1 ;
DROP TABLE IF EXISTS `usr_restriction`;
CREATE TABLE IF NOT EXISTS "usr_restriction" (
"usr_restriction_id" int(11) NOT NULL AUTO_INCREMENT,
"usr_restriction_user_id" int(11) DEFAULT NULL,
"usr_restriction_ip" varchar(39) DEFAULT NULL,
"usr_restriction_valid_from" date NOT NULL,
"usr_restriction_valid_to" date DEFAULT NULL,
PRIMARY KEY ("usr_restriction_id"),
UNIQUE KEY "usr_restriction_id_UNIQUE" ("usr_restriction_id"),
KEY "user_id" ("usr_restriction_user_id"),
KEY "usr_user_id" ("usr_restriction_user_id")
) AUTO_INCREMENT=1 ;
What are my options? Should I export the dump with a different software and force it to be plain SQL? Which software could do that? Or am I doing something wrong?
The problem is that H2 doesn't support AUTO_INCREMENT=1, which you have specified in the SQL statement. Try removing it. I don't think it's necessary for MySQL either.
The source SQL exported from MySQL has double-quotes surrounding it's literals. The first DROP statement also has a "back-tick" (`). But when H2 is reporting the error, H2 is showing the literals surrounded by double-double quotes. I think this is the problem.
Try a couple of things. First, take the back-tick in the DROP statement and convert it to single quotes. If that doesn't work, convert all of the double-quotes to single-quotes. If that doesn't work, remove all of the quotes.
I think H2 is trying to create tables with the double-quotes as a part of the actual table names/column names and this is causing it to bomb.
H2 doesn't support AUTO_INCREMENT=1.
Use this instead:
ALTER TABLE table_name ALTER COLUMN id RESTART WITH 1;
I have the following mysql query in PHP and it works fine.
$strUpdate = "INSERT INTO `batchfolder`.`newbatch` (`BatchID` ,`Batch` ,`barcode` ,`PG`)VALUES (NULL , '', '1', '')";
and is also fine when i run it directly in database.
However when I run it in java,
try {
conn = DriverManager.getConnection (url, userName, password);
Statement st = conn.createStatement();
st.execute("INSERT INTO `batchfolder`.`newbatch` (`BatchID` ,`Batch` ,`barcode` ,`PG`)VALUES (NULL , '', '1', '')");
st.close();
}
It gives the following error.
Exception in thread "main" java.sql.SQLException: Field 'Pre' doesn't have a default value
Pre is the the next row in the database and it does not have a default value.
My question is, then how does this query run fine in mysql and php.
P.S BatchID is an int(10) autoincremented value in newbatch.
This is my table structure.
CREATE TABLE IF NOT EXISTS `newbatch` (
`BatchID` int(10) NOT NULL AUTO_INCREMENT,
`Batch` varchar(100) NOT NULL,
`barcode` varchar(5) NOT NULL,
`Ly` varchar(5) NOT NULL DEFAULT '0',
`PG` varchar(5) NOT NULL,
`Pre` varchar(5) NOT NULL,
`Flu` varchar(5) NOT NULL,
`FluID` varchar(100) DEFAULT NULL,
`DateCreated` varchar(100) DEFAULT NULL,
`Comments` varchar(500) DEFAULT NULL,
PRIMARY KEY (`BatchID`),
UNIQUE KEY `FluID` (`FluID`)
) ENGINE=MyISAM DEFAULT CHARSET=latin1 AUTO_INCREMENT=1000160 ;
Are you absolutely sure this actually runs in PHP? You may just not be seeing the error message.
Try this again with display_errors = 'on' and error_reporting(E_ALL | E_STRICT);
This is a SQL error, it doesn't depend on the driver. The driver just displays the message to you.
UPDATE:
It looks like Java is turning off MySQL's strict mode for some reason. The only thing that (to my knowledge) should be able to cause this behavior is MySQL strict mode being off.
If strict mode is not enabled, MySQL sets the column to the implicit
default value for the column data type.
You can check the mode that your server is running on by SELECT ##GLOBAL.sql_mode;. Try that in both Java and PHP. If the results differ then that's your answer.
UPDATE2: Jep!
Looked at your table definition, you have Pre defined as not null, but you didnt specify a value for it in the insert, so it should show an error proper.
You have several database fields that are 'NOT NULL', so you must specify these in your INSERT statement. I couldn't imagine how this would work in PHP either.