We Keep Coding

Inserting millions of records from local to godaddy hosting

I'm trying to insert about 8 million records from an access database to a mysql database in GoDaddy.
I built a desktop java app to manage the inserts chunking 5000 records each time. I use Laravel 5.1 to manage the inserts in the server. So, basically, the java app send a request to a laravel php route, then it take care of inserting directly in my MySQL database.
Edit: people will insert data directly into the Access, so I have to watch the MDB file for changes. This is why I can't just export from Access and import into the MySQL.
The first part of records inserts successfully, but than when I send another request, I get this error:
2015-10-28 10:43:57.844 java[3475:280209] Communications error: <OS_xpc_error: <error: 0x7fff7298bb90> { count = 1, contents =
"XPCErrorDescription" => <string: 0x7fff7298bf40> { length = 22, contents = "Connection interrupted" }
}>
Got xpc error message: Connection interrupted
org.apache.http.NoHttpResponseException: api.mydomain.com:80 failed to respond

Make sure you have MySQL odbc connector is installed:
https://dev.mysql.com/downloads/connector/odbc/
create your DNS to your MySQL server (you can do this via odbc manager or just with a notepad)
your DNS file will look like this:
[ODBC]
DRIVER=MySQL ODBC 5.3 Unicode Driver ' check what driver is installed
UID=username to the server
PORT=3306
PWD= password to the server
DATABASE=dbname
SERVER=serverip/name
Save the DNS file somewhere lets call it GoDaddy_MySQL.dns
Open up you ACCESS database.
External Data
ODBC
Select "link to the data source by creating"
select the godaddy_mysql.dns file
if all your connection details are correct Access will show you tables & views to import. Click the tables you would like to upload data from your Access dtabase.
Now you have linked the actual MySQL table in your MS Access database.
All you need to do is uploading data from your local table to your linked table simply by:
you can chunk your upload by using the TOP keyword. if you add a where condition with (is not already on the linked table) you can always upload new records automatically to your MySQL server.
if you are and will still use your Access database you can also switch from local to linked tables so all new entry will automatically uploaded to your godaddy server.
Pseudo:
insert into linked_table select top 5000 from your local table where local_records_are not in linked table.

Try a staggered MySQL importer that is run on the GoDaddy server. Like Bigdump: http://www.ozerov.de/bigdump/
Export from Access, import to MySQL. It's run directly on the godaddy server, so the data is already on the server and then just read. I've used this numerous times on GoDaddy for large imports.
Cheers!

How to set cache size in hsql database

I am trying to setup the cache size for an existing HSQL database as specified in their website, but i am unable to do so.
Here is what i am trying.
1)Through SQL, i am trying to run the query.
SET FILES CACHE SIZE 100.
The database is giving me an error that, unexpected files token e.t.c.
2)Through java code while opening up the database connection.
jdbc:hsqldb:file:enrolments;hsqldb.cache_size=100;
But according to their website , we cannot change properties of existing database except for only some properties.
Please tell me how can i do it?
Thanks.

Importing and Connecting to Derby Database at Netbeans IDE error

I am total novice in this topic. In general, my friend created derby database in netbeans. All databases were stored in directory:
home/.netbeans-derby
He ziped the folder with database he created, and sent to me. I entered unziped database folder to my directory (screen with it:https://dl.dropboxusercontent.com/u/108321090/DATABASE_FOLDERS.png) :
C:\Users\Fisher\.netbeans-derby
Netbeans finds this database and shows it in the tree under Java DB. When use "Connect" on this database I get error (screen with it: https://dl.dropboxusercontent.com/u/108321090/DB_ERROR.png):
Unable to connect. Cannot estabilish a connection to jdbc:derby//localhost:1527/mydatabase using org.apache.derby.jdbc.ClientDriver (DERBY SQL error: SQLCODE -1, SQLSTATE: XJ040, SQLERRMC: Starting database 'mydatabase' failed. Details are containing exception.::SQLSTATE: XSAI2. Demanded conglomeration (16) doesnt exist.).
And also I can provide you with log txt from my folder which is rly long so: https://dl.dropboxusercontent.com/u/108321090/derby.log
Please tell me is there any easy way to connect to that database or is there any EASY way to export base if it was done in wrong way? Please help me.

Table view does not exist exception on Netbeans Derby connection

I followed the instructions in this blog to create a basic embedded database application.However, although checking the steps over and over and searching for the problem on the web , I'm still getting the exception : Table view blabla does not exists. Table seems to exists when I expand my driver's app schema. Netbeans version is 7.3.

The most common reason for a table-does-not-exist error with Derby is confusion over the location of the database. The database that you are accessing via Netbeans is probably different than the database that your application is accessing.
The location of your database is controlled by the JDBC connection URL, so if you provide some details about your JDBC connection URL, that might help others to help you.

In process hsqldb error only when debugging the code

I'm running in process hsqldb in my java code for testing purposus, and i'm getting the following error:
user lacks privilege or object not found: MY_TEMP_TABLE
only when i debug the code. If i run the (junit) test without debug, everything is working.
I do need the debugger for some other areas.
I'm getting the error on this query:
INSERT INTO MY_TEMP_TABLE VALUES ('a','a','a')
I don't think the problem is in sql syntax because all queries are working when running the test without debug.
Any help would be appriciated..

Obviously your table is not present when you debug your code.
One possibility is the table is not created at all. To see what is happening, use the jdbc:hsqldb:file:yourdb as the database URL and check the yourdb.log file for the CREATE TABLE statements.
The other possibility is the table is not in the PUBLIC (or default) schema and you are not changing the schema to the one that contains the table.