I am not allowed to compile the java class into the instance of Oracle we're running on, per architects request, so I am looking for alternatives. The requirement is to utilize a java library located on an application server on the network. Is it possible to call a java method located on another machine from PL/SQL? I found this article talking about external procedures in Oracle, but I'm not sure that it allows for this. As a side not, the performance would also have to be fast enough to be used in batch processing of thousands or millions of calls.
I suspect the best you can do is add entries to another table which your Java process polls to get each or batches of messages. Oracle is not really designed for message processing.
In any case, I would discuss this with your Architect what to do as he is the expert. ;)
If your Oracle system can't do the job, you may need to have a solution which doesn't use Oracle.
You have three options:
1.) We solved a similar problem by making PL/SQL call HTTP using UTL_HTTP and then let the app-server call the java procedure. We did this to interface our Oracle Database with Oracle Reports. The PL/SQL fired an HTTP Request which was received by the app-server which called Java. The Java can call back PL/SQL via normal JDBC.
2.) You might not be able to load that java proc, but maybe you can create some other java stored procedure that can invoke it using RMI.
3.) AQ is another method. Basically you can Queue a message using AQ and use JMS on the App Server to Dequeue it and use it.
Option 3 would be the fastest, though we have tried option 1 and the latency for this is not as much as you might it. It also offers a way to do some parallel processing by running multiple requests in parallel.
Related
I am trying to execute my java application once a record is inserted, deleted or update into a database table. For that sake, I am planning to write a Trigger which will be called while inserting. My question is that, will I be able to call a java application from this trigger?
While Python and Perl, among others, have a PL equivalent that can be written within an SQL function (albeit with sometimes considerable overhead), Java does not. Even if the Java app could be launched via command line or something after a trigger, the overhead in launching the JVM alone would be a performance disaster.
I would recommend looking into having Postgres emit notifications via notify. This would let you send a payload (although there are size limitations) that can be listened for by a client (that client could be your Java app, or an intermediate program that performed some additional ETL to get it ready for your app). This should allow for the data to be served up and processed in a performant way.
Here is my situation:
I wrote two REST clients in Java which are running on my server. And these clients are packaged into Runnable Jar files. I set a schedule for running it. Every time data has been synced around 3MB.
Recently, I needed to write two more clients to sync from other resources. Before that, I didn't have software architecture experience to build an efficient client System.
My problem is my server is not good for running Microsoft windows server 2003 R2. The hardware info is following:
CPU: Intel Xeon E5649 2.53GHZ
RAM: 2GB
I use MySQL database, basically the sql writes per second around 20. It is very slow. From now, doing one synchronization takes 2 hours. I could not imagine 4 REST clients running on Monday.
I need help with how to deal with four clients running on a low capacity server. However, please don't convince me to change a new powerful server :)
I have been thinking for a long time, could I only build a client which sync data from different resource? Or build four clients which running on a different schedule? The other problem is in the future more resources will be added. I don't know how to build a system with strong scalability because of my lack of knowledge.
If you could give me some advice to push my learning a little bit, it will be very appreciated. Thank you.
Further information:
The goal is to grasp data from different RESTful servers with different API and GET query rules and insert these data into Database. Thus, basically the application's job is to put data into MySQL via RESTful call.
In addition, I only focus on it. I do not need to consider about how to deal with inserted data. The structure is simply:
Get Restful call and get JSON format result
parsing JSON
insert into DB
I used (JAX-RS) Jersey api to implement RESTful call, and use JDBC to manage the database, but I am working on the next version which will use Hibernate to implement insert, delete, update, and search functions.
This application implementation does not use any application server. I package the program to runnable jar file, and set schedule to run it on windows server 2003.
Being a recently graduated student, that is my first REST client, but with my deep research and studying, I know more about it. However, I don't have experience on it. I just want to make it work better. Any suggestion, I appreciate it.
Use a profiler to gain insight into the actual memory and CPU usage of your application. You can then decide how to improve it using the appropriate means. (F.e. multithreading, caching, compression, ...)
I am interested in creating a module which accepts TCP connections and handles OCI calls made from db tools like Toad, or PL/SQL Developer.
My aim is to manipulate data before sending it to client, hide some stuff etc. Is it possible to do something like that ?
More specifically, we created a data migration tool coded in java. If i can get calls from a db tool and call the tool's methods using JNI, i achive what i am intended to do. As far as i know there is no api to create a OCI server to connect.
First decide if you are binding to oci or jdbc. The solutions are very different depending on the technology.
Second, keep in mind that while you simply "connect" to a socket, the oci or jdbc protocol is going to request data according to it's protocol. In other words, it is not enough to quickly create an OCI server, you need to have code to actually respond to the requests.
The simplest way of doing that is to actually use a database, but one with a different configuration. The only other way to do that is to write something that simulates a database. With enough simulation, you actually find that you are moving closer to implementing a database.
Perhaps it would be much easier to find a small, compact database like HSQLDB to actually provide a database for your testing via TOAD; however, it will be a different database (which may introduce other issues) it won't support oci (only Oracle supports the Oracle Call Interface) and odds are you will eventually have to test against a live Oracle database anyway.
You can use the Oracle Express edition for testing purposes: http://www.oracle.com/technetwork/products/express-edition/downloads/index.html
Since working with databases requires input/output, may take unbounded amount of time, etc. it seems natural to want a non-blocking, asynchronous API. Is there one for Java?
I do not think that such API exists but there are 2 different things: DB access libraries and a lot of ways to perform asynchronous calls in java.
You can use either plain JDBC or any other higher level tool that simplifies DB access implementation to access your database.
You can make asynchronous calls using JMS (if you are in Java EE environment) or using queues and executors from concurrency package if your are in JSE environment. Obviously a lot of other solutions available too.
There is no standard API like JBDC which would allow you to asynchronously call any DB. However there is this Google Project which tries to do exactly this for PostgreSQL and MySQL.
You may also take a look at this question, which addresses similar stuff:
Is asynchronous jdbc call possible?
For Couchbase I came across Reactive Couchbase which claims to do this and has a Java Wrapper. Didn't try it but there are several examples in the links.
We are trying to connect to a SQL Server instance used by the ACT CRM system. They have managed to lock things down so it isn't possible to connect to the SQL back-end using ODBC (there is some special utility that will add ODBC support that you can install if you purchase the primo version of the software, but that's just crazy).
The recommended method of connecting to theses databases is using an OLEDB connection.
Does anyone have any tricks/ideas/etc... for how to make and use an OLEDB connection from Java?
This doesn't have to be JDBC if that's not possible. All we really need to do is execute a SELECT query that returns two fields and parse those field values out for each row. I have very little experience with OLEDB, so 'use JACOB' might be a good answer, but I'd appreciate some details on what the COM calls would actually have to be.
I know this is old, but could help someone to know how I did it
I described in more detail how to do it Here.
That's not your problem. The problem is the way they have locked down the server. Basically on startup it looks for logins other than ACTUSER and removes them.
You can unlock it pretty easily though, then you will be able to connect in the usual way.
https://serverfault.com/questions/77712/sqlserver-need-to-access-an-act-database-for-data-migration
I've managed to unlock mine but I forget how... I think I started it in single-user mode then did some funny stuff involving decrypting a stored proc in the master database and editing it to remove that "functionality". That in turn involved using SQLTrace to see what commands ACT was sending.
I suggest you ask on Server Fault.
Java can not access OLEDB directly. You need to do this in another language like C++ or C#. Then you can access via JNI or external process. If you does not want write the native part of JNI self then you can use JACOB how you suggest it. But I think an external process take the request seem be simpler.
Or use: http://uda.openlinksw.com/jdbc/mt/jdbc-sqlserver-mt/
They develop all kinds of drivers. I used this company before...
Two ways to solve this issue.
Spawn from java an external proc (c#, c++, etc) that connects to SQLSrv using OLEDB and redirect the stdin, stdout and stderr to your java program.
Create a C# listener on a particular port and have java pass all the requests via a client to that listener.