As per my understanding java.net talkes with glibc.
I want to know in Java JDBC source, which method is talking with glibc socket methods for implementing socket in Java.
And where can I get the definition of socketRead0
It isn't in the JDBC source code at all.
It is in the java.net source code, and its native JNI past, and the JVM library that supports it. All over the place. Keep looking.
Why do you think you need to know? You don't.
JDBC is an interface.
The actual code implemented by a JDBC provider. Each provider depending on there background and what was flavor of the month when they first implemented thier JDBC library will probably be using a different method.
Of the two I am most familiar with SQLITE and DB2, the sqlite implementation just calls the sqlite C api. The DB2 implementation makes extensive use of java asynchronous IO libraries as well as the standard JAVA sockets libraries.
Related
My goal here is to make two separate applications (one in Java and other in C++, both on the same machine) read from the same SQLite database. The C++ implementation already works and has all the methods that I need for that communication. It uses the sqlite3.h libraries.
The first rational thing to do would be to use a JDBC or a SQLite wrapper in the Java application. The problem is that my embedded system (POSIX) has very limited resources and takes very long to execute a simple query when I have included the necessary *.jar into it. I have tried out the Xerial JDBC, sqlite4java, sqljet and the Javasqlite Wrapper/JDBC driver from Christian Werner. The JavaVM just takes too long to load everything and execute it and performance is a critical issue.
As a workaround, I have managed the Java application to use system commands and run the sqlite3 command shell to execute the query and obtain the answer. I am looking for a more "stylish" and secure solution.
I actually need the Java application to use the methods from C++. They just return a string as the methods are implemented to return only one value. After a lot of IPC reading, I have reached the conclusion that I have to use named Pipes. The thing is that I would have to use JNI but I have a beginner Java level and by this time, JNI is just too complex for me. Is JNI an overkill in this case?
What other solution could I implement here?
Not sure about the performance you need over the IPC but there are several approaches:
use sockets
use pipes
use memorymappedfiles (using memorymappedfiles you will have a performance gain)
In either case you will need a serializer/deserializer for the objects(data) you pass from java to c++ and viceversa.
Depending on the data format you might need serializer/deserializer only on Java side. (e.g. you send out binary data that C++ will read without needing to decode it anymore). A good tutorial on how to use memorymapped file in java can be found here and in C++ you need to use mmap function.
You could use swig. Swig can parse your C/C++ header and generate Java clases/functions of it. The generated code has jni calls to call your c++ clases or your c functions.
Actually I was wrong. I don't need to use JNI to use named pipes in Java. I have successfully communicated these two processes using basic techniques. In java I have just used FileOutputStream and FileInputStream to communicate with the named pipes.
This link was specially useful to me:
http://carminedimascio.com/2014/01/named-pipes-with-java/
I am interested in creating a module which accepts TCP connections and handles OCI calls made from db tools like Toad, or PL/SQL Developer.
My aim is to manipulate data before sending it to client, hide some stuff etc. Is it possible to do something like that ?
More specifically, we created a data migration tool coded in java. If i can get calls from a db tool and call the tool's methods using JNI, i achive what i am intended to do. As far as i know there is no api to create a OCI server to connect.
First decide if you are binding to oci or jdbc. The solutions are very different depending on the technology.
Second, keep in mind that while you simply "connect" to a socket, the oci or jdbc protocol is going to request data according to it's protocol. In other words, it is not enough to quickly create an OCI server, you need to have code to actually respond to the requests.
The simplest way of doing that is to actually use a database, but one with a different configuration. The only other way to do that is to write something that simulates a database. With enough simulation, you actually find that you are moving closer to implementing a database.
Perhaps it would be much easier to find a small, compact database like HSQLDB to actually provide a database for your testing via TOAD; however, it will be a different database (which may introduce other issues) it won't support oci (only Oracle supports the Oracle Call Interface) and odds are you will eventually have to test against a live Oracle database anyway.
You can use the Oracle Express edition for testing purposes: http://www.oracle.com/technetwork/products/express-edition/downloads/index.html
Since working with databases requires input/output, may take unbounded amount of time, etc. it seems natural to want a non-blocking, asynchronous API. Is there one for Java?
I do not think that such API exists but there are 2 different things: DB access libraries and a lot of ways to perform asynchronous calls in java.
You can use either plain JDBC or any other higher level tool that simplifies DB access implementation to access your database.
You can make asynchronous calls using JMS (if you are in Java EE environment) or using queues and executors from concurrency package if your are in JSE environment. Obviously a lot of other solutions available too.
There is no standard API like JBDC which would allow you to asynchronously call any DB. However there is this Google Project which tries to do exactly this for PostgreSQL and MySQL.
You may also take a look at this question, which addresses similar stuff:
Is asynchronous jdbc call possible?
For Couchbase I came across Reactive Couchbase which claims to do this and has a Java Wrapper. Didn't try it but there are several examples in the links.
Where I can find information how Oracle performs communication with database on the lowest level, I mean at level of sockets? I want to write a program(without jdbc) that simply perform some statement(select or create). So I need to know what protocol Oracle uses to do this.
Why would you want / need to eliminate JDBC for this? This is exactly why JDBC exists, and why Oracle provides JDBC drivers for their databases. Even if there would exist some low-level protocol documentation for what you're trying to do, you'd have no guarantee of it being portable between Oracle releases, etc.
I'm going to make an assumption in that you're looking to eliminate an install of the Oracle client. If this is the case, make sure you're using the type 4 JDBC driver, or Oracle's "thin" driver - available at http://www.oracle.com/technetwork/database/enterprise-edition/jdbc-112010-090769.html. This does exactly what you're probably trying to do - connecting to the Oracle database through pure Java, without requiring any other installed software - though it technically will still be JDBC.
If you're still insistent on proceeding without any Oracle client components (even including just the Oracle type 4 JDBC driver), here are a few additional links that may be of interest:
Wire-Level Network Protocol Specification for Oracle? Including:
Where can I get Oracle’s Wire-Level Protocol Specification?
Oracle.
Outside of Oracle, only a few wire-level driver vendors have it.
Likewise, unlike the driver vendors, Oracle is the only one with
permission to distribute it. And, before you ask, I’ve never seen the
specification myself. All of my protocol knowledge is based on years
of research and significant trial-and-error.
SibylNet: An Open Source Wire-level Client Library for Oracle
This project is (was) available at http://sourceforge.net/projects/sibylnet/ , but hasn't seen any updates since 2008-10-07, and doesn't have any files or source code available.
You could take a look at Oracle's documentation on the Oracle Call Interface. It is for use with C, not Java, but if you wrapped it in some Java Native Interface stuff you might be able to workout how to call it.
Not saying it would be painless but ... it's a direction to try.
/b
"DMA" here means: Direct memory access, and "RDMA" is: remote direct memory access.
I used Java to created an application to transfer stock data, but I found the latency is bigger than I expected. I heard someone developed same type application used "DMA/RDMA", which has good performance, so I wonder if I can use "DMA/RDMA" in Java?
If not, what language should I use, and if there are any good libraries to use?
This article from IBM developers work give a great overview of how DMA access can be achieved using java
You have two options (that I know of):
Java SDP -- has been deprecated
JXIO [1], a high-performance messaging library built on RDMA and supported by Mellanox
[1] https://github.com/accelio/JXIO
There is DiSNI, a new library to program RDMA in Java. DiSNI is the open source variant of IBM's jVerbs. There are some performance numbers here that illustrate how RDMA with Java compares to sockets in C and Java, and also to RDMA in C.
RDMA as I know it is a property of the networking infrastructure (along with the relevant kernel modules etc), not of the application-level programming language.
In other words, you'd have to get specialized kit to make use of RDMA to reduce network latency. Here is an example of a 10GbE network card that supports RDMA: link.
Java applications are capable of mmaping files via nio packages. Those mmaped files can be accesses by multiple programs - I affraid this is closes thing to DMA available in java
Java 7 supports SDP protocol on Solaris and Linux (with OpenFabrics.org drivers). Unfortunately SDP protocol has been deprecated in the 2.x version of OFED. People resort to JNI wrappers like jVerbs.