Is there a way to easily create Java Input/Output streams for unnamed pipes created in native code?
Motivation: I need my own implementation of the Process class. The native code spawns me a new process with subprocess' IO streams redirected to unnamed pipes.
Problem: The file descriptors for correct ends of those pipes make their way to Java. At this point I get stuck as I cannot create a new FileDescriptor which I could pass to FileInput/FileOutput stream.
I have used reflection to get around the problem and got communication with a simple bouncer sub-process running. However I have a notion that it is not the cleanest way to go. Have you used this approach? Do you see any problems with this approach? (the platform will never change)
Searching around the internets revealed similar solution using native code.
Any thoughts before I dive into heavy testing of this approach are very welcome. I would like to give a shot to existing code before writing my own IO stream implementations...
Thank you.
I have run into this before as well. The only way I know to create FileDescriptor objects is using reflection (or from JNI) to set the private "int" field inside the FileDescriptor class.
But this works fine and is surely how it's done elsewhere in the standard library, so I believe it's as legitimate as you could expect.
Related
I am developing a Minecraft plugin which uses a class that I made called customPlayer. When I save the plugin data from a running instance, I put all of these objects into a HashMap<String,customPlayer> and save them with ObjectOutputStream. Loading these classes back into the same version of the plugin works great, but my problem arises when I modify the class and try to read the object using that modified class (usually associated with a new version of my plugin).
I thought about it for a bit, and thought I came up with a clever solution. My idea was to just include the old class files as an External Library inside the new version of the plugin, cross my fingers and hope it worked. It didn't.
Is there a better way to do this? I'm new to serialization and this kind of stuff, so any suggestions would be greatly appreciated. Below I will include a few Screenshots of the customPlayer class and the crash log of the server. Ideally any solution that is presented should be able to be used easily with future modifications to the class (Updates to the Jar downloaded Via a Github repo).
Instance Variables and Constructor of customPlayer.java
Is there a better way to do this?
There certainly is. Stop using Serialization and ObjectOutputStream. These classes are a disaster (even OpenJDK core team effectively agrees with this assessment). The output they generate is not particularly efficient (it's more bytes than is needed), it is not human readable, nor (easily) read by anything except java code, and it results in such hairy situations as you ran into.
Instead use e.g. Jackson to turn your objects into JSON, or use google's protobuf to turn it into efficient binary blobs.
You can read this JSON or these binary blobs in any language you want and you'll have your pick of the litter as far as libraries go. You will need to write some explicit code to 'save' an object (turn it into JSON / protobuf), and to 'read' one, but now you are free to change your code.
If you insist on continuing with serialization, you need to add a field named serialVersionUID, and set up readObject and writeObject. it's convoluted rocket science that's hard to get right. The details are in the javadoc of java.io.Serializable.
Do yourself a favour though. Don't do it.
So I have a neural network in tensorflow (python2.7) and I need to retrieve its output using Java. I have a simple python function getValue(input) which starts the session and retrieves the value. I am open to any suggestions. I believe Jython wont work cause tensorflow is not in the library. I need the call to be as fast as possible. JNI exists for Java calling C so can I convert with cython and compile then use JNI? Is there a way to pass the information in RAM or some other way I haven't thought of?
In Python, save the model (using saver.save) and the graph (using tf.train.write_graph).
In Java, use the org.bytedeco.javacpp-presets library to instantiate a GraphDef from the saved protobuf file and pass in your input features and get the output features within a Session.
See https://medium.com/google-cloud/how-to-invoke-a-trained-tensorflow-model-from-java-programs-27ed5f4f502d#.4su1s26fz for example code.
I've had the same problem, Java+Python+TensorFlow. I've ended up setting up a simple http server. If that's too slow for you, you can shave off some overhead by employing sockets directly.
Encapsulate your calling for TensorFlow into a script.py and then:
Process proc = Runtime.getRuntime().exec("python script.py");
Not sure whether it solves your case.
I am designing an API where one of the methods needs to provide a way to get file content.
I do not want the method to return a file location as potentially files will be stored in the database. I also do not want to return file content as Array[Byte] as the files can be quite big.
In Java I would create a method that returns an InputStream.
What is the recommended way of doing it in Scala?
Scala doesn't really have much in the way of I/O. You could return a Scala.io.Source, which is really a wrapper around InputStream (as pretty much anything will have to be, since that's all any JVM program has access to).
There's an interesting Scala I/O library, not a standard library, based on Java 7 stuff, iirc. And there's Scalaz Stream, which is the best thing out there, though it still uses an InputStream under the covers if you are handling files.
The recommended way of doing it in Scala is by using InputStream instead of reinventing the wheel.
i need to detect if a class relies on another class programatically,to detect inappropriate intimacy code smell(i want to analyze other java programs ,using my program).Any directions on
how to achieve this will be a great help.
And
How to identify all the objects created in a java program?
How to identify all the called methods in a java program?
Any help would be appreciated.
You might want to use what's already there instead of building something yourself. Especially if you're not very familiar with the internals of Java and the JVM.
Have a look at JDepend: http://clarkware.com/software/JDepend.html
Use a profiler as JConsole or VisualVM. With the use of profilers you can pretty much see everything that happens at runtime.
One way i think of is using logger, Put some log statement in the construct and in the methods you want to monitor. So through logs you can find out the objects created and methods accessed
I have found very useful the ObjectWeb asm-all Java bytecode manipulation and analysis library, also known as asm-all.jar
It allows you to convert any *.jar application into equivalent XML file. You can fully inspect the application structure, change it in the XML format and convert back into *.jar file
In order to use the XML files you'll need to understand what it contains. Oracle's The Java® Virtual Machine Specification is very good reference to start with
BTW: one thing you can do with this tool is to instrument the bytecode so that it creates runtime profiling information - which methods were called and by whom (as suggested by #upog)
I have a javaagent attached to a remote process using agentmain which sends serialized java objects via socket to a client application.
There appears to be a bug in the formatting of the stream which I am not sure if it is an error in my writeExternal implementation or if it is something to do with the java versions. I am experimenting with different combinations of java 1.6 32 / 64 bits.
Basically the object stream is throwing a class not found exception, but the package name printed is something like org.mycode.b.e.b., the b.e.b part is nothing to do with any packages I own and I have searched all over and there is nothing in my code or libraries with that string... b.e.b.
So I am suspicious it is an encoding issue caused by either a encoding format discrepancy or a 32/64 bit word length issue, but I am not that experienced in these areas.
I am quite used to opening up the debugger to track down code issues, but less experienced diagnosing problems of this nature, with serialization and byte formats so I am asking to see what are the recommended approaches to debugging?
I have control/compilation/source of the client which uses code like:
objectInput = new ObjectInput(inputStream);
Object object = objectInput.readObject();
which currently throws class not found for the org.mycode.b.e.b.
I was thinking about re-writing it to store the serialized stream to a file and taking a look ( not sure what format the file will actually be in in regard to reading it), is that the right angle?
The format for ObjectOutputStream is unchanged whether you use 32-bit or 64-bit or a big endian or little endian machine. Where there have been problems in the past has been between Java 1.3 and Java 1.4 using slightly different formats. Also Eclipse and Sun used different algos to generate serialVersionUID so you can run into problems if you didn't hard code it.
However if you have packages like b.e.b you most likely have obfuscated your code which is designed to make it difficult to work with. I suggest you only debug/monitor your un-obfuscated code.
I addition to commend of #Peter Lawrey.
I think that to simplify your debugging you should work with non-obfuscated code. This will give you at least the tip where the problem is.
Moreover probably the fact of partial obfuscation is the root cause of your problem. Think: you object is serialized on client side that is not obfuscated. Let's say your class named MyFirstClass depends on MySecondClass and is serialized accordingly. But on server side the MySecondClass is named q. Why q? This is up to obfuscator. Moreover this name may be different for each build. So, the server side cannot find the class MySecondClass.
Other possible problem is serialiVersionUID. Probably you have to define this variable for all relevant classes and handle its value.
But first just avoid obfuscation and try again. I believe everything will work. Good luck.