Passing parameter from Java to Python and vise versa [closed] - java

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I have a python code that waits for input parameter. This input is the output of another java code. Is there any way I can pass the parameters between the two codes like a bridge between the two?
Thanks in advance

The best way to do these kind of tasks is to use a message broker like RabbitMQ. It provides support for Java, Python, PHP, etc. You can send messages (a json message for example, or in any other format) between processes implemented with different languages. Here you can find tutorials implemented with different languages.
RabitMQ
RabbitMQ is a message broker. It sits between producers and consumers. Producers are components which produces messages and publish those messages to a queue in the RabbitMQ. RabbitMQ takes those messages and by the rules you defined routes and delivers those messages to the consumers. Consumers are task runners which wait to receive messages and run tasks. Here is a simple producer which connects to RabbitMQ, sends a single message, and exit.
import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
public class Send {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws java.io.IOException {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = "Hello World!";
channel.basicPublish("", QUEUE_NAME, null, message.getBytes());
System.out.println(" [x] Sent '" + message + "'");
channel.close();
connection.close();
}
}
And here we have a simple java consumer which we keep it running to listen for messages. You are not limited to java. You can use any language which there are RabbitMQ libraries for (Python, PHP, C#, JavaScript, etc):
import com.rabbitmq.client.*;
import java.io.IOException;
public class Recv {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(" [*] Waiting for messages. To exit press CTRL+C");
Consumer consumer = new DefaultConsumer(channel) {
#Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
throws IOException {
String message = new String(body, "UTF-8");
System.out.println(" [x] Received '" + message + "'");
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
}
}
To run it:
$ javac -cp rabbitmq-client.jar Send.java Recv.java
$ java -cp .:commons-io-1.2.jar:commons-cli-1.1.jar:rabbitmq-client.jar Send
$ java -cp .:commons-io-1.2.jar:commons-cli-1.1.jar:rabbitmq-client.jar Recv
Note: you'll need rabbitmq-client.jar and its dependencies on the classpath.

This kind of process interconnection have two main problems: serialization and transport.
When you passing data from Java process to Python application most likely you want to see a object of specific type rather than raw-bytes or text. There is number of frameworks that was designed to handle cross-language behavior. Take a look at Apache Avro, Protobuf or Apache Thrift. Each one have it's own pros and cons.
Second problem is transport. Like #Mustafa Shujaie already said message-oriented transport like RabitMQ is a good choice. But also have a look at REST services for Java and Python.
By the way, if you take byte-oriented serialization like Protobuf then plain TCP transport could be a good choice - data overhead would be significantly lover in comparison to any application-leveled protocol like HTTP. See this articles: python and java.

If you have deployed both programs on a single device you can consider using libraries that supports handling interprocess communication (IPC) such as ZeroMQ.
It has bindings for both:
Java bindings
Python bindings
I've successfully applied communication with 0MQ on embedded system with ARM processors. It acs as a communication bus between programs written in multiple languages (mostly in C and Python in my case).
While prototyping you can use human readable data format (like JSON). There were a lot of libraries that supports (de)serializing JSONs in Java (like GSON). Python has native modules for that.

Related

Sending objects between Python/Java server/client

I have a Python client and a Java server. I would like the client to send an object to the server. How to implement this?
How to also implement the other way around (Java client - Python server)?
Here's an attempt I made with Python server and Java client:
PYTHON SERVER SIDE
import pickle
import socket
from simple_message import SimpleMessage
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 9999))
s.listen(1)
while True:
print("Waiting for a message...")
conn, addr = s.accept()
data = conn.recv(4096)
incoming_message = pickle.loads(data)
conn.close() # Close connection, not needed anymore
print(SimpleMessage.get_payload(incoming_message))
The object it refers to (of class SimpleMessage) is defined as follows:
#!/usr/bin/env python
class SimpleMessage:
dest_address = str()
message_type = int()
payload = str()
def __init__(self, dest_address, message_type, payload):
self.dest_address = dest_address
self.message_type = message_type
self.payload = payload
def get_payload(self):
return self.payload
JAVA CLIENT SIDE
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.io.OutputStream;
import java.net.InetSocketAddress;
import java.net.Socket;
public class JavaClient {
public static void main(String[] args) throws IOException {
Socket sendingSocket = new Socket();
sendingSocket.connect(new InetSocketAddress("127.0.0.1", 9999));
OutputStream outputStream = sendingSocket.getOutputStream();
ObjectOutputStream objectOutputStream = new ObjectOutputStream(outputStream);
SimpleMessage message = new SimpleMessage("127.0.0.1", 1, "Test message!");
objectOutputStream.writeObject(message); // Write Message on socket
sendingSocket.close();
}
}
And the class SimpleMessage:
import java.io.Serializable;
public class SimpleMessage implements Serializable {
private String destAddress;
private Integer messageType;
private String payload;
public SimpleMessage(String destAddress, Integer messageType, String payload) {
this.destAddress = destAddress;
this.messageType = messageType;
this.payload = payload;
}
}
OUTPUTS
Here is the output I get on the Python server side:
Waiting for a message...
Traceback (most recent call last):
File "python_server.py", line 16, in <module>
incoming_message = pickle.loads(data)
_pickle.UnpicklingError: invalid load key, '\xac'.
And here's the output I get on the Java client side:
Exception in thread "main" java.net.SocketException: Broken pipe (Write failed)
at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
at java.base/java.io.ObjectOutputStream$BlockDataOutputStream.drain(ObjectOutputStream.java:1883)
at java.base/java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(ObjectOutputStream.java:1792)
at java.base/java.io.ObjectOutputStream.writeNonProxyDesc(ObjectOutputStream.java:1287)
at java.base/java.io.ObjectOutputStream.writeClassDesc(ObjectOutputStream.java:1232)
at java.base/java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1428)
at java.base/java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1179)
at java.base/java.io.ObjectOutputStream.writeFatalException(ObjectOutputStream.java:1583)
at java.base/java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:352)
This is a specific instance of choosing a serialization format.
https://en.wikipedia.org/wiki/Comparison_of_data-serialization_formats
This is a potentially overbroad topic, so I will hold back from a long and possibly redundant answer considering all possible formats.
JSON is a good and currently fashionable format which will get you started, but also potentially work well more involved use cases in the future. There are well-known libraries in Python and Java.
No matter what you choose, if there is any use outside a small prototype or assignment, defining an explicit schema lays a good foundation for future work (eg using JSON schema for JSON).
pickle, as in your original example code, is a Python-specific serialization format. So unless you have some specific use for a tool like Jython in your Java world, it's not a great choice for communicating over a network with a service that can be written in another language.
You should also consider whether low level sockets are the best choice for your use case, or a higher level network library like HTTP may be a better fit.
What you are talking about here is also known as decoupling of elements in the systems.
This gives you a lot of flexibility to change languages and implementation in the system and this is also the method that is used when different backend services talk with each other in a micro-services architecture.
The common way to do it is to select a JSON protocol that the two sides transfer between each other: For example:
{
fName: "David",
lName: "Gold"
}
And then to make HTTP calls to GET or POST data between your two elements.
By doing so, you are free to change the implementation on each side (let's say you discover that you better write your client in JavaScript and the server in R).
As long as both sides keep working with the same protocol, they are agnostic to the implementation that the other side uses.

C# TcpClient to netty not always arriving

Me and a friend are working on a project which requires us to communicate between a C#.NET application and a Java application.
We're using the TcpClient and BinaryWriter classes on the .NET side of things to send and receive things. We're using code similar to this to send things:
byte[] content = //we're getting our content here
Writer.Write(new byte[9]); //this is the BinaryWriter with the NetworkStream of the TcpClient
Writer.Flush();
On the Java side of things, we're using netty to handle our networking. To receive the content we send from the .NET side, we add a ChannelInboundHandlerAdapter to the pipeline and use the channelRead method to read the content:
public void channelRead(ChannelHandlerContext ctx, Object received)
{
ByteBuf receivedByteBuf = (ByteBuf)received;
this.bytesRead = receivedByteBuf.readableBytes();
System.out.println("Received " + this.bytesRead + " bytes.");
final byte[] buffer = new byte[this.bytesRead];
receivedByteBuf.markReaderIndex();
receivedByteBuf.readBytes(buffer);
receivedByteBuf.resetReaderIndex();
}
Now the strange thing is, that when we try sending content, it doesn't always arrive in one piece. Sometimes we only receive all but some bytes we originally sent, which arrive in a new call of channelRead. In this example, only 6-8 bytes would arrive. This is very strange, as this only happens when using .NET. We tried sending content using python and everything worked fine and it arrived in one channelRead call.
import socket
import string, random
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1", 8888))
s.send(''.join(random.choice(string.lowercase) for x in range(500)))
s.close()
Unfortunately, the nature of our project prevents us from changing our Java networking library, so we're stuck with netty.
Did we miss some setting in netty or does this have to do with the nature of the .NET TCP libraries? We would appreciate any help we can get.

Broadcast Message in Java

I need some mechanism that allows me to transfer some data from one java program to another within a same PC. I already investigated RMI but I want 1st app to broadcast some message for 2nd without request of 2nd application. In RMI only client can initiate a communication.
Raw sockets are also not desirable (very low level).
I need something like RMI with a different scheme of starting communication: 1 server broadcasts messages for several clients without requests from clients.
Could you please suggest me some libs/technologies (for desktop app)?
I suggest you to use java messaging service and one of it's implementations such as ApacheMQ.
A good starting point is here.
I would suggest using a database with a trigger and store procedure. Here is an example of calling java methods from the database. A message queue will work, but that is an overly complex solution.
Here's an excerpt from an example of how to create a procedure and call it via a trigger:
First, you add the following Java method to the class DBTrigger
CREATE OR REPLACE PROCEDURE add_emp (
emp_no NUMBER, emp_name VARCHAR2, dept_name VARCHAR2)
AS LANGUAGE JAVA
NAME 'DBTrigger.addEmp(int, java.lang.String, java.lang.String)';
Then, you create the INSTEAD OF trigger:
CREATE OR REPLACE TRIGGER emps_trig
INSTEAD OF INSERT ON emps
FOR EACH ROW
CALL add_emp(:new.empno, :new.ename, :new.dname);
Since you tagged this with CORBA, you could use the Event Service to broadcast a notice to all interested clients.
To enable the server to send packets of information to the client/clients. A datagram, by definition, is “an independent, self-contained message sent over the network whose arrival, arrival time, and content are not guaranteed”. Essentially, we are opening a DatagramSocket in order to send DatagramPacket messages to the client. We are using the datagram classes (instead of standard sockets) because they allow us to broadcast information to multiple clients, that are all connected to a MulticastSocket.
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.UnknownHostException;
public class MulticastSocketServer {
final static String INET_ADDR = "224.0.0.3";
final static int PORT = 8888;
public static void main(String[] args) throws UnknownHostException, InterruptedException {
// Get the address that we are going to connect to.
InetAddress addr = InetAddress.getByName(INET_ADDR);
// Open a new DatagramSocket, which will be used to send the data.
try (DatagramSocket serverSocket = new DatagramSocket()) {
for (int i = 0; i < 100; i++) {
String msg = "Sent message no " + i;
// Create a packet that will contain the data
// (in the form of bytes) and send it.
DatagramPacket msgPacket = new DatagramPacket(msg.getBytes(),
msg.getBytes().length, addr, PORT);
serverSocket.send(msgPacket);
System.out.println("Server sent packet with msg: " + msg);
Thread.sleep(500);
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
One thing that we need to take into consideration here, is that there are specific addresses that allow us to use a MulticastSocket are limited, specifically in the range of 224.0.0.0 to 239.255.255.255. Some of them are reserved, like 224.0.0.0. The address that we are using, 224.0.0.3, can be used safely.
Regarding the client, we are going to move a little bit differently. We are going to create a client class, that will accept incoming messages from the server, and then we are going to duplicate this class. The point here is that by using the same code, we can connect to the server seamlessly, while having as many clients as we like.
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.InetAddress;
import java.net.MulticastSocket;
import java.net.UnknownHostException;
public class MulticastSocketClient
{
final static String INET_ADDR = "224.0.0.3";
final static int PORT = 8888;
public static void main(String[] args) throws UnknownHostException {
// Get the address that we are going to connect to.
InetAddress address = InetAddress.getByName(INET_ADDR);
// Create a buffer of bytes, which will be used to store
// the incoming bytes containing the information from the server.
// Since the message is small here, 256 bytes should be enough.
byte[] buf = new byte[256];
// Create a new Multicast socket (that will allow other sockets/programs
// to join it as well.
try (MulticastSocket clientSocket = new MulticastSocket(PORT)){
//Joint the Multicast group.
clientSocket.joinGroup(address);
while (true) {
// Receive the information and print it.
DatagramPacket msgPacket = new DatagramPacket(buf, buf.length);
clientSocket.receive(msgPacket);
String msg = new String(buf, 0, buf.length);
System.out.println("Socket 1 received msg: " + msg);
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
First, we start the clients, which will keep waiting for incoming packets of information. As soon as we start the server, it will send the information packets and the clients will receive them and print the information on the screen
ZeroMQ could be what you are searching, but I have only used it in C, C++ and Python, so I am not totally sure of how well it is usable in Java. But they have implemented the Publisher-Subscriber pattern for their sockets and at least the static library of the version 3.2.2 under ubuntu 12.04 is stable and works very well.
As a "quick fix" write the output to file and have the second app read the file.
This is not elegant at all, but if you code to interfaces you can later replace the implementation with something better.
A good solution would be an implementation of the OMG Data Distribution Standard (DDS). With DDS there is just a global data space with a dynamic discovery protocol. See for example RTI DDS, there is a free community edition.

Can ZeroMQ be used by two JVM processes to communicate on same machine?

I've always used either ActiveMQ or RabbitMQ, but have started digging into ZeroMQ lately because of the attention its getting. If what I'm reading is correct, then it seems to be a message broker as well as a mechanism for interprocess communication (IPC)?
I have a situation where I have 2 Java processes that need to communicate with each other on the same machine. I don't want to use a shared file approach because the dialog/protocol between them is pretty sophisticated and a file solution seems clumsy.
So I was going to start heading down the road of using something like Netty or MINA to defines my own comm protocol, and have them converse over ports, but then I started reading about ZeroMQ and am wondering if I can accomplish the same but with less work.
So I ask: can ZeroMQ be used for IPC between the Java processes, and if so, are there any concrete code examples or articles explaining exactly how to do this?
The first three lines of the web site tell you every thing you need to know.
Distributed Computing Made Simple
 Ø  The socket library that acts as a concurrency framework.
 Ø  Carries messages across inproc, IPC, TCP, and multicast.
I don't see any reason to suspect that this doesn't work over loopback, and it would be pretty bizzare if it couldn't.
Yes, zmq can be used to connect two Java processes. You can use pure Java implementation JeroMq or ZeroMq with Java client. JeroMq is easier to install as you need only the appropriate dependency. Here is simple example for listener:
import org.zeromq.ZMQ;
public class Subscriber {
public static void main(String[] a){
final ZMQ.Context ctx = ZMQ.context(1);
final ZMQ.Socket sub = ctx.socket(ZMQ.SUB);
// sub.connect("tcp://localhost:6001");
sub.connect("ipc://001");
sub.subscribe("".getBytes());
while (true){
String msg = sub.recvStr();
System.out.println(msg);
}
}
}
and for publisher:
import org.zeromq.ZMQ;
public class Publisher {
public static void main(String[] a) throws InterruptedException {
final ZMQ.Context ctx = ZMQ.context(1);
final ZMQ.Socket pub = ctx.socket(ZMQ.PUB);
pub.bind("ipc://001");
// pub.bind("tcp://*:6001");
while (true){
System.out.println("Publishing");
pub.send("Test");
Thread.sleep(1000);
}
}
}
IPC and TCP both work.

How do I send a message over a network? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking for code must demonstrate a minimal understanding of the problem being solved. Include attempted solutions, why they didn't work, and the expected results. See also: Stack Overflow question checklist
Closed 8 years ago.
Improve this question
I want to be able to send a simple message across a network. Any message.
Specifically, I want a server that's running all the time, and a client that can connect to and communicate with the server on demand.
I don't know where to begin. What is the simplest way to send a message over a network?
Starting with the tutorial Reading from and Writing to a Socket, you could start with a simple echo server which uses a ServerSocket like so
public static void main(String[] args) throws Exception {
// create socket
int port = 4444;
ServerSocket serverSocket = new ServerSocket(port);
System.err.println("Started server on port " + port);
// repeatedly wait for connections, and process
while (true) {
// a "blocking" call which waits until a connection is requested
Socket clientSocket = serverSocket.accept();
System.err.println("Accepted connection from client");
// open up IO streams
BufferedReader in = new BufferedReader(new InputStreamReader(
clientSocket.getInputStream()));
PrintWriter out = new PrintWriter(clientSocket.getOutputStream());
// waits for data and reads it in until connection dies
// readLine() blocks until the server receives a new line from
// client
String s;
try {
while ((s = in.readLine()) != null) {
out.println(s);
out.flush();
}
} catch (Exception e) {
e.printStackTrace();
}
// close IO streams, then socket
System.err.println("Closing connection with client");
out.close();
in.close();
clientSocket.close();
}
Then you can use "telnet localhost 4444" to test it, or write an entire client; perhaps like the client in the tutorial above.
Okay, this question is really open. First there are a few questions to answer: Do you want to implement your own server or use an existing technology. The JMS - API oviously would be a starting point in the later case. There exist many implementation providers, the reference implementation within Glassfish itself, HornetQ both standalone or as part of a JBoss - Instance, just to name two. Sending messages with JMS is pretty straight forward when understanding some basic concepts and they can contain both textual and binary data.
Another way of using existing solution is to implement a WebService your client can connect to, wich would be the JAX-RS for REST - based WebServices e.g. This approach is also more platform - independend, as of this writing I don't know a library for Android clients supporting the JMS, and most of the other programming language have full support for REST - based services. When you don't want to use any existing protocoll, though, you have to stick with an own implementation of a server and client. So this would be the mentioned socket - programming. Well, this task won't be to easy: You would won't your server to handle multiple clients at the same time, not blocking one request until another one is finished, provide some mechanism for authorizing your client for application access against the server. You will propably have resources that are shared between your clients on the server when handling multiple clients simutanliously so the access to this resources must be synchronized, these topics relate to concurrent programming and is a quiet interesting field of information technology of its own. Other intersting aspects of your application design could be decisions as using an supporting the JNDI for resource binding or implementing a CDI - container so that clients can easily adapt your message exchanging format. It's fun, but not a within - one - day - task. ;)

Categories