I want to write to a named pipe (already created) without blocking on the reader. My reader is another application that may go down. If the reader does go down, I want the writer application to keep writing to that named pipe. Something like a this in Java
fopen(fPath, O_NONBLOCK)
So that when the reader comes up, it may resume from where it failed.
First I try to answer your questions. Next I will try to show you a code snippet I created that solves your problem using blocking IO.
Your questions
I want to write to a named pipe
(already created) without blocking on
the reader
You don't need non blocking IO to solve your problem. I think it can not even help you solve your problem. Blocking IO will also run good(maybe even better then non blocking IO because of the low concurrency). A plus is blocking IO is easier to program. Your reader can/should stay blocking.
My reader is another application that
may go down. If the reader does go
down, I want the writer application to
neep writing to the named pipe. So that when the reader comes up, it may resume from where it failed.
just put the messages inside a blocking queue. Next write to the named pipe only when the reader is reading from it(happens automatically because of blocking IO). No need for non-blocking file IO when you use a blocking queue. The data is asynchronous delivered from the blocking queue when a reader is reading, which will sent your data from your writer to the reader.
Something like a fopen(fPath,
O_NONBLOCK) in Java
You don't need non-blocking IO on the reader and even if you used it. just use blocking IO.
CODE SNIPPET
A created a little snippet which I believe demonstrates what your needs.
Components:
Writer.java: reads lines from console as an example. When you start program enter text followed by enter which will sent it to your named pipe. The writer will resume writing if necessary.
Reader.java: reads lines written from your named pipe(Writer.java).
Named pipe: I assume you have created a pipe named "pipe" in the same directory.
Writer.java
import java.io.BufferedWriter;
import java.io.Console;
import java.io.FileWriter;
import java.io.PrintWriter;
import java.util.concurrent.BlockingDeque;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.LinkedBlockingDeque;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Writer {
private final BlockingDeque<StringBuffer> queue;
private final String filename;
public static void main(String[] args) throws Exception {
final Console console = System.console();
final Writer writer = new Writer("pipe");
writer.init();
while(true) {
String readLine = console.readLine();
writer.write(new StringBuffer(readLine));
}
}
public Writer(final String filename){
this.queue = new LinkedBlockingDeque<StringBuffer>();
this.filename = filename;
}
public void write(StringBuffer buf) {
queue.add(buf);
}
public void init() {
ExecutorService single = Executors.newSingleThreadExecutor();
Runnable runnable = new Runnable() {
public void run() {
while(true) {
PrintWriter w = null;
try {
String toString = queue.take().toString();
w = new PrintWriter(new BufferedWriter(new FileWriter(filename)), true);
w.println(toString);
} catch (Exception ex) {
Logger.getLogger(Writer.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
};
single.submit(runnable);
}
}
Reader.java
import java.io.BufferedReader;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Reader {
private final BufferedReader br;
public Reader(final String filename) throws FileNotFoundException {
br = new BufferedReader(new FileReader(filename));
}
public String readLine() throws IOException {
return br.readLine();
}
public void close() {
try {
br.close();
} catch (IOException ex) {
Logger.getLogger(Reader.class.getName()).log(Level.SEVERE, null, ex);
}
}
public static void main(String[] args) throws FileNotFoundException {
Reader reader = new Reader("pipe");
while(true) {
try {
String readLine = reader.readLine();
System.out.println("readLine = " + readLine);
} catch (IOException ex) {
reader.close();
break;
}
}
}
}
If you want pipes to stay active and queue up messages, you probably want a messaging system rather than a raw pipe. In Java, the standard API is called "Java Messaging System" (JMS), and there are many standard implementations-- the most common of which I've seen being Apache ActiveMQ. If you want a cross-platform, sockets-like interface that does buffering and recovery I might suggest 0MQ, which while not being "pure Java" has bindings for many languages and excellent performance.
If there was such a thing as non-blocking file I/O in Java, which there isn't, a write to a named pipe that wasn't being read would return zero and not write anything. So non-blocking isn't part of the solution.
There's also the issue that named pipes have a finite buffer size. They aren't infinite queues regardless of whether there is a reading process or not. I agree with the suggestion to look into JMS.
You should be able to use NIO's asynch write on a UNIX FIFO, just as you can to any other file:
AsynchronousFileChannel channel = AsynchronousFileChannel.open(...);
Future<Integer> writeFuture = channel.write(...);
... or...
channel.write(..., myCompletionHandler);
However, it's not clear to me what you want to happen when the FIFO isn't accepting writes. Do you want it to buffer? If so you'll need to provide it within the Java program. Do you want it to time out? There's no simple timeout option on Java file writes.
These aren't insurmountable problems. If you're determined you can probably get something working. But I wonder whether you'd not find life much easier if you just used a TCP socket or a JMS queue.
Related
I was trying to do a chat and I notice that readUTF() and writeUTF() did not work. readUTF() stays waiting when I already used writeUTF(). I made a simple test and does not work, what I am doing bad?
(I know that I could use Data Streams instead of Object Streams but in my chat I want to write objects and strings)
Server code:
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.net.ServerSocket;
import java.net.Socket;
public class Server {
public static void main(String[] args) throws IOException {
ServerSocket server = new ServerSocket(40001);
Socket s = server.accept();
ObjectOutputStream out = new ObjectOutputStream(s.getOutputStream());
ObjectInputStream in = new ObjectInputStream(s.getInputStream());
System.out.println(in.readUTF());
out.writeUTF("E");
}
}
Client code:
import java.io.IOException;
import java.io.ObjectInputStream;
import java.io.ObjectOutputStream;
import java.net.Socket;
public class Client {
public static void main(String[] args) throws IOException {
Socket s = new Socket("localhost", 40001);
ObjectOutputStream out = new ObjectOutputStream(s.getOutputStream());
ObjectInputStream in = new ObjectInputStream(s.getInputStream());
out.writeUTF("A");
System.out.println(in.readUTF());
}
}
I could use writeObject("A") and cast the String with readObject(), but I want to know why this way does not work.
You need to call flush() after writing using writeUTF. The reason that writeObject just seems to work is that writeObject will switch to a specific mode before it starts writing, and switch back after it is done. This switching back will automatically flush the buffered data (at least in Java 11). This is not the case for writeUTF, and an explicit call to flush() is needed.
ObjectOutputStream uses an internal buffer, so you should try out.flush() after writes if you want content to be available to read on the InputStream.
The javadoc for ObjectOutputStream includes:
callers may wish to flush the stream immediately to ensure that constructors for receiving ObjectInputStreams will not block when reading the header
I'm trying to create a scenario on my Macbook with Mojave in Anylogic, which is part of agent-based simulation with many different tools. My idea is to connect Anylogic via Java Interface to Eclipse.
The main problem is, that Anylogic somehow does not respond.
I have tried many different codes with sockets, but could not find one, which worked for Anylogic. I'm using the free version of Anylogic and created a Java Interface under my Main Project. To run the Java Interface I press right-click on the file and select 'run with Java Editor'
In comparison to that I create two files in Eclipse, with one being the Server and one the Client and it worked.
so this is my Eclipse, which I guess should be fine https://www.minpic.de/i/7wbk/nv00b
this is my main model in Anylogic https://www.minpic.de/i/7wbn/pzuut
and there is the Java interface with the server code in it. https://www.minpic.de/i/7wbo/1mxsl4
Im pretty new to coding so hopefully you guys can help me.
public class server{
public static void main(String[] args) throws IOException {
ServerSocket ss = new ServerSocket(4995);
Socket s = ss.accept();
System.out.println("Client connected");
DataInputStream dout = new DataInputStream(s.getInputStream());
BufferedReader br = new BufferedReader (new InputStreamReader (System.in));
while(true) {
String yoo = dout.readUTF();
System.out.println("client" + yoo);
if(yoo.equalsIgnoreCase("exit"));
break;
}
ss.close();
}
}
public class client{
public static void main(String[] args) throws IOException {
Socket s = new Socket("localhost",4995);
DataOutputStream dout = new DataOutputStream(s.getOutputStream());
BufferedReader br = new BufferedReader (new InputStreamReader (System.in));
while (true)
{
String so= br.readLine();
dout.writeUTF(so);
System.out.println("client" + so);
if(so.equalsIgnoreCase("exit"));
break;
}
s.close();
}
}
I was expecting the consoles of both programs to show me messages I have send via the console, but neither of the programs show me the messages of what I wrote in the other program.
The Java code itself is fine, at least for creating a simple connection.
For the server side in Eclipse, you can leave it like that.
For the client side in AnyLogic however, there is an issue:
You can't run the code directly like this, because you have a main method in there. AnyLogic is no "normal" Java IDE like Eclipse, it is a very specific IDE. It automatically creates you exactly one project and puts everything in there that is needed to run it, including one main method. This means you don't need a second main method. You rather have to make your client become "part" of the bigger programm that AnyLogic is building for you. When you clicked on "Open with Java Editor" that only showed you the code, you cannot run any code like that in AnyLogic!
Therefore we do the following steps:
Create of a Java class (a simple one, without main method) Client in AnyLogic
Add a function to the class to start the client procedure (before it was triggered by it's own main method)
Create an instance from the Class Client
The class code, already including the function is the following:
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.ServerSocket;
import java.net.Socket;
public class Client implements Serializable {
public Client() {
}
public void activate() {
try {
Socket s = new Socket("localhost",4995);
DataOutputStream dout = new DataOutputStream(s.getOutputStream());
BufferedReader br = new BufferedReader (new InputStreamReader (System.in));
while (true)
{
String so= br.readLine();
dout.writeUTF(so);
System.out.println("client" + so);
if(so.equalsIgnoreCase("exit"));
break;
}
s.close();
}
catch(IOException e) {
System.out.println(e);
}
}
/**
* This number is here for model snapshot storing purpose<br>
* It needs to be changed when this class gets changed
*/
private static final long serialVersionUID = 1L;
}
Creating the instance and activating the client can be done with this code, add it for example in a button or the OnStartup code of the AnyLogic Main Agent:
Client client = new Client();
client.activate();
I'm trying to perform communication between server and client using Genson library. I've detected the following problem: trying to send a message to the server my application stalls when genson on the server is trying to read the message.
Meanwhile, if I shutdown the client, the message is perfectly read and processed. I've thought it to be deadlock but not sure.
There is no such a problem with native Java serialization.
Here is my server:
import com.owlike.genson.Genson;
import java.io.IOException;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Arrays;
public class Server {
public static void main(String[] args) throws IOException {
Genson genson = new Genson();
try (ServerSocket server = new ServerSocket(9991)) {
try (Socket socket = server.accept()) {
int[] loc = genson.deserialize(socket.getInputStream(), int[].class);
System.out.println("Server: " + Arrays.toString(loc));
genson.serialize(loc, socket.getOutputStream());
}
}
}
}
Here is the client:
import com.owlike.genson.Genson;
import java.io.IOException;
import java.net.Socket;
import java.util.Arrays;
public class Client {
public static void main(String[] args) throws IOException {
Genson genson = new Genson();
try (Socket socket = new Socket("localhost", 9991)) {
genson.serialize(new int[] {1, 2, 3}, socket.getOutputStream());
int[] loc = genson.deserialize(socket.getInputStream(), int[].class);
System.out.println("Client: " + Arrays.toString(loc));
}
}
}
I wound highly appreciate any help with this question. Thanks in advance.
Edit: This is really wierd. I've made some additional tests and here is what i get:
Additional class:
import com.owlike.genson.annotation.JsonProperty;
import java.io.Serializable;
public class Tester implements Serializable {
public static final Tester TEST = new Tester(Math.E);
private double val = Math.PI;
public Tester(#JsonProperty("val") double val) {
this.val = val;
}
public Tester() {}
public String toString() {
return "" + val;
}
}
Having written genson.serialize(Tester.TEST, socket.getOutputStream()) in the client request I have the same strange result. But having written genson.serialize(new Tester(Double.NaN), socket.getOutputStream()) the result is the expexted one.
Furthermore, if I define the only field in Tester class to be of type int[], lets say, it only works with values of null or new int[0].
In addition to that, if I'm trying to serialize and transmit int for integers in range 0..9 I observe the following behaviour: the same strange thing except that when I shutdown the client, server always shows 0 value.
Besides, for constants like Double.NaN, Double.POSITIVE_INFINITY, Integer.MAX_VALUE and similar there is nothing strange at all (everything works as expected).
For those additional tests Genson class was defined as follows:
Genson genson = new GensonBuilder()
.useMethods(false)
.setFieldFilter(VisibilityFilter.PRIVATE)
.create();
Note that there is no such issue when ser/deser to/from a file using streams:
import com.owlike.genson.Genson;
import com.owlike.genson.GensonBuilder;
import com.owlike.genson.reflect.VisibilityFilter;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
public class FileTest {
private static final String FILENAME = "test.json";
public static void main(String[] args) throws IOException {
Genson genson = new GensonBuilder()
.useMethods(false)
.setFieldFilter(VisibilityFilter.PRIVATE)
.useIndentation(true)
.create();
try (OutputStream stream = new FileOutputStream(FILENAME)) {
genson.serialize(Tester.TEST, stream);
}
try (InputStream stream = new FileInputStream(FILENAME)) {
System.out.println(genson.deserialize(stream, Tester.class));
}
}
}
Looks like it was a mistake of mine all the time. I've forgotten that socket's stream can't be closed (unless you want to close the socket too). So in this case Server tries to get as much data from InputStream as it can but it can't consume all the stream (because it is always opened and data can be sent at any time from the client). So the Server basically freezes waiting for data but there is no more data to come. As a result we have the very situation described above.
A solution would be to specify some kind of protocole to denote query size so the Server can know how much data it should consume. See this answer for more details.
Some code paths in the reading API will try to eagerly ensure that there are at least N bytes available or EOF has been reached. This happens all the time when parsing a number (that is not a constant) as you noted.
An option could be to implement a small layer that serializes to a byte array or string, gets the message length and writes it to the output, followed by the payload. On the reading side you would first read the length and then read from the stream in a while loop until you reached that length or EOF.
Then you would just pass to Genson.deseralize this in memory message.
I was wondering if i could get help making or finding a program that has the ability to send keyboard presses and receive them on another computer. I want to use this to play multiplayer flash-player games with friends across computers. I know there are some programs out there like "logmein" but both users cannot use the keyboard at the same time. (When i press a key the computer user cannot press a key at the same time because it wont respond.) I only know java and I am quite novice at it. Im guessing if i need to write it ill have to send the information through a port or onto a web-server. I would like to know your opinions and suggestions for this program, thanks guys.
Basically what you are looking for is a chatroom program? Have you tried looking into mIRC?
mIRC is a free internet relay chat. What exactly are the requirements for the program? Is there a certain size that it must be? Are these flash games that you and your friends are playing taking up your full computer screen?
Building a program would require a web-server(any computer with internet access would do), and you would have to open the ports on your network to allow the traffic to go through.
A basic server in java would look something like this:
Please note that after the first connection this "server" will close the connection.
import java.net.ServerSocket;
import java.net.Socket;
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.IOException;
public class Server
{
private static ServerSocket serverSocket;
private static Socket clientSocket;
private static BufferedReader bufferedReader;
private static String inputLine;
public static void main(String[] args)
{
// Wait for client to connect on 63400
try
{
serverSocket = new ServerSocket(63400);
while(true){
clientSocket = serverSocket.accept();
// Create a reader
bufferedReader = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
// Get the client message
while((inputLine = bufferedReader.readLine()) != null)
{System.out.println(inputLine);}
serverSocket.close();
System.out.println("close");
}
}
catch(IOException e)
{
System.out.println(e);
}
}
}
And a client would almost be the same:
import java.net.Socket;
import java.io.PrintWriter;
public class client
{
private static Socket socket;
private static PrintWriter printWriter;
public static void main(String[] args)
{
try
{
//change "localhost" to the ip address that the client is on, and this number to the port
socket = new Socket("localhost",63400);
printWriter = new PrintWriter(socket.getOutputStream(),true);
printWriter.println("Hello Socket");
}
catch(Exception e)
{
System.out.println(e);
}
}
}
If I am not mistaken printWriter is a 16-bit operation, and in order to reduce lag, if you were just sending text then you might want to use printStream(). I believe that this might be a bit quicker.
I have no idea why this is hanging. I'm trying to capture output from a process run through commons-exec, and I continue to hang. I've provided an example program to demonstrate this behavior below.
import java.io.DataInputStream;
import java.io.IOException;
import java.io.PipedInputStream;
import java.io.PipedOutputStream;
import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
import org.apache.commons.exec.ExecuteException;
import org.apache.commons.exec.PumpStreamHandler;
public class test {
public static void main(String[] args) {
String command = "java";
PipedOutputStream output = new PipedOutputStream();
PumpStreamHandler psh = new PumpStreamHandler(output);
CommandLine cl = CommandLine.parse(command);
DefaultExecutor exec = new DefaultExecutor();
DataInputStream is = null;
try {
is = new DataInputStream(new PipedInputStream(output));
exec.setStreamHandler(psh);
exec.execute(cl);
} catch (ExecuteException ex) {
} catch (IOException ex) {
}
System.out.println("huh?");
}
}
According to the javadoc, execute(CommandLine command) is synchronous, execute(CommandLine command, ExecuteResultHandler handler) on the other hand is asynchronous.
The command you invoked, java, produces output to its standard output stream. That stream must be pumped into an input stream by your invoking program. This does not happen in your program.
You have to read the input stream (is in your code) in a separate thread, because that is how piped streams work. Note that you must start the reading thread before calling execute().
See also Capturing large amounts of output from Apache Commons-Exec
According to your other question Streaming output with commons-exec? you expect large data, so you must use the piped streams and cannot use the simpler approach of using a ByteArrayInputStream as output. The answer you give yourself there, suffers from the same problem as your code here.