I have this class which opens a HTTP-Server and listens on a port. The reply is a http-header plus a json object. The class takes the input stream from the server, converts it to a string, filters the http header and parses the json object.
The thing is that the conversion from input stream to string is taking about 3 seconds. Is there a way to read the inputstream faster?
public class GamestateListener {
private static int PORT = 3001; // Port specified in your cfg
public static ServerSocket listenServer;
private static JSONObject MYJSONOBJ;
public GamestateListener() {
try {
listenServer = new ServerSocket(PORT); // open new Server Socket
System.out.println("Started Socket..."); // printing out started
// listening
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public JSONObject listenAndParseJSON() throws IOException {
System.out
.println("Listening for connection on port " + PORT + " ...."); // printing
// out
// started
// listening
try (Socket socket = listenServer.accept()) { // wait for connection
System.out.println("Start get From Socket "
+ System.currentTimeMillis());
InputStream mis = socket.getInputStream();
System.out.println("Stop get From Socket "
+ System.currentTimeMillis());
String responseString = IOUtils.toString(mis, "UTF-8");
System.out.println("Stop to String "
+ System.currentTimeMillis());
MYJSONOBJ = new JSONObject(responseString.substring(responseString
.indexOf("{")));// split the response string
return MYJSONOBJ;// return the json obj
} catch (Exception e) {
MYJSONOBJ = new JSONObject("{ERROR:True}");// create error obj
return MYJSONOBJ;// return it
}
}
}
You're not measuring what you think you are. Here:
System.out.println("Start get From Socket "
+ System.currentTimeMillis());
InputStream mis = socket.getInputStream();
System.out.println("Stop get From Socket "
+ System.currentTimeMillis());
... you seem to think that you've read all the data when getInputStream() returns. You haven't. You've just got a stream you can read from. It means there's a connection, but that's all.
If you want to measure how long it takes just to read all the data (and not actually process it) you could do something like:
System.out.println("Start get From Socket "
+ System.currentTimeMillis());
InputStream mis = socket.getInputStream();
byte[] buffer = new byte[8192];
// Read all the data (but ignore it)
while ((mis.read(buffer)) != -1) ;
System.out.println("Stop get From Socket "
+ System.currentTimeMillis());
Of course, there then won't be any data to read for the rest of the code, but you'll see how long just the data reading takes. My guess is that that'll be about 3 seconds... which could be due to slow networking, or the client taking a long time to even send all the data. (It could connect, sleep for 2.9 seconds and then send a bunch of data.)
Related
I have connection to TCP server (ip,port) to which meter is connected. I'd like to read the specified data from this port because when I'm using standard read method it sends me the whole data stream which takes about 15 minutes to read. So my question: is there any method I can use to get one specified register's value using his OBIS code (1.1.1.8.0.255 - active energy taken) in java via TCP server?
import java.net.*;
import java.io.*;
public class scratch {
public static void main(String[] args) {
String hostname = "ip (hidden)";
int port = port (hidden);
try (Socket socket = new Socket(hostname, port)) {
OutputStream out = socket.getOutputStream();
InputStream input = socket.getInputStream();
InputStreamReader reader = new InputStreamReader(input);
int character;
StringBuilder data = new StringBuilder();
String test = "/?!\r\n";
byte[] req = test.getBytes();
out.write(req);
while ((character = reader.read()) != '\n') {
data.append((char) character);
}
System.out.println(data);
} catch (UnknownHostException ex) {
System.out.println("Server not found: " + ex.getMessage());
} catch (IOException ex) {
System.out.println("I/O error: " + ex.getMessage());
}
}
}
The message "test" send initiation request to meter and his respond is correct but I dont' know how to put flags (ACK STX ETX) in my request, I've tried something like this:
String test2 = (char)0x6 + "051\r\n";
byte[] req2 = test2.getBytes("ASCII");
out.write(req2);
But meter doesn't recognize it.
I have a integration test that sends 800 messages to a client. The client receives all the messages fine. The issue is my test fails periodically because the socket (towards the end of the messages), reads no bytes (-1) from the stream, closes the socket, reopens the socket, and gets the bytes. The test 'completes' before the final bytes are read (and so fails), but I can see in the log the last messages do get to the client successfully. This is intermittent. It happens about every half dozen runs or so. In other words, maybe 5 runs straight will get no errors (the socket never had to be closed/re-opened), but the 6th run will have this issue.
So far I have tried increasing/decreasing message sending speed.
Server:
try
{
srvr = new ServerSocket(PORT);
Socket socket = srvr.accept();
...
OutputStream out = socket.getOutputStream();
for (String msg : Messages)
{
byte[] bytes = new BigInteger(msg, BASE_16).toByteArray();
out.write(bytes);
// Delay so client can keep up.
sleep(SOCKET_CLIENT_DELAY);
}
}
catch (IOException ioe)
{
fail(ioe.getMessage());
}
finally
{
handleClose(srvr);
}
Client:
#Override
public void run()
{
final long reconnectAttemptWaitTimeMillis = 5_000;
Socket socket = null;
while (true)
{
try
{
socket = new Socket(host, port);
boolean isConnected = socket.isConnected();
if (isConnected)
{
read(socket);
}
}
catch (ConnectException ce)
{
LOGGER.warn(
"Could not connect to ADS-B receiver/antenna on [" + host + ":" + port
+ "]. Trying again in 5 seconds...");
try
{
sleep(reconnectAttemptWaitTimeMillis);
}
catch (InterruptedException ie)
{
LOGGER.error(ie.getMessage(), ie);
break;
}
}
catch (IOException ioe)
{
LOGGER.error(ioe.getMessage(), ioe);
}
}
LOGGER.info("A total of " + totalEnqueued + " ADS-B messages were enqueued.");
}
/**
* Reads binary ADS-B messages from the {#link Socket}. Assumption is the given {#link Socket} is connected.
*
* #param socket
* where to read ADS-B messages from.
*/
private void read(final Socket socket)
{
LOGGER.info(getName() + " connected to " + host + ":" + port);
DataInputStream in = null;
try
{
in = new DataInputStream(socket.getInputStream());
while (true)
{
byte[] rawBuffer = new byte[MESSAGE_BUFFER_SIZE];
int bytesRead = in.read(rawBuffer);
if (bytesRead == -1)
{
LOGGER.warn("End of stream reached.");
break;
}
/*
* The Mode-S Beast receiver's AVR formatted output for every message begins with 0x1A.
*/
if (rawBuffer[0] == MODE_S_BEAST_PREFIX_NUM)
{
/*
* AdsbDecoder will expect a hexadecimal String representation of the ADS-B message
* without the prefix and suffix.
*
* First, we will convert the raw bytes into a hexadecimal String.
*
* Then, we will remove the Mode-S Beast metadata from the AVR format.
*
* For example:
* "1A33000000000000008DA44E325915B6B6A2FACB45988A" will look like "8DA44E325915B6B6A2FACB45988A"
*
* Finally, we enqueue the ADS-B hex message.
*/
// 1A33000000000000008DA44E325915B6B6A2FACB45988A
String modeS = new BigInteger(rawBuffer).toString(BASE_16).toUpperCase();
// Remove Mode-S Beast info
final int modesBeastPrefixLength = 18;
String adsbMessage = modeS.substring(modesBeastPrefixLength, modeS.length());
LOGGER.info("Message read from receiver/antenna: [" + adsbMessage + "]");
rawAdsbMessages.offer(adsbMessage);
++totalEnqueued;
}
}
}
catch (
IOException ioe)
{
LOGGER.error("Problem encountered reading from Socket. " + ioe.getMessage());
}
finally
{
if (Objects.nonNull(in))
{
try
{
in.close();
}
catch (IOException ex)
{
LOGGER.error("Problem encountered closing the Socket reader. " + ex.getMessage());
}
}
}
}
I expect to not see the bytesRead value become -1 and it have to re-establish the connection since this is all in one test. I have very limited knowledge of socket programming, so maybe this is an unrealistic expectation. If so, please tell me why. Maybe putting a buffered reader/writer in .?? Any suggestions would be fantastic.
I'm having the following problem in java: I am developing and app using java.net.Socket. It looks like that: There is a server with a thread which accepts and adds new client, and another thread which reads data from sockets and does "something" with it. Next to it there are clients. Client has data reader thread as well as a separate thread. I send the data as simple as:
socket.getOutputStream().write((content+"\n").getBytes());
on the client side and read it on the server like:
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
String received;
while(true) {
try {
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
received = reader.readLine();
if(received == null) {
break;
}
System.out.println("SERVER " + received);
increaseReceivedCounter(1);
} catch(SocketException e) {
break;
} catch (IOException e) {
e.printStackTrace();
}
}
} catch (IOException e) {
e.printStackTrace();
} finally {
System.out.println("SERVER RECEIVED "+ getReceivedCounter() + " MESSAGES!");
}
Now I just set the client to send some amount of messages like this:
try {
int n = 1000;
System.out.println("sending "+ n +" messages to " + client);
for(int i=0 ; i<n ; ++i) {
socket.getOutputStream().write((content+"\n").getBytes());
}
System.out.println("done sending " + n + " messages");
} catch (IOException e) {
e.printStackTrace();
}
The problem is that not all of the messages are transferred to a server. I have been looking for some solution for this but didn't manage to achieve 100% reliability. Is it even possible? I also tried with read instead of readLine but the result is the same: sometimes even 90% data loss. I think while server is working on the received data it ignores incoming packets and they're just lost.
Edit
Sockets initializations:
serverSocket = new ServerSocket(Server.PORT);//PORT = 9876, whatever
for the data reader on server side:
socket = serverSocket.accept();
on the client:
socket = new Socket("127.0.0.1", Server.PORT)
This is not an 'efficiency issue'. It is a bug in your code.
The problem is that not all of the messages are transferred to a server.
No, the problem is that you are losing data at the server. This is because you keep recreating BufferedReaders. You should create it once for the life of the socket.
reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
Remove this line.
The way you have it, you will lose data every time the prior BufferedReader has, err, buffered.
You also need to close the socket.
I try to implement a Java Proxy for Http (Https will be the extension after Http works). I found a lot of resources on the Internet and try to solve all problems on my own so far. But now I come to a point where I stuck.
My Proxoy does not load the full http websites. I get a lot of error messages with the socket is already closed. So I think I try to send something over a Socket that is closed.
My Problem is now. I can not see why it is like this. I think a lot over the problem but I can not find the mistake. From my side The Sockets only get closed when the server close the connection to my Proxy Server. This happen when I read a -1 on the input stream from the server.
I would be happy for any help :-)
greetings
Christoph
public class ProxyThread extends Thread {
Socket client_socket;
Socket server_socket;
boolean thread_var = true;
int buffersize = 32768;
ProxyThread(Socket s) {
client_socket = s;
}
public void run() {
System.out.println("Run Client Thread");
try {
// Read request
final byte[] request = new byte[4096];
byte[] response = new byte[4096];
final InputStream in_client = client_socket.getInputStream();
OutputStream out_client = client_socket.getOutputStream();
in_client.read(request);
System.out.println("---------------------- Request Info --------------------");
System.out.println(new String(request));
Connection conn = new Connection(new String(request));
System.out.println("---------------------- Connection Info --------------------");
System.out.println("Host: " + conn.host);
System.out.println("Port: " + conn.port);
System.out.println("URL: " + conn.URL);
System.out.println("Type: " + conn.type);
System.out.println("Keep-Alive:" + conn.keep_alive);
server_socket = new Socket(conn.URL, conn.port);
InputStream in_server = server_socket.getInputStream();
final OutputStream out_server = server_socket.getOutputStream();
out_server.write(request);
out_server.flush();
Thread t = new Thread() {
public void run() {
int bytes_read;
try {
while ((bytes_read = in_client.read(request)) != -1) {
out_server.write(request, 0, bytes_read);
out_server.flush();
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
};
t.start();
int bytes_read;
while ((bytes_read = in_server.read(response)) != -1) {
out_client.write(response, 0, bytes_read);
out_client.flush();
//System.out.println("---------------------- Respone Info --------------------");
//System.out.println(new String(response));
}
//System.out.println("EIGENTLICH FERTIG");
} catch (IOException e) {
e.printStackTrace();
} finally {
try {
client_socket.close();
server_socket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
EDIT:
My HTTP Proxy now works. The Answer is pretty helpfull once you understand what is ryl going on. If you come hear to find a solution this questions may help you:
Does the client send a request only to one Website / Webserver? Means do we always have the same port / hostname?
The Loop from the answer is very usefull but think where to place it?
Last think: Thanks #EJP its working your reply was very usefull. It only tooks a time to understand it!
You are making all the usual mistakes, and a few more.
The entire request is not guaranteed to arrive in a single read. You can't assume more than a single byte has arrived. You have to loop.
You aren't checking for end of stream at this stage.
You need a good knowledge of RFC 2616 to implement HTTP, specifically the parts about Content-length and transfer encoding.
You cannot assume that the server will close the connection after sending the response.
Closing either the input or the output stream or a socket closes the socket. This is the reason for your SocketException: socket closed.
When you get to HTTPS you will need to look at the CONNECT verb.
Flushing a socket output stream does nothing, and flushing inside a loop is to be avoided,
I got to implement a chat in my application. Connection to a server is made using sockets. I should register to that server and the server will aknowledge that with a reply.
I have implemented this in a single method where I send the command using a BufferedWriter, and then start reading from the input stream until it tells me there is no more data.
I read properly the server reply. However, I never get the negative value from the second in.read call and thus my method stays blocked in the while loop (in the conditionnal statement where I make that call).
How should this be done with sockets? I usually do that with files or other input streams without problem.
If I should read only the bytes I am supposed to read, does that mean that I either have to:
Know in advance the length of the server response?
or make the server send a code to notify it has finished to send its response?
Currently I am doing the following:
private String sendSocketRequest(String request, boolean skipResponse) throws ChatException {
if (!isConnected()) openConnection();
try {
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(
socket.getOutputStream()), 2048);
out.append(request);
out.flush();
out = null;
} catch (IOException e) {
LogHelper.error("Unable to send socket request: " + request, e);
throw new ChatException("Unable to send socket request: " + request, e);
}
try {
BufferedReader in = new BufferedReader(new InputStreamReader(
socket.getInputStream()), 2048);
StringBuffer response = new StringBuffer();
char[] buffer = new char[2048];
int charsRead = -1;
// >>>>>>>> This is where it gets blocked <<<<<<<<<
while ((charsRead = in.read(buffer)) >= 0) {
if (charsRead > 0) response.append(new String(buffer, 0, charsRead));
}
return response.toString();
} catch (IOException e) {
LogHelper.error("Unable to read socket response: " + request, e);
throw new ChatException("Unable to read socket response: " + request, e);
}
}
Connection to the server is made with the following method:
public synchronized void openConnection() throws ChatException {
try {
socket = new Socket(Constants.API_CHAT_SERVER_ADDRESS, Constants.API_CHAT_SERVER_PORT);
socket.setKeepAlive(true);
LogHelper.debug("CHAT >> Connected to the chat server: " + Constants.API_CHAT_SERVER_ADDRESS);
} catch (UnknownHostException e) {
LogHelper.error("Unable to open chat connection", e);
throw new ChatException("Unable to open chat connection", e);
} catch (IOException e) {
LogHelper.error("Unable to open chat connection", e);
throw new ChatException("Unable to open chat connection", e);
}
}
The amount of data to be sent/received over a socket based connection is protocol dependend and not known to the TCP/IP stack, but only to the application layer.
The protocol used is developer dependend ... ;-) so coming to your questions:
If I should read only the bytes I am supposed to read, does that mean that I either have to:
Know in advance the length of the server response?
Yes, this is one possibility.
or make the server send a code to notify it has finished to send its response?
Also yes, as this is another possibility. Common markers are \n or \r\n. The NUL/'\0' character also might make sense.
A third option is to prefix each data chunk with a constant number of bytes describing the amount of bytes to come.
Instead of dealing with bytes, maybe it's simpler handling instances of ad-hoc classes, like - for instance - a Message class:
The server:
// Streams
protected ObjectInputStream fromBuffer = null;
protected ObjectOutputStream toBuffer = null;
// Listening for a new connection
ServerSocket serverConn = new ServerSocket(TCP_PORT);
socket = serverConn.accept();
toBuffer = new ObjectOutputStream(socket.getOutputStream());
fromBuffer = new ObjectInputStream(socket.getInputStream());
// Receiving a new Message object
Message data = (Message)fromBuffer.readObject();
The client then sends a message by simply:
// Sending a message
Message data = new Message("Hello");
toBuffer.writeObject(data);
Message can be as complex as needed as long as its members implement Serializable interface.