We Keep Coding

How to handle HTTP request using Java Socket?

I am trying to implement sample HTTP server using Java socket and executor service for concurrency. However every 2nd request is failing when I run the test using JMeter with 2 or more requests or browser for example.
How to properly handle the request? Here is the sample source code:
public class Service {
public static void main(String[] args) throws Exception {
var serverSocket = new ServerSocket(8080);
var executors = Executors.newFixedThreadPool(4);
while(true) {
try {
var server = serverSocket.accept();
executors.submit(() -> {
try {
var text = "sample";
System.out.println("Waiting for client on port " +
serverSocket.getLocalPort() + "...");
System.out.println("Getting empty request");
var response = "HTTP/1.1 200 OK\r\n" +
"Content-Type: text/plain\r\n" +
"Content-Length: " + text.length() + "\r\n\r\n"
+ text;
server.getOutputStream().write(response.getBytes(StandardCharsets.UTF_8));
} catch (Exception e) {
System.out.println("Executor error:" + e.toString());
e.printStackTrace();
} finally {
try {
System.out.println("Closing server");
server.close();
} catch (Exception e) {
System.out.println("Executor error2: ");
e.printStackTrace();
}
}
});
} catch (Exception e) {
e.printStackTrace();
break;
}
}
serverSocket.close();
}
}

Your first problem lies in your response.
"HTTP/1.1 200 OK\r\n"
That allows for keep-alive, which you're not handling. A basic JMeter sampler tries to use keep alive, that is why you always fail on the second attempt.
You can change it to
"HTTP/1.0 200 OK\r\n"
That does not support keep alive, so you'll get a lot more successes with your current code. For me I only get a couple 1000 responses before JMeter has another error, but I don't know what the error is.
To support keep alive, I need to parse the request. Here is an example.
int clients = 0;
while(true) {
try {
System.out.println("Waiting for client on port " +
serverSocket.getLocalPort() + "...");
var server = serverSocket.accept();
final int client_no = clients++;
System.out.println("handling: " + client_no);
executors.submit(() -> {
int sent = 0;
try {
var is = server.getInputStream();
var os = server.getOutputStream();
var text = "sample";
byte[] tb = text.getBytes(StandardCharsets.UTF_8);
char[] buffer = new char[256];
int cr_count = 0;
while( true ){
int i=0;
int r = is.read();
if(r == -1) break;
while( r != -1 ){
char c = (char)r;
if( c == '\n' ){
cr_count++;
} else if( c != '\r' ){
cr_count = 0;
}
buffer[i++] = c;
if(cr_count == 2) break;
r = is.read();
}
//System.out.println("request: " + new String(buffer));
var response = "HTTP/1.1 200 OK\r\n" +
"Content-Type: text/plain\r\n" +
"Content-Length: " + tb.length + "\r\n\r\n";
os.write(response.getBytes(StandardCharsets.UTF_8));
os.write(tb);
os.flush();
sent++;
}
} catch (Exception e) {
System.out.println("Executor error:" + e.toString());
e.printStackTrace();
} finally {
try {
System.out.println("Closing connection!");
server.close();
} catch (Exception e) {
System.out.println("Executor error2: ");
e.printStackTrace();
}
System.out.println("sent " + sent + " responses to client " + client_no);
}
});
} catch (Exception e) {
e.printStackTrace();
break;
}
}
This will run a JMeter test for me. It can use either 1.0 or 1.1 and finish 10's of thousands of requests. If I use keep alive (1.1) each client handles many requests, if I don't use keep alive (1.0) each client handles 1 request.
If I dont read the request header, then the http 1.0 version will stop after a couple thousand requests.
It is also a separate issue from your original "dies after the second request." which is because your are not actually using HTTP 1.1 !

WebSocket in Plain Java

I am coding a Web Framework for me with an WebServer and an WebSocket Server.
My Current Problem since days is, that the Response Content of my WebSocket Client is very funny...
It sends me not the Content as bytes, every time the value is another.
Web Response for normal HTTP and the Socket request works perfectly.
My current Code:
poolIO.execute(new Thread() {
#Override
public void run() {
try {
InputStream inputIO = clientIO.getInputStream();
StringBuilder builderIO = new StringBuilder();
while (clientIO.isConnected()) {
int cacheIO = 0;
int countIO = 0;
byte[] binaryIO = new byte[0];
while ((inputIO.available() != 0 && (cacheIO = inputIO.read()) != 0)) {
binaryIO = new byte[binaryIO.length + 1];
binaryIO[countIO] = (byte) cacheIO;
builderIO.append((char) cacheIO);
countIO++;
}
if (builderIO.length() > 0) {
string(clientIO, builderIO.toString());
binary(clientIO, binaryIO);
binaryIO = new byte[0];
builderIO.delete(0, builderIO.length());
}
}
inputIO.close();
disconnect(clientIO);
this.stop();
} catch (IOException errorIO) {
if (errorIO.getMessage().equalsIgnoreCase("Stream closed.")) {
logIO.debug(GuardianLog.Type.INFO, "Client with IP " + clientIO.getInetAddress().getHostAddress() + " disconnected from Server with Port " + networkIO.getPort() + ".");
} else logIO.append(GuardianLog.Type.ERROR, "Socket throw an Error ", errorIO);
}
}
});
Regards
Jan

Fixed getting raw bytes with following Code:
byte[] bufferIO = new byte[inputIO.available()];
inputIO.read(bufferIO);
if (bufferIO.length != 0) {
binary(clientIO, bufferIO);
string(clientIO, new String(bufferIO));
}

downloading files in java in several parts or segments

I'm trying to download files in java in a multi-segment way (i.e., dividing it to several parts and downloading each part in a separate thread parallelly) but when I use the code below, it seems each thread is downloading the whole file instead of just a part of it but when it finishes, file is downloaded correctly.
note that "downloadedSizeCombined" is sum of all bytes which are downloaded by all the threads and ArrayList "downloadedSize" keeps track of bytes which are downloaded by a single thread.
this method is in class Download which extends SwingWorker.
public Void doInBackground() {
ExecutorService es = Executors.newCachedThreadPool();
for (int i = 0; i < MAX_NUMBER_OF_PARTS; i++) {
int numOfThePart = i;
es.execute(new Runnable() {
#Override
public void run() {
RandomAccessFile file = null;
InputStream stream = null;
try {
while (Download.this.getStatus() == WAITINGLIST) {
Thread.sleep(1);
}
// Open connection to URL.
HttpURLConnection connection =
(HttpURLConnection) url.openConnection();
// Specify what portion of file to download.
int startByte = numOfThePart * sizeOfFile / MAX_NUMBER_OF_PARTS;
int endByte = ((numOfThePart + 1) * sizeOfFile / MAX_NUMBER_OF_PARTS) - 1;
if (numOfThePart == MAX_NUMBER_OF_PARTS)
endByte = ((numOfThePart + 1) * sizeOfFile / MAX_NUMBER_OF_PARTS);
connection.setRequestProperty("Range",
"bytes=" + ((startByte + downloadedSize.get(numOfThePart))) + "-" + endByte);
// Connect to server.
connection.connect();
// Check for valid content length.
int contentLength = connection.getContentLength();
if (contentLength < 1) {
System.out.println("1");
}
/* Set the size for this download if it
hasn't been already set. */
if (sizeOfFile == -1) {
sizeOfFile = contentLength;
}
file = new RandomAccessFile(new File(s.getCurrentDirectory(), getFileName(url)),
"rw");
file.seek(startByte + downloadedSize.get(numOfThePart));
fileLocation = new File(s.getCurrentDirectory(), getFileName(url));
stream = connection.getInputStream();
while (status == CURRENT) {
file.seek(startByte + downloadedSize.get(numOfThePart));
byte buffer[];
buffer = new byte[MAX_BUFFER_SIZE];
// Read from server into buffer.
int read = stream.read(buffer);
if (read == -1)
break;
// Write buffer to file.
file.write(buffer, 0, read);
downloadedSizeCombined += read;
downloadedSize.set(numOfThePart, downloadedSize.get(numOfThePart) + read);
publish(numOfThePart);
while (status == PAUSED) {
Thread.sleep(1);
}
}
} catch (Exception e) {
e.printStackTrace();
} finally {
// Close file.
if (file != null) {
try {
file.close();
} catch (Exception e) {
e.printStackTrace();
}
}
// Close connection to server.
if (stream != null) {
try {
stream.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
});
}
return null;
}
Thanks in advance.

Can't we use UDP connection? So if we use DatagramSocket class, it will anyways send the data in packets. Try this.
Will get back on this soon..

Java nio read bytebuffer from channel

How to receive the data from server example "cutting" in parcels of 1024 bytes, cause when the answer comes from server in packets like 2 parts I don't know how solve that.
Ex. When the first packet arrives the size informed by the server is 1988 and received is 1444, that it's ok, but when the second packet arrives the size informed is something like 808333347 and received 540, the sum of 1444 + 540 = 1984 that is right. I don't know where this number 808333347 is coming from.
I'm googling for this solution and all then teachs using udp and I need this for a tcp/ip connection.
The class:
public class Connection implements Runnable {
private static Connection instance;
private SocketChannel channel = null;
private int port = 0;
private int service = 0;
private final int SOCKET_TIMEOUT = 15 * 1000;
private final int SOCKET_BYTES = 16 * 1024;
private final Charset CHARSET = Charset.forName("ISO-8859-1");
private String host = null;
private String message = "";
public Connection(String host, String port){
this.host = host;
this.port = Integer.parseInt(port);
}
public static Connection createConnection(String host, String port) {
if(instance == null){
instance = new Conexao(host, port);
}
return instance;
}
public void connect(){
try{
instance.channel = SocketChannel.open();
instance.channel.socket().setSoTimeout(SOCKET_TIMEOUT);
instance.channel.socket().setTcpNoDelay(false);
instance.channel.socket().setKeepAlive(true);
instance.channel.connect(new InetSocketAddress(host, port));
instance.channel.configureBlocking(false);
} catch (IOException ioe){
Log.d(TAG, ioe.getMessage() + " " + ioe.toString());
}
}
#Override public void run() {
if(null != instance.channel){
if(instance.channel.isConnected()){
Log.d(TAG, "CHANNEL CONNECTED = TRUE");
} else {
Log.d(TAG, "CHANNEL CONNECTED = FALSE");
}
} else {
instance.connect();
Log.d(TAG, "CHANNEL CONNECTED");
}
sendMessage();
while(true){
receiveMessage();
}
}
public void sendMessage() {
int size = message.length();
ByteBuffer buffer = ByteBuffer.allocate(4 + 4 + size);
buffer.putInt(service).putInt(size).put(message.getBytes());
buffer.flip();
for(int i = 0; i < size; i++){
try {
instance.channel.write(buffer);
} catch (IOException ioe) {
Log.d(TAG, ioe.getMessage() + " " + ioe.toString());
}
}
}
public void receiveMessage(){
ByteBuffer buffer = ByteBuffer.allocateDirect(SOCKET_BYTES);
int bytesReaded = 0;
String received = "";
buffer.clear();
try {
do {
bytesReaded = instance.channel.read(buffer);
} while (bytesReaded == 0);
} catch (IOException ioe) {
Log.d(TAG, ioe.getMessage() + " " + ioe.toString());
}
buffer.flip();
int size = buffer.getInt();
received += CHARSET.decode(buffer);
Log.d(TAG,"SERVIÇE: " + size + "/" + received.length() + " MSG: " + received);
}
public int getService() {
return service;
}
public void setService(int service) {
this.service = service;
}
public String getMessage() {
return message;
}
public void setMessage(String message) {
this.message = message;
}
I changed the function like this:
public void receiveMessage(){
ByteBuffer buffer = ByteBuffer.allocateDirect(SOCKET_BYTES);
int bytesReaded = 0;
String received = "";
buffer.clear();
try {
bytesReaded = instance.channel.read(buffer);
} catch (IOException ioe) {
Log.d(TAG, ioe.getMessage() + " " + ioe.toString());
}
buffer.flip();
if(bytesReaded >= 4){
if(size == 0 && size < 5000) size = buffer.getInt();
received += CHARSET.decode(buffer);
answer += received;
if(size == answer.length()){
Log.d(TAG,"SERVICE: " + size + "/" + answer.length() + " " + answer);
}
}
}
But it's very ugly now.

You can't control how data arrives on a TCP connection, from either end. It can arrive a byte at a time or in any other quanta up to the size of the data. You have to loop at the receiving end until you have everything you need. You need to use the same read buffer for the size of the socket so you can accumulate data for this loop, and to preserve any data that might belong to a subsequent message.
The 'size informed' cannot be as huge as you state. That only results from a bug. Probably you are getting out of sync.

The problem is that with TCP you have no control how the data is sent (fragmented). You do not know how much data is read with channel.read(...).
You call receiveMessage() in a loop, there you fill read data into a buffer.
You can not be sure that
int size = buffer.getInt();
is the size of the message you receive(only in the first call, if you receive at least 4 bytes.). You have to remember the first 4 received bytes, the size, (since you use getInt()), then you have to channel.read(...) until you have received size bytes, after that -> process the next message.
Also re-use your buffer. Since you use NIO (and non-blocking) i also suggest you to use select(), instead of busy reading.

Streaming chunks of audio (mp3) using Java&JSP for real-time playback through servletOutputStream

I am trying to playback audio and keep it continuous and free from skips or blank spots. I have to first receive as bytes in chunks and convert this to mp3 to be streamed by the servletOutputStream. I only start playing once enough bytes have been collected by the consumer in an attempt to maintain a constant flow of audio. As you can see I have hard coded this buffer but would like it to work for any size of audio bytes. I was wondering if anyone had come across a similar problem and had any advice?
Thanks in advance. Any help would be greatly appreciated.
public class Consumer extends Thread {
private MonitorClass consBuf;
private InputStream mp3InputStream = null;
private OutputStream OutputStream = null;
public Consumer (MonitorClass buf, OutputStream servlet)
{
consBuf = buf;
OutputStream = servlet;
}
public void run()
{
byte[] data;
byte[] tempbuf;
int byteSize = 60720; //This should be dynamic
int byteIncrement = byteSize;
int dataPlayed = 0;
int start = 0;
int buffer = 0;
boolean delay = true;
AudioFormat generatedTTSAudioFormat = getGeneratedAudioFormat();
try
{
while(true)
{
try
{
data = consBuf.get(); //gets data from producer using a shared monitor class
if(data.length >= byteSize) //Buffer size hit, start playing
{
if(delay) //help with buffering
{
System.out.println("Pre-delay...");
consBuf.preDelay();
delay = false;
}
tempbuf = new byte[byteIncrement];
arraySwap(data, tempbuf, start, byteSize);
System.out.println("Section to play: " + start + ", " + byteSize);
mp3InputStream = FishUtils.convertToMP3( new ByteArrayInputStream(tempbuf), generatedTTSAudioFormat);
copyStream(mp3InputStream, OutputStream);
System.out.println("Data played: " + byteSize);
System.out.println("Data collected: " + consBuf.getDownloadedBytes() );
dataPlayed = byteSize;
start = byteSize;
byteSize += byteIncrement;
}
if( consBuf.getIsComplete() )
{
if (consBuf.checkAllPlayed(dataPlayed) > 0)
{
System.out.println("Producer finished, play remaining section...");
//mp3InputStream = convertToMP3(new ByteArrayInputStream(tempbuf), generatedTTSAudioFormat);
//copyStream(mp3InputStream, OutputStream);
}
System.out.println("Complete!");
break;
}
}
catch (Exception e)
{
System.out.println(e);
return;
}
}
}
finally
{
if (null != mp3InputStream)
{
try
{
mp3InputStream.skip(Long.MAX_VALUE);
}
catch (Exception e)
{}
}
closeStream(mp3InputStream);
closeStream(OutputStream);
}
}
}