i've a java servlet that makes some reports. When a user choose a report it makes a query on a db and stream the xls report to the client. All in synchronous way. The problem is that sometimes i've a lot of records to fetch from the db and i would like to give a better user experience, allowing the user to do something else while the report is processing and popping out in some way the link when the process is finished. Is there a java library or some techniques to avoid the long waiting and achieve that goal?
Right now i've prepared a piece of code that in a asynchronous way completes the report and sends an email to the registered client, with the url from wich download the file, but it has to be replaced with something else because i can no longer communicate by email.
Thanks in advance
heres my take on this, i dont know of a single library that will exactly match you needs, youd probably need some custom development here.
I believe you have implemented async service that on completion sends
out an email for notification. Instead of sending out an email, let
that thread update a job table of some sort -- an entry in a db table
or some application/session scoped map.
Have a servlet/restful ws
expose that job table at some url. Poll the url at regular
intervals. Ajax poll is a standard feature in js libraries JQuery,
Prototype.
When you get a response that some report is complete, show
some popup or may be a facebook you-have-notification kind of thing
on the client side.
i have not considered authentication/authorization issues here, you need to take care of that as well.
Hope this helps
A multithreaded client server program to download my image files.
Since there are four files to download the client makes 4 connection attempts. This is not limited to 4 but the files sent by the FileServer will get repeated after the fourth attempt. The save dialog and file saving is done in different threads so as to not hamper the file downloading.
Here is the FileServer...
public class FileServer {
private final ExecutorService exec = Executors.newCachedThreadPool();
final String[] fileNames = {
"C:\\Users\\clobo\\Pictures\\Arpeggios\\Ex 1.jpg",
"C:\\Users\\clobo\\Pictures\\Arpeggios\\Ex 2.jpg",
"C:\\Users\\clobo\\Pictures\\Arpeggios\\Ex 3.jpg",
"C:\\Users\\clobo\\Pictures\\Arpeggios\\Ex 4.jpg"
};
public void start() throws IOException {
ServerSocket socket = new ServerSocket(7777);
System.out.println("Waiting for client message...");
while (!exec.isShutdown()) {
try {
for (final String fileName : fileNames){
final Socket conn = socket.accept();
exec.execute(new Runnable() {
public void run() {
sendFile(conn,fileName);
}
});
}
} catch (RejectedExecutionException e) {
if (!exec.isShutdown())
log("task submission rejected", e);
}
}
}
public void stop() {
System.out.println("Shutting down server...");
exec.shutdown();
}
private void log(String msg, Exception e) {
Logger.getAnonymousLogger().log(Level.WARNING, msg, e);
}
public void sendFile(Socket conn, String fileName) {
File myFile = new File(fileName);
if (!myFile.exists()) {
log("File does not exist!",null);
}
// file does exist
System.out.println(Thread.currentThread().getName());
System.out.println("AbsolutePath:" + myFile.getAbsolutePath());
System.out.println("length: " + myFile.length());
if (myFile.exists()) {
try {
ObjectOutputStream oos = new ObjectOutputStream(
conn.getOutputStream());
oos.writeObject(myFile);
oos.close();
} catch (IOException e) {
log("IOException Error", e);
e.printStackTrace();
}
}
}
public static void main(String[] args) throws IOException {
FileServer fs = new FileServer();
fs.start();
}
}
here is the FileServerClient...
public class FileServerClient {
private final ExecutorService exec = Executors.newCachedThreadPool();
Frame myFrame = new Frame();
List<File> fileList = new ArrayList<File>();
public void receiveFileFromServer() throws Exception{
Socket sock = null;
InputStream socketInputStream = null;
String host = "localhost";
int port = 7777;
for (int i=0;i<4;i++) {
sock = new Socket(host, port);
socketInputStream = sock.getInputStream();
System.out.println("Connection successful...");
// recieve the file
ObjectInputStream ois = new ObjectInputStream(socketInputStream);
// file from server is deserialized
final File myfile = (File) ois.readObject();
fileList.add(myfile);
// deserialized file properties
System.out.println("AbsolutePath: " + myfile.getAbsolutePath());
System.out.println("FileName:" + myfile.getName());
System.out.println("length" + myfile.length());
exec.execute(new Runnable() {
public void run() {
saveFile(myfile);
}
});
}
}
private void saveFile(File myfile) {
FileDialog fileDialog = new FileDialog(myFrame,
"Choose Destination for "+ myfile.getName(), FileDialog.SAVE);
fileDialog.setDirectory(null);
fileDialog.setFile("enter file name here");
fileDialog.setVisible(true);
String targetFileName = fileDialog.getDirectory()
+ fileDialog.getFile() + ".jpg";
System.out.println("File will be saved to: " + targetFileName);
copyBytes(myfile, targetFileName);
}
private void copyBytes(File originalFile, String targetFileName) {
try {
FileInputStream in = new FileInputStream(originalFile);
FileOutputStream out = new FileOutputStream(targetFileName);
int c;
while ((c = in.read()) != -1) {
out.write(c);
}
out.close();
in.close();
} catch (Exception e) {
log("IOException Error", e);
}
}
private void log(String msg, Exception e) {
Logger.getAnonymousLogger().log(Level.WARNING, msg, e);
}
public static void main(String[] args) throws Exception {
FileServerClient client = new FileServerClient();
client.receiveFileFromServer();
}
}
You could make an asynchronous request from the client. Lets assume that you client is an html page. When the user selects a report and clicks on 'submit' you could fire an ajax request with the report parameters (jquery can be useful for this). It would be good to keep a section on the user homepage that says something like 'prepared reports'. The client can then goto the prepared report section to download the report. As specified in the comments above, you may also have to implement a popup that informs the user that the requested report is ready. the popup is shown when the ajax requests returns successfully. However, the client may have logged out by the time the report finishes, so it may be a good idea to make the download link available again in the 'prepared reports' section when the user logs in.
Related
I'm trying to build a server client application using jetty. I have setup a jetty server and configured websockets. Sending text messages works fine between client and server. But how binary data as inputstream could be sent from client endpoint. I cannot find any snippets regarding websocket client. Below is what i have tried
ServerEndPoint:
#OnMessage
public void handleBinaryMessage(InputStream input, Session session) {
logger.info("onMessage::inputstream");
try {
byte[] buffer = new byte[2048];
try (OutputStream output = session.getBasicRemote().getSendStream())
{
int read;
while ((read = input.read(buffer)) >= 0)
output.write(buffer, 0, read);
}
} catch (IOException e) {
e.printStackTrace();
}
ClientEndpoint:
#OnOpen
public void onOpen(Session s) throws IOException {
logger.info("Client Connected ... " + s.getId());
this.session=s;
session.getBasicRemote().sendText("Ping from client");
// size of the file 200~500MB
File source= new File("/tmp/Setup.exe");
try(InputStream input = new FileInputStream(source)) {
session.getAsyncRemote().sendObject(input);
}
}
Any help is appreciated
EDIT:
I have modified clientendpoint and serverendpoint. Trying to send data as chunks but the zip file is partial or sometimes even very smaller than source file.
source size : 1.5gb
after writing data from buffer using stream : 20kb
#ClientEndpoint
private static void sendFileToRemote(File source) throws FileNotFoundException, IOException {
// TODO Auto-generated method stub
Session session=null;
final WsClient wc = new WsClient("ws://localhost:2714/events/","test");
session=wc.getSession();
try (
InputStream inputStream = new FileInputStream(source);
) {
byte[] chunk = new byte[102400];
int chunkLen = 0;
while ((chunkLen = inputStream.read(chunk)) != -1) {
session.getAsyncRemote().sendBinary(ByteBuffer.wrap(chunk, 0, chunkLen));
}
} catch (IOException ex) {
ex.printStackTrace();
}
}
#serverendpoint
#ServerEndpoint("/events/")
public class EventListener {
static Logger logger = Logger.getLogger(Initservice.class.getName());
private OutputStream os=null;
#OnOpen
public void Init(Session session) throws FileNotFoundException {
this.user_session = session;
logger.info("onOpen:: server" + session.getId());
this.os = new FileOutputStream(new File("/tmp/silicon_test.zip"));
logger.info("instantiate zip files");
}
#OnMessage
public void onMessage(Session session, ByteBuffer byteBuffer) throws IOException {
try {
os.write(byteBuffer.get());
} catch(Exception e) {
close();
logger.log(Level.SEVERE,"Exception occured in onMessage :: ", e);
throw e;
}
}
}
The code in your ServerEndpoint looks like it should work fine, however in your ClientEndpoint you are only sending text data to the ServerEndpoint and this can only be read by a server onMessage method configured to receive text messages.
Instead of using session.getRemoteEndpoint().sendText(...) you should use the method session.getRemoteEndpoint().sendBinary(...). This will send the data in binary frames instead of text frames and you will be able to receive it in your servers handleBinaryMessage method.
As for the session.getAsyncRemote().sendObject(input), to make this work you will also need to provide an Encoder.Binary or Encoder.BinaryStream in order to send the object as binary data.
Edit:
WebSocket is a message based protocol and you are sending your data from the file over multiple websocket messages. You could use session.getBasicRemote().sendBinary(ByteBuffer, boolean) to send partial messages and send all the data in the same message.
Or you could try something like this code which may be simpler.
try (InputStream inputStream = new FileInputStream(source))
{
try (OutputStream sendStream = session.getBasicRemote().getSendStream())
{
inputStream.transferTo(sendStream);
}
}
I have Java client which connects to my Java server. Java clients are in different networks and are running on MTX-GTW (Embedded Linux).
When I start server and the clients everything works fine, and clients are sending data every 1 minute. But after a day or more, clients will stop sending data one by one. Time varies.
But rest of the program runs fine, since program uses HTTP to communicate with some API and there we are still receiving data.
I checked server debug output and I can't see any errors or exceptions. I tried restarting the server and it also didn't help. My next step will be, to have similar client on my PC, so that I can see debug log, but that can take some time. So would any of you have any idea what could be the problem?
I use Java 7, here I call method to open socket:
static private void createHomeCallTimer()
{
new java.util.Timer().schedule(
new java.util.TimerTask()
{
public void run()
{
log.info("homeCall run");
Main main = new Main();
String data = "xxxxx";
try
{
log.info("Start of HOMECALL with data: " + data);
new TCPClient().openSocketAndSendData(data);
createHomeCallTimer();
} catch (Exception e)
{
log.error("Exception on homeCall: " + e);
createHomeCallTimer();
}
}
},
HOMECALLTIME
);
}
And this is client which is called by that method:
public class TCPClient
{
public void openSocketAndSendData(String data) throws IOException
{
Logger log = Logger.getLogger(TCPClient.class);
String ip = "xx.xx.xx.xx";
int port = 9000;
Socket clientSocket = new Socket(ip, port);
log.info("SOCKET TO IKU SERVER OPENED");
DataOutputStream outToServer = new DataOutputStream(clientSocket.getOutputStream());
BufferedReader inFromServer = new BufferedReader(new InputStreamReader(clientSocket.getInputStream()));
log.info("IKU SERVER: " + inFromServer.readLine());
outToServer.writeBytes(data);
clientSocket.close();
log.info("SOCKET CLOSED");
}
}
After you said that device has really small memory, then here's my suspect.
I didn't run your code but stripped it a little and speed it up to see what happens.
Here's code:
public class Main {
private static final int HOMECALLTIME = 10;
static private void createHomeCallTimer() {
new java.util.Timer().schedule(new java.util.TimerTask() {
public void run() {
// log.info("homeCall run");
// Main main = new Main();
String data = "xxxxx";
System.out.println(data);
// log.info("Start of HOMECALL with data: " + data);
// new TCPClient().openSocketAndSendData(data);
createHomeCallTimer();
}
}, HOMECALLTIME);
}
public static void main(String[] args) {
createHomeCallTimer();
}
}
And here's output after few minutes:
Exception in thread "Timer-21424" java.lang.OutOfMemoryError: unable
to create new native thread
at java.lang.Thread.start0(Native Method) at
java.lang.Thread.start(Thread.java:714) at
java.util.Timer.(Timer.java:160) at
java.util.Timer.(Timer.java:132) at
pkg.Main.createHomeCallTimer(Main.java:13) at
pkg.Main.access$0(Main.java:12) at pkg.Main$1.run(Main.java:22) at
java.util.TimerThread.mainLoop(Timer.java:555) at
java.util.TimerThread.run(Timer.java:505)
I suspect recursive call prevents freeing up memory and you run out of memory on your device. That's just a suspicion, but doesn't quite fit in a comment.
Here's the same code without recursion, using Timer:
static private void createHomeCallTimer() {
new java.util.Timer().scheduleAtFixedRate(new java.util.TimerTask() {
public void run() {
// log.info("homeCall run");
// Main main = new Main();
String data = "xxxxx";
System.out.println(data);
// log.info("Start of HOMECALL with data: " + data);
// new TCPClient().openSocketAndSendData(data);
}
}, 0, HOMECALLTIME);
}
What I'm trying to achieve:
I'm trying to make a very simple camera surveillance system. In this case, the camera will be the server, and there will be a client application to watch the video feed on the other end.
For simplicity sake, I will emulate the camera by capturing frames from a saved video file, and sending these frames one by one through sockets to all the connected clients (Yes, the camera can handle more than one client). On the client side, I will receive the frames and then I will display them in a jPanel one after another to create the effect of a video playing.
I've already done all that but it works only for a couple of frames then it suddenly stops without an exception.
The Server-Side:
This is the main function in the Camera class:
public static void main(String[] args) throws InterruptedException, IOException, RemoteException, AlreadyBoundException {
ServerSocket ssock = new ServerSocket(1234);
System.out.println("Listening");
Camera.getInstance().startCamera(); // Starts reading the frames from the video file
while (true) {
Socket sock = ssock.accept();
System.out.println("Connected");
ClientConnection con = new ClientConnection(sock); // Creates a new connection
// Runs the connection on it's own thread
Thread conThread = new Thread(con);
conThread.start();
// Keeps a reference to the connection so it can be used later to send frames
Camera.getInstance().connections.add(con);
}
}
Snippets from the ClientConnection class:
The constructor:
public ClientConnection(Socket csocket) throws IOException {
this.csocket = csocket;
outStream = new PrintStream(csocket.getOutputStream());
objectOutStream = new ObjectOutputStream(csocket.getOutputStream());
}
The ClientConnection class implements the runnable interface so it can work on a separate thread. The run method will be responsible for receiving predefined messages (ex. "SET_MOVIE") from the client and do some actions accordingly. These actions and what they do are irrelevant to the question so we can safely ignore them. Here is the run method:
#Override
public void run() {
try {
inStream = new Scanner(csocket.getInputStream());
String msg;
while (inStream.hasNext()) {
msg = inStream.nextLine();
if (msg.equals("SET_MOVIE")) {
setMovie();
} else if (msg.equals("SET_IDLE")) {
setIdle();
} else if (msg.equals("FORCE_STATE_ON")) {
forceStateOn();
} else if (msg.equals("FORCE_STATE_OFF")) {
forceStateOff();
} else if (msg.equals("DISCONNECT")) {
// TO-DO
}
}
} catch (IOException ex) {
Logger.getLogger(ClientConnection.class.getName()).log(Level.SEVERE, null, ex);
}
}
This is the sendFrame method in the ClientConnection class. It gets called every time a new frame is available and ready to be sent.
// SEND_FRAME here works as an indicator to the client so that it can expect
// the image and start reading it
public void sendFrame(Frame _frame) throws IOException {
outStream.println("SEND_FRAME"); //tells the client there is a new frame
outStream.println(_frame.getCaptureTime()); //sends the time in which the frame was captured
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
ImageIO.write(_frame.getFrame(), "jpg", byteArrayOutputStream);
byte[] size = ByteBuffer.allocate(4).putInt(byteArrayOutputStream.size()).array();
outStream.write(size);
outStream.write(byteArrayOutputStream.toByteArray());
outStream.flush();
}
The Client-Side:
This is the main method, it simply creates a new CameraConnection and run it on it's own thread.
public static void main(String[] args) throws InterruptedException, IOException {
Thread client = new Thread(new CameraConnection("Cam_1", 1234));
client.start();
}
This is the CameraConnection constructor:
public CameraConnection(String name, int port) throws IOException {
this.name = name;
clientSocket = new Socket("localhost", port);
// This scanner will be used to read messages sent from the server
// such as "SEND_FRAME"
inStream_scanner = new Scanner(clientSocket.getInputStream());
// This inputStream will be used to read the bufferedImage in a array of bits
inStream = clientSocket.getInputStream();
// This is the outStream used to send messaages to the server
outStream = new PrintStream(clientSocket.getOutputStream());
}
This is the run method inside the CameraConnection:
#Override
public void run() {
String msg;
while (inStream_scanner.hasNext()) {
// Stores the incoming message and prints it
msg = inStream_scanner.nextLine();
System.out.println(msg);
// Irrelevant
if (msg.equals("NOTIFY_MOTION")) {
onMotion();
}
// Here is where the image gets read
else if (msg.equals("SEND_FRAME")) {
Frame f = new Frame();
long capturedTime = inStream_scanner.nextLong();
try {
byte[] sizeAr = new byte[4];
inStream.read(sizeAr);
int size = ByteBuffer.wrap(sizeAr).asIntBuffer().get();
byte[] imageAr = new byte[size];
inStream.read(imageAr);
BufferedImage image = null;
image = ImageIO.read(new ByteArrayInputStream(imageAr));
long receivedTime = System.currentTimeMillis();
// Prints out the image dimension and the time in which it was received
System.out.println("Received " + image.getHeight() + "x" + image.getWidth() + ": " + receivedTime);
f.setCaptureTime(capturedTime);
f.setFrame(image);
f.setRecievedTime(receivedTime);
} catch (Exception e) {
System.out.println(e.toString());
}
}
}
}
The Output:
As mentioned above, it works fine for a couple of frames then it stops without an exception, also the scanner from the inputStream starts reading and printing weird symbols on the console as if it's corrupted. It keeps printing these weird symbols as long as the server keeps sending the frames. Here is an image to the output:
screenshot from the output
You can't mix two types of stream or reader or writer on the same socket. Buffering will foul you up completely. You need to use the object streams for everything.
You can't assume that read() fills the buffer.
For reading a 4-byte integer you should use readInt() (and writeInt() for writing it), not home-grown code.
For reading the body of the image you should use readFully().
I don't see any need for object streams here: you should use DataInputStream and DataOutputStream.
I need to write a server for stock exchange in Java, the server will get many requests from couple of servers per second, my goal is to write this server and find out how many HTTP GET requests it can handle after executing couple of conditions and send a response in less then 100ms.
Each request contains a token that will be valid for 100ms.
My goal is to fetch data from cache (Redis/Memcached) using the request parameters, execute couple of O(1) conditions, send a response to the same server with the token and a boolean answer in max 100ms and save the request and response in a database (asynchronously).
This server will sit on AWS ec2 at the same region as the requesting servers.
Its needs to be written in Java, low level as it should be, btw, I come from Python/NodeJS world for the past 2 years.
I think its a classic producer-consumer design pattern.
Can anyone guide me to the technical parts ? like.. use "this' for handling requests (SocketInputStream), and use "that" for handling the queues, or maybe use "OpenMQ" framework ? monitor requests with "this" ? any reference for an implementation for similar problem ?
Update 1:
Thanks #mp911de, I saw LMAX (Disruptor), someone already invented the wheel but it feels like over-programming for now, I want to check how many 1k json object can be sent and response 100bytes back from simple ec2-server that run this java code I attached ? BTW, how many threads to put in Executors.newFixedThreadPool ?
public class JavaSimpleWebServer {
private static final int fNumberOfThreads = 100;
private static final Executor fThreadPool = Executors.newFixedThreadPool(fNumberOfThreads);
public static final int PORT = 81;
public static void main(String[] args) throws IOException {
ServerSocket socket = new ServerSocket(PORT);
while (true)
{
final Socket connection = socket.accept();
Runnable task = new Runnable() {
#Override
public void run() {
HandleRequest(connection);
}
};
fThreadPool.execute(task);
}
}
private static void HandleRequest(Socket s) {
BufferedReader in;
PrintWriter out;
String request;
try {
String webServerAddress = s.getInetAddress().toString();
System.out.println("New Connection:" + webServerAddress);
in = new BufferedReader(new InputStreamReader(s.getInputStream()));
request = in.readLine();
System.out.println("--- Client request: " + request);
out = new PrintWriter(s.getOutputStream(), true);
// do some calculation
out.println(jsonResponse);
out.flush();
out.close();
s.close();
}
catch (IOException e) {
System.out.println("Failed respond to client request: " + e.getMessage());
}
finally {
if (s != null)
{
try {
s.close();
}
catch (IOException e) {
e.printStackTrace();
}
}
}
}
}
In my program I want to send some information to another computer with sockets. The first socket send some text throw the server and when the second socket receive this information it send a answer to the first socket. The problem is that another thread receive the answer and the only thing which I get is deadlock.
This is the server side:
else if(msgArray[0].compareTo("game_request") == 0){
if(userMap.containsKey(msgArray[1])){
Socket socketPlayerTwo = userMap.get(msgArray[1]);
otherPlayer = msgArray[1];
sendResult(socketPlayerTwo, "game_offer?"+username);
Boolean willPlay =Boolean.valueOf(recieveRequest(socketPlayerTwo).split("?")[1]);
if(willPlay)
playingUser.put(username,msgArray[1]);
sendResult(socket, "game_accept?"+willPlay);
}
}
This is the client side:
private void showGameRequest(String usernameOther) {
try{
int status = GameWindow.getInstance().showNotification("Game Offer","The user " + usernameOther + " offers you a game!\nDo you agree?",SWT.ICON_QUESTION|SWT.YES|SWT.NO);
if (status == SWT.YES){
otherPlayer = usernameOther;
sendRequest("send_message_to_user?user="+usernameOther+"&value=true");
GameWindow.getInstance().getDisplay().asyncExec(new Runnable() {
#Override
public void run() {
GameWindow.getInstance().startNewGame();
}
});
}
else
sendRequest("send_message_to_user?user="+usernameOther+"&value=false");
}
catch (IOException exc){
}
}
Here is the sendRequest method:
private void sendResult(Socket socket,String request) throws IOException{
DataOutputStream writer = new DataOutputStream(socket.getOutputStream());
writer.writeUTF(request);
writer.flush();
}
The client socket is created in the class Server
while (true) {
try {
Socket socket = serverSocket.accept();
new GameThread(socket,databaseManager);
} catch (IOException e) {
e.printStackTrace();
}
}
and it is put in hashmap when the user pass the login level:
if(msgArray[0].compareTo("user_info") == 0){
Integer flagUser = -1;
String[] userInfo = {msgArray[1].substring(msgArray[1].indexOf("=") + 1, msgArray[1].indexOf("&")),msgArray[1].substring(msgArray[1].lastIndexOf("=")+ 1, msgArray[1].indexOf(";"))};
Boolean status = databaseManager.checkUser(userInfo[0], userInfo[1]);
if(status){
if(!userMap.containsKey(userInfo[0])){
userMap.put(userInfo[0], socket);
username = userInfo[0];
flagUser = 0;
}
else
flagUser = 1;
}
sendResult(socket, "user_info_status?"+flagUser.toString()+"");
}
I thing I know what is the reason of the deadlock but I can't solve it. When the first user send the information to the other user he wait for response. Of course the other user send a response but this response is handle from other thread. So the deadlock is from a read method which block the thread. How can I send information from one socket to another without deadlock?
public GameThread(Socket socket, DatabaseManager databaseManager) {
this.socket = socket;
this.databaseManager = databaseManager;
parser = new RequestParser();
authorizationControl = new AuthorizationControl(databaseManager);
communication = new SocketCommunication(socket);
start();
}
Could you show us more of your code?
sendResult(socketPlayerTwo, "game_offer?"+username);
recieveRequest(socketPlayerTwo);
sendRequest("send_message_to_user?user="+usernameOther+"&value=true");
for starters.
Judging by the little i'm seeing you have a problem with the order in which you're using them. As far as i can tell you have a inputSteram.readObject() method that has blocked somewhere waiting for a message from the other side.