I have a quick general question. I am currently using Netty to handle TCP and UDP packets coming from a client. I have the socket listeners each on a separate thread and it works great.
My concern now is, when traffic starts hitting it heavily, I don't think a single thread for each handler to manage messages will suffice. Is it correct to generate a new thread per message (I feel like it's not)? Or should I use something like a Threadpool for this? Any advice would be greatly appreciated.
Here is some example code for the message handler. I wrote some pseudo code to visualize this process. It might help you guys too.
#Override
public void messageReceived(ChannelHandlerContext ctx, DatagramPacket packet) {
//This will likely need made into a thread
//SocketDecoder.decode(packet.content().toString(CharsetUtil.US_ASCII));
//Handle decoded message, will return CharSequence
DatagramPacket response = Namespace.Decoder.createDatagram(packet, "hello");
ctx.writeAndFlush(response);
}
You need to make an educated guess.
By that I mean to say that you should consider the following programming standpoints for server sided design.
1) What does this server do? Does it just relay messages or is there processing to be done?
If there is processes that need to be done it is highly likely that handling incoming message will not be a concern as the processes may be the limiting factor.
2) How much "traffic" do you expect? What is the estimated server load at any time?
I know this is hard when you build your first server. But you need to know you target. If you think the whole world will use it you really need to prepare for it. Otherwise I would suggest just trying to test yourself.
Now as you know many game servers/messaging servers are multi-servers they have one main server and they delegate work to others which all can communicate with each other if needed. Perhaps that is what you need depending on the type of program you are after.
But really you should try to create a server to test. I would suggest a single threaded message listener/packet handler. And if you need to process something shoot off another processing thread.
Really server design principles like the only you ask only come into full force over thousands of users. And even then it heavily depends on the processes you need to run on the server.
Related
I have a requirement in my java web application where I need to send email alerts for certain conditions. For this I have used javax mail api and sending email works just fine. But the problem is the programs executions waits until the methods for sending the email are executed. As there are hundreds of email to be sent at various points ... this reduces the performance significantly.
I am using spring and have also used spring aop. Can anyone suggest me how can I separate my business logic and sending email functionality. It should be like -
Sending emails is my advice which gets executed when xyz method is called - So main execution should not wait for advice to finish its execution rather it should return back and execute further business logic thus email sending executed separately.
Here creating new threads seems obvious choice. But I think there could be some better way, is there? Thanks.
You can make the mail sending method #Async. This way Spring will execute this in a seperate thread. Read this blog post about it: Creating Asynchronous Methods
What you describe is asynchronous execution and natural way to do async execution is Java is to use threads.
You can introduce some Executor, e.g., Executors.newFixedThreadPool(), and use it to offload mailing task into separate threads.
Aspect itself is a unsuitable place for this, since this would introduce state into aspect, for example, you may want to check if mail task was successful by using returned Future:
class Mailer {
private final ExecutorService executor = Executors.newFixedThreadPool(maxMailingThreads);
//...
public void doMail(MailTask anEmail) {
Future<MailTaskResult> future = executor.submit(new MailTask(anEmail));
future.get().isSuccessful(); // handle success or failure somehow
}
Better move this logic into separate class and call it from aspect somehow.
Treat the email sending functionality like an IO device. Make it a plugin to your business logic. Do not allow any knowledge of the fact that you're even talking to the email code into your business logic. Make the email logic depend on the business logic. Never the other way around.
Here's a very good talk about this kind of architecture:
https://vimeo.com/97530863
Here's a series debating it:
https://www.youtube.com/watch?v=z9quxZsLcfo
Here's a ruby master demonstrating it with real code. We miss him.
https://www.youtube.com/watch?v=tg5RFeSfBM4
If your business rules are interesting enough to be worth respecting than this is the way to make them the masters of your application. Express them only using java. Don't accept any help. No spring, no weird annotations, just business rules. Push all that "help" out to the mail code.
Do this and your app will scale well. I think this is the best way to put it:
That's from a hexagonal architecture post. But the idea of giving your business rules a safe place to live removed from implementation detail shows up in many architectures. This answer rounds them up nicely.
Use a localhost MTA (like OpenSMTPD) and then relay to your real SMTP server, like Amazon SES ("Satellite" mode). It won't block.
I did a test, and sent 1000 emails in 2.8 seconds this way
It's simpler than doing async in java, and is useful across multiple applications.
As for separating logic, raise a Spring Application Event when needed, and make another class to listen to it, and send your email from there. Or consider something like Guava's EventBus
Consider creating a separate thread to send emails within your application. This will allow parallel execution(application+email sending).
If you would want another approach you can create a separate back end application that only sends emails. Although you will need to submit the email messages to the application. An asynchronous way to do this is to send a JMS message to the email application.
I'm trying to build a Java Bittorent client. From what I understand after peers handshake with one another they may start sending messages to each other, often sending messages sporadically.
Using a DataInputStream connection I can read messages, but if I call a read and nothing is on the stream the peers holds. Is there a way I can tell if something is being sent over the stream? Or should I create a new thread that reads the stream for messages continuously from each peer until the client shuts them down shut down?
I think you need to do some major experimenting so that you can start to learn the basics of socket I/O. Trying to answer your question "as is" is difficult, because you don't yet understand enough to ask the question in a manner that it can be answered.
If you want to be able to tell if there is data to read, then you should not use the blocking I/O approach. Instead, you will need to use the APIs known as "NIO", which allow you to "select" a socket that has data to read (i.e. a socket that is associated with a buffer that already has data in it).
This will make much more sense after you write a lot of code and mess it up a few times. The underlying I/O primitives are actually quite primitive (pun intended). In this industry, we just made up lots of complicated terms and function names and API descriptions so that people would think that network communication is magic. It's not. It's generally no more complicated than memcpy().
There is a function in C called select(). In the scenario you've described, you need an equivalent of select in Java. And that is, as cpurdy mentioned, Non-blocking Socket I/O or NIO. Cursory googling returned following links:
http://tutorials.jenkov.com/java-nio/socket-channel.html
http://www.owlmountain.com/tutorials/NonBlockingIo.htm
http://rox-xmlrpc.sourceforge.net/niotut/index.htm
You might want to take a look at the Netty project: http://netty.io/
It is very easy with Netty to get started on network programming.
I was attempting to communicate to a device server with Sockets, after having my little Java program hang when on readLine I ended up having to inject my target application with a packet sniffer and found out that os.writeBytes("notify\n"); was being split to two packets, the first containing n and the next otify, of which the server did not like. I fixed this by adding another writeBytes before hand:
os.writeBytes(" ");
os.writeBytes("notify\n");
os.flush();
This to me seems a bit hacky and potentially unstable, could someone shed some light why I'm having to do this and give me a better solution.
Cheers
When working with raw socket connections, you can never assume that you will get your messages in discrete chunks. In a production environment, its entirely possible you will receive partial messages, or multiple messages at a time.
If you don't want to deal with this, you should consider using a library like Netty which handles these concerns for the programmer.
Having said that, I agree with Thomas that your problem is probably related to your choice of writeBytes.
I am making an application that will work much like a real time chat. A user will be constantly writing on lets say a text area and messages will be send to other users. On the communications class I have set up a receiver. When a message from someone reaches the client, the receive method will be invoked and will get the message. What I can't understand is how the code will be executed. What happens if, while the user is typing/sending a message the receive message is invoked ? What do I need to do in order for this to work properly ?
Hope the question is clear enough.
ps : Im still in the design phase thats why I haven't tested it to see what happens.
Also atm I only use a second thread to receive messages which calls the receive method.
There should not be a problem at all.
When a message from someone reaches the client, the receive method
will be invoked and will get the message. What I can't understand is
how the code will be executed?
You should have a Receiver class that will encapsulate a socket (from which your receive data) and keep a set of listeners (see Observer pattern). A GUI can be one of the listeners. When a message is received via the socket, you need to notify all listeners by forwarding the data received. This way, you have a clean and nice way to notify the GUI about new messages arrivals.
What happens if, while the user is typing/sending a message the
receive message is invoked ?
This depends on the type of IP protocol you are using but in general your don't have to worry about this although I suggest you protect your sockets using lock mechanisms.
What do I need to do in order for this to work properly ?
Here is a nice example that can give you some inspiration :)
EDIT: As for your question regarding execution flow, sending and receiving are two different and uncorrelated operations that can happen at the same time. This can be achieved by implementing send and receive operations in two different threads. Here is an article on socket communications and multithreading.
You should either do what traditional Java EE app servers have done, which is assign a separate thread for processing each incoming message, or try a Java NIO solution along the lines of Netty.
I am writing the code for a server that would help two different applications in different platforms communicate with each other. To visualize it a bit it would be something like this :
App1<------>Server<------>App2
What server does is rear var1 from app2, write it to app1, then read var2 from app1 and write it to app2. Like this :
while(true){
var1 = app2stream.readInt();
app1stream.writeInt(var1);
var2 = app1stream.readDouble();
app2stream.writeDouble(var2);
}
My problem is that at some point i have this code at my server :
app1.accept();
app2.accept();
What this means is that no matter what, and given the fact that the server is always running, app1 is the one that should connect first since app1.accept() is a blocking method.
Is there any way around this? It would be great to allow the two applications to connect to the server regardless of who "came" first and then wait for the server to proceed with the above code. Can i use threads just for the accept() part and then pass the streams to the other thread? I read a bit about channels but got a bit buffled, any examples would be great.
Use NIO
It allows you to do non-blocking sockets (including accept) using the Selector class.
Basically, it gives you much more native access to the system libraries and the ability to handle your task without having to multi-thread.
Only have one accept call, and one server socket. You can make the determination which app has connected once they connect. If you can't get it from the connection details, have them send an authcode (probably a good idea anyway) which you can map to your app.
You should probably treat them both the same unless they say otherwise.
For example when the each socket connects send a "what client?" message.
Then check whether the client responds with 1 or 2.
If both respond with 1 or something just disconnect both.
I think the "standard" way to do this is to have the server listening on a port, and when a message comes in, immediately spin off a new thread to handle it, then go back to listening for another message. Then, as Glowcoder says, make all the connections in the same loop and make it figure out which is which after connecting.
I suppose the alternative is to have multiple threads, each listening on different ports. I've never tried to do that, I'm not sure if one would block until a connection was made and so you'd never get to the other thread.