We have a string processing service (c++, uses stdin/out for in/output) that has different layouts, each layout runs separately (eventually will run on separate machines), each layout takes time to load, thats why it must keep running after first run.
I must implement a system with client that will ask the master server to connect it to a relevant slave server which actually runs the relevant layout service. The slave server will communicate the data passed from the client to the service, and when finished will become available on the master server for other clients.
The question is what is the best way to go about implementing the servers? Should I keep an open connection between slave/master until the process is complete to notify the master that the connection is over or keep some sort of var in a synchronized function to check that?
Any other important inputs (or other designs) I have overlooked are also very welcomed, Thanx!
Assuming you can't replace the C++ stuff, here is how I would do it off the top of my head.
I would setup one master server. That server would run a process that accepts requests (probably by HTTP, so it'd be a webservice) and I would have it read the request, parse out what it is, and then call the correct slave. Basically it acts as a proxy. Once it receives the response from the slave it forwards it back to the caller. The simplicity here means that if you start getting more of one type of request, you can set up additional servers for that and round-robin requests to them.
The slaves would be webservices that open the C++ program and forward input and retrieve output. That's all it would do.
I wouldn't bother keeping open connections (except between the slave and the C++ program based on your description). Just using a web request for this stuff will keep the connection between the master and the slave open during the process, but it shouldn't be a problem. This way you don't need to worry about this detail.
Now if I were you I would seriously look at reimplementing the C++ code in Java or calling it via JNI or something. If you can avoid it, I think avoiding the Java wrapper around C++ thing would be a good design goal. The Java could do whatever expensive process it is during start up once, and then hold things ready in memory like the C++ code does.
I hope this helps.
Depending on your scalability needs, you may want to take a look at the Java NIO package. This will give you a starting point to build a scalable, non-blocking server implementation.
Related
I need to secure the connection between my primary java app and my MYSQL server. Right now I have a class in my primary java app with the info about my SQL server (login details; user, password, schema etc).
I tried obfuscating that class but it didn't succeed. Then I heard something about calling an external java app with the connection info, and retrieve that info securely.
How can I execute such a thing?
Runtime run;
Process pr = null;
run = Runtime.getRuntime();
pr = run.exec("your program.jar");
pr.getInputStream().close();
InputStream eos = pr.getErrorStream();
and you can use a file to pass your info to the jar application
When dealing with a client/server style application, all the business logic, including the persistence layer, should be maintained on the server side.
That is, the client connects to some server process and makes requests. It should never care about how the data is managed or stored. It just cares about getting and manipulating the data. This also means that you centralise the business associated with that data, which means that should it change, you are less likely to need to change the client.
This also means that all the access information for the database never leaves the domain of the server.
Now the question is, how do you achieve. This answer will come down to exactly what it is you want to achieve an the means by which you want to achieve it, but, I would also add, the client should be authenticating with the server first, meaning that the user must be made to enter and user name and password in order to be able access the data (unless it's a public accessible API, then you probably don't care).
You could use
RMI. This would allow you to expose server objects that the client could interact with. This is good if you wish to send objects from the server to the client. It allows the client to interact with Java objects as if they were local objects.
From a coding point of view, this is a (relatively) simple solution, as you are dealing with Java Objects. The problem is though, only Java clients (with the right libraries) will be able to access the server.
You could use
Plain Sockets. This will allow you to connect to a service on the server and communicate with it.
You can even serialize objects between the client and server, allow the application to deal with Java Objects as well.
This is also a much more difficult approach, as you become responsible for dealing with the low level protocol and error handling (which RMI takes care of for you).
This approach does, however, provide you with the opportunity for other clients to connect to your server (so long as you are using just a plain sockets and serializing objects ;)).
This is a lot of work...
You could use
Some kind of web service (Servlet's under Tomcat for example or event a J2EE server), that would use simple HTTP requests to list of available services/functions that would return either something like JSON or XML response which the client would then need to parse.
This is, by far, the most open and probably the most common solution. It would take some work to get running, but is far less involved then using something like sockets and is also the most flexible, as you wouldn't need release no libraries each time you want to change or update a service.
Now all these allow you to provide secure connections over the wire, through SSL, you just need to establish the correct connection from the client to the server, so you've got an added level of security.
Each hides the database access behind a server layer, adding additional protection to the database.
I need to write an application in java which communicates with a web server.
I know how to do that, using PHP, but I'm afraid it won't cut it in this one.
Here's my situation.
I have multiple clients, when one of them sends a specific message to the server (so far, no problem on PHP), I want it to send a certain message to all other connected clients.
The problem is, I could hardly find any information regarding server socket in PHP, which led me to believe this isn't the proper way of achieving that. I'm using a paid hosting (x10premium) to host my servers so far, so I was thinking of doing it with this one, however, I'm not sure it's even possible with PHP.
At the moment I'm having each of the clients periodically check with the server if he received a message from any of the other clients, but I don't like this solution...
I hope someone could point me to the right direction. I don't know too much about Ruby and other languages which are used to do stuff like that, But if necessary, I would gladly learn it.
Thanks in advance
EDIT:
Forgot to mention, the server (currently the PHP) would also communicate with a MySQL table. If it matters.
This is a good example of PHP socket server/client:
http://tech.navarr.me/2010/07/how-to-create-a-socket-server-in-php.html
You could do it just like with JAVA, but remember that PHP does not support multithreading or multiprocessing so if 10 clients connect at once to your server, you will process them one by one, so eventually 1 will have to wait for the other 9 to finish - everything, database and connection overhead.
If you do it with JAVA or Python for example, you could handle each request in separate thread so that DB & Network communication overhead is handled simultaneosly.
Python has build in socket server components and nice and easy to use mysql component, that would make it a breez to achieve this, without even having to understand threading at all.
For the python socket server see here:
http://docs.python.org/library/socketserver.html
Basically you just define a function that will be executed for each new client connection and tell the server to serve_forever() - until it dies, it will do what you want.
I have to make an university project that involves a client/server architecture.
There should be a server where a client can login and search or save some stuff.
What's the best way to implement a stuff like that?
I think it can be done using RMI or ServerSockets or even WebServices, but what's the easiest way to implement this project?
Using Web Sevrvices i think it can be troublesome the authentication/session handling, using ServerSockets i have done some tests where i pass some custom serialized objects, but It doesnt seem to me a good way to go.
Any help is appreciated
Since this is a project for university I will not post an solution, but give you an good direction.
The most basic Way (what may be a good thing for a university project, and for understanding th whole matter...) would be with the Server listening in his Mainthread on a ServerSocket for Requests to connect to the Server and then for every (correct) Request (you need to specify somehow what is correct in this case) starting a new Thread with a Socket connected to the Client. This Threads should be hosted in some sort of List or whatever in the Mainthread of the Server...
Update:
So if this Server provides different functionalities to its clients, which are of course methods in our Server Code, you can specify the Objects which are crated when a new Client connects (I'm calling these "ClientServerConnection" from now on, and which run in its own Thread) in the Way that the Server Object is passed to it, so if one of the "ClientServerConnection"s get a Request for whatever they can call the matching method on the Server-Object and give an according response to the client...
Here some pseudo-code:
in Server:
//request for Connection came in
ClientServerConnection csc = new ClientServerConnection(this, "and everything you need, at least client IP for connecting the socket");
csc.run(); //running in its own thread, of cause ClientServerConnection should extend Thread
connectionList.add(csc); //a list of the connections the Server holds
in ClientServerConnection:
//A request to the use a functionality of the Server come in, in the easiest way you are sending a String, and than trying to match it here
if(recievedString=="doWhatever"){
Server server.doWhatever(); //calling the according method on the Server Object you passed by creation of the ClientServerConnection Object
//now return something to the client, according to whatever the Method did
}else if(recievedString=="doSomethingElse"){
//same again, according to whatever the now requested method does
}else{
//the client requested something you do not provide, need some sort of handling here
}
Hope I got you right and this helps...
'Easy' is a subjective thing, depending on what you already have experience on.
If you have experience in Java related technologies, you could pick a tech stack like Servlets, JSP and JQuery, and use GAE to keep things simple from the 'troublesome' aspects you mentioned. GAE is a platform as a service so you woudnt have to worry about those things, as google takes care of the authentication, scaling etc. You can use GAE with PHP too, if you are into that.
I think RMI is the easiest solution since you define all your interfaces and don't have to care about the protocol used to communicate.
You can also use web service with SOAP which is also a RPC (remote procedure call ) interface.
But by using Socket and ServerSocket you will learn how to build a server / client software from scratch, which is very important to know (because this is the basics).
Situation : 2 small java applications both of them connecting to remote service and sending some data there (first application listens to local socket, process data, sends it for verification to remote service and process response; second application starts on scheduled time, process some data for database and send that data to remote service).The problem is that remote service allows only one connection (that connection is SMPP session), that mean if one application is running and other application starts and try to make connection then bad things will happens...
The idea is to combine that 2 applications in 1 (maybe there are other solutions?) and create some kind of control workflow functionality which responsibility will be to manages applications to avoid collisions in connecting to remote service.
Can someone give me any advice about that idea? Maybe there is some kind of design pattern which allows me to avoid some pitfalls when I will be implementing that? (it would be even better if there are some open source applications which manages similar kind of problem so I could browse source code and gather some good information).
Thank you.
Wrap data to classes together with necessary metadata.
Place your applications to separate threads and instead of sending the data add them to a queue.
Then, in another thread, read the queue and send data from the queue to the service.
I'd give a try to BlockingQueue (http://download.oracle.com/javase/1,5.0/docs/api/java/util/concurrent/BlockingQueue.html) .
Most obvious solution is to write a simple reverse proxy server that will gather requests in to queue and send them one by one to your remote service. Or proxy could run new instance of service for each request.
http://en.wikipedia.org/wiki/Reverse_proxy
if i write the comet push with php but use this code on a java server via quercus, will that solve the one process per request problem that apache had and scale well with lot of users using my chat?
Yes, Quercus solves the one process per request Apache bottleneck. However, you need to understand the possible bottlenecks of the JVM. In my opinion, though, you should write the service or app in C/C++ using something like libevent, in Erlang, in Google Go, or simply as a Java servlet simply for portability's sake.
Well, Quercus runs on the (J)VM so it can run with other code that can start threads. But why do you need threads to do chat? You simply set the timeout on a vanilla PHP request to 0 (no timeout) and wait for there to be something to send back to the user.
That something else will be in response to someone else's request (ie A says "hello" which interrupts B's wait for something to happen). That doesn't require multithreading.
Also you could keep using Apache/PHP and do the above and instead connect to a Java (or other) service via something like XML RPC, which could wait forever. That server could do run multiple threads or do whatever it needs to.