I am trying to create a web server that provides web service functions, communicates with android devices using GCM, uses a database and probably also has some background work to do.
I am currently unsure on the architecture of such a server.
I know how to create a simple web service, but have not found tutorials or descriptions that go beyond simple "Hello World" examples. As far as i know i can create a class with the #WebService annotation, and once deployed to e.G. Tomcat the server will create at least one instance of this class and provide the annoteted functions in this class as web services.
Now i wonder how to best implement database connection. From what i know the server would create an instance of this class for every request, wich will be garbage collected once the connection is closed. Since the web server needs the database for nearly every function it provides i think it would not be a good idea to create a database connection for every instance, but rather use a global connection to query the database.
From what i found out so far this could be achieved by injecting a class that handles the database connection as #Resource or #Singleton into the web service class.
But is this the proper way to do this ? Or am i worring too much and just creating a new database connection for each request is fine ?
Then i want to send GCM messages (the simple POST ones using the HTML google server), so i would probably create a controller or manager class to handle these requests. Would this also be injected as a #Resource or #Singleton into the web service class ?
And last but not least the server probably has to do some work periodicaly, wich would be some kind of background thread, that is independant of the requests the web service is recieving. Here i am at a loss on how to do this. A web service does not have a main() method i am aware of, so i am unsure on how to create this.
Can anyone give me a guick overview on how to design such a web server or can point me to documentation that describe on how to achieve this ?
All i found so far were simple examples that don't cover advanced stuff like this.
Ok, so you have 3 questions/problems.
How to create a Webservice
How to manage database connections
Execute batch process
All of this, in the same App. First of all, I should advice you to split into 2 app, one for consume (Web Services) and another for batch processing, including push notifications (IMHO). But lets go one step at a time.
1. Webservice: It's depends on the framework you choose. I usually made a choice between Spring-MVC or Jersey
2. How to manage database connections When you are querying a database, you usually don't want to open and close connections crazily. You want to use a connection pool. In a connection pool you'll ask for an open connection, will use it, and will free it when you are done. Normally, a connection pool is managed por the application server. If you want to manage the connections manually, you have to use a singleton to centralise acquiring and releasing.
3. Execute batch process You probably should use a singleton to manage batch process. This job manager will launch the job executions on other threads
Don't know if that answers your question/concern.. please let me know.
Ryu,
I found myself in a similar scenario. After going through the webs for over 2 days, I stumbled upon this solution of running a background thread which is triggered during the initialization of a servlet(init method).
Perhaps you can give this a shot and let me know if it works for you.
Here's the link which has an example to try - http://docstore.mik.ua/orelly/java-ent/servlet/ch03_05.htm
Cheers!
Related
I am implementing an android based chat. I want to make it almost as low level as possible.
The reason for that is simple - i want to gain more knowledge on how things work.
I am using sockets to connect to a server. With a single socket everything is working quite well but my question is:
Will i need multiple connections when the user using the application opens multiple chat windows. If so - what is the best aproach for making those connections.
I was thinking about using something like a Util class that opens a connection when needed but i'm still not quite sure what architecture this class must have. For example would it make sense to make it a singleton class? Will i be able to keep track of all the opened connections and close them when they are no longer needed.
Any help would be appreciated.
P.S. If i missed something feel free to tell me what and i will try to edit the question to be as clear as possible.
This is clearly more of an architectural question, but I'll offer some thoughts. I would say this depends on your setup.
It sounds like you are connecting to your "contacts" directly, instead of using a central server. I assume you are determining and using the IP address directly to initiate a chat session. If this is the case, then yes, you will need to open a connection for each chat session you have going on.
If instead you are using a chat server, then theoretically, you only need one connection to that server. This server, of course, will require a connection to each user. Using a chat server requires more work on your part, but it could allow for a more user friendly experience. For example, registering your user name on the server would allow you to speak with other people via their usernames instead of having to know their IP. You still need to connect to your server via a well known IP address or DNS name, however.
As for class architecture, I highly recommend you check out something called "Dependency Injection". Dependency injection in practice usually means you interact with services and providers through an interface. The actual class that implements the interface is also written by you, and "injected" at run time. This allows you to decouple your application from a specific technology or protocol, which means that someday, you can replace your custom sockets implementation with, say, a web service implementation without having to make changes to the code that uses the service. In addition, most dependency injection frameworks allow you to specify how classes are instantiated and used when they are injected. You can use configuration to specify if one and only one class will ever be instantiated (effectively a singleton), or if a new class will be instantiated each time the service is requested.
I'm experimenting with grails in order to interface with an online trading platform.
specifically Interactive Brokers (IB) http://interactivebrokers.com/en/p.php?f=programInterface&ib_entity=llc
The way the API works is you need to have their client program, Trader Workstation (TWS http://interactivebrokers.com/en/p.php?f=tws&ib_entity=llc) running and then we consume the API to do stuff. Consuming the API basically involves creating a "broker" object, calling a connect() member function (this makes a local port connection to the TWS software) and calling something like getData()
The value of grails in this scenario are the GORM features and the web framework provided. I want to be able to define objects abstracted from db implementation, easily do persistance operations and easily provide users with a UI to do CRUD and custom actions.
My challenge is the IB API uses asynchronous communication for requests and replies. i.e. when i call getData(), the API knows to use the callback function dataResults() when it is ready to send them. In order for dataResults() to be callable, the broker object I created still needs to be around to receive the reply.
Inside a controller function, if i create a broker object and call getData(), when the request finishes, the broker object obviously also disappears. So I'll never be able to receive the reply.
I think there might be some way to do this by kicking off background threads but i'm not sure this is the path i want to go down.
Does anyone have any recommendations on what the best approach is?
I'm not married to grails, the reasons i'm using it are above. If there is a desktop app framework that I can also easily make a web interface on top of later, I'm definitely open to that.
thanks in advance.
Create your object in Service and make the Service singleton (which is by default):
static scope = "singleton"
In terms of web UI Grails is definitely a good choice.
Then, the asynchronous operations could be handled by Ajax calls as you shouldn't block the controller waiting for results.
The following [presentation][1] has some good examples
1: http://skillsmatter.com/podcast/java-jee/high-volume-scalable-ajax-with-grails
I have a local web app that is installed on a desktop PC, and it needs to regularly sync with a remote server through web services.
I have a "transactions" table that stores transactions that have been processed locally and need to be sent to the remote server, and this table also contains transactions that have retrieved from the remote server (that have been processed remotely) and need to be peformed locally (they have been retrieved using a web service call)... The transactions are performed in time order to ensure they are processed in the right order.
An example of the type of transactions are "loans" and "returns" of items from a store, for example a video rental store. For example something may have been loaned locally and returned remotely or vice versa, or any sequence of loan/return events.
There is also other information that is retrieved from the remote server to update the local records.
When the user performs the tasks locally, I update the local db in real time and add the transaction to the table for background processing with the remote server.
What is the best approach for processing the background tasks. I have tried using a Thread that is created in a HTTPSessionListener, and using interrupt() when the session is removed, but I don't think that this is the safest approach. I have also tried using a session attribute as a locking mechanisim, but this also isn't the best approach.
I was also wondering how you know when a thread has completed it's run, as to avoid lunching another thread at the same time. Or whether a thread has ditched before completing.
I have come accross another suggestion, using the Quartz scheduler, I haven't read up on this approach in detail yet. I am going to puchase a copy of Java Concurrency in Practice, but I wanted some help with ideas for the best approach before I get stuck into it.
BTW I'm not using a web app framework.
Thanks.
Safest would be to create an applicationwide threadpool which is managed by the container. How to do that depends on the container used. If your container doesn't support it (e.g. Tomcat) or you want to be container-independent, then the basic approach would be to implement ServletContextListener, create the threadpool with help of Java 1.5 provided ExecutorService API on startup and kill the threadpool on shutdown. If you aren't on Java 1.5 yet or want more abstraction, then you can also use Spring's TaskExecutor
There was ever a Java EE proposal about concurrency utilities, but it has not yet made it into Java EE 6.
Related questions:
What is the recommend way of spawning threads from a servlet?
Background timer task in a JSP web application
Its better to go with Quartz Scheduling framework, because it has most of the features related to scheduling. It has facility to store jobs in Database, Concurrency handling,etc..
Please try this solution
Create a table,which stores some flag like 'Y' or 'N' mapped to some identifiable field with default value as 'N'
Schedule a job for each return while giving loand it self,which executes if flag is 'Y'
On returning change the flag to 'N',which then fires the process which you wanted to do
I know 'normal' Java, but am new to the world of servlets, containers etc. Because of that I am not sure which approach is most sensible.
Situation: I have created a Servlet that receives information and stores it in a database. This database gets read by other applications.
Now what I need is an application that receives the exact same information and stores it in the same database. However this new application needs to pull this information from another server (I'll be using httpClient for this) instead of it being pushed to it. Both applications will co-exist.
For this new applications I see the following two options:
Make a stand alone application. For this I can copy paste a lot of the existing back-end code, but I will need to make some modifications (the servlet container offers a context, easy database connection pooling etc.) Further I might need to use some wrapper so this can work like a proper daemon that I can start, but also gracefully stop/restart etc.
Make the new application part of a Servlet. That is: just start a new Thread in the init() of the servlet that will run the new application. This would allow me to reuse all the backend code I already have, without needing to rewrite any of it. I only need to write the code that does the HTTP-GET requests to the other server. With this approach it will also be easier to start and stop the service, because I can use the Servlet container for that.
Some info about the project: the backend code that parses and writes the data to the database has a few threads, but is not very complicated. Writing the code for the original servlet was about one week of work. With the existing code base I feel this new application should probably be 1, 2 days of work max.
The way I see it option 2 is easier. But it feels a bit like I would 'abuse' servlets.
So my question is: Aren't servlets for applications that should handle requests, instead of applications that make request? Are there some huge drawbacks I don't see here? Which option would make most sense?
tl;dr: Can I write an application that doesn't serve requests as a Servlet?
Don't copy and paste code.
Write a re-usable class/module which handles storing the information in the database which can be used by both 1) the servlet and 2) standalone code which retrieves information from a HttpClient.
This way the same piece of code handles the same logic - how to store the information in the database - whether the information in question is being pushed to a servlet or being fetched from a remote URL.
Servlet containers are thread-managed environments. In general, don't start your own threads in a servlet, or bad things can happen... starting and stopping the application context, for example - the appserver doesn't know about the threads you might have started so won't stop them with your app... (More detail in this SO question)
I would try to extract the logic I need from the servlet into classes that don't depend on the Servlet API, and redesign the servlet to make use of these classes. (Refactoring). The servlet API, as you note, is all about receiving requests and sending responses.
I can re-use the logic in my new non-servlet classes anywhere I like, including a non-servlet part of the application that polls out pull that info.
You could use, but you shouldn't, it is a very poor design.
If you have two different ways to access your application (one through servlets and other as standalone), you should create at leats three classes:
One class that does all the work relating database, etc.
One servlet that calls the first class
One stand alone class (or whatever) that calls the first class
In this way, you don't copy/paste, and you can reuse your code (even you can have a third way to call the class that do the heavy work
If you want to reuse code, have this code as part of a "Service" or "Business Logic" layer that will be used both by your servlet and non servlet application.
Package the code as a jar and use it in both applications.
I currently have a tomcat container -- servlet running on it listening for requests. I need the result of an HTTP request to be a submission to a job queue which will then be processed asynchronously. I want each "job" to be persisted in a row in a DB for tracking and for recovery in case of failure. I've been doing a lot of reading. Here are my options (note I have to use open-source stuff for everything).
1) JMS -- use ActiveMQ (but who is the consumer of the job in this case another servlet?)
2) Have my request create a row in the DB. Have a seperate servlet inside my Tomcat container that always runs -- it Uses Quartz Scheduler or utilities provided in java.util.concurrent to continously process the rows as jobs (uses thread pooling).
I am leaning towards the latter because looking at the JMS documentation gives me a headache and while I know its a more robust solution I need to implement this relatively quickly. I'm not anticipating huge amounts of load in the early days of deploying this server in any case.
A lot of people say Spring might be good for either 1 or 2. However I've never used Spring and I wouldn't even know how to start using it to solve this problem. Any pointers on how to dive in without having to re-write my entire project would be useful.
Otherwise if you could weigh in on option 1 or 2 that would also be useful.
Clarification: The asynchronous process would be to screen scrape a third-party web site, and send a message notification to the original requester. The third-party web site is a bit flaky and slow and thats why it will be handled as an asynchronous process (several retry attempts built in). I will also be pulling files from that site and storing them in S3.
Your Quartz Job doesn't need to be a Servlet! You can persist incoming Jobs in the DB and have Quartz started when your main Servlet starts up. The Quartz Job can be a simple POJO and check the DB for any jobs periodically.
However, I would suggest to take a look at Spring. It's not hard to learn and easy to setup within Tomcat. You can find a lot of good information in the Spring reference documentation. It has Quartz integration, which is much easier than doing it manually.
A suitable solution which will not require you to do a lot of design and programming is to create the object you will need later in the servlet, and serialize it to a byte array. Then put that in a BLOB field in the database and be done with it.
Then your processing thread can just read the contents, deserialize it and work with the ressurrected object.
But, you may get better answers by describing what you need your system to actually DO :)