i am trying to upload a file into the server and storing the information of the file into an Access database, is there any need to handle the threads while database connectivity for multiple user. If yes how to do it?
Your webserver is inheritly multithreaded, that saves you from implementing you own threads to handle the uploads.
Do however make sure that multiple requests dont use same resources (dont write all uploaded file in the same tmp file,....)
To avoid problems saving the data to your db, use a Connection Pool.
So yes you need threads but if your design is good then all the threading will be handled by your frameworks
Exactly. Each HTTP request is already a thread at its own. Keep in mind that the web container will create only one servlet instance during application's lifetime and that the servlet code is been shared among all requests. This implies that any class-level variables or static variables are going to be shared among all requests. If you have such one variable, it is not threadsafe. You need to declare request-specific variables threadlocal at method-level.
As to JDBC: just write solid code and everything should go well. Using a connection pool is only useful to improve connecting performance (which is really worth the effort, believe me, connecting the DB is a fairly expensive task which may account up to at least 200ms or even more, while reusing a connection from the pool costs almost nothing). It only doesn't change anything to the threadsafety of the code you write, it's still in your control/hands. To get a clear picture of how to do the basic JDBC coding the right way, you may find this article useful.
Related
Referring to similar question :
Pattern for connecting to different databases using JDBC
I am using different Connection Strings/Drivers for each database.This is what I am doing, not very sure if it's the most efficient way to do it:
Create separate classes for each db's Connection with a getConnection(String URl,String userid,String password) method in it
In main class get connection object for DB1,DB2,DB3, open connections
Fetch data from DB1, write it to a flat file, repeat for DB2 and DB3
Close all three connections.
NOTE:I read about using Spring/Hibernate/DataSources/ConnectionPooling Dont know what shoud be the best option
The way I understand it is that you want your application to run some (SELECT?) queries on different databases and dump the results. I presume this is a part of a larger application since otherwise you would probably get results quicker by simply writing a command-line script that automates the client tools for the specific databases.
Hibernate, Data Sources (in the Java DataSource object sense) and Connection Pooling won't solve your problem - I guess it's the same for Spring but I don't know which part of Spring you're referring to. The reason for this is that they all are designed to abstract over a single (or a pool/collection of connections) to a single database - connection pooling simply allows you to keep a pool of ready-to-use (TCP) connections to a given database in order to improve performance, for example by avoiding connection and authentication overhead. Hibernate does the same in the sense that it abstracts a connection to a single database (and can use connection pooling for performance reasons on top of that).
I would suggest to maybe take a different approach to thinking about your problem:
Since you want to run some queries on some datasource and write the results to some destination, why don't you start your design this way: Come up with an interface/class DataExtractionTask that requires a database connection, a set of queries to run and some output stream. Instead of using java.sql.Connection directly you could choose some framework to make your life easier, there are heavy-weights like Hibernate and light-weights like jdbi. Then come up with code that establishes your database connection, decides which queries to run and the outputs to write to and feed all of that into your thought-out DataExtractionTask to run the logic of processing (orchestrating the individual parts).
Once you have the basic stuff in place you can add other features on top of it, you could make it configurable, you could choose to run multiple DataExtractionTasks in parallel instead of sequentially, et cetera.
This way you can generalize the processing logic and then focus on getting everything (database connections, query definitions, etc.) ready for processing. I realize that this is very broad-picture but maybe it makes things a bit easier.
Regarding efficiency: If you mean high performance (relative terms!), the best way would be what #Elliott Frisch wrote -- keeping it all in a single database that you connect to using a single connection pool.
You don't need to use separate classes just for connecting, just build up a util class which holds all the JDBC URLs and obtain a connection from it.
Besides that, you should consider using JPA instead, which you can do as well in Java SE as in Java EE. With that, you can abstract from the low level connection and define a named datasource. See for example this Oracle tutorial.
I have to create an mysql database to be used by several applications in parallel for the first time. Up until this point my only experience with mysql databases have been single programs (for example webservers) querying the database.
Now i am moving into a scenario where i will have several CXF java servlet type programs, as well as a background server editing and reading on the same schemas.
I am using the Connector/J JDBC driver to connect to the database in all instances.
My question is this: What do i need to do in order to make sure that the parallel access does not become a problem. I realize that i need to use transactions where appropriate, but where i am truly lost is in the management.
For example.
Do i need to close the connection every time a servlet is done with a job?
Do i need a unique user for each program accessing the database?
Do i have to do something with my Connector/J objects?
Do i have to declare my tables in a different way?
Did i miss anything or is there something i failed to think about?
I have a pretty good idea about how to handle transactions and the SQL itself, but i am pretty lost when it comes to what i need to do when setting up my database.
You should maintain a pool of connections. Connections are really expensive to create think on the order of of several hundred milliseconds. So for high volume apps it makes sense to cache and reuse them.
For your servlet it depends on what container you are using. Something like JBoss will provide pooling as part of the container. It can be defined through the datasource definition and accessed through JNDI. Other containers like tomcat may rely on something like C3PO.
Most of these frameworks return custom implementations of JDBC connections that implement the close() methods with logic that returns the connection to the pool. You should familiarize yourself with the details of your concrete implementation to make sure you are doing things in a way that is supported
As for the concurrency considerations, you should familiarize yourself with concepts of optimistic/pessimistic locking and transaction isolation levels. These have trade offs where the correct answer can only be determined given the operational context of your application.
Considering the user, Most applications have one user that represents the application called the read/write user. This user should only have privilege to read and write records from the tables,indices,sequences, etc. that are associated with your application. All the instances of the application will specify this user in their connection string.
If you familiarize yourself with the concepts above, you'll be about 95% of the way there.
One more thing. As pointed out in the comments on the administration side your database engine is a huge consideration. You should familiarize yourself with the differences and the tuning/configuration options.
I am writing a moderately complex Java desktop app, including an embedded database. I do not see any reason why, after the app establishes a connection to the database, why it should close the connection until the app is going to shut down.
Practically everything one does with the database requires a connection; transactions can be started and completed serially in the connection, the app is not doing anything fantastically complicated with the database.
Is there any reason why I should not create the connection and put a reference to it in a static variable in a class known and used by database-specific classes? It would save having the connection have to be passed around among all kinds of methods without ever changing value.
Is there a design-level consideration I'm missing somewhere?
rc
I would suggest using a library such as c3p0 or dbcp which handles connection pooling for you. It gives you the flexibility to scale up your application later if necessary.
Anything static usually makes it harded to write proper test cases, since you never know if the static resource has been altered or not.
Three months down the road, you're going to want to be able to connect to two databases at the same time - maybe you're doing some import / export work, or an upgrade job, or merging two customers together. And then you're going to want two of them. And now suddenly that static field everyone uses is a nightmare.
You could look into an IoC container like Guice or Spring to ensure that you can keep track of "singleton" objects without abusing static fields to enforce their "Singleton"ness.
Avoid statics. Think on concurrency and multithread issues with this kind of variables. A good point is handle your connections with a database pool. Spring is your friend to reach a simple and nice configuration
I do not see any reason why, after the app establishes a connection to the
database, why it should close the connection until the app is going to shut down.
That seems completely fine to me. It's an embedded database; it is at the service
of your application. Create the connection when you start, use it as long as you
need, shut it down when your application closes down.
I plan to implement a GAE app only for my own usage.
The application will get its data using URL Fetch service, updating it every x minutes (using Scheduled tasks). Then it will serve that information to me when I request it.
I have barely started to look into GAE, but I have a main question that I am not able to clear. Can state be maintained in GAE between different requests without using jdo/jpa and the datastore?
As I am the only user, I guess I could keep the info in a servlet subclass and so I can avoid having to deal with Datastore...but my concern is that, as this app will have very few request, if it is moved to disk or whatever (don't know yet if it has some specific name), it will loose its status?
I am not concerned about having to restart the whole app and start collecting data from scratch from time to time, that is ok.
If this is an app for your own use, and you're double-extra sure that you won't be making it multi-user, and you're not concerned about the possibility that you might be using it from two browsers at once, you can skip using sessions and use a known key for storing information in memcache.
If your reason for avoiding datastore is concern over performance, then I strong recommend testing that assumption. You may be pleasantly surprised.
You could use the http session to maintain state between requests, but that will use the datastore itself (although you won't have to write any code to get this behaviour).
You might also consider using the Cache API (like memcache). It's JSR 107 I think, which Google provide an implementation of. The Cache is shared between instances, but it can empty at anytime. But if you're happy with that behaviour this may be an option. Looking at your requirements this may be the most feasible option, if you don't want to write your own persistence code.
You could store data as a static against your Class or in an application scoped Object, but doing that means when your instance spins down or your instance switches to another instance, the data would be lost as your classes would need to be loaded into the new instance.
Or you could serialize the state to the client and send it back in with each request.
The most robust option is persistence to the datastore - the JPA code is trivial. Perhaps you should reconsider?
We have a gwt-client, which recieves quite a lot of data from our servers. Logically, i want to cache the data on the client side, sparing the server from unnecessary requests.
As of today i have let it up to my models to handle the caching of data, which doesn't scale very well. It's also become a problem since different developers in our team develop their own "caching" functionality, which floods the project with duplications.
I'm thinking about how one could implement a "single point of entry", that handles all the caching, leaving the models clueless about how the caching is handled.
Does anyone have any experience with client side caching in GWT? Is there a standard approach that can be implemented?
I suggest you look into gwt-presenter and the CachingDispatchAsync . It provides a single point of entry for executing remote commands and therefore a perfect opportunity for caching.
A recent blog post outlines a possible approach.
You might want to take a look at the Command Pattern; Ray Ryan held a talk at Google IO about best practices in GWT, here is a transcript: http://extgwt-mvp4g-gae.blogspot.com/2009/10/gwt-app-architecture-best-practices.html
He proposes the use of the Command Pattern using Action and Response/Result objects which are thrown in and out the service proxy. These are excellent objects to encapsulate any caching that you want to perform on the client.
Here's an excerpt: "I've got a nice unit of currency for implementing caching policies. May be whenever I see the same GET request twice, I'll cache away the response I got last time and just return that to myself immediately. Not bother with a server-side trip."
In a fairly large project, I took another direction. I developed a DtoCache object which essentially held a reference to each AsyncCallback that was expecting a response from a service call in a waiting queue. Once the DtoCache received the objects from the server, they were cached inside the DtoCache. The cached result was henceforth returned to all queued and newly created AsyncCallbacks for the same service call.
For an already-fully-built, very sophisticated caching engine for CRUD operations, consider Smart GWT. This example demonstrates the ability to do client-side operations adaptively (when the cache allows it) while still supporting paging for large datasets:
http://www.smartclient.com/smartgwt/showcase/#grid_adaptive_filter_featured_category
This behavior is exposed via the ResultSet class if you need to put your own widgets on top of it:
http://www.smartclient.com/smartgwtee/javadoc/com/smartgwt/client/data/ResultSet.html
There are two levels of caching:
Caching during one browser session.
Caching cross browser sessions, e.g the cached data should be available after browser restarted.
What to cache: depend on your application, you may want to cache
Protected data for particular user
Public static (or semi-static, e.g rarely to change) data
How to cache:
For the first caching level, we can use GWT code as suggested in the answers or write your own one.
For the second one, we must use Browser caching features. The standard approach is put your data inside html (whether static html files or dynamic generated by jsp/servlet for example). Your application then use http://code.google.com/webtoolkit/doc/latest/DevGuideCodingBasicsOverlay.html techniques to get the data.
I thought Itemscript was kind of neat. It's a RESTful JSON database that works on both the client (GWT) and server.
Check it out!
-JP