I plan to implement a GAE app only for my own usage.
The application will get its data using URL Fetch service, updating it every x minutes (using Scheduled tasks). Then it will serve that information to me when I request it.
I have barely started to look into GAE, but I have a main question that I am not able to clear. Can state be maintained in GAE between different requests without using jdo/jpa and the datastore?
As I am the only user, I guess I could keep the info in a servlet subclass and so I can avoid having to deal with Datastore...but my concern is that, as this app will have very few request, if it is moved to disk or whatever (don't know yet if it has some specific name), it will loose its status?
I am not concerned about having to restart the whole app and start collecting data from scratch from time to time, that is ok.
If this is an app for your own use, and you're double-extra sure that you won't be making it multi-user, and you're not concerned about the possibility that you might be using it from two browsers at once, you can skip using sessions and use a known key for storing information in memcache.
If your reason for avoiding datastore is concern over performance, then I strong recommend testing that assumption. You may be pleasantly surprised.
You could use the http session to maintain state between requests, but that will use the datastore itself (although you won't have to write any code to get this behaviour).
You might also consider using the Cache API (like memcache). It's JSR 107 I think, which Google provide an implementation of. The Cache is shared between instances, but it can empty at anytime. But if you're happy with that behaviour this may be an option. Looking at your requirements this may be the most feasible option, if you don't want to write your own persistence code.
You could store data as a static against your Class or in an application scoped Object, but doing that means when your instance spins down or your instance switches to another instance, the data would be lost as your classes would need to be loaded into the new instance.
Or you could serialize the state to the client and send it back in with each request.
The most robust option is persistence to the datastore - the JPA code is trivial. Perhaps you should reconsider?
Related
If I have a distributed java web application deployed in a cluster and I have say 10 servlets & 10 JSPs running the show, and if I want to share some data, say a variable or a simple POJO between all the threads of all the servlets on all the machines, what is the way to do it?
No framework like Spring/Struts is used and let's say I'm only using the basic Servlets and JSPs. Usually we think about ServletConfig, ServletContext, HttpSession and HttpServletRequest objects to store information which needs to be passed/shared from one component to another. ServletContext has the largest scope because it's accessible from all the servlets and JSPs in the web app. But in case of distributed application I guess ServeltContext object would be created one per JVM, so even for a single web app every machine in the cluster will have a different java object for ServletContext, correct? So in such a scenario what should be done to share a POJO between all the servlets on all the machines of a single web app?
If it's not possible using plain Servlets and JSP, do any frameworks make is possible? Would appreciate any inputs. Many thanks!
In a distributed architecture, it is useful to think beyond objects and think about "services". There are several possible solutions for this but all of them would include some form of service you could access from any of your 10 nodes.
So, you could for example create an 11th machine and host an API for putting and getting objects (values/maps/etc?). That would create a shareable region between the nodes.
However, this opens a whole world of possible issues if not done correctly, because you need to think about sinchronization, deadlocks, dirty reads and other concurrent processing stuff in a cross-JVM mindset.
Also, many systems sinchronize their nodes via the database, but this approach is somehow deprecated nowadays in favor of the more recent "microservices" approach where persistence is distributed, not monolithic.
you are using spring already, so maybe spring session project is a right choice for you - http://projects.spring.io/spring-session/. For sure its the easiest one to run.
You can use hazelcast, a framework as memcache but with auto-discovery for clustering . I use to used for the session and cache sharing on my Amazon cluster and works like a charm
http://hazelcast.com/use-cases/caching/
But if you want keep in simple you can always use as I said before memcached
http://memcached.org/
Sharing things between servers is:
error prone
sometimes complicated
The most common thing to want is user session data across a load balanced cluster of servers. If someone is talking to one server, then gets load balanced to a different server, you want to keep their session going. Tomcat Clusters does this, and it's already built in.
https://tomcat.apache.org/tomcat-7.0-doc/cluster-howto.html
The last time I played with that, it was touchy; don't count on session replication always working in any servlet container, and you'll be better off. Also, session replication is crazy expensive; once you're past a few machines, the cost (in RAM) of having all session data everywhere... starts to add up quickly, and you can't add more users easily anymore.
Wanting to share things between multiple JVMs is a code smell; if you can architect around it, do so. But other than clustering, you have the two normal options:
a database. Tried, true, tested; keep details that need to change there.
an in-memory store. If it gets called on every request, and/or must be really fast for whatever reason, just consider keeping it in memory; memcached is a multi-machine in-memory key-value-store that does just this.
The simplest solution is ConcurrentHashMap https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ConcurrentHashMap.html
If you want to scale your application - you will need something like hazelcast - http://hazelcast.com/
I'm currently stuck between two options:
1) Store the object's information in the file.xml that is returned to my application at initialization to be displayed when the GUI is loaded and then perform asynchronous calls to my backend whenever the object is edited via the GUI (saving to the file.xml in the process).
-or-
2) Make the whole thing asynchronous so that when my custom object is brought up for editing by the end-user it queries the backend for the object, returns the xml to be displayed in the GUI, and then do another asynchronous call for if something was changed.
Either way I see many cons to both of these approaches. I really only need one representation of the object (on the backend) and would not like to manage the front-end version of the object as well as the conversion of my object to an xml representation and then breaking that out into another object on the flex front-end to be used in datagrids.
Is there a better way to do this that allows me to only manage my backend java object and create the interface to it on the front-end without worrying about the asynchronous nature of it and multiple representations of the same object?
You should look at Granite Data Services: http://www.graniteds.org If you are using Hibernate: it should be your first choice, as BlazeDS is not so advanced. Granite implements a great facade in Flex to access backend java objects with custom serialization in AMF, support for lazy-loading, an entity cache on the flex-side with bean validation. Globally, it is a top-down approach with generation of AS3 classes from your java classes.
If you need real-time features you can push data changes on flex client (Gravity module) and solve conflicts on the front side or implement conflict resolvers on the backend.
Still you will eventually have to deal with advanced conflicts (with some "deprecated" flex objects to work with on the server: you don't want to deal with that), a basic feature for instance is to add a version field and reject manipulation of such objects on the backend automatically (many ways to do that): you will have to implement a custom way for a flex client to update itself to the current changes implying that some work could be dropped (data lost) on the flex client.
If not so many people work on the same objects on your flex application, this will not happen a lot, like in a distributed VCS.
Depending on your real-time needs (what is the frequency of changes of your java object? This is the most important question), you can choose to "cache" changes in the flex side then updating the whole thing once (but you'll get troublesome conflicts if changes have happened) or you can check everytime the server-side (granite enables this) with less conflicts (and if one happens: it is simpler) but you'll generate probably more code to synchronize objects and more network traffic.
During processing the doGet/doPost methods in my servlet, I made some Java objects. But unfortunately, all of the created objects will be gone when the application instance is shutdowned. Is there anyway that I save it and use for for later requests, beside the datastore option, which does not work well in my case because my objects are not supported properties (according to error log on google app engine).
Thank you very much.
You can use appengine memcache service. Memcache is managed independent of the application instances and is available across all application instances. But memcache is not very reliable and your data could be swiped off by appengine.
So if you can't reconstruct the objects reliably when they are not available in memcache, you may need to look at persistent storage options like datastore.
What error were you seeing in your logs?
One standard approach to this problem is to serialize your object, using either Java's Serializable facilities, or something like JSON (e.g., GSon), then stash it in the datastore.
Treating memcache as anything other than a short-term cache is risky.
We have a gwt-client, which recieves quite a lot of data from our servers. Logically, i want to cache the data on the client side, sparing the server from unnecessary requests.
As of today i have let it up to my models to handle the caching of data, which doesn't scale very well. It's also become a problem since different developers in our team develop their own "caching" functionality, which floods the project with duplications.
I'm thinking about how one could implement a "single point of entry", that handles all the caching, leaving the models clueless about how the caching is handled.
Does anyone have any experience with client side caching in GWT? Is there a standard approach that can be implemented?
I suggest you look into gwt-presenter and the CachingDispatchAsync . It provides a single point of entry for executing remote commands and therefore a perfect opportunity for caching.
A recent blog post outlines a possible approach.
You might want to take a look at the Command Pattern; Ray Ryan held a talk at Google IO about best practices in GWT, here is a transcript: http://extgwt-mvp4g-gae.blogspot.com/2009/10/gwt-app-architecture-best-practices.html
He proposes the use of the Command Pattern using Action and Response/Result objects which are thrown in and out the service proxy. These are excellent objects to encapsulate any caching that you want to perform on the client.
Here's an excerpt: "I've got a nice unit of currency for implementing caching policies. May be whenever I see the same GET request twice, I'll cache away the response I got last time and just return that to myself immediately. Not bother with a server-side trip."
In a fairly large project, I took another direction. I developed a DtoCache object which essentially held a reference to each AsyncCallback that was expecting a response from a service call in a waiting queue. Once the DtoCache received the objects from the server, they were cached inside the DtoCache. The cached result was henceforth returned to all queued and newly created AsyncCallbacks for the same service call.
For an already-fully-built, very sophisticated caching engine for CRUD operations, consider Smart GWT. This example demonstrates the ability to do client-side operations adaptively (when the cache allows it) while still supporting paging for large datasets:
http://www.smartclient.com/smartgwt/showcase/#grid_adaptive_filter_featured_category
This behavior is exposed via the ResultSet class if you need to put your own widgets on top of it:
http://www.smartclient.com/smartgwtee/javadoc/com/smartgwt/client/data/ResultSet.html
There are two levels of caching:
Caching during one browser session.
Caching cross browser sessions, e.g the cached data should be available after browser restarted.
What to cache: depend on your application, you may want to cache
Protected data for particular user
Public static (or semi-static, e.g rarely to change) data
How to cache:
For the first caching level, we can use GWT code as suggested in the answers or write your own one.
For the second one, we must use Browser caching features. The standard approach is put your data inside html (whether static html files or dynamic generated by jsp/servlet for example). Your application then use http://code.google.com/webtoolkit/doc/latest/DevGuideCodingBasicsOverlay.html techniques to get the data.
I thought Itemscript was kind of neat. It's a RESTful JSON database that works on both the client (GWT) and server.
Check it out!
-JP
My requirement is I have server J2EE web application and client J2EE web application. Sometimes client can go offline. When client comes online he should be able to synchronize changes to and fro. Also I should be able to control which rows/tables need to be synchronized based on some filters/rules. Is there any existing Java frameworks for doing it? If I need to implement on my own, what are the different strategies that you can suggest?
One solution in my mind is maintaining sql logs and executing same statements at other side during synchronization. Do you see any problems with this strategy?
There are a number of Java libraries for data synchronizing/replication. Two that I'm aware of are daffodil and SymmetricDS. In a previous life I foolishly implemented (in Java) my own data replication process. It seems like the sort of thing that should be fairly straightforward, but if the data can be updated in multiple places simultaneously, it's hellishly complicated. I strongly recommend you use one of the aforementioned projects to try and bypass dealing with this complexity yourself.
The biggist issue with synchronization is when the user edits something offline, and it is edited online at the same time. You need to merge the two changed pieces of data, or deal with the UI to allow the user to say which version is correct. If you eliminate the possibility of both being edited at the same time, then you don't have to solve this sticky problem.
The method is usually to add a field 'modified' to all tables, and compare the client's modified field for a given record in a given row, against the server's modified date. If they don't match, then you replace the server's data.
Be careful with autogenerated keys - you need to make sure your data integrity is maintained when you copy from the client to the server. Strictly running the SQL statements again on the server could put you in a situation where the autogenerated key has changed, and suddenly your foreign keys are pointing to different records than you intended.
Often when importing data from another source, you keep track of the primary key from the foreign source as well as your own personal primary key. This makes determining the changes and differences between the data sets easier for difficult synchronization situations.
Your synchronizer needs to identify when data can just be updated and when a human being needs to mediate a potential conflict. I have written a paper that explains how to do this using logging and algebraic laws.
What is best suited as the client-side data store in your application? You can choose from an embedded database like SQLite or a message queue or some object store or (if none of these can be used since it is a web application) files/ documents saved on the client using Web DB or IndexedDB through HTML 5's LocalStorage API.
Check the paper Gold Rush: Mobile Transaction Middleware with Java-Object Replication. Microsoft's documentation of occasionally connected systems describes two approaches: service-oriented or message-oriented and data-oriented. Gold Rush takes the earlier approach. The later approach uses database merge-replication.