We Keep Coding

Back end suggestions for android application

I am creating an image/graphics intensive application on android. Thus I have decided to keep images at server side and fecth them in batches when needed for each user. Apart from this I would like to manage some minor user data at backend for any future extension to the app or dynamic loading of some content.
For this I am looking out for the easiest but not a very rigid back-end solution. After some research I have boiled down to below mentioned options(In the order of priority):-
Amazon SDK for android :- It looks like this provides a lot of pre-built components but I am not sure how flexible it is when doing some custom back-end coding/feature implementation.
Parse :- Easy to understand and use but not flexible when it comes to custom feature development.
Amazon EC2 Java Backend:- I will have to do all the server side coding from scratch here but this will provide complete independence in feature implementations. Though I would love if I can find some code samples relates to user management, backend db management and java restful web services.
Any suggestions or pointers that you guys have in the above choice would be great
Thanks in advance

I have been using Parse but I haven't explored the other 2. So, this may not be a comprehensive answer but I would try to give you some pointers based on my experience with Parse.
I have been into Android development for quite some time now but I do not have any significant expertise (I would say very minimal) on the backend. Also, you mentioned you wish to work on graphics/image intensive application. As far as the application I use Parse for is more of user data and minimal images, (requiring extensive relational database).
Parse makes it really simple to create the backend structure. And the client SDK is also very powerful. Their API's are very straight-forward and doesn't require you to worry about writing complex queries, caching them and saving the data. Given my background as I mentioned above, I would say there is no learning curve involved into getting started with the dev. You can simply start building your app right away!
Also, Parse uses AWS S3 on the backend with Mongo-DB. So, I believe computation on the server side should not be a problem. Server side logic can be implemented using ParseCloud (requires some javascript). But, if you plan to write some complex algorithms, I am not very sure how much can that be done.
Documentation of Parse on Android is quite good to get through most of the dev. Extensive doc for iPhone dev.
As far as cost structure goes, it allows 1 million free API requests per month and this is very much sufficient to get through quite a number of users. In your case, the storage should be of more concern. Parse allows 1GB free and some 20 cents above per GB.
Hope this helps!

I am looking out for the easiest but not a very rigid back-end solution
Have you considered AppEngine? Here's a tutorial about how to get app engine working for you fast
You can store up to 5 GB of blob storage for free, should be more than enough for experimenting. If you go over you can pay the $0.13/GB/mo extra for blob storage, which is more than reasonable.

I don't know what kind of app you are doing, but I'll propose one approach.
Use https://imageshack.com/ for images.
Create your user saving data application with a lightweight webservice (REST+JSON)
and expose it at heroku (https://www.heroku.com/) with your prefered language/plataform.
It could be java or ruby.
Using imageshack for images will save cloud space for you and the service is quite fast.

Lotus notes agents - registering a Agent on a database

I am writing a remote Lotus/Domino NSCO (Notes CORBA API) Java client for reading and writing to a Domino server. The client should roughly be able to act and simulate all the features of Lotus Notes desktop client for a user's mailbox (Mail, Calendar, Tasks).
Those of you experienced with NSCO.jar are probably already aware of many limitations it has. One example is marking a document as read/unread, which isn't implemented with this API. To bypass this, my latest direction is writing a Java Agent on the server side, which would use Java Notes local API to mark a document as read/unread. I can later call this client using the NSCO API. I would like to be able to call this agent for every user (every user has his own database), but I dont like the idea of creating an instance of this agent on each database. According to this, my question is:
- How (if possible) can I register an Agent in Domino so that is available for every user?
Apart from this specific question, I would very much appreciate any links towards good documentation or books on this topic. (I believe I have already browsed through most of online documentation, and it's quite poor or out-of-date so books might be more useful)
Thanks.

Your question is very broad but I will attempt to answer what I can.
First, I don't think you realise the huge task you are attempting to do in simulating the Notes Client.
Much of the functionality in the front end will not be available for you, and creating back end agents to get to that functionality is going to put undue stress on the server. You are going to have to do some serious load testing to see what impact it has.
How (if possible) can I register an Agent in Domino so that is available for every user?
The proper way is to create the agent once in a template and then have the mail files update their design (Admin related help). The agent is then run within each users mail file as they need it. You also need to factor in how the agent runs. For example if you run it scheduled then AMGR may not run it as you expect it to.
If you plan to have one agent you kick off to process all databases, then you start having to deal with security of your agent.
Alternatively you can go the route of DOTS tasklets. These are OSGi bundles which can run like a service on the server.
I would very much appreciate any links towards good documentation or books on this topic.
The help within the Domino Designer client will be the most up to date on the API. The Domino Wiki will have a lot of resource material you are looking for.
Personally I think what you are attempting to achieve with just NCSO is not going to cut it. I would recommend to leverage already existing standards to talk to the server. For example POP3/SMTP/ICAL/RnR/DDS (REST API). Or use iNotes which would have much less overhead then what you are trying to achieve.

P2P file sharing application

I have developed a simple file sharing application in java using TCP socket. Now, the question is how do i make this application P2P? Can Distributed Hash Table (DHT) do that or there are other options that i can implement in my application to make it P2P? I have been trying to get ideas on this for a long time but i only get more confused. Please help.

The Forest platform which is in early development is targeting applications like your. You might want to take a look at it.
The only problem is that it is still in development and is not yet usable. I advise you to keep an eye on it and try to use it once it reaches it first release.
EDIT to answer the comment under your question:
Your application would need to connect the some peers. Depending on how you choose them (random people or know contacts), you are using respectively a peer-to-peer ('p2p') network or a friend-to-friend ('f2f') network.
Forest is providing applications a f2f network, but application are free to route messages on the top of this network to achieve (anonymous) p2p.
For a classical p2p network, you would need to connect a DHT, and you would need to bootstrap to it via a (or some) server(s) which take part in the DHT.
In general, p2p application are not so simple to do, and you will need to deal with a lot more details here and there. If you really want to experiment with them, you can go to check existing DHT libraries and see how they work. You might first want to go to read the article on Wikipedia about Kademlia which is the base for the most used DHT algorithms.

Based upon your description it sounds like you have already created a simplistic P2P application. If you are looking for the next step I would setup a DHT server (bamboo). Next, modify your original application to get a list of available files from the DHT instead of connecting directly to the other peers for file lists.

I develop a simple P2P file sharing application in java too,but i find it hard . If you write it in java . You can search for Vuze or jBittorrent , which may help you develop it faster.

Complex data-driven web application in Java - Decision on technologies

Dear Stack Overflow Community,
I am a Java programmer in front of a task of building a complex, data-driven, web application (SaaS) and I'm searching for technologies to use. I have already made some decisions and I believe I'm proficient enough to build the appliaction using just the technologies I have decided for (I'm definitely not saying it would be perfect, just that it would be working). However, I'd like to make my task easier and that's why I need your help.
Brief description of the project
Back-end
The application will be heavily data-driven, meaning that everything will be stored in a self-descripting database. This means the database itself will be entirely described with metadata and the application will not know what data it reads and writes. There won't probably be any regular entities (in terms of JPA #Entity) because the application won't know the structure of the data; it will obtain it from the metadata. Only the metadata will have a pre-determined structure. To put it simply, the metadata is the alpha-omega of the application because it will tell the application WHEN and WHAT to display and HOW to display it.
The application will probably utilize stored procedures to perform some low-level tasks on the data, such as automatical auditing, logging and translating to user's language, thus most likely eliminating any possibility to use ORM frameworks because there won't be just simple CRUD operations. Therefore, JDBC seems like my only option (doesn't it?).
Front-end
The UI will be "dumb" in terms that it will not know what data it is displaying (to some extent, of course). It will just know how to display it based on the metadata which it will obtain from the database. All UI controls (like menu items, buttons, etc) will be created based on current application's state and the UI will NOT know what the controls do. This means that clicking a menu item or a button will just send an identifier of associated action to the back-end and the server will decide what to do.
My goals
My main goal is to have the application as lightweight as possible with as least dependencies as possible. Because the application will be very complex, I'd like to avoid any heavy framework(s) because there is a very high probability that I'd need to customize a lot of its functionality.
What I have already decided for
Please object to the following decisions only if you think they're absolutely non-viable for my application, as I have already implemented some core functionality using these technologies:
Servlets on Tomcat, Guice DI, AOP (AspectJ)
I believe all of these technologies are lightweight enough and I don't need to learn J2EE.
GWT with GIN-jection on the front-end
Seems like the best option for me because I'm very familiar with Java and Swing and don't want to write any Javascript, PHP or learn a new language. GIN is a little brother of Guice and I will be using the same syntax and principles on both the client and server.
MSSQL RDBMS
This is actually a requirement from company management as I'd much rather like to go with an open-source solution. Too bad for me..
Maven 2
I think no-one can object to this :)
What I need help with
DB communication
I think that ORM is ruled out (is it?) so I need to use JDBC. Do you think Spring JDBC is lightweight and flexible enough for my use? I would often need to "blindly" read data from database, mapping it to some generic entity (because I won't assume any pre-determined structure), and then send the data using some generic DTO to the client along with the metadata telling it what data it is and how to display it. Or do you know any alternatives? Or should I do this myself?
Client/Server communication
GWT and its GWT-RPC mechanism seems not very suited for sending the generic data I need. Although I'm convinced that it's doable using GWT-RPC, are there any alternatives? But I definitely want to use GWT.
Security
Do you know any security libraries / frameworks that would help me? I'm aware of the existence of Spring-security; do you think it's flexible enough for my use or I'd be better off implementing that myself? Also, is Spring's IoC an integral part of the Spring framework, or would I be able to continue to use Guice?
Anything else that you think might be useful?
I really appreciate any advice and suggestions because I wouldn't dare to try to make such decisions myself. Please ask me if you need more information.
Thank you in advance!
eQui

I think you are over-engineering the solution. Take a look at
http://thedailywtf.com/Articles/Programming-Sucks!-Or-At-Least,-It-Ought-To-.aspx
If everything is driven by the DB, you are going to have immense difficulty making things happen in the UI, and you aren't going to be able to use many of the tools that make UI development easier.
I also suggest you take a look at Spring Roo, if your application is mainly just updating data in a database.

UI framework and implications for client/server communication
You say that any UI action will triger the backend (and potentially the DB). This mean that UI interraction will be somewhat slow anyway, and more than that will require a round trip to the server.
GWT is especially suited to avoid as much as possible round trips to the server and do all UI work on client side. In this model, only information that will transit from client to server is real data, and not UI metadata. GWT will do the job, but you'll be using a somewhat low level tool, needed for advenced optimisation you'll be unable to perform anyway...
Framework like ZK or Vaadin seems more suited to what you want to do. The client side has nice widgets with a rich UI, but you manipulate the UI from the server side. The framework manage client/server communication for you (no need of REST, RPC or javascript). The main limitation of theses framework is scalability, with all theses chatty round trip. But because your requirement impose that chatty behaviour anyway, you could really benefit from the abstraction they provide, are they are at not cost in your case.
I have tried both GWT and Zk to do some proof of concept for my company. We ended choosing GWT, because of it's hability to be embedded nicelly into any existing UI and to fine tune what you do... In particular avoid as much as possible rountrip to server. But ZK is really easier and faster in term of developmeent hours.
The side effect is that would totally solve your client/server communication concern, leting the framework performing it in an optimized way (Zk is able to intelligently regroup several UI event before sending them to server).
DB and ORM
For DB design, i tend to think that using fine granularity things in DB will make it very very slow. If each widget is one or several rows in the database you'll have to perform many lookup to perform the simpliest thing.
Problem is if your UI is just a little complex with a few dozen of elements (a few button, checkboxes, labels and widgets), compositing a screen will require lot of requests to the DB. Rendering just one page might be very slow and scalability would be very very bad.
I know this because i worked on somewhat generic bug tracking system with similar (but simpler) requirements than yours and we had exactly this problem.
So i would try to describe UI in some templating or XML format. Maybe you'll not show this data to the user, providing it with a nice abstraction, but instead of performning many queries for just one screen, you'll save the whole screen as one blob.
A really dumb and basic implementation of this would be to store HTML/CSS/PNG file in your DB and load it as needed, with user being responsible for making theses HTML file by hand. Of course this would be terrible for the user. That's why you need a nice and fancy editor UI editor that would work on an intermediary format of your own. Another dumb implementation would be some sort of wiki templating. This is not what you need, you need more. But you have the idea, I would seek in that direction...
For the maintenance and debugging too, this would be far easier to the whole UI description to a few file, to understand what is really implemented than to read lot of tabuled data in your prefered SQL editor. Users would have they export/import format to easily version, backup or experiment.
Security
I would say by hand... Because you have a generic UI generated by user it seem likely that the security will be generic too and dependant of database content.
Hope it help...

For the backend, i implemented a program which had a similar interaction with the database. the code was database structure oblivious, instead, it read a config file describing the db and could construct complex sql queries based on this information. most of the code is proprietary, but one bit of it got pushed into an open source project called sqlbuilder. may be useful to you on the backend.

I think you're on the right track, with your tool seclection. Your 100% data driven model is going to be hard to maintain. But I understand that's a requirement not an option. Normal source control is going to fail you becuase of the ui application logic all being in the meta-data. You'll need some good test databases and some way to maintain them, such as regularily mysqldump them out and check them in to souce code control to handle all the differences, etc..
You're wise to stay away from various ORM solutions and just use JDBC for this type of app.
Let me give you some warnings about GWT. On the surface it will abstract all the uglyness of html, javascript away and give you clean heirarchy's BUT...
1) If the abstraction fails you how do you easily debug?
2) Do you want any of your site to be visible to Google or other search engines, if yes GWT is not for you
3) Do you want to use any HTML5 technogies or do you want to be stuck in IE 5 compatability mode?
So...
I think you'll be much better off Implementing the UI as simple HTML controls with a small set of jQuery ajax interactions with the server. You can define an input type in your database, your serverlet can generate an input tag and then you have two options you can have some standard event bindings in jquery to tell your server that button1. is clicked, or that select2 has changed, etc.
Your server can send back javascript to change the state of the ui - simply load the javascript inside a div so it runs on the client. or 2) You let the input submit the data to the server and do an old school page refesh and the servlet build the next ui screen based on the database.
Building an interface dynamically in HTML from a database is easy and straight foward compared to doing the same in SWING or Windows Forms. You just have to write out a big text string, been doing that since 1999.
That approach is going to be much more lightweight - simpler to debug, understand and modify in the long run than going with the "GWT automatically compiles to unreadable javascript that doesn't run in my browser for some unknown reason" solution.

Need a Java compatible GIS library/data

I'm building a webapp that will need to provide some real-time tracking information overlayed on to some maps. My requirements are fairly simple:
I need to take a series of lat/long co-ordinates, and overlay simple markers at those positions with a small bit of associated textual information (ie, just a label and/or tooltip at the marker)
allow the user to zoom/pan
use open client side technologies (eg, AJAX, not flash/active-x)
a client-side control to allow them to configure an auto-refresh interval would be nice (but not necessary)
be deployable in a standard servlet container like Tomcat
by deployable in a private network (ie, can't go to a web service on the internet to fetch data)
provides basic map data (I do not have my own)
commercial or open source is fine
This question suggests GeoServer + OpenLayers, but required an open-source solution (I do not). A casual glance at those sites seems like it would be a reasonable amount of effort to get that going, and it's not clear to me that GeoServer actually comes with map data. This is key, I do not have my own map data. I want a library that provides basic map data. I don't need detailed street information, just basic geography, political boundaries and major cities is enough.
I found Dundas Maps which looks like it would serve my needs fine, however, it is for ASP.NET so it is not an option.

Geoserver doesn't come with map data, but then no map server does. Depending on which area of the world you need data for, you can check e.g. Openstreetmap, Natural Earth Data, and local authorities. You'll have to serve the data (= background map) from your own servers to meet your "by deployable in a private network" requirement.
Setting up Geoserver with an OpenLayers front-end is pretty straight-forward. Labeling and customizing markers is simlple (Geoserver uses standard SLDs). The "preview" pages Geoserver provides by default are a good basis for development of your website and Tomcat is the recommended container for Geoserver afaik.

I have worked on a inhouse project a work that provides almost all the features in ur bullets, but we use GeoExt for the UI and geoserver as our server. GeoExt provides a very rich UI using Ext and OpenLayers. Also some of our data has been take from Open street Maps.
Another option is Mapguide, an open source GIS platform.

Without knowing exactly what you are doing on the server side, I think you might find GeoServer usable. I have recently read through the licensing of GeoServer and my understanding is that you can use it as a service with unreleased software without issue which is how the software is intended. If you alter the GeoServer code and redistribute, you would have to contribute your changes.
GeoServer used in the case you are describing is different from a scenario like ExtJS where it is a part of the application. Have a look at the licensing again and look for scenarios that explain the interpretation to set your mind at ease. The company I work for uses open source GIS software quite a bit and this is how we interpret it. BTW I work with one of the contributors.

+1 for OpenLayers & GeoServer. There are other viable open source options.
If you are willing to spend money, do at least consider the commercial GIS solutions like ArcGIS Server. They are very successful, and it's not just due to marketing muscle. The deal is you pay them money, sometimes serious money, but you save developer time - which of course is also money.

You should have a look at Geomajas (http://geomajas.org/). This is a solution which is all-Java and integrates both front-end (GWT) and back-end (using amongst others Spring, Geotools, JTS,...). It is open source (AGPL), but other licenses are also available (from Geosparc).
I think it servers all your needs (though for map data, you would indeed need OpenStreetMap, Natural Earth or similar).
I know some people have worked on something similar to what you mention. There was even a talk at foss4g about display of a soccer match using Geomajas (the presentation should be online, but the foss4g site seems to be down at the moment).

Try http://www.opengts.com/ an opensource project for free GPS tracking built on Java