In the context of a university project, we have to develop a Java distributed application with these requirements:
The application will follow the classic client-server schema, with
multiple clients connecting to a central server on a different
machine, which also hosts a rdbms to which the server connects
The relational database we must use is postgresql (latest version)
Both client and server must be written in Java
We must use native JDBC to access the database (we can't use frameworks like Spring)
DISCLAIMER:please understand we are just a group of students and this is our first big project involving all these aspects and we aren't experts by any means, so please be patient with us :) (also English is not our first language, sorry for any mistake you might find)
We are currently in the design phase of the application (class diagrams, sequence diagrams etc) and we're stuck with a possible concurrency problem with the database:
ideally our server would listen for any requests and for each client that logs in the application, the server launches a dedicated thread that provides implemented services to the user (implementing pattern proxy-skeleton with basic socket programming). Each of these service providers (threads), upon completing the requested task should update/insert/delete data to the database. Here is the problem: how should we manage the concurrency here?
We tried to search the internet for this kind of issue and we found some things but we're still very confused:
Since we actually interact with the database from one single central
server (with one admin profile) we could implement a queue sistem for
the various transactions coming from the different threads we
launched
We manage concurrency at database level with some well-known mechanism such as MVCC, which is apparently a lot more complicated
Ideally we would like that read requests don't block other reads or writes, and writes only block other writes (which seems to be the case with MVCC). Which alternative would be best? Are there any other options that we could implement with the restrictions above mentioned? Thanks in advance for any suggestion
Related
I have 3 different applications
ASP.NET web application
Java Desktop application
Android Studio mobile application
These 3 applications have the same database and and they need to connect from any part of the world with an internet connection. They share almost all the information, so, if you move something in one application it has to update the information in the other 2 applications.
I have the database on a physical server and I want to know how best to make this connection.
I have searched but I couldn't find if I have to connect directly to the server with some SQL Server, using Web Service, or something like that.
I hope someone could help.
Thank you.
I believe the best way is to first create a Web API layer (REST/SOAP) that will be used to perform all the relative operations in the centralized DB. Once that is setup, any of your applications written in any language can use the exposed web API methods to manipulate the data of the same DB.
If you are looking at a global solution - will you have multiple copies of the applications in different parts of the world as well?
In this scenario you should be looking at a cloud-hosted database with some form of geo-replication so that you can keep latency to a minimum.
There are no restrictions on the number of applications that can connect to a specific database - you do not have to create a different database for each and you may be able to reuse Stored Procedures between applications if they perform the same task.
I would however look at the concept of schemas - any database objects that are specific to one app should be separated from other - so put them in a schema for "App1". Shared objects can be in a shared schema.
I'm developping an Android application for offering and claiming certain items (sort of a market) for a course that mainly focusses on user-centered design and the UIs (which mostly means we have to quickly implement all the logic and management underneath).
It's my first application that effectively needs to be shipped. So now I wonder what are the best strategies and best decisions regarding the R DBMS (because this area seems a very dense forest for some newbies) ?
The application will not target >1000 users in total
Remotely accessible
Transactions and ACID properties need to be supported
SQL queries must be possible
Secondly the DMBS must be contacted remotely. So I wonder if there's a sort of plug and play (blackbox) solution available for setting up the backend (free/cheap, few implementations, ...).
scalability, availability and security aren't really issues (at the moment)
First you need to decide how the android devices will talk to the central database. Options, from easiest to hardest:
Direct JDBC connection (over TCP). Some networks might block direct TCP connections. No offline capability.
Tunnel JDBC over HTTP(S). No offline capability.
Run a local database on Android (Sqlite) and synchronize to remote database when can (allows for offline). See SymmetricDS.
Put REST API in front of remote database and use android's built-in syncing capability.
Second you need to decide how to segregate remote database data between users:
No segregation, users can do what they want to others data.
Database segregation, aka Row-Level Security
Application segregation (no segregation in database, but control segregation in say your REST API)
You need database segregation if choosing options A, B or C from question 1.
How many concurrent writers for remote database?
PostgreSQL is a great choice these days, but MariaDB might be a better choice if you require row-level security as it supports the WITH CHECK OPTION clause, which isn't coming in PostgreSQL until the summer.
Not entirely sure what you are specifically asking because you can get a lot of info just by googling a few things. But if you might want to look at couple of google options. I haven't tried all of these so grain of salt:
App Engine: https://cloud.google.com/developers/articles/how-to-build-mobile-app-with-app-engine-backend-tutorial
Google Drive: http://developer.android.com/google/play-services/drive.html
and more generally, storage options from the Android docs: http://developer.android.com/guide/topics/data/data-storage.html#netw
I need to implement a proof of concept application Swing application where there's a server having a list of users and several clients which connect to the server and do CRUD operations on the database and hence on the list of users.
I have an obvious synchronization dilemma of keeping all clients lists updated so that if one client removed a user another one who still has it in his list cannot change its name.
Now I know a protocol in which before updating a user the client asks the server whether it still exists would work.
However this is just a simple example but in the real application I might have junction tables and complex references between objects which need always to be kept consistent and I don't want to reinvent the wheel.
What I'm wondering if there's some ready made solutions or some library which does this job which doesn't require me to change database or load extremely complex dependencies.
I did some research on Google but nothing seems to fit and the most similar example of client server synchronization I found was "chat programs", however chat programs are inherently simple because a message is never modified or deleted and all you have to keep consistent is the chronological order. I would need something more involved than that or some useful hints on the subject.
What you need is some sort of messaging between server and clients. Clients will either long-poll the server, asking for the updates, or subscribe to some streaming endpoint on your server.
You can take a look on Comet model, long-polling itself, websockets, etc.
Alternatively - there are couple of data management servies - BlazeDS, GraniteDS or any similar purpose solution. You can integrate one of those in your application and use for complete data management cycle.
We're trying to design a new addition to our application. Basically we need to submit very basic queries to various remote databases accessed over the internet and not owned or controlled by us.
Our proposal is to install a small client app on each of the foreign systems, tiered in 2 basic layers, 1 that is tailored to the particular database its talking to, to handle the actual query in SQL or whatever, the other tier would be the communication tier to handle incoming requests and send back responses. This communication interface would be the same over all of the foreign systems, ie all requests and responses have the same structure.
In terms of java remoting I guess this small client app would be the 'server' and our webapp (normally referred to as the server) is the 'client'.
I've looked at various java remoting solutions (Hessian, Burlap, RMI, SOAP/REST WebServices). However am I correct in thinking that with all of these the 'server' must run in a container, ie in a tomcat/jetty etc instance?
I was really hoping to avoid having to battle all the IT departments controlling the foreign systems to get them to install very much. The whole idea is that its thin/small/easy to install/pain free. Are there any solutions that do not require running in a container / webserver?
The communication really is the smallest part of this design, no more than 10 string input params (that have no meaning other than to the db) and one true/false output. There are no complex object models required. The only complexity would be from security/encryption etc.
I wamly suggest somethig based on Jetty, the embedded HTTP server. You package a simple runnable JAR with dependency JARs into a ZIP file, add a startup script, and you have your product. See for example here.
I often use Sprint-Remoting in my projects and here you find a description how to use without a container. The guy is starting the jetty from within his application:
http://forum.springsource.org/showthread.php?12852-HttpInvoker-without-web-container
http://static.springsource.org/spring/docs/2.0.x/reference/remoting.html
Regards,
Boskop
Yes, most of them runs a standard servlet container. But containers like Jetty have very low footprint and you may configure and run Jetty completely out of your code while you stay with servlet standards.
Do not fail to estimate initial minimal requirements that may grow with project enhancement over time. Then have a standard container makes things much more easier.
As you have tagged this question with [rmi], RMI does not require any form of container. All you need is the appropriate TCP ports to be open.
I want to make my web application able to work offline and as soon as it becomes online or gets connected again, then it should be able to transfer the modifications made by user in offline mode.
I have seen Google Gears as an ideal solution for my problem, which is not recommended to be used as it is now deprecated.
What is a good way to make my application work offline, both in terms of technology to use and application design?
Gears is deprecated because the HTML5 standard allows for equivalent features to be present in compliant browsers.
With respect to your current problem at hand of handling offline web application access, you can look into the support offered by HTML5 for offline web applications via support for client-side SQL database access, and the client-side application HTTP cache.
The features will have to be used in conjunction, as the client-side database access will allow for storage of data (generated when the application is offline) in a structured format, while the offline application cache will allow for caching of HTTP responses from the server; you should not be caching responses that are dynamic in nature which depend on any user-provided inputs.
The details of the proposed APIs can be found in the W3C HTML5 specification, which is in draft at the moment, although it appears that certain user-agents have already implemented this feature.
Firstly, you will need some form of offline storage. HTML5's capabilities are the successor to Google Gears, as stated on the google gears developer blog; essentially, the purpose of Google Gears was just to push the development & subsequent adoption of HTML 5 features.
Specifically you should be looking at the HTML5 offline (here's a tutorial) APIs, and the Storage APIs may also come in handy (relevant tutorial).
With regards to design, you will essentially need to maintain your complete web application state client side, and then send over the differences (i.e. update the server-side state) as soon as the connection to the server is available again.
Off the top of my head, there's 2 simple ways to design this:
Explicitly maintain separate application states for the client and server. Essentially, when the user takes an action, it's applied to the client application state first, and then at specified intervals (and/or triggers, e.g. the user clicks the save button), the client sends over the differences between the last known state of the server and the current state of the client. This is probably best suited to highly interactive web applications, and I suspect Google Docs works on this kind of design. Depending on your application (if "conflicting changes" can occur), you'll need to also account for merging application state: do you override with the last received client state, or do you intelligently try to merge? (you'll have to decide which makes more sense for your particular application.)
Record user actions while offline, and replay them once the connection becomes available again. You essentially implement the Command design pattern, and have both your client-side code and server-side code able to handle each command. The client-side code always handles each command, and while the connection to the server is available, your client side code also sends off the commands to the server. You'll probably want to implement some batching, to avoid continual requests to the server, and also some roll-back functionality when requests to the server fail (e.g. conflicting changes). This ends up looking more or less like GMail's main email managment user interface, where you can undo operations.
This has not much to do with J2EE, but rather how you code your web-client. One possible solution would be to use a javascript client that does save the data in the local storage introduced with html5 (see http://diveintohtml5.ep.io/storage.html ). That is also basically the reason why google gears was stopped ...