I'm developping an Android application for offering and claiming certain items (sort of a market) for a course that mainly focusses on user-centered design and the UIs (which mostly means we have to quickly implement all the logic and management underneath).
It's my first application that effectively needs to be shipped. So now I wonder what are the best strategies and best decisions regarding the R DBMS (because this area seems a very dense forest for some newbies) ?
The application will not target >1000 users in total
Remotely accessible
Transactions and ACID properties need to be supported
SQL queries must be possible
Secondly the DMBS must be contacted remotely. So I wonder if there's a sort of plug and play (blackbox) solution available for setting up the backend (free/cheap, few implementations, ...).
scalability, availability and security aren't really issues (at the moment)
First you need to decide how the android devices will talk to the central database. Options, from easiest to hardest:
Direct JDBC connection (over TCP). Some networks might block direct TCP connections. No offline capability.
Tunnel JDBC over HTTP(S). No offline capability.
Run a local database on Android (Sqlite) and synchronize to remote database when can (allows for offline). See SymmetricDS.
Put REST API in front of remote database and use android's built-in syncing capability.
Second you need to decide how to segregate remote database data between users:
No segregation, users can do what they want to others data.
Database segregation, aka Row-Level Security
Application segregation (no segregation in database, but control segregation in say your REST API)
You need database segregation if choosing options A, B or C from question 1.
How many concurrent writers for remote database?
PostgreSQL is a great choice these days, but MariaDB might be a better choice if you require row-level security as it supports the WITH CHECK OPTION clause, which isn't coming in PostgreSQL until the summer.
Not entirely sure what you are specifically asking because you can get a lot of info just by googling a few things. But if you might want to look at couple of google options. I haven't tried all of these so grain of salt:
App Engine: https://cloud.google.com/developers/articles/how-to-build-mobile-app-with-app-engine-backend-tutorial
Google Drive: http://developer.android.com/google/play-services/drive.html
and more generally, storage options from the Android docs: http://developer.android.com/guide/topics/data/data-storage.html#netw
Related
In the context of a university project, we have to develop a Java distributed application with these requirements:
The application will follow the classic client-server schema, with
multiple clients connecting to a central server on a different
machine, which also hosts a rdbms to which the server connects
The relational database we must use is postgresql (latest version)
Both client and server must be written in Java
We must use native JDBC to access the database (we can't use frameworks like Spring)
DISCLAIMER:please understand we are just a group of students and this is our first big project involving all these aspects and we aren't experts by any means, so please be patient with us :) (also English is not our first language, sorry for any mistake you might find)
We are currently in the design phase of the application (class diagrams, sequence diagrams etc) and we're stuck with a possible concurrency problem with the database:
ideally our server would listen for any requests and for each client that logs in the application, the server launches a dedicated thread that provides implemented services to the user (implementing pattern proxy-skeleton with basic socket programming). Each of these service providers (threads), upon completing the requested task should update/insert/delete data to the database. Here is the problem: how should we manage the concurrency here?
We tried to search the internet for this kind of issue and we found some things but we're still very confused:
Since we actually interact with the database from one single central
server (with one admin profile) we could implement a queue sistem for
the various transactions coming from the different threads we
launched
We manage concurrency at database level with some well-known mechanism such as MVCC, which is apparently a lot more complicated
Ideally we would like that read requests don't block other reads or writes, and writes only block other writes (which seems to be the case with MVCC). Which alternative would be best? Are there any other options that we could implement with the restrictions above mentioned? Thanks in advance for any suggestion
Is it possible to start up and shut down multiple H2 databases within a JVM?
My goal is to support multi-tenancy by giving each user/account their own database. Each account has very little data. Data between the accounts is never accessed together, compared, or grouped; each account is entirely separate from the others. Each account is only accessed briefly once a day or a few times a month. So there are few upsides to housing the data together in a single database, and some serious downsides.
So my idea is that when a user logs in for a particular account, that account’s database is loaded. When that user logs out, or their web app session (Vaadin app) times out, that account’s database is closed, it's data flushed to storage, and possibly a backup performed. This opening and closing would be happening for any number of databases in parallel.
Benefits include minimizing the amount of memory in use at any one time for caching data and indexes, minimizing locking and other contention, and allowing for smooth scaling.
I'm new to H2, so I'm not sure if its architecture can support this. I'm asking for a denial or confirmation of this capability, along with any tips or caveats.
Yes it is possible to do so. Each database will contain its own mini environment, no possible pollution between databases.
You could for example use a jdbc url based on the user id or login from the user:
jdbc:h2:user1 in H2 1.3.x embedded mode
jdbc:h2:./user1 in H2 1.4.x embedded mode
jdbc:h2:tcp://localhost/user1 in tcp mode
You can use any naming convention for the database name, provided your OS allows it: user1, user2, etc... or truly the name of the login.
Tips:
use the server mode rather than the embedded mode, allowing for same user multiple connections from multiple sessions/hosts
have a schema migrator (like flyway) to initialize each newly created db
ensure you manage name collisions at the top level of your app, and possibly store these databases and corresponding logins in a dedicated database as well
Caveats:
do not use a connection pool as connections will be difficult to reuse
You must make sure IFEXISTS=TRUE is not used on the server
avoid using tweaks on the jdbc url, like turning LOG=0, UNDO_LOG=0, etc...
I do not know if you'll have a limitation from your OS or the JVM on how many db files could be opened like this.
I do not know if such setting can be tweaked from the manual pages. I could not find one.
Please refer to H2 manual in doubts of url parameters.
I have a SQL databse on the internet which has information
I need my Android app to be able to access that information
The app needs to know the username and password of the database
How can it know?
If i code it in, anyone can get it
In general, databases should not be publicly accessible, nor should they be directly accessed by a user application, for several very good reasons:
There is generally no easy way to implement row-level access control. Views and triggers can only get you so far - in general application-level users do not map well to database users, since the latter usually have access to far more data than the former should have.
The DB clients are tied to the actual database schema. Having clients not under your control like, say, an Android application is a very good way to tie yourself up in ways that would disallow any and all future development.
Having a DB port open to the world is not considered by any means secure. Any potential security hole would give straight access to all of your data. The MySQL security guidelines explicitly warn against opening the DB port to the internet.
There is no way to protect the DB credentials or the data from a sufficiently determined and knowledgeable user. If your application can access something, so can they.
Database access protocols are mostly designed with local-area networks in mind, rather than the inherently unreliable nature of the Internet. Even encryption and security are often more of an afterthought...
The standard way to approach this issue is to create an intermediate web service with separate user accounts and a restricted set of operations on the data. The web service would let each user access only the data that relate to them, and even that indirectly. This approach separates the data from the user application layer, allows you the flexibility of storing and accessing your data however you wish and provides an additional layer of security for your DB.
I want to make my web application able to work offline and as soon as it becomes online or gets connected again, then it should be able to transfer the modifications made by user in offline mode.
I have seen Google Gears as an ideal solution for my problem, which is not recommended to be used as it is now deprecated.
What is a good way to make my application work offline, both in terms of technology to use and application design?
Gears is deprecated because the HTML5 standard allows for equivalent features to be present in compliant browsers.
With respect to your current problem at hand of handling offline web application access, you can look into the support offered by HTML5 for offline web applications via support for client-side SQL database access, and the client-side application HTTP cache.
The features will have to be used in conjunction, as the client-side database access will allow for storage of data (generated when the application is offline) in a structured format, while the offline application cache will allow for caching of HTTP responses from the server; you should not be caching responses that are dynamic in nature which depend on any user-provided inputs.
The details of the proposed APIs can be found in the W3C HTML5 specification, which is in draft at the moment, although it appears that certain user-agents have already implemented this feature.
Firstly, you will need some form of offline storage. HTML5's capabilities are the successor to Google Gears, as stated on the google gears developer blog; essentially, the purpose of Google Gears was just to push the development & subsequent adoption of HTML 5 features.
Specifically you should be looking at the HTML5 offline (here's a tutorial) APIs, and the Storage APIs may also come in handy (relevant tutorial).
With regards to design, you will essentially need to maintain your complete web application state client side, and then send over the differences (i.e. update the server-side state) as soon as the connection to the server is available again.
Off the top of my head, there's 2 simple ways to design this:
Explicitly maintain separate application states for the client and server. Essentially, when the user takes an action, it's applied to the client application state first, and then at specified intervals (and/or triggers, e.g. the user clicks the save button), the client sends over the differences between the last known state of the server and the current state of the client. This is probably best suited to highly interactive web applications, and I suspect Google Docs works on this kind of design. Depending on your application (if "conflicting changes" can occur), you'll need to also account for merging application state: do you override with the last received client state, or do you intelligently try to merge? (you'll have to decide which makes more sense for your particular application.)
Record user actions while offline, and replay them once the connection becomes available again. You essentially implement the Command design pattern, and have both your client-side code and server-side code able to handle each command. The client-side code always handles each command, and while the connection to the server is available, your client side code also sends off the commands to the server. You'll probably want to implement some batching, to avoid continual requests to the server, and also some roll-back functionality when requests to the server fail (e.g. conflicting changes). This ends up looking more or less like GMail's main email managment user interface, where you can undo operations.
This has not much to do with J2EE, but rather how you code your web-client. One possible solution would be to use a javascript client that does save the data in the local storage introduced with html5 (see http://diveintohtml5.ep.io/storage.html ). That is also basically the reason why google gears was stopped ...
Sometimes we deploy applications behind customer firewall and we need read only access to their DB for debugging issues as sometimes their IT people are not SQL savvy. We want to bundle our application with some web based application that will expose the database and allow us to fire adhoc SQL queries and show their output in HTML table. We dont want to write home grown code and we can bundle Java/JSP based applications.
Our backend is Oracle so we need a solution that can connect to oracle server and expose it over the web.
If you want the database behind a firewall, and believe me, you do want your database behind a firewall, see if you can have a VPN for going directly into the box. Once you are on the VPN, you can use whichever management tool you currently use for managing the database. So if you use SQL Server, you can connect via the VPN, and use Enterprise Manager to manage the database. Oracle probably has a similar tool, although I'm not that familiar. While having a VPN does incur an extra cost, it will probably make things many times easier.
phpMyAdmin is a good favourite if you're using MySQL. Its always a bit dangerous opening up an application like that if the db is intended to live behind a firewall, but as you say you could set it up ith a read-only account and possibly add extra layers of security on top like simple HTTP Basic auth.
Oracle Tool is a pretty decent oracle web front end. But so is enterprise manager.
Either way, you don't want your database open to the outside world in any way, either via some website or a listener or anything really.
The VPN solution is the way to go, just get VPN and then you can use whatever tools you normally use, if VPN is not an option then a simple firewall rule allowing just your IP's access would be ok, but not quite as nice as VPN.
On my current project, we use SQL Navigator in order to run queries on our Oracle database. I've also heard decent things about TOAD. I would advise against writing a web application just to run ad hoc SQL queries, because it seems to me that plenty of tools can support your debugging needs.