I am developing an Android application where users will see lists of Groups and join 1-* Groups. Within the Groups will be members and users can message each other within the group they have joined. Sending messages 1 person to many and reply all.
I want this to support multiple devices. So, if i join a group and someone sends me a message, i want to see that message on my phone and tablet. If both have the app installed.
I want the user to be able to see group details while offline.
I have been doing some research and drawing diagrams on how this could work.
What are my options for how to set this up with a remote database?
Here are a couple of the options that i have finally narrowed down to.
Have the remote DB store everything. Have a local DB on the device that reflects the remote DB. Have the remote DB contain a version number. At some point, either scheduled or triggered, the app compares the version numbers in the local and remote and if the local is out of date it will update to the current remote DB.
Have the remote DB store everything. Have a local DB on the device that reflects everything in the remote DB except the messaging system. This would behave similar to #1. Except the tables and stuff that are related to the messaging system would "somehow, gcm maybe" send a message to all devices associated with the user that a message relating to the user has changed and the app will then update to the most recent version of the remote messaging tables in the remote database. Then their local database contains the new data and their "message box" would be updated.
What would be the best way to do this? How is this done currently in most systems? Are there more options then what i listed?
Remote DB: MySql
Local DB: sqlite
I am aware of some similar questions here on SO, but i am wanting to know if either of these options or something else would be best for my specific scenario. Mainly how to handle the messaging part.
Thanks
I think Option 1 is a pretty standard way of handling this problem.
A DB version # could work, but if it changes frequently you will end up resync all the data very often. Its probably better if you just store "Last Modified Date" on records and only pull down things that have changed since the last sync. That way you can return a smaller set of new records.
You would only want to pull down public information (the groups) and the information specific to the user. The remote DB would have everything, but the local DB would only consist of data that is related to the user.
I found this tutorial that walks through how this might be implemented using php on the remote server for API access. You could streamline this process by using an ORM or "Out of the Box" API solution.
I'm sure there are more elegant solutions out there for syncing a local Android DB with a remote but this is likely the most practical approach for the non-enterprise solutions.
Related
I am planning to make a software which requires two or more databases to be in sync with a main server database ( located on a web server ). Suppose one user have installed my software on more than one computers. Suppose he updates some info on PC-1 then the change should be locally updated and if he is connected with the internet, it should get in sync with the main web server. When he opens the software on PC-2, latest into must be shown there ( assuming he is connected with the internet ).
Now suppose both PC-1 and PC-2 have same data, and both are offline. User deletes the data from PC-1 but don't deletes it from PC-2 ( he assumes it'll get managed automatically ), now when both of the systems go online, the record must be deleted from PC-2 too.
Basically, I am trying to find the database which was most recently updated or modified and sync it with the main database ( on the server ). How can I do it ?
It sounds like you want to synchronize some user data whenever the users or application becomes "online" again.
A couple of suitable services are:
Amazon Cognito Sync is an AWS service and client library that enables cross-device syncing of application-related user data. You can use it to synchronize user profile data across mobile devices and the web without requiring your own backend. The client libraries cache data locally so your app can read and write data regardless of device connectivity status. When the device is online, you can synchronize data, and if you set up push sync, notify other devices immediately that an update is available.
AWS AppSync enables developers to interact with their data by using a managed GraphQL service. GraphQL offers many benefits over traditional gateways, encourages declarative coding style, and works seamlessly with modern tools and frameworks, including React, React Native, iOS, and Android. GraphQL is a data language that was developed to enable apps to fetch data from servers. It has a declarative, self-documenting style. In a GraphQL operation, the client specifies how to structure the data when it is returned by the server. This makes it possible for the client to query only for the data it needs, in the format that it needs it in.
I am creating an app that logs data. I am creating documents that have the data and sending those documents to a couchbase server. Or I am trying to anyways. One major concern I have is how do I confirm a document is stored on the server so that it can be immediately deleted on the device? I am hoping there is a quick and efficient way to do this. The end result is to have a thread constantly checking if there is a connection to couchbase, and if so start sending data up to clear it off the device. Most documentation seems to be regarding syncying the database, however I don't want to do this because I don't want to keep a copy of the data on the device. It would take up too much storage. Thanks for any help.
EDIT: For clarification, I currently have the app storing many data points in documents. I want to send these documents to a couchbase server. I don't want to "sync" the documents, but rather just insert them into the database then immediately delete them off the device. How would one go about doing this? Most examples I have seen typically sync documents such as profile information where changes can be made in various synced databases and all those changes would appear in every database. Instead I want a 1 way relationship with the database were information is sent, confirmed as received, then immediately deleted from the device.
There are at least a few possibilities.
If you are expecting a solid network connection, or are ok with handling errors yourself, you can achieve this with a direct REST call to Sync Gateway. You can, of course, always write your own REST server that talks directly to Couchbase Server, too.
The second way relies on an older version of Couchbase Lite. Couchbase Lite 2.x is a major rewrite of the product. As of the current shipping version (2.1), it does not support this approach, so you'll need to use the 1.x version (1.3 or later, IIRC). See further down for how to approach this with 2.1.
Set up a push only replication. After replication, cycle through the docs and purge all the ones that are not still pending. (This uses the isDocumentPending method on the Replication class. That's the key piece not available as of 2.1.) You can either run one shot replications and do this after the replication completes, or monitor the replication state of a continuous replication.
Purging a document from the local CB Lite database effectively makes it act as if it never existed on that device. By running a push only replication, you don't have to worry about the docs getting sent back to the device.
Using 2.1, you can't as easily determine if a document has been replicated. So you need to run a replication to completion while avoiding a race condition with writing something new.
One approach here is to pause writing documents, run a one shot replication, then purge the documents before starting up again. You could also work out something with alternating databases, or tracking documents yourself somehow, etc.
For completeness, if you were in a situation where you had a mixed use, that is, wanted only some documents pushed up off the device and forgotten, and some synced, you would control this through Sync Gateway channels.
I don't know Lite and Sync Gateway well enough, but from a Server perspective:
You could use the new Eventing service in Couchbase. When a document is created in bucket A, you could write an event to copy that document over to bucket B. Then, if the documents are deleted on the device, it wouldn't matter if they get deleted from bucket A.
I have a bucket "staging" and a bucket "final". I created a function called "moveIt" with "final" (I aliased as 'f').
The OnUpdate function could be as simple as:
function OnUpdate(doc, meta) {
f[meta.id] = doc;
}
My main concern would be the timing. I don't think there's an easy way for your mobile app to know that the event has finished copying a document before you decide to delete it in Lite and start a Sync. But it might be worth a try. Check out the docs to learn more about the Eventing service details.
In Couchbase Lite 2.5, you can use replicated events to detect when a document has synced (pushed to server or pulled from server). You can register a callback on the Couchbase Lite replicator to detect if the documents have been pushed to the sync gateway and then use the purge API to locally purge
I'm developing a chat application in Java.
The architecture used is Server - Client(s) architecture.
The majority of the code is in Java, JavaFX for the GUI and PostgreSQL as the Database.
As this is a chat application (desktop), I'd like to know which is the best way to store chat history:
Locally in a text file, that the client has to read every-time
In the database as of type String (VarChar)
In the server as Lists
Some questions based on the three ways:
If a client connects from a different machine the text file will not be there
Is it possible to store every text entry in the database with a chatroomID?
How many objects can be stored in the server for as long as it runs?
Out of your three choices, I recommend you choose option #2 for storing chat history: A database, and here is why:
If you store the chat history locally in a text file, you run into issues such as how to sync with others. Also, you can modify the contents of the text file without going through your Java program (such as with an editor). If this file contains chats with sensitive information and someone has access to your computer, they can read it. This spells trouble.
Storing in a database is a great idea because it provides a central location for all your Java program. This is especially handy if multiple people are using your Java client, that way they can fetch chat history, as well as easily transfer chats to others! I wouldn't only use a type String (VarChar), but try to think of some other useful fields or columns that might be useful (i.e. timeSent, chatUserID, timeRead, etc). This also brings up the point that by using a database, you are able to set up some sort of user access rights (username and password) so that specific people can read specific chats.
If you store the chats on the server as a list within the Java server itself, and if your server restarts, you lose all your chat history. Bummer.
To sum up, keeping your Java client-server-database architecture is perfectly fine, and technically all 3 options could work, but databases is the way to go for storing your chat history! Even if setting up the database takes a little bit of work, it proves to be superior in efficiency and security out of the other 2 methods described since databases are built for archiving data.
I'm in a similar situation, I'm also developing a chat from 0, the only difference is that I'm doing it for iOS.
The way I'm developing my chat is:
I use an Ubuntu web server.
The server has a database in Mysql and the communication with the user is done through NodeJS.
In NodeJS I have a socket "Socket.io" which facilitates notifications between users.
On the iOS device, I store all the messages that it receives in Core Data, which is an extension of SQLite.
In order to obtain the pending messages depending on the device where the user is connected and I use an ID for each device, this ID is created and identified by the MAC physical address and thus what messages to obtain from the server and what not.
Initially I base myself on this database to know how to structure my application: https://github.com/yoosuf/Messenger
Socket.IO is incredibly easy to use and the best thing is that it has libraries for different programming languages, here is its page:
https://socket.io/
https://github.com/socketio/socket.io
I would say Nr2 - for safety reasons (if you care about it) and because it is a pretty easy way.
For the beginning a database with 4 columns should be enough (Date/Time, ChatroomID, UserID or just a name and the message itself). If a user sends a new message it creates a new row containing all the information that is needed for the columns. You can easily iterate through it as well when your client reloads (maybe every 10 seconds)
I want to create java desktop application, which stores it's data offline in a database (not just some config files). The application should work fine when the user is offline. When the user becomes online, the offline database should be able to sync with the online master.
Any ideas which technologies can be used to achieve this?
This has been discussed on stackoverflow a lot and it usually boils down to: don't roll out your own solution - This is a very specialized field - Look up SymmetricDS. It does what you want.
One of my fav discussions is Strategy for Offline/Online data synchronization
Use one of the available pure java DB implementations as a local DB. Use any other DB as a remote one.
Implement logic that tries to connect to remote DB and fall backs to local one on failure. If it connects successfully to remote DB implement the data synchronization.
When the local application operates, it should not only change database, but also log the changes. That changes are sent to the server when the connection is available. Also, the application receives logged changes stored on the server (from other application instances).
The main problem is how to merge changes made by different instances. There can be 3 variants:
1) Each application instance can modify only its private part of the whole database. Your are lucky, no merging needed, and server can store only logs and not run the whole database.
2) modifications always can be merged automatically (for example, application can add a value to a common variable, but cannot set it directly). The server runs the whole database, accepts partial logs from clients, generates its own log and sends it to clients.
3) Clients are allowed to do arbitrary modifications. This leads to potential conflicts. In case of conflicts, one of conflicting changes should be rejected. That means, that if a client made local modifications, that modifications can be rejected later by the server. The user interface must reflect this issue. In the rest, this is similar to the variant 2.
i am writing an application in java and i want to enable it to access a mysql remote server.
my problem is that if the application have the user name and password someone can take them and use my db with a different software.
is there a way of preventing it ?
UPDATE
i found this workaround for connecting to a remote MySQL database from an android device.
you put a service in the middle. like a php page that code the data in JSON format which the android app gets via http.
here is the example i found :
Connecting to MySQL database
Having the username and password is designed specifically to grant access to the database. That's the whole point.
You could go to some extra lengths like restricting database connectivity to specific hosts, so at least your paying customers get access to the database and no else can access it, but your customers might choose to use different software to access the database. There's no way around it except licensing terms.
What you could do is run an intermediary program on your own hardware that connects to the database and restrict access to the database to software that is under your direct administrative control. Your program would validate all requests from software under control of your customers and allow the queries that you want to allow and refuse (and log) the queries you do not have to allow. (You do not need to send raw SQL data back and forth -- you can do any amount of processing on the data and queries.)
You can setup JDBC Data Source on your application server. Here you can see example for Oracle Glassfish.
So, your credential won't be used in your code/resources.
If you are saying that you have an application trying to access a MySQL remotely (not in the same box), then i think you need not worry, as the connection that will be established by your application codes will not expose the username and password when it is trying to authenticate and authorize itself to the MySQL server.
You can limit the access to the MySQL-server so that only certain IP-addresses or IP-ranges have access to it.
As a side note, make sure that the user you use in your application only has the needed permissions to the database. For example, the user might not need to be able to create or delete tables. You can even specify permissions for the user on table and column level.