Note:There is a good chance I'm not using the correct terminology here and that maybe the reason I'm not finding the answers to my question. I apologize upfront if this has been already answered, so please just direct me there.
I am looking for an open source framework written in Java that would allow me to build pluggable data connectors (and obviously have some built in already) and almost have a query language (abstraction layer) that would translate into any of those connections.
For example: I would be able to say:
Fetch 1 record from a Mongo DB that matches name='John Doe'
and get JSON as a response
or I could say
Fetch all records from a MySQL DB that matches name='John Doe'
and get a JSON as a response
If not exactly what I described, I am willing to work with anything that would have a part of this solved.
Thank you in advance!
You're not going to find a "Swiss army knife" data abstraction framework that does all of the above. Perhaps the closest things to what you ask for would be JPA providers for both Mongo and MySQL (Hibernate is a well-regarded JPA provider for MySQL, and a quick google search shows Kundera, DataNucleus and Hibernate OGM for Mongo). This will let you map your data to Java Objects, which might be a step further than what you ask for since you explicitly asked for JSON; however, there are numerous options for mapping the resulting objects into JSON if you need to present JSON to a user or another system (Jackson comes to mind for this).
Try YADA, an open source data-abstraction framework.
From the README:
YADA is like a Universal Remote Control for data.
For example, what if you could access
any data set
at any data source
in any format
from any environment
using just a URL
with just one-time configuration?
You can with YADA.
Or, what if you could get data
from multiple sources
in different formats
merging the results
into a single set
on-the-fly
with uniform column names
using just one URL?
You can with YADA.
Full disclosure: I am the creator of YADA.
Related
Fortunatly we have evolution in play framework 2 so we don't care about schema (if we want). My question is, is there any practice with loading data to the database on application startup? I suppose that I can load it with Global object, but what the approach needs less code and more flexible(in java)?
Take a look on the same question (and answer), the official way is showed in Zentasks sample.
you can also use common evolutions to insert data in raw SQL format (it doesn't work only with schema)
I´m building Java backends with Spring, Hibernate and RDBMSs for a while now. Also I´m regularily working on mobile applications for iOS and Android.
So I have a full stack of technology to use for this task, however I am looking for something maybe more advanced that better fits the requirements. I was having some thoughts about it, but I better first explain how my current systems work and then how I want my upcoming systems to look like.
Currently using
Spring Framework to connect everything together
Hibernate with Entity beans for persistence
MySQL or others as RDBMS
DTO objects created with Dozer
RESTful API to expose services
DTOs are transferred in JSON format
This setup works. But I have the feeling that it´s just too much work and life could be simpler with other technologies.
What I am looking for
On the mobile site, I want to receive data for the current screen that I could easily cache. JSON is something that is already serialized and that would be easy to save to disk in the mobile application, without using yet another database. So the question is, how could I store the data in the backend, so that I can more easily receive it, without using entity beans, DTOs and Dozer to convert between them? Isn´t there another database solution which already delivers JSON? What about graph databases for example, like OrientDB or Neo4J?
I definitely want to go with Java and Spring, and I am open to a replacement for Hibernate, RDBMS and entity beans and DTOs.
Looking forward to your answers!
Your current design (This setup works) has niceties which a good system should have. tiered and good separation of concerns.
If I understand your requirement correctly then, you argument is, if my end data format is JSON then why not store the data in JSON format which will get you rid of lot of plumbing code/effort in the middle tier.
It will directly enable you to fetch the data from the storage and pass it on the requesting client. These are your requirement in nutshell. Please correct me if I am wrong.
Now JSON is more of textual notation and less of storage format. Jason is generally consumed by the View tier of MVC architecture as its easy to render on the screen using Javascript.
Your reasoning of using a NoSQL DB which directly delivers JSON is credible given that tye end client is going to be mobile app.
Overall architecture looks good and highly optimized for Mobile access.
Now coming on the NoSQL JSON storage, following are the Document Store NoSQL DBs which support JSON interface
i. CouchDB
ii. JasDB
iii.SchemaFreeDB
8.You can evaluate any of these to suite your needs.
(full disclosure - I'm an engineer with Kinvey, a BaaS provider)
One option you might consider is using Backend-as-a-service. Most BaaS providers use JSON to transfer the data over the wire, which sounds like it would be compatible with your requirements.
In addition, you'll typically get a lot of common mobile app functionality baked in (i.e. push notifications, file storage and CDN infrastructure, user management, etc). This could be especially useful if you are building multiple apps, each with their own backend; rather than reinventing the wheel each time, simply spin up a new backend.
One last, but important note, would be pricing. A lot depends on your use case, but from what I've seen, a BaaS provider is usually significantly cheaper that rolling your own solution on AWS or some other cloud provider, especially since most providers offer a free tier.
Even though this question is a bit old, maybe a quick alternative for RDBMS: MongoDB. It is a document database with document-level locking. It scales really well.
Main point: it uses JSON as its document storage (actually the Binary JSON a.k.a. BSON, but that is just a superset). Inserting a document into the database is as easy as
db.collection.insert(JSON);
on the mongo shell and
DBObject bson = (DBObject) JSON.parse(JSONstr);
collection.insert(bson);
in the java driver.
I'd like to save persistent objects to the file system using Hibernate without the need for a SQL database.
Is this possible?
Hibernate works on top of JDBC, so all you need is a JDBC driver and a matching Hibernate dialect.
However, JDBC is basically an abstraction of SQL, so whatever you use is going to look, walk and quack like an SQL database - you might as well use one and spare yourself a lot of headaches. Besides, any such solution is going to be comparable in size and complexity to lighweight Java DBs like Derby.
Of course if you don't insist absolutely on using Hibernate, there are many other options.
It appears that it might technically be possible if you use a JDBC plaintext driver; however I haven't seen any opensource ones which provide write access; the one I found on sourceforge is read-only.
You already have an entity model, I suppose you do not want to lose this nor the relationships contained within it. An entity model is directed to be translated to a relational database.
Hibernate and any other JPA provider (EclipseLink) translate this entity model to SQL. They use a JDBC driver to provide a connection to an SQL database. This, you need to keep as well.
The correct question to ask is: does anybody know an embedded Java SQL database, one that you can start from within Java? There are plenty of those, mentioned in this topic:
HyperSQL: stores the result in an SQL clear-text file, readily imported into any other database
H2: uses binary files, low JAR file size
Derby: uses binary files
Ashpool: stores data in an XML-structured file
I have used HyperSQL on one project for small data, and Apache Derby for a project with huge databases (2Gb and more). Apache Derby performs better on these huge databases.
I don't know exactaly your need, but maybe it's one of below:
1 - If your need is just run away from SQL, you can use a NoSQL database.
Hibernate suports it through Hibernate OGM ( http://www.hibernate.org/subprojects/ogm ).
There are some DBs like Cassandra, MongoDB, CouchDB, Hadoop... You have some suggestions Here
.
2 - Now, if you want not to use a database server (with a service process running always), you can use Apache Derby. It's a DB just like any other SQL, but no need of a server. It uses a singular file to keep data. You can easily transport all database with your program.
Take a look: http://db.apache.org/derby/
3 - If you really want some text plain file, you can do like Michael Borgwardt said. But I don't know if Hibernate would be a good idea in this case.
Both H2 and HyperSQL support embedded mode (running inside your JVM instead of in a separate server) and saving to local file(s); these are still SQL databases, but with Hibernate there's not many other options.
Well, since the question is still opened and the OP said he's opened to new approaches/suggestions, here's mine (a little late but ok).
Do you know Prevayler? It's a Java Prevalence implementation which keep all of your business objects in RAM and mantain Snapshots/Changelogs in the File System, this way it's extremely fast and reliable, since if there's any crash, it'll restore it's last state and reapply every change to it.
Also, it's really easy to setup and run in your app.
Ofcourse this is possible, You can simply use file io features of Java, following steps are required:-
Create a File Object
2.Create an object of FileInputStream (though there are ways which use other Classes)
Wrap this object in a Buffer object or simply inside a java.util.Scanner.
use specific write functions of the object created in previous step.
Note that your object must implement Serializable interface. See following link,
I am using Spring Data (Mongo) for my web application (close to a social networking website). Now, I wish to provide search capabilities over the content written within the application (such as posts, tags, friends, etc.).
I believe Lucene/Solr is one of the better libraries to go for such cases, but am not sure how to use (integrate?) it with Spring Data (or maybe there is some inherent support within Spring for it).
Would appreciate help (documentation, links, blog posts, etc.) on this!
Though the post has been around for a while, you may have a look at this one https://github.com/SpringSource/spring-data-solr/
The Spring Data for Solr project provides a natural Spring Data like API for querying data from Solr. Read the examples for a quick overview.
I found a good read here - http://adeithzya.wordpress.com/2011/08/25/using-apache-solr-with-spring-framework - that hits the nail on its head!
Integrating them is relatively easy, the difficult part is maintaining data consistency between them. For example, how would you answer these questions:
How and when do you intend to perform CRUD with mongo and sorl? Do you write to Mongo first (with/without waiting for a confirmation?) and then to Solr?
if you're using async writes with mongo, what happens when you send the data to solr, and then get an exception for mongo (data exist in solr, but doesn't exist in mongo)?
What happens if you get an error while trying to write to solr (data exist in mongo but not in solr)?
if you delete something from mongo, and right after that someone performs a search where solr returns that very deleted document because solr stil has that document indexed?
The point is there'll be an inconsistency window where mongo and solr are not in sync, and you probably want to handle at least some of the issues.
I have a reasonably simple a reasonably system that are conceptually 100,000's of small/simple JSON documents. I want to switch from mysql to a distributed system for redundancy and backup purposes.
Its currently written in java. What will be the simplest/easiest nosql solution to install and use? i.e. I want something I can just install on a few servers, point to each other, and start using.
I don't want to spend hours learning, tweaking and configuring stuff. (If its going to be too large a project I may as well just write some java code that synchronizes the relevant tables)
mongoDB is easy to install and start to use document database for me.
It is easy to make it work with java and it has a nice mapper from java objects to mongo called morphia.
If you can tolerate hosting your data on Google's AppEngine, Ubud-db can be something for you:
https://bitbucket.org/f94os/ubud-db/wiki
Ubud-db is a document store on AppEngine with a REST-JSON API. Spring/Jackson maps from JSON to a Map, and then Ubud's service maps from the Map to Entity, persisted by the Datastore.
How can I ask this question if you've already asked it!?
I'll add BaseX to the list -- it handles JSON fine. Has a nice GUI, or you can just use a console through bash et. al.
Geared towards XML with XQuery and Xpath, but has JSON modules.
I'm only looking at alternatives because I'm not quite sure how to use some of the features...but that's on me, not the db.
usage example:
let $content := file:read-text('x.json')
let $json := json:parse($content)
return admin:write-log($json)