I want to develop a desktop application that allows users to search through json files.
These files (around 50.000) are predefined. They should be shipped with the application itself.
My question is, what would be the best way to ship these documents with the application and at the same time allow users to search for documents containing certain values, e.g. in sql terms: show all documents where some json value within the document like %Example%.
I thought about using some kind of NoSQL solution, preloading the files into the db and bundle it with the app. I've looked at some solutions, but I'm not really sure which one would be best suited for my needs or if it's even the best approach.
Bottom line is, I can't have my users install a db on their system, that is way too complicated.
I'd prefer a solution suitable for java or python.
Thanks for your help!
You can use an embedded database, memory based database (like hsql) or a file-based database like sqlite.
Neither require any installation from your end users. You just have to package the libraries as part of your application install bundle (and of course, the engine itself).
If you are looking for a k/v store, then the good ol' Berkeley DB should suffice. If you are really looking for a "embedded NoSQL solution", try MooDB.
Mongo DB comes in an embeddable version: https://github.com/flapdoodle-oss/embedmongo.flapdoodle.de
I've used it for integration testing (mocking a Mongo server) and it works really well!
Anytime I read document and search, I also think of Solr: http://lucene.apache.org/solr/
Related
I'm currently getting into Socket Programming and building a multi-threaded console application where I need to register/login users. The data needs to be saved locally, but I can not seem find the right structure for it.
Here are the ideas I though about:
Simply saving the data to .txt file. (will be troublesome to search and authenticate the logins)
Using the Java Preferences API but since the application is multi-threaded I keep on overwriting the data each time a new client connects to my server. Can I create a new node for each new user?
What do you guys think is the ideal structure for saving login credentials? (security isn't currently a concern for this application)
I would consider the H2 database engine.
quote:"Very fast, open source, JDBC API Embedded and server modes; in-memory
databases Browser based Console application Small footprint: around 2
MB jar file size"
http://www.h2database.com
It really depends on what you want to do with the application. The result would be different, depending on what you would answer to the following questions:
Do you want/need to persist the databases?
Is there any other data which you need to store along with that?
are you using plain java or a framework like Spring?
Some options:
if you're just prototyping and you don't have any persistence: consider using an in-memory storage for it. For simplicity in coding/dependencies, something like a ConcurrentMap can be completely sufficient. If you wrap it properly, you can exchange it later - and you don't add dependencies and complexities at an early state.
If you're prototyping but you still need persistence, using properties files on top of the ConcurrentMaps can give you a quick win.
There might be some more stages to this, depending on where you want to go with this, choosing a database at one point can be an option. Depending on your experience and needs, you can use a SQL or NoSQL database. Personally, I get faster results with NoSQL (MongoDB in my case) but prefer SQL in production for use cases like account management.
I'm planning to write a Java application wich relies on a small (Around 3000 nodes) graph to represent its structure. The data should be loaded from a custom file at startup to create an in-memory graph database. I've looked into Neo4j but saw that you can't make it run directly as in-memory. Googling around a bit I found Google JIMFS (Java in-memory file system) may suit my needs.
Does anyone have experience with getting Neo4j to work on a JIMFS FileSystem?
Are there more suited alternatives wich work in Java (possibly in-memory out of the box like HSQLDB) for small-scale graphs and still provide a declarative query language like Cypher?
Note that performance is not so much of an issue to me, it's more of a playground to gather some experience with graph databases, but I don't want the application to create a Database file system on disk.
Note that performance is not so much of an issue to me,
In that case you can go for ImpermamentGraphDatabase of neo4j, which is created like this:
graphDb = new TestGraphDatabaseFactory().newImpermanentDatabase();
It doesn't create any files on filesystem.
Source:
http://neo4j.com/docs/stable/tutorials-java-unit-testing.html
I don't know why you wouldn't want the application to create a Database file system on disk but I can easily tell that there are many options. I used neo4j and for most cases found its query methodology clear and visualizer very useful, thereby in my limited knowledge, make it my number one choice. However considering your requirements you might find this interesting :
https://bitbucket.org/lambdazen/bitsy/wiki/Home
I want to implement a search functionality in my web application that I am building using java technology. I would have to search through the database, depending on the user query and will display the results. Which way can I go about doing this(please take note I am using java technology)??.Thanks.
You can use a product like http://lucene.apache.org/core/ or http://lucene.apache.org/solr/ for this instead of writing this on your own.
Lucene is a high-performance search engine for documents.
SOLR is built on top of Lucene and provides additional features (like hit highlighting, faceted search, database integration or rich document (Word, PDF, ..) search)
Lucene will analyze your text data and build up an index. When performing a search you run a lucene query against this index.
Assuming you mean free text searching of the data in the database...
For free text searching Lucene and/or SOLR are very good solutions. These work by creating a separate index of the data in your database. It is up to you to either pull the data from the database and index it using Lucene/SOLR or arrange your code that writes to the database to also update the Lucene/SOLR index. Given what you have said it sounds like this is being retrofitted to an existing database so pulling the data and indexing it may be the best solution. In this case SOLR is probbaly a better fit as it is a packaged solution.
Another option would be Hibernate Search. Again this would be a solution to use if you are starting out. It would be more difficult to add after the fact.
Also bear in mind some databases support free text searching in addition to normal relational queries and could be worth a look. SQL Server certainly has text search capabilities and I would imagine other databases have some sort of support. I am not too sure how you access these but I would expect to be able to do it using SQL via JDBC. It is likely to be database specific though.
If you just mean normal SQL searching then there are a whole load of Java EE technologies, plain JDBC, Spring templates, ORM technologies (JPA, JDO, Hibernate etc). The list goes on and it would be difficult to suggest any particular approach without a lot more info.
I am currently working on a project that was not made by me but it makes use of a lot XML files instead of MySQL in place of it.
Because of that it makes me wonder if there is really any benefits of using XML over MySQL here.
The scene is, the XML files are loaded only ONCE and used on the server for N things it does.
The XML is only reload if the admin issue a command to the server to reload it.
All the XML files together have an average of maximum 100 mb size.
If you could as well give me a little brief of the above in regards the usage of XML over MySQL would appreciate.
What should I consider to know when a XML would be a better option over a simple innodb or myisam table ?
If your data is read-only and brought into memory only at the command of the admin, then I don't think it's much of an advantage for either technology.
MySQL would have the advantage of SQL queries if you have to search the data. Even in that case it's the type of data that matters. If you have long reference chains/object graphs, then a relational database may be slow because of all the JOINs.
But XML has its own issues. You can easily parse it into a DOM object, but then you only have XPath to search it.
XML is used as one of the ways of storing data. one of using xml is, it makes the data easy to be readable. you can use mysql if there are lot of users need the access to the data at the same time and mysql also supports transactional processing of data whereas xml does not have such features.
just adding the option in between - you could also use some form of xml database like
eXist (http://exist-db.org/index.html) or sedna (http://modis.ispras.ru/sedna/)
XML stored at local storage, and readable only by local server (don't argue me you can use memcache, replicated via rsync or so)
No doubt you can open the XML via a http server, but it will be slow.
While, mysql support port communication, and replication, it basically don't have boundaries if you expanding to multiple servers.
And even at 5.1, mysql support XML
I am making a desktop application in java and using MSAccess in data base.
I want that if i run the setup of the application the database should be created on client machine because there can be different client using the application and how can i create the setup?
is there any tools available for this free of cost?
please explain me in detail..
thanks
Java 6 (enhanced for desktop application work) comes with a built-in database called JavaDb (formerly IBM's Derby). That will do what you want.
Here's a set of guides and tutorials on how to use it.
I would suggest that when your application first starts, it checks for the presence of the created database, and if it doesn't exist, it builds the database (via the appropriate SQL). I've used this approach before and it works quite well.
I prefer nullsoft. Take a look at Open Source Installers Generators in Java
#pratap: database should be created on client machine..
Add an empty access database to your setup.
Have a look at SQLite, which is used by Mozilla (Firefox stores all bookmarks and history in a database) and several other major applications.
When you say
access in database
do you mean Microsoft Access or access the data in a database.
I would advise against MS Access if that is the case. If not, you could either use the JavaDB or HSQLDB and the use SQL scripts to create the database. As a summary
Package the application in one of the installers (InnoSetup or NSIS are good ones)
When installing, extract all the files in proper folders
Execute the SQL scripts before first running the application to ensure the database is setup, you can do other housekeeping tasks along with this step (refer to installer documentation for after-install steps)
Your application is good to go
In the last distribution of NetBeans I used, there was a wizard to create such application. The application used the Java Persistence API to store the Data.
My option is HSQLDB since it's fast, reliable and easy to use.
In the documentation it's explained how to use the standalone database mode, this is primarily used for unit testing but it fits your case too. The good thing with this is that you just connect to the file based database without any special set up and if the files doesn't exist, they're created.
I would second the posters who recommend JavaDB.
It is absurdly easy to adminster from inside your application. Whats more because everything is native Java you dont get the char->unicode little-endian->big-endien and all the other conversion malarky you normally get when reading SQL into java.
The one tip is that with JavaDB is prepare your SQL statements. Prepared statements get cached and the resulting access program (similar to an access plan but actually a jvm program) is reused, the programs for "executed' statements are not cached.
If you are really set on MSAccess then I would suggest you package an "default.mdb" file with all your required tables defined and your classifcation tables populated. If the user's table does not exist then simply copy over the default .mdb file and open that.
I recommend the H2 database because it is simple, fast, pure Java, and small. See this page for how H2 compares to other Java databases, including those mentioned here in other answers. It has many features Derby/JavaDB and HSQLDB do not.