Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
What I am trying to do is cache all the data that I have written into mongodb. So that all client requests are served from the cache. Should I consider ehcache or memcache.
Note the mongodb data is queried a lot, that is why I have thought to cache all of it at server start time, no writes are permitted to this data. I am using java for the application.
It makes very little sense to use a cache in front of MongoDB if you are using it for reads only. An extra cache is just going to take up more memory. MongoDB uses memory-mapped files and the Operating System will keep the most requested data in memory. If all of your data fits in memory, then MongoDB will returns all the documents straight from it - just like an additional cache would.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I was creating an application with java and database is mysql and it was running well but when data get large (more than 500 fields) application running slowly , so How Can I make it fast ?
For db
Add indexes for frequently searched fields
Think about table partitioning, rarely searched data should be stored in archive tables
For backend
Optimize queries
Minimize cursor fetching
For client
Use pagination to avoid large data loading
Use async loading (SwingWorker for swing, Service for javafx) to avoid UI hanging
Don't mix archive and working data in one UI form
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
What is the best way to post large data for processing within enterprise applications?
Data
will be of size upto 1 GB Data
when consumed, can be removed (if processed); need not be persisted.
Can we look at JMS technologies / Kafka Cluster to receive and distribute the data? Data has to be consumed fully, only-ones, and cant be shared(partitioned) across multiple consumers.
What are the other options can be explored?
Apache KAFKA is more designed to handle realtime streams of data rather than large data transfer. Also messages do not delete, rather they are commited. Also exactly once processing would need to be implemented by you. KAFKA by itself is not JTA aware. I would recommend against using large message sizes.
No matter what other queuing technology you use you will need to use message sizes smaller than 1GB (i.e. you will need to chunk your data and reassemble or make your processing stream like instead of bulk).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I want to know what is the best way to store uploaded file.
Is there any performance related issues and benefits between storing in these two ways: (storing in DB and storing in project path)
Which kind of risks can be occurred in each way when retrieving files?
If volume of files (the overall storage that you need) is not high better to use DB. But if there are a lot of large files that need to be uploaded, better to use filesystem.
In case of DB, pros:
All data is in one place so you can easily read all you need from a single place.
DB level permissions can be applied to stored files too.
You can parse/process file contents using DB level procedures.
You can use full-text search of the DB system.
Backup/resore of data will be easier because everything is in DB.
cons:
In case of larger volume of data, storage/retrieval of data will take a lot of time.
For filesystem, pros:
You can handle larger storages.
Cons:
Opposite of all cons of storing data in the database, especially the backup/restore process will be much more difficult and complex.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
Improve this question
Went I read rss from diffrent sources such as bbc, voa,... How can i plan they 'by source', 'by time',... Should I use database ? I think it make my app slowly.
Not using a database would make your app slower. Without a database every time you restart the app it needs to fetch the rss feeds from the servers. It's sensible to store the data offline and update it with the new items on startup / with a button.
Think about additional operations which need a database: labeling, starring, saving for later, etc.
This provides an additional benefit: you can store the news offline so the user can read them without an active connection.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
I am currently working on a project in which I am storing the name of program/application window titles and my knowledge of databases and datastores is fairly limited.
The idea is that I want to query the database with a string to see if it is present in the database. I am currently using MongoDB to do this but I have seen that MongoDB is mostly used to be run on a server which isn't what I'm looking for.
My question is - if I am just storing strings / searching for strings would a custom Array or HashMap be sufficient or would search times make it inefficient meaning that SQLite would be more ideal for this situation.
SQLite is perfect for this application. Firefox, for example, uses SQLite for storing its internal configuration settings (the about:config page). SQLite databases are single files, and it can be transparent to the user and requires very little in the way of system resources--unlike most server/client database solutions.
i would suggest to use java preferences api, if the data to be saved is not too much and if it needs to be available even when the application is terminated and restarted.,