I have seen application to have clustered web server(like 10 to 20 server) to have scalability where they can distribute the
load among webservers. But i have always seen all webserver using single DB.
Now consider any ecommerce or railways web application where million users are hitting the application at any point of time.
To scale at webserver side, we can have server clustering but how we can scale DB ? As we can not have multiple DB like multiple webserver as one dB will have different state than other one :)
UPDATE:-
Is scaling the db not possible in relational DBMS but only in NO SQL DB like mongo db etc ?
There is two differend kind of scalability on database side. One is read-scalability and other one is write scalability. You can do both with scaling vertically means adding more CPU and RAM to some level. But if you need to scale on very large data more than the limit of a single machine you should use read replicas for need of read-scalability and sharding for write-scalability.
Sharding is not working like putting some entities(shoes) to one server and others(t-shirts) to another servers. It works like putting some of shoes and some of t-shirts to one machine and doing that for the rest of entities also.
Another solution for high volume data management is using microservices which is more similar to your example. I mean having a service for shoes another service for t-shirts. With microservices you divide your code and data to different projects and to different application and database servers. So you can deal with scalability of different part of your data differently.
Related
So i have a problem. Currently my application connects with single database and supports multi user. So for different landscapes, we deploy different application all together.
I need a solution that my application remains the same (single WAR deployment) but is able to connect to different DB across different landscapes.
For example a user in UK is using same application but underlying DB is in UK and subsequently another user logs in from Bangladesh and he sees the data of DB schema for Bangladesh and so on.
Currently we are creating JDBC connections in a connection pool created out of java and taking the same through out the application. Also we load static datas in hashmaps during the start up of server. But the same would not be possible with multiple DB since one would overwrite the other static data.
I have been scratching here and there , If some one can point me in the right direction, would be grateful.
You have to understand that your application start up and a user's geography are not connected attributes. You simply need to switch / pick correct DB connection while doing CRUD operations for a user of a particular geography.
So in my opinion, your app's memory requirement is going to be bigger now ( than previous ) but rest of set up would be simple.
At app start up, You need to initialize DB Connection pools for all databases and load static data for all geographies and then use / pick connection & static data as per logged in user's geography.
There might be multiple ways to implement this switching / choosing logic and this very much dependent on what frameworks & libraries you are using.
I have 2 differents applications (Web app and Desktop app) with differents database servers but same structure.
I want to have the same data in all databases, no matter where the user insert/update/delete a record. This is the easiest for me but I don't think is the optimal.
So, for example, if I insert a record in the desktop app, this record must be inserted into the web app server ("cloud") and vice versa.
I'm using Spring+Hibernate+PostgreSQL for the web app and JavaFX+Hibernate+PostgreSQL for the desktop app.
I'm considering 2 options at the moment:
Use sockets to send messages between servers every time a record has been inserted/deleted/updated.
Use Triggers and Foreign Data Wrapper in PostgreSQL. I'm not too familiarize with this, so I don't know if I can make what I want with this option. Can they work together?
There is another option? What do you think is the best?
The simplest and maybe best solution is to have one central read-write database and several physical replication standbys.
The standby servers will be physical copies of the main database, and you can read from them (but not modify data).
i want to optimize my game servers in Minecraft. I have 150k users in database, when daily on my servers join 15k users.
I have read about Redis, and i also read that Redis is faster than MySQL, i know that i can't give up from MySQL because my websites are using same database.
But what if i will load every 15 minutes all MySQL data to redis, then all my server plugins will work on this data, then after next 15 minutes redis will export that data to MySQL? I load same data to 4 servers and to 3 plugins on every server, so maybe loading it all to one redis server will be faster than send requests to MySQL from 4 servers * 3 Plugins?
Thanks for help.
Redis is an effective way to cache data from a MySQL database. Even though Redis has persistence options, many will still favor using a MySQL database for this task. As Redis operates in memory, it will be much faster than a MySQL database which (for the most part) does not operate in memory. Often, people will favor storing cache data with HashMaps, but since you have 3 servers, Redis would be a much better option. This way, you wouldn't have to create 3 near identical caches for each server.
Hi as much I can understand you have 4 mysql servers and 3 plugins.
As Redis is extremely fast no doubt but Redis use case is different than mysql. my advice is to load data in Redis which you'll use very frequently it'll be much faster than mysql, but to make it faster you have to design your keys intelligently so that Redis can search it faster. You can refresh keys and values after certain interval, defiantly your system's performance will improve.
we are building a web application, multiple dashboards and reports, that uses multiple datasources to read the data
we are planning to use redis cache to make performance better of the application
what should be the better solution for this
1) loading data periodically into the redis cache and serve it from the cache
this seems very good at first but has problem that for dashboards and reports that are not frequently used we will be pulling the data after the certain time hence creating unnecessary load on application and cache
2) not loading the data into the cache will take minutes of time to load for one dashboard or report
what should be the ideal approach we can take in such system where number of users are very much likely to expand to tens of thousand in a year after the release
we are using java, spring, hibernate, and redis cache on server side and angular2 on client side
I'm definitely not an expert Java coder, I need to implement sessions in my java Servlet based web application, and as far as I know this is normally done through HttpSession. However this method stores the session data in the local filesystem and I don't want to have such constraints to horizontal scalability. Therefore I thought to save sessions in an external database to which the application communicates through a REST interface.
Basically in my application there are users performing some actions such as searches. Therefore what I'm going to persist in sessions is essentialy the login data, and the meta data associated to searches.
As the main data storage I'm planning to use a graph noSQL database, the question is: let's say I can eventually also use another database of another kind for sessions, which architecture fits better for this kind of situation?
I currently thought to two possible ways. the first one uses another db (such as an SQL db) to store sessions data. In this way I would have a more distributed workload since I'm not using the main storage also for sessions. Moreover I'd also have a more organized environment being session state variables and persisten ones not mixed up.
The second way instead consists in storing every information relative to any session into the "user node" of the main database. The sessionid will be at this point just a "shortcut" for an authentication. This way I dont have to rely on a second database, however I move all the workload to the main db mixing the session data with the persistent ones.
is there any standard general architecture to which I can ake reference? DO I miss some important point which should constraint my architecture?
Your idea to store sessions in a different location is good. How about using an in-memory cache like memcached or redis? Session data is generally not long-lived so you have other options other than a full-blown database. Memcached & Redis can both be clustered and can scale horizontally.