I am developing a simple REST API using Spring 3 + Spring MVC. Authentication will be done through OAuth 2.0 or basic auth with a client token using Spring Security. This is still under debate. All connections will be forced through an SSL connection.
I have been looking for information on how to implement rate limiting, but it does not seem like there is a lot of information out there. The implementation needs to be distributed, in that it works across multiple web servers.
Eg if there are three api servers A, B, C and clients are limited to 5 requests a second, then a client that makes 6 requests like so will find the request to C rejected with an error.
A recieves 3 requests \
B receives 2 requests | Executed in order, all requests from one client.
C receives 1 request /
It needs to work based on a token included in the request, as one client may be making requests on behalf of many users, and each user should be rate limited rather than the server IP address.
The set up will be multiple (2-5) web servers behind an HAProxy load balancer. There is a Cassandra backed, and memcached is used. The web servers will be running on Jetty.
One potential solution might be to write a custom Spring Security filter that extracts the token and checks how many requests have been made with it in the last X seconds. This would allow us to do some things like different rate limits for different clients.
Any suggestions on how it can be done? Is there an existing solution or will I have to write my own solution? I haven't done a lot of web site infrastructure before.
It needs to work based on a token included in the request, as one client may be making requests on behalf of many users, and each user should be rate limited rather than the server IP address.
The set up will be multiple (2-5) web servers behind an HAProxy load balancer. There is a Cassandra backed, and memcached is used. The web servers will be running on Jetty.
I think the project is request/response http(s) protocol. And you use HAProxy as fronted.
Maybe the HAProxy can load balancing with token, you can check from here.
Then the same token requests will reach same webserver, and webserver can just use memory cache to implement rate limiter.
I would avoid modifying application level code to meet this requirement if at all possible.
I had a look through the HAProxy LB documentation nothing too obvious there, but the requirement may warrant a full investigation of ACLs.
Putting HAProxy to one side, a possible architecture is to put an Apache WebServer out front and use an Apache plugin to do the rate limiting. Over-the-limit requests are refused out front and the application servers in the tier behind Apache are then separated from rate limit concerns making them simpler. You could also consider serving static content from the Web Server.
See the answer to this question How can I implement rate limiting with Apache? (requests per second)
I hope this helps.
Rob
You could put rate limits at various points in the flow (generally the higher up the better) and the general approach you have makes a lot of sense. One option for the implementation is to use 3scale to do it (http://www.3scale.net) - it does rate limits, analytics, key managed etc. and works either with a code plugin (the Java plugin is here: https://github.com/3scale/3scale_ws_api_for_java) which pushes or by putting something like Varnish (http://www.varnish-cache.org) in the pipeline and having that apply rate limits.
I was also thinking of the similar solutions a couple of day's ago. Basically, I prefer the "central-controlled" solution to save the state of the client request in the distributed environment.
In my application, I use a "session_id" to identify the request client. Then create a servlet filter or spring HandlerInterceptorAdapter to filter the request, then check the "session_id" with the central-controlled data repository, which could be memcached, redis, cassandra or zookeeper.
We use redis as leaky bucket backend
Add a controller as entrance
google cache that token as key with expired time
then filter every request
It is best if you implement ratelimit using REDIS. For more info please look this Rate limiting js Example.
Related
We have 13 years old monolithic java application using
Struts 2 for handling UI calls
JDBC/Spring JDBC Template for db calls
Spring DI
Tiles/JSP/Jquery for UI
Two deployables are created out of this single source code.
WAR for online application
JAR for running back-end jobs
The current UI is pretty old. Our goal is to redesign the application using microservices. We have identified modules which can run as separate microservice.
We have following questions in our mind
Which UI framework should we go for (Angular/React or a home grown one). Angular seems to be very slow and we need better performance as far as page loading is concerned.
Should UI/Javascript make call to backend web services directly or should there be a spring controller proxy in deployed WAR which kind of forwards UI calls to APIs. This will also help if a single UI calls requires getting/updating data from different microservice.
How should we cover microservice security aspect
Which load balancer should we go for if we want to have multiple instance of same microservice.
Since its a banking application, our organization does not allow using Elastic Search/Lucene for searching. So need suggestion for reporting using Oracle alone.
How should we run backend jobs?
There will also be a main payment microservice which will create payments. Since payments volume is huge hence it will require multiple instances. How will we manage user logged-in session. Should we go for in-memory distributed session store (may be memcache)
This is a very broad question. You need to get a consultant architect to understand your application in depth, because it is unlikely you will get meaningful in-depth answers here.
However as a rough guideline here are some brief answers:
Which UI framework should we go for (Angular/React or a home grown one). Angular seems to be very slow and we need better performance as far as page loading is concerned.
That depends on what the application actually needs to do. Angular is one of the leading frameworks, and is usually not slow at all. You might be doing something wrong (are you doing too many granular calls? is your backend slow?). React is also a strong contender, but seems to be losing popularity, although that is just a subjective opinion and could be wrong. Angular is a more feature complete framework, while React is more of a combination of tools. You would be just crazy if you think you can do a home grown one and bring it to the same maturity of these ready made tools.
Should UI/Javascript make call to backend web services directly or
should there be a spring controller proxy in deployed WAR which kind
of forwards UI calls to APIs. This will also help if a single UI calls
requires getting/updating data from different microservice.
A lot of larger microservice architectures often involve an API gateway. Then again it depends on your use case. You might also have an issue with CORS, so centralising calls through a proxy / API gateway, even if it is a simple reverse proxy (you don't need to develop it) might be a good idea.
How should we cover microservice security aspect.
Again no idea what your setup looks like. JWT is a common approach. I presume the authentication process itself uses some centralised LDAP / Exchange or similar process. Once you authenticate you can sign a token which you give to the client, which is then passed to the respective micro services in the HTTP authorization headers.
Which load balancer should we go for if we want to have multiple
instance of same microservice.
Depends on what you want. Are you deploying on a cloud based solution like AWS (in which case load balancing is provided by the infrastructure)? Are you going to deploy on a Kubernetes setup where load balancing and scaling is handled as part of its deployment fabric? Do you want client-side load balancing (comes part of Spring Cloud)?
Since its a banking application, our organization does not allow using
Elastic Search/Lucene for searching. So need suggestion for reporting
using Oracle alone.
Without knowledge of how the data on Oracle looks like and what the reporting requirements are, all solutions are possible.
How should we run backend jobs?
Depends on the infrastructure you choose. Everything is possible, from simple cron jobs, to cloud scheduling services, or integrated Java scheduling mechanisms like Quartz.
There will also be a main payment microservice which will create
payments. Since payments volume is huge hence it will require
multiple instances. How will we manage user logged-in session. Should
we go for in-memory distributed session store (may be memcache)
Not really. It will defeat the whole purpose of microservices. JWT tokens will be managed by the client's browser and expire automatically. You don't need to manage user logged-in session in such architectures.
As you have mentioned it's a banking site so security will be first priory. Here I have few suggestions for FE and BE.
FE : You better go with preactjs it's a react like library but much lighter and fast as compare to react. For ui you can go with styled components instead of using some heavy third party lib. This will also enhance performance and obviously CDNs for images and big files.
BE : As per your need you better go with hybrid solution node could be a good option.e.g. for sessions.
Setup an auth server and get you services validate user from there and it will be used in future for any kinda service .e.g. you will expose some kinda client API's.
User case for Auth : you can use redis for session info get user validated from auth server and add info to redis later check if user is logged in from redis this will reduce load from auth server. (I have used same strategy for a crypto exchange and went pretty well)
Load balancer : Don't have good familiarity with java but for node JS PM2 will do that for you not a big deal just one command and it will start multiple instances and will balance on it's own.
In case you have enormous traffic then you better go with some messaging service like rabbitmq this will reduce cost of servers by preventing you from scaling your servers.
BE Jobs : I have done that with node for extensive tasks and went quite well there you can use forking or spanning this will start a new instance for particular job and will be killed after completing it and you can easily generate logs along with that.
For further clarification I'm here :)
Within our corporate intranet, we have a few end-point service platforms like BPM, document management system, etc. These end-point services expose REST API. We develop web applications using AngularJS as front end.
There are two options on how we can make calls from AngualJS to these end-point services.
Option 1: Given these end-point services expose REST, call these REST API directly from AngualrJS.
Option 2: Introduce a middle layer (on an application server like WebLogic or Tomcat). Build a Java application layer that calls into the end-point REST API; and host it on this millde layer. The AngularJS calls into REST provided by this middle layer; this middle layer inturn calls into the end-point REST.
I personally prefer the Option 1; however I invite your openion on this matter. I have listed the pros and cons of Option 1 as I see them.
Pros of Option 1:
Better performance (throughput) given one less hop for HTTP requests.
Lesser development/deployment efforts due to one less component.
Lesser number of points of failure. If there is an issue, we know its either in AngualrJs or the end service.
Cons of Option 1:
Security issues? Not sure of this - would like expert comments on this.
CORS: the end services will need to enable Access-Control-Allow-Origin to appropriate domains.
Poor logging? If something goes wrong, the logs will be available only on user machines (IE/Chrome development tool) or on the end service.
Too much processing in AngualJS layer? This processing is mainly parsing the result from end service. This also depends on the kind of end service that is being used.
option 2 in my opinion is a better option in long run. There are few reasons for that.
Security is first and foremost, If you have a middleware in between, you can have inherent security, which means you can expose only those REST APIs which your angular webapp needs. You can also include a security mechanism like oAuth since you control the middleware.
Logging is another one. for sure any application nowadays do need some sort of auditing. both security and logging are layers before your actual REST calls.
You would be able to add some aspects on any key REST API, such that in case if that API is called trigger a mail, it's always handy to have those flexibility even we don't need at the moment.
You can include response transformation and error handling efficiently. Once you get the response from service, in your middleware you can transform the response, remove unnecessary or critical fields, conjure some values etc. This all can be done with angular also but then the real response or error is exposed to the client.
On the downside you rightly mentioned performance is one but imo keeping your REST middlware in sync with services REST is more bane. any new API added by services, needs to be included in middleware, recompiled and redeployed. But it also depends what are the likelihood and frequency of those changes? for any those changes you anyhow might need to change in angular webapp to include it.
You mention "Within our corporate intranet". Depending on how the end-points are secured, option 1 could be challenging.
Angular will run in a web-browser so if those services are only accessible via VPN / intranet, the web-app will only work if your computer is connected to that intranet (i.e. it won't work if you run it from home).
Another security challenge with option 1 is that if the end-points require special authentication "secrets" (API tokens, passwords, certificates, etc.), those secrets will be exposed and visible to anyone who uses the web-app since anyone can see the traffic between their browser and the server. With option 2, those secrets can stay hidden behind your middle layer.
Lastly, even if Angular talks to those end-points directly, you will still need to have the HTML / JS / CSS hosted on some web-server. You may not need a full blown application server but you'll need something to point your web browser at.
If those concerns don't apply to your case, then it's really up to you to pick whichever option you and your team are the most comfortable with.
Thanks for such a nice article.
If you are concern with security and your project requirement is focused on Security. One must go with Option 2.
If Security is not a big concern. Options 2 is better.
We are using Tomcat 7 for our web application. We provide an XML based API so that our customers can communicate with our server in a machine-to-machine way (no web browser needed). The requests are processed by a servlet.
We need to prevent users from sending too many requests in a row. Some of the services we provide involve polling for results and users may make requests in a loop without any pauses, making dozens of requests per second for nothing.
How can we protect ourselves from being flooded with useless requests? Is there a simple way to block requests at the servlet entry level when there are too many requests originating from the same IP? Is there something built-in Tomcat to deal with this problem?
Assuming that you are using an apache reverse-proxy in front of tomcat (if you aren't you should be), use mod_cband on the apache layer.
You could code your own.
Starting points for looking at this would be the Servlet API, in particular the Filter interface and the getRemoteHost() method of the SerlvetRequest interface.
Should be easy enough to write a Filter implementation which stores a count of requests from each host and takes action if a limit exceeded.
Spring Security has a lot of the features of Apache httpd's mod_security if you want a Java-only solution.
Apache's mod_evasive
or mod_security
could cover for your need here. You may consider Cloudflare for more complexly serious attacks that will require hardware protection.
Consider the following architecture that has two web-servers(for balancing) and around 10 java app servers for servlet processing.
Where do you think is the right place for doing authentication. In ApacheWebserver or custom code at one of the app servers?
If I do authentication at Webserver how should we handle custom things like OPenId?
EDIT: Also what are the implications of choosing either on Performance, Scalability and Security?
As usual with architecture question the answer depends on many things.
Do the web servers do anything that needs protection (like serving static content. Then you'll have to do authentication on (or before) the webserver
are the web servers or the app servers close to maximum capacity? Try to do authentication on the side
Can the web server handle all the authentication requirements? Or is logic/information needed that is not available on the web server (like the reputation here at SO which affects, what user may do). The requirement to support OpenId might be such a limiting factor.
Finally note that there are not so obvious attacks which might be affected by where you do authentication. e.g. if the web server handles 404 (not found) but the app server handles authentication (401) an attacker can find out if a resource is available or not even if he doesn't gain access. Since replying will take longer if the app server needs to be accessed this information might even leak to the attacker when in both cases an identical response is returned.
I have a Java SOAP data service which sits on top of a Sybase database which, for reasons out of my control, has unreliable performance. The database is part of a vendor package which has been modified by an internal team and most of the issues are caused by slow response times at certain times of the day.
The SOAP service provides data to a calculation grid and when I request data, I need the response time to be both fast and consistent. The service provides basic CRUD functionality, but the ratio of reads to writes is approximately 100:1.
What is the best strategy to isolate myself from the database's unreliable performance and ensure that the SOAP service is fast and reliable?
I have seen this issue a few times, normally with a vendor database.
If this is on Windows, you could create a Windows service as an intermediary between the SOAP service and the database. Then put a message queue (either MSMQ or a JMS implementation such as MQ Series) between the SOAP service and Windows service for asynchronous communications. In this way the database performance issues will no longer affect the SOAP service. This solution does, however, come at the cost of increased complexity.
Note that a .NET web service can be called by, and respond asynchronously to, its clients. I'm not sure if that's possible with a Java SOAP service.
If this is on some flavour of Unix, I assume it has similar functionality to a Windows service - maybe a daemon.
Why not use a thread? That way, the application could gently wait even if the database is slow.
RoadWarrior's response is right on. Requests to do any operation get put in a queue. The user comes in once to make the request, and once to pick up the request. This is in fact what is happening on sites like Expedia where it is talking to an unreliable service (the backend). The user's browser is pinging the server until the red light turns green.
How about caching the responses from the web service (either on the client invoking the WS request, or by setting up a proxy web service in between)?
You could cache the results from the DB if the DB Is not too big.
Get the other internal team to tune that database, so everyone using the app benefits. I do love me some indexes!