I have a question form my brand new Java web application. In a web server, what is the biggest problem for a site that have increased your visits every day? Is memory a problem in the future? My application uses a simple J2EE, Tomcat, JPA and Hibernate.
I was a PHP developer, and for each visitor of my site, I use a little bit more memory, and in Java, how works?
Like PHP a Java web applications uses a bit of memory for each concurrent request. Thus the more concurrently running requests, the bigger the memory foot print becomes. The total required memory under certain loads depends on how fast each request is processed, because faster processing means less concurrent requests.
I also assume that a PHP web application will use very little initial memory at startup, but will use more memory for each request compared to a Java web application. The cause is that Java web application typically keeps more object preloaded and API’s like Hybernate are often configured to use database connection pooling and object caches.
It depends on how many objects you are using...in java it is usually memory issue, which is caused by a fact that you are creating DOM model of documents for example.
But if it is a simple web application, then the issue should be the fact, that there is always one servlet instance handling requests, so you wouldn't go out of memory, but it would get very slow. The threads from tomcat would have to wait until the request is processed for another one to be executed.
There are simply limits for the number of requests per second...But as I said, it is more likely then that you would get out of memory.
Related
In Vaadin Flow web apps, the state of the entire user-interface is maintained in the session on the web server, with automatic dynamic generation of the HTML/CSS/JavaScript needed to represent that UI remotely on the web browser client. Depending on the particular app, and the number of users, this can result in a significant amount of memory used on the web container.
Is it possible to limit the amount of memory a session and requests related to it can use?
For example, I would like to limit each user session to one megabyte. This limit should apply to any objects created when handling requests. Is that possible?
It is theoretically possible, but it is not practical.
As far as I am aware, no JVM keeps track of the amount of memory that (say) a thread allocates. So if you wanted to do this, you would build a lot of infrastructure to do that. Here are a couple of theoretical ideas.
You could use bytecode engineering to inject some code before each new to measure and record the size of the object allocated. You would need to run this across your entire codebase ... including any Java SE classes and 3rd-party classes that you app uses.
You could modify the JVM to record the information itself. For example, you might modify the memory allocator that new uses.
However, both of these are liable be a lot of work to implement, debug and maintain. And both are liable to have significant performance impact.
It is not clear to me why you would need this ... as a general thing. If you have a problem with the memory usage of particular types of requests, then it would be simpler for the request code itself to keep tabs on how big the request data structures are getting. When the data structures get too large, the request could "abort" itself.
As the correct Answer by Stephen C explains, there is no simple automatic approach to limiting or managing the memory used in Java.
Given the nature of Vaadin Flow web apps, a large amounts of memory may be consumed on the server for user sessions containing all the state of each user’s user-interface.
Reduce memory usage of your codebase
The first step is to examine your code base.
Do you have data replicated across users that could instead be shared across users in a thread-safe manner? Do you have cached data not often used that could instead be retrieved again from its source (database, web services call)? Do you cache parts of the UI not currently onscreen that could instead be instantiated again later when needed?
More RAM
Next step is to simply add more memory to your web server.
Buying RAM is much cheaper than paying for the time of programmers and sysadmins. And so simple to just drop in more stocks of memory.
Multiple web servers
The next step after that is horizontal scaling: Use multiple web servers.
With load balancers you can spread the user load across servers fairly. And “sticky” sessions can be used to direct further user interactions to the same server to continue a session.
Of course, this horizontal scaling approach is more complicated. But this approach is commonly done in the industry, and well-understood.
Vaadin Fusion
Another programming step could involve refactoring app to build parts of your app using Vaadin Fusion.
Instead of your app being driven from the server as with Vaadin Flow, Fusion is focused on web components running in the browser. Instead of writing in pure Java, you write in TypeScript, a superset of JavaScript. Fusion can make calls into Vaadin Flow server as needed to access data and services there.
Consulting
The Vaadin Ltd company sells consulting services, as do others, to assist with any of these steps.
Session serialization
Be aware that without taking these steps, when running low on memory, some web containers such as Apache Tomcat will serialize sessions to disk to purge them from memory temporarily.
This can result in poor performance if the human users are actively still engaged with those sessions. But the more serious problem is that all the objects in your entire sessions must be serializable. And you must code for reconnecting database connections, etc. If supporting such serialization is not feasible, you likely can turn off this serialize-sessions-on-low-memory feature of the web server. But then your web server will suffer when running out of memory with no such recourse available.
We have a costumer that have 3 stores with different databases. In every store has a wildfly running some webservices which communicate between them. Each json request with 10/30 rows spends 1 seconds in average. Every wildfly uses 1,5 gb of RAM. I know that memory is always a problem in Java, but can I be more economic using some microframework like Javalin or microservices rather than a java ee app server? And node.js would be a option for better performance?
Before you start looking into a different architecture, which would probably make for a major rewrite, find out where all that time is going. Set up profiling on the WildFly servers. Start by doing that on one, then have some calls coming in. Check how much time is spent in various parts of the stack. Is one call to the web service handled rather slowly? Then see where that time goes. It might be the database access. Is one such call handled pretty quickly on the server itself once it comes in? Then your best bet is your losing time on the network layer.
Check the network traffic. You can use Wireshark or a similar tracing tool for this. See how much time actually passes between a request coming in and the response going out. Is that slow but the processing on Wildfly itself seems fast enough? Maybe there's some overhead going on (like security). Is the time between request and response very fast? You're definitely looking at the network as the culprit.
Eventually you may need to have profiling and network tracing active on all three servers simultaneously to see what's going on, or for each combination of two servers. It may turn out only one of them is the bottleneck. And if you have servers A, B and C, from the sound of it your setup might cause a call from A to B to also require a call from B to C before some result can be returned to A. If that is the case, it's little wonder you may see some serious latency.
But measure and find the root of the problem before you start deciding to change the entire framework and a different programming language. Otherwise you may put a lot of time into something for no improvement at all. If the architecture is fundamentally flawed you need to think of a different approach. If this is still in the prototyping phase that would be substantially easier.
Well, first you may prune your WildFly installation or try Quarkus :)
I am trying to determine what part of my app code is needing a large amount of memory when handling client requests.
For this I am using VisualVM and JConsole, in the local development server, but the requests have become pretty complex and it is very hard to track down the memory consumption of the requests, and I have no idea how to proceed.
One request, from start to finish, usually uses: Search API, Entity Low level API (datastore access), Java reflection for entity conversion (low level to java plain objects), GWT RPC. so there are tens or hundreds of classes to look for (my code) at least.
I would like to know:
is it ok to make tests in local dev server, since the environment is very different from the one in production mode? I believe it should't be a problem if I know specifically "where /how to look" for memory.
what tools or patterns can I use to track down the memory used by one request (and then use it to estimate for N clients running simulateous requests).
I believe that indeed the memory needed has become very large, but I need to know how much of my code can I optimize (or where do I have code problems, garbage, libs etc) and at what point should I increase the instance types (to F4) or event switch from standard to flexible environment.
Also, if there are java tools/APIs to programmatically determine memory consumption, please advise!
Thank you.
First of all, I have a conceptual question, Does the word "distributed" only mean that the application is run on multiple machines? or there are other ways where an application can be considered distributed (for example if there are many independent modules interacting togehter but on the same machine, is this distributed?).
Second, I want to build a system which executes four types of tasks, there will be multiple customers and each one will have many tasks of each type to be run periodically. For example: customer1 will have task_type1 today , task_type2 after two days and so on, there might be customer2 who has task_type1 to be executed at the same time like customer1's task_type1. i.e. there is a need for concurrency. Configuration for executing the tasks will be stored in DB and the outcomes of these tasks are going to be stored in DB as well. the customers will use the system from a web browser (html pages) to interact with system (basically, configure tasks and see the outcomes).
I thought about using a rest webservice (using JAX-RS) where the html pages would communicate with and on the backend use threads for concurrent execution.
Questions:
This sounds simple, But am I going in the right direction? or i should be using other technologies or concepts like Java Beans for example?
2.If my approach is fine, do i need to use a scripting language like JSP or i can submit html forms directly to the rest urls and get the result (using JSON for example)?
If I want to make the application distributed, is it possible with my idea? If not what would i need to use?
Sorry for having many questions , but I am really confused about this.
I just want to add one point to the already posted answers. Please take my remarks with a grain of salt, since all the web applications I have ever built have run on one server only (aside from applications deployed to Heroku, which may "distribute" your application for you).
If you feel that you may need to distribute your application for scalability, the first thing you should think about is not web services and multithreading and message queues and Enterprise JavaBeans and...
The first thing to think about is your application domain itself and what the application will be doing. Where will the CPU-intensive parts be? What dependencies are there between those parts? Do the parts of the system naturally break down into parallel processes? If not, can you redesign the system to make it so? IMPORTANT: what data needs to be shared between threads/processes (whether they are running on the same or different machines)?
The ideal situation is where each parallel thread/process/server can get its own chunk of data and work on it without any need for sharing. Even better is if certain parts of the system can be made stateless -- stateless code is infinitely parallelizable (easily and naturally). The more frequent and fine-grained data sharing between parallel processes is, the less scalable the application will be. In extreme cases, you may not even get any performance increase from distributing the application. (You can see this with multithreaded code -- if your threads constantly contend for the same lock(s), your program may even be slower with multiple threads+CPUs than with one thread+CPU.)
The conceptual breakdown of the work to be done is more important than what tools or techniques you actually use to distribute the application. If your conceptual breakdown is good, it will be much easier to distribute the application later if you start with just one server.
The term "distributed application" means that parts of the application system will execute on different computational nodes (which may be different CPU/cores on different machines or among multiple CPU/cores on the same machine).
There are many different technological solutions to the question of how the system could be constructed. Since you were asking about Java technologies, you could, for example, build the web application using Google's Web Toolkit, which will give you a rich browser based client user experience. For the server deployed parts of your system, you could start out using simple servlets running in a servlet container such as Tomcat. Your servlets will be called from the browser using HTTP based remote procedure calls.
Later if you run into scalability problems you can start to migrate parts of the business logic to EJB3 components that themselves can ultimately deployed on many computational nodes within the context of an application server, like Glassfish, for example. I don think you don't need to tackle this problem until you run it to it. It is hard to say whether you will without know more about the nature of the tasks the customer will be performing.
To answer your first question - you could get the form to submit directly to the rest urls. Obviously it depends exactly on your requirements.
As #AlexD mentioned in the comments above, you don't always need to distribute an application, however if you wish to do so, you should probably consider looking at JMS, which is a messaging API, which can allow you to run almost any number of worker application machines, readying messages from the message queue and processing them.
If you wanted to produce a dynamically distributed application, to run on say, multiple low-resourced VMs (such as Amazon EC2 Micro instances) or physical hardware, that can be added and removed at will to cope with demand, then you might wish to consider integrating it with Project Shoal, which is a Java framework that allows for clustering of application nodes, and having them appear/disappear at any time. Project Shoal uses JXTA and JGroups as the underlying communication protocol.
Another route could be to distribute your application using EJBs running on an application server.
I have written a standalone Java application that I've packaged into a jar file that takes in some command line arguments, does some hardcore computations, and then writes out the result to a file along with some output to the default output stream pointing to where the file with the results are.
I now want to create a website around this technology. The idea is that the user can fill in an html form, post it to a webpage, which would then call the Java application, parse the results from the Java app, and display it to the user.
Currently, I am using a little bit of PHP to collect the data from the post request, and then just using an exec call: java -jar -Xmx128m myapplication.jar command-line-arguments
Is this bad?
I do have several thousand visits to my website each day and each execution of the Java application can take upwards of 30 seconds to a minute, so I don't want to be overly inefficient. It seems like there would be a better solution than having to call Java directly for every request.
I keep hearing things like java servlets, beans, tomcat, glassfish, etc., but I don't understand what they are and how they would benefit me. What do these get me? Faster results because the Java JVM doesn't have to be created each time I run the application? Less memory usage? I obviously want it to run as fast as possible with as little memory footprint as possible.
So, what is the best approach that I can take here? I don't want to do any serious rewriting of my application as there is a lot of code (so rewriting it to C or C++ is out of the question).
Thanks.
Ok, servlets are smalish applications that are designed to run inside of a container. They provide an extension point for you to insert your java code into either a simple servlet container like tomcat, or a more fully featured application server like glassfish. You want to do this because the application server does the heavy lifting of dealing with the http interaction and provides other features like security, logging, session management, error handling, and more (see the servlet specification).
When you make your application live within an application conatiner (web server with all those other extra features), you can also manage the lifecycle of your application better. You'll be able to start and stop the application without shutting down the web server, redeploy, start more instances, etc. Plus, when you come up with that great second application, its easy to drop it in right next to the first one. Or, you can cluster several machines together for easy redundancy and load balancing, features of the application server.
This is just a start, there are many more features, technologies, and frameworks out there to help you make container based applications. Servlet tutorial.
[Do these get me] "Faster results because the Java JVM doesn't have to be created each time I run the application?"
Yes.
And -- bonus -- you can replace PHP so your entire site is in a single language: Java.
Further, you can consider revising your use cases so it isn't a painful 30-60 seconds in one shot, but perhaps a series of quicker steps executed interactively with the user.
Run your code inside a servlet container.
Assuming that you need to keep your website in PHP and as you already have java installed on your machine, simply install a free servlet container (such as Apache Tomcat, or Jetty). Configure to run the servlet container on an unused port. (8080) is their default.
These servlet containers are really java based webservers, just like Apache, however specialized in serving java code.
The most obvious advantage of using a java webserver rather than a new java.exe invocation for each request, is that your java virtual machine (jvm) will always be "hot", up and running. Each new start of the java.exe (jvm) will give you those extra seconds of waste.
The second advantage of using a servlet container, is that the container will enable your code to run in a new thread, inside the jvm, for each new request. You will have no problem providing your service to your thousands of users a day. Most likely, your machine will crash if you were to start hundreds of java instances rather than one.
Place your code inside a servlet. It really is easy even for a newcomer. You will talk to the servlet via HTTP (doGet or doPost methods of the servlet). Pass the php request form to this servlet and have the servlet give you back whatever: a page, a json object, xml or plain text.
You probably don't want to invoke the java app directly from the website. Like you said, if the java process takes 30 seconds to run, your web server is going to get way bogged down, especially if your site is getting pounded.
You may want to look into web-services (and possibly a message queue) for dispatching back-end processing requests. The PHP page could call the web-service on the server, which could then put a processing request on a queue, or just kick off the java app in an asynchronous fashion. You don't want the HTTP request to wait for the java app to finish, because, while it's processing, the user will just have a hung browser, and the HTTP request might timeout.
Once the java app finishes, it could update a database table, which the user could then access from the website.
The easiest thing to start with would be to embed a webserver in you application. Have a look at Jetty.