I have built a java program within an android app that parses some data from an online website. This data is to be used from the app to show the content(organized) to the user. I tried to optimize my code as much as I could but there's not much I can do since it's a lot of data. So what I did instead was deploying the java program on HEROKU and let the server side do the work and give the result in a simple html which I can easily parse with no major delay. The thing is that this worked out pretty fine. I got a high increase in performance yet with one little problem. When I open the app for the first time in say 2 days it results to be a lot slower, but on the second run just after that it seems to be a lot faster. Now I am guessing that the HEROKU server works on some cache-like way in that the least recent run dynamic web sites get no priority until a request comes from the client side, considering that in 2,3 consequent runs I get a very high increase in performance. Now, my question is, is there a way I can sort of "give priority" to my HEROKU java program or is there another free dynamic web site that allows you to deploy a war and presents no such performance issues. To some it might seem as a no big deal. In particular I get performance increase of say from 6 seconds to 2 seconds which is actually quite a big deal since app users usually do not tolerate such kinds of delays.
Heroku puts free apps to sleep. The first request after it sleeps will restart the app, which means you have to wait longer.
For more info see Heroku's sleeping policy for Free dynos.
Related
Context: I am learning to code with a course run by my country's government. We are using Angular, Java Spring Boot and SQL to create a portfolio. They did not go into much detail on different things, and they pointed us to Google (and Stack Overflow), as well as cooperation between students, for any doubts. I managed to create an app, and is currently hosted on Firebase (frontend), Heroku (backend) and CleverCloud (database), with the free versions of each.
When running the app locally, it starts in around 10 seconds. When running the app in Heroku or doing clean and build in Apache Netbeans while connected to CleverCloud, it starts in around 2 minutes. I would like to reduce the start up as much as possible. The obvious solution would probably be to use a non free version of Heroku and not let the app stop, but given both my country and the course aiming for the free version, that is not an option.
What I tried:
Deleting unused imports on entities and adding the "Excluding test
dependencies" part in this Heroku guide:
https://devcenter.heroku.com/articles/reducing-the-slug-size-of-java-applications,
but after doing a clean and build in Apache Netbeans, the load time
was around 2 minutes, and it took like another minute to do other
processes that were not there before. I ended rolling back the changes.
Added a loading screen to the frontend to tell users that Heroku takes 2 minutes on startup, and a timeout of like 1800000 on the HTTP calls made from the frontend, but the HTTP call still crashes after like a minute, making it so user have to refresh the page and wait.
I would like, as the app is going to be checked at some point, to either have the app running nonstop on weekdays on workhours or for it to start fast. I saw some pages like https://kaffeine.herokuapp.com/ that ping the app, yet I dont know how to use them o if I can set them for free to be used on weekdays and workhours.
Given the startup is slow on both Heroku and while doing a clean and build in Apache Netbeans while connecting to CleverCloud, I dont even know if the issue is on Heroku itself. As someone who is completely new to coding, and its exposure has been a "do this" without much explanation, this is kinda frustrating. I have seen something called "cron", but I dont understand how they work or to create them, or if they would work on a Java app that only supports CRUD (maybe doing this cron thing on the frontend, but still I dont know anything about them).
TLDR: Are there any ways to speed up the build or start up of a Java app, or guides to autoping free Heroku apps on weekdays and workhours for dummies?
Sorry if the question is obvious, as I am a total beginner that had to rely on code along guides and videos.
For the Free & Hobby version on Heroku you application on Free version will go to sleep which after arriving on the site will take a while to load up.
Sleeps after 30 mins of inactivity, otherwise always on depending on your remaining monthly free dyno hours.
For 7 bucks a month you can remove this restriction and get more power Hobby version - I think it's totally worth it and I use this for my personal projects quite a lot. https://www.heroku.com/pricing
If you use something to ping Heroku to keep it alive you will run out of dyno hours (hours that your app can run) and even then the free tier will have hard time running applications.
Also Heroku supports SQL (Postgres) so having db, frontend and backend in the same place is a bonus. Everything runs smoothly, environment variables and automatic deploys are easy to set up.
If it's your first time on AWS you could also use free tier over there for some time and deploy using elastic beanstalk. But AWS infrastructure can be overwhelming for one simple free project - gotta watch out when they start charging you.
I have a rest service based on Spark Java 2.5, which uses Jetty server under the hood.
My problem is that it doesn't work on a constant performance and suddenly "decides to trade" cpu to memory and vise versa some time later.
Plots are created using Java melody.
As you can see - at about 18:00 performance plots abruptly changed. Memory consumption began to grow and processor load goes down. At the same time, request latencies didn't change as well as request per|second and request types. Additional parameters also changed - especially used buffered memory and number of opened files.
A week later everything will change back and maybe two weeks or a month later the cycle repeats, I saw this cycle for the last three months.
I tried to use profiler, but didn't find anything useful.
I'm pretty sure such change is not provoked by business logic, because nothing is changed in user's interaction with web server and no background tasks are active, so probably it's jetty's or java's internals or misconfiguration.
Server runs on a Java 8 in a Docker container in AWS EC2 (we use AWS ECS for docker autoscaling). There is about 50 requests per second load. The api itself uses spring and hibernate with postgresql 9.4 driver. For hibernate second level ehcache is used. Some of api requests are multipart with size about 100kb and they are uniformly distributed on a request timeline. Java is started with parameters: -server -Xmx6000m -XX:+UseG1GC. If more details needed, please ask me.
What I want is a constant performance. If my problem resonates with your experience, please reply.
ps: on ~23:30 change is not related to the problem, so don't analyze it.
Finally the solution was simple. Increasing -Xms to 500mb helped, after it behavior became stationary.
I've developed a web application using the following tech stack:
Java
Mysql
Scala
Play Framework
DavMail integration (for calender and exchange server)
Javamail
Akka actors
On the first days, the application runs smoothly and without lags. But after 5 days or so, the application gets really slow! And now I have no clue how to profile this, since I have huge dependencies and it's hard to reproduce this kind of thing. I have looked into the memory and it seems that everything its okay.
Any pointers on the matter?
Try using VisualVM - you can monitor gc behaviour, memory usage, heap, threads, cpu usage etc. You can use it to connect to a remote VM.
`visualvm˙ is also a great tool for such purposes, you can connect to a remote JVM as well and see what's inside.
I suggest you doing this:
take a snapshot of the application running since few hours and since 5 days
compare thread counts
compare object counts, search for increasing numbers
see if your program spends more time in particular methods on the 5th day than on the 1str one
check for disk space, maybe you are running out of it
jconsole comes with the JDK and is an easy tool to spot bottlenecks. Connect it to your server, look into memory usage, GC times, take a look at how many threads are alive because it could be that the server creates many threads and they never exit.
I agree with tulskiy. On top of that you could also use JMeter if the investigations you will have made with jconsole are unconclusive.
The probable causes of the performances degradation are threads (that are created but never exit) and also memory leaks: if you allocate more and more memory, before having the OutOfMemoryError, you may encounter some performances degradation (happened to me a few weeks ago).
To eliminate your database you can monitor slow queries (and/or queries that are not using an index) using the slow query log
see: http://dev.mysql.com/doc/refman/5.1/en/slow-query-log.html
I would hazard a guess that you have a missing index, and it has only become apparent as your data volumes have increased.
Yet another profiler is Yourkit.
It is commercial, but with trial period (two weeks).
Actually, I've firstly tried VisualVM as #axel22 suggested, but our remote server was ssh'ed and we had problems with connecting via VisualVM (not saying that it is impossible, I've just surrendered after a few hours).
You might just want to try the 'play status' command, which will list web app state (threads, jobs, etc). This might give you a hint on what's going on.
So guys, in this specific case, I was running play in Developer mode, which makes the compiler works every now and then.
After changing to production mode, everything was lightning fast and no more problems anymore. But thanks for all the help.
We have an Java ERP type of application. Communication between server an client is via RMI. In peak hours there can be up to 250 users logged in and about 20 of them are working at the same time. This means that about 20 threads are live at any given time in peak hours.
The server can run for hours without any problems, but all of a sudden response times get higher and higher. Response times can be in minutes.
We are running on Windows 2008 R2 with Sun's JDK 1.6.0_16. We have been using perfmon and Process Explorer to see what is going on. The only thing that we find odd is that when server starts to work slow, the number of handles java.exe process has opened is around 3500. I'm not saying that this is the acual problem.
I'm just curious if there are some guidelines I should follow to be able to pinpoint the problem. What tools should I use? ....
Can you access to the log configuration of this application.
If you can, you should change the log level to "DEBUG". Tracing the DEBUG logs of a request could give you a usefull information about the contention point.
If you can't, profiler tools are can help you :
VisualVM (Free, and good product)
Eclipse TPTP (Free, but more complicated than VisualVM)
JProbe (not Free but very powerful. It is my favorite Java profiler, but it is expensive)
If the application has been developped with JMX control points, you can plug a JMX viewer to get informations...
If you want to stress the application to trigger the problem (if you want to verify whether it is a charge problem), you can use stress tools like JMeter
Sounds like the garbage collection cannot keep up and starts "halt-the-world" collecting for some reason.
Attach with jvisualvm in the JDK when starting and have a look at the collected data when the performance drops.
The problem you'r describing is quite typical but general as well. Causes can range from memory leaks, resource contention etcetera to bad GC policies and heap/PermGen-space allocation. To point out exact problems with your application, you need to profile it (I am aware of tools like Yourkit and JProfiler). If you profile your application wisely, only some application cycles would reveal the problems otherwise profiling isn't very easy itself.
In a similar situation, I have coded a simple profiling code myself. Basically I used a ThreadLocal that has a "StopWatch" (based on a LinkedHashMap) in it, and I then insert code like this into various points of the application: watch.time("OperationX");
then after the thread finishes a task, I'd call watch.logTime(); and the class would write a log that looks like this: [DEBUG] StopWatch time:Stuff=0, AnotherEvent=102, OperationX=150
After this I wrote a simple parser that generates CSV out from this log (per code path). The best thing you can do is to create a histogram (can be easily done using excel). Averages, medium and even mode can fool you.. I highly recommend to create a histogram.
Together with this histogram, you can create line graphs using average/medium/mode (which ever represents data best, you can determine this from the histogram).
This way, you can be 100% sure exactly what operation is taking time. If you can't determine the culprit, binary search is your friend (fine grain the events).
Might sound really primitive, but works. Also, if you make a library out of it, you can use it in any project. It's also cool because you can easily turn it on in production as well..
Aside from the GC that others have mentioned, Try taking thread dumps every 5-10 seconds for about 30 seconds during your slow down. There could be a case where DB calls, Web Service, or some other dependency becomes slow. If you take a look at the tread dumps you will be able to see threads which don't appear to move, and you could narrow your culprit that way.
From the GC stand point, do you monitor your CPU usage during these times? If the GC is running frequently you will see a jump in your overall CPU usage.
If only this was a Solaris box, prstat would be your friend.
For acute issues like this a quick jstack <pid> should quickly point out the problem area. Probably no need to get all fancy on it.
If I had to guess, I'd say Hotspot jumped in and tightly optimised some badly written code. Netbeans grinds to a halt where it uses a WeakHashMap with newly created objects to cache file data. When optimised, the entries can be removed from the map straight after being added. Obviously, if the cache is being relied upon, much file activity follows. You probably wont see the drive light up, because it'll all be cached by the OS.
I'm planning on hosting a JRuby on Rails app on Google AppEngine/Java. I found a great blog post by Ola Bini on how to to this, but in the summary he says:
Overall, JRuby on Rails works very
well on the App Engine, except for
some smaller details. The major ones
are the startup cost and testing. As
it happens, you can’t actually get
GAE/J to precreate things. Instead
you’ll have to let the first release
take the hit of this. Now, GAE/J does
a let of preverifying of bytecodes and
so on, so startup is a bit more heavy
than on other JDKs. One runtime takes
about 20 seconds wall time to startup,
so the first hit takes some time.
I don't fully understand this. How often, under what circumstances, will a runtime need to be started up? A regular 20 second lag is likely to be an issue.
App Engine will start new runtimes for you whenever demand is outstripping the currently running instances. It will then shut down instances when demand is lower. Ultimately, this means that all of your instances could be shut down if your app is not used for a certain amount of time. Then, the next time a user tries to access your app, a new instance will need to be started, or "spun up" as some people call it.
As of March, the app engine team wouldn't give any official estimate on how long an instance will stay up:
7:40pm] nwinter: Is it possible to get a rough estimate of how long an app
instance will stick around once spawned?
[7:40pm] marzia_google: #nwinter, not really
[7:40pm] marzia_google: there are no garuntees
[7:41pm] nwinter: No average time or anything?
[7:42pm] marzia_google: #nwinter i'm not sure an average time would be
meaningful, even if i knew off hand what it was ( i don't)
[7:42pm] marzia_google: since it really can be quite variable
[7:42pm] Kardax: Re instance lifetime: So an app instance could last a few
seconds or a few hours? Just curious
[7:43pm] dan_google: nwinter: Apps are evicted by least-recently-used on an
app server. As someone noted recently (forums or chat I forget), low
traffic could mean lots of "restarts", but so could spikes in traffic which
may start new instances on multiple app servers.
[7:43pm] nwinter: #dan_google: good to know!
[7:43pm] dan_google: Kardax: Yes, depending on the weather. By which I
mean, request patterns, other apps on each app server, and so forth. Not
really predictable.
This is the transcript of a chat with the app engine team. I have deleted the non-relevant lines in the transcript like "so and so logged in." The full transcript can be found here