I'm developing a game for Android. It uses a surface view and uses the sort of standard 2D drawing APIs provided. When I first released the game, I was doing all sorts of daft things like re-drawing 9-patches on each frame and likewise with text. I have since optimised much of this by drawing to Bitmap objects and drawing them each frame, only re-drawing onto the Bitmap objects when required.
I've received complaints about battery drain before, and following my modifications I'd like to know (scientifically) if I've made any improvements. Unfortunately, I don't have any prior data to go by, so it would be most useful to compare the performance to some other game.
I've been running Traceview, and using the results of it mostly for the purposes of identifying CPU-time-consuming methods.
So -- what's the best way of determining my app's battery performance, and what's a good benchmark?
I know I can look at the %s of different apps through the settings, but this is again unscientific, as the figure I get from this also depends on what's happening in all of the other apps. I've looked through (most of) Google's documentation, and although the message is clear that you should be saving battery (and it gives the occasional tip as to how), there is little indication of how I can measure how well my app is performing. The last thing I want are more complaints of battery drain in the Android Market!
Thanks in advance.
EDIT
Thanks for all your helpful advice/suggestions. What I really want to know is how I can use the data coming from Traceview (ie: CPU time in ms spent on each frame of the game) to determine battery usage (if this is at all possible). Reading back on my original question, I can see I was a bit vague. Thanks again.
Here is my suggestion:
I watch power consumption while developing my apps (that sometimes poll the sensors at rates of <25ns) using PowerTutor. Check it out, it sounds like this maybe what you are looking for, the app tells you what you are using in mW, J, or relative to the rest of the system. Also, results are broken down by CPU, WiFi, Display, (other radios installed). The only catch is that it is written for a specific phone model, but I use it with great success on my EVO 4G, Galaxy S (Sprint Epic), and Hero.
Good luck,
-Steve
There is a possibility that your game is draining battery. I believe this depends on several reasons, which reads as follows:
Your application is a game. Games drains battery quickly.
You're iterating with help from a Thread. Have you limited the FPS to make the CPU skip unnecessary iterations? Since you're working with 2D I assume you're using the SurfaceView. 60 FPS will be enough for a real-time game.
You don't stop the Thread when your application terminates. Hence you reiterate code when your application isn't alive.
Have you an iterate lock that does wait(); during onPause?
The people commenting that your game is leaking battery probably aims when your application isn't in use. Otherwise, it would be wierd because every game on Android Market drains battery - more or less.
If you're trying to gauge the "improvement over your previous version", I don't think it makes sense to compare to another game! Unless those two games do the exact thing, this is as unscientific as it gets.
Instead, I would grab the previous version of your app from source control, run it, measure it, and then run it with the latest code and compare it again.
To compare, you could for example use the command line tool "top" (definitely available in busybox if your phone is rooted, not sure if it comes with a stock phone. Probably not). That shows you the CPU usage of your process in percent.
There is a battery profiler developed by Qualcomm called Trepn Profiler: https://developer.qualcomm.com/mobile-development/increase-app-performance/trepn-profiler
how I can use the data coming from Traceview (ie: CPU time in ms spent on each frame of the game) to determine battery usage (if this is at all possible)
In theory it would be possible to extrapolate the battery usage for your app by looking at the power consumption on a frame by frame basis. The best way to accomplish this would be to evaluate the power consumption of the CPU (only) for a given period (say two seconds) while your app is running the most CPU intensive operation, (additionally, GPU power usage could be gleaned this way also) while recording TraceView data (such as frames per second or flops per second) giving you the the traffic across the CPU/GPU for a given millisecond. Using this data you could accurately calculate the average peak power consumption for your app by running the above test a few times.
Here is why I say it is theory only: There are many variables to consider:
The number and nature of other processes running at the time of the above test (processor intensive)
Method of evaluating the power draw across the CPU/GPU (while tools such as PowerTutor are effective for evaluating power consumption, in this case the evaluation would not be as effective because of the need to collect time stamped power usage data. Additionally, just about any method of collecting power data would introduce an additional overhead (Schrödinger's cat) but that strictly depends on the level of accuracy you require/desire.)
The reason for the power consumption information - If you are looking to define the power consumption of your app for testing or BETA testing/evaluation purposes then it is a feasible task with some determination and the proper tools. If you are looking to gain usable information about power consumption "in the wild", on user's devices, then I would say it is plausible but not realistic. The vairables involved would make even the most determined and dedicated researcher faint. You would have to test on every possible combination of device/Android version in the wild. Additionally, the combinations of running processes/threads and installed apps is likely incalculable.
I hope this provides some insight to your question, although I may have gone deeper into it than needed.
-Steve
For anyone looking, one resource we've been using that is extremely helpful is a free app from AT&T called ARO.
Give it a look: ARO
It has helped me before and I don't see it mentioned very often so thought I'd drop it here for anyone looking.
"I know I can look at the %s of
different apps through the settings,
but this is again unscientific, as the
figure I get from this also depends on
what's happening in all of the other
apps."
The first thing I'd do is hunt for an app already out there that has a known, consistent battery usage, and then you can just use that as a reference to determine your app's usage.
If there is no such app, you will have to hope for an answer from someone else... and if you are successful making such an app, I would suggest selling your new "battery usage reference" app so that other programmers could use it. :)
I know this question is old and it's late, but for anyone who comes here looking for a solution I suggest you take a look at JouleUnit test:
http://dnlkntt.wordpress.com/2013/09/28/how-to-test-energy-consumption-on-android-devices/
It integrates into eclipse and gives you a great amount of detail about how much battery your app is consuming.
I know of three options that can help you for a having scientific measure:
Use a hardware specifically built for this. Monsoon HIGH VOLTAGE POWER MONITOR.
https://msoon.github.io/powermonitor/PowerTool/doc/Power%20Monitor%20Manual.pdf
Download and install Trepn Profiler (a tools from Qualcomm) on your phone. You wont need a computer for reporting. Reports are live and realtime on the phone. You can download Trepn Profiler from the following link: https://play.google.com/store/apps/details?id=com.quicinc.trepn&hl=en_US
Please take note that for recent phone (with android 6+) it works in estimation mode. If you need accurate numbers, you need to a list of select devices. Check the following link for the list:
https://developer.qualcomm.com/software/trepn-power-profiler/faq
You can profile apps separately, and the whole system.
Use Batterystats and Battery Historian from google.
https://developer.android.com/studio/profile/battery-historian
Related
The title asks the crux of the question. Additionally, if there's some pretty graphics in JProfiler that illustrate whether a slow app is CPU, Memory, or Network-bound then I would like to know that as well.
I suspect that RTFM would help but it's TL;DR. However, I am running JProfiler and trying to stumble into the solution.
I can see thread activity for the app under measurement and, yes, there are occasional blue spans of network I/O idling for a few of the working threads -- but hardly enough to account for the extensive network communication that the app is in the midst of performing.
Is there then a way to see, in addition to the hotspot statistics for code that is run on the CPU, how that CPU time compares with overall wallclock time and its relation to network and memory usage time?
The CPU call tree and hot spots views have a thread selector in the top-right corner. By default it's set to "Runnable", but you can change it to "Net I/O" and compare the call tree or the hot spots for the different thread states.
I am trying to make a visual stimulus for an EEG study. The video is simply a flicker between a black frame and a white frame, and the alternation should occur at a range of rates: 12Hz, 24Hz, 48Hz, 72Hz.
Our monitors have a refresh rate of 144Hz and the computers are also fancy, and I am measuring the success of the videos with an oscilloscope to ensure accuracy. So, the hardware should not be an issue; theoretically, up to half the monitor's refresh rate should be possible. However, I have failed in both Java and MatLab.
I have tried using MatLab:
1) using imwrite() to make a gif
2) using the VideoWriter
3) using getframe() and movie2avi().
In all of these methods, the extra-high framerate is declared, and I can see in my command window that all the frames were inserted during the run. However, the final output file never exceeds 48Hz.
On top of that, the 48Hz as well as 24Hz and even 12Hz files also have serious timing issues with the frames.
I have also tried to make the files using Processing's MovieMaker: I set the framerate to 72Hz -- input a list of 72 .png files as frames -- and it should output a 1-second file that flickers at 72Hz.
However, the result only goes at 48Hz, and again the timing of the frames is not reliable.
I wouldn't be posting here if I hadn't exhausted my search; I'm really out of ideas. MatLab and Processing were both recommended ways of achieving this kind of high fps file, and both have big timing issues even with lower flicker frequencies. If anyone has any tips on increasing the temporal fidelity of the high-Hz flicker (graphics settings? codecs?), or of how to make it all the way to 72Hz, we'd greatly appreciate it!
As I said, I have only used Processing/Java and MatLab, so please feel free to recommend another platform.
THis is not an answer. It needs more than the comment-box, though, so please bear with me.
There are fundamental issues involved:
Simply drawing to whatever facility your OS/Graphics combo exposes, does not at all guarantee the drawed element to be present starting from the next frame (in all systems I know of).
This simply stems from the fact, that all these combos were explicitly NOT ment for an EEG Stimulus, but for consumption by visual understanding
Many combos offer lower-level facilities (e.g. OpenGL), that do carry such a promise, but come with other sets of problems, one of which is the less comfortable programming environment
With most OS/Harware combos, it might be less than trivial to sustain this stimulus - 144 Hz translates to less than 7ms - a time slot, that might be missed by a single bad scheduling decision of the OS or a need for a double-read even on a fast spinning disk. You would need to aim for some realtime-oriented OS dialect.
EDIT
After re-reading your question, I see you use Java. Forget it. a single GC break can easily be more than 7 ms.
There are a couple of free (as in beer and freedom) toolboxes for Matlab that wrap the low level openGL commands that you need in order to gain the type of control you want
MGL only runs on Mac but:
mgl is a set of matlab functions for displaying full screen visual stimuli from matlab. It is based on OpenGL functions, but abstracts these into more simple functions that can be used to code various kinds of visual stimuli.
Psychtoolbox runs on Mac, Windows, and Linux
The attraction of using computer displays for visual psychophysics is
that they allow software specification of the stimulus. Programs to
run experiments are often written in a low-level language (e.g. C or
Pascal) to achieve full control of the hardware for precise stimulus
display... The Psychophysics Toolbox is a software package that adds this capability to the Matlab and Octave application on Macintosh, Linux and Windows computers
It sounds like you are just starting out, in which case I would also suggest looking at the Python based PsychoPy
PsychoPy is an open-source package for running experiments in Python (a real and free alternative to Matlab). PsychoPy combines the graphical strengths of OpenGL with the easy
Python syntax to give scientists a free and simple stimulus
presentation and control package.
STORY
I've been coding in OpenGL for about a year now (on the same hardware), and I've only recently been getting artifacts like the ones in the image above. They show up after running my program consecutively in a short period of time (couple of mins), and appear anywhere: from WordPad (see picture) to my desktop and taskbar or even other games (league of legends launcher, it's a software renderer i think).
They can take any form similar to what you see in the image: black/white blocks or pieces of texture from my application. The artifacts dissapear after the affected application refreshes it's screen.
INFO
My Application looks just Fine itself, no artifacts
I am using an ATI m5800 card with the latest driver (I have an hp elitebook 8540w), windows 7 64bit and Opengl 3.3 or lower.
Why am I not posting a code sample? Because (as obnoxious as it sounds) it does not directly seem to be a specific part of my code that is causing the artifacts, I can easily run the program for 15 minutes without a problem. My Graphics card will heat up to 68 degrees celcius and no artifacts will occur. The artifacts start to happen after multiple consecutive runs, though my video card will never heat up past 68 degrees even when the artifacts occur.
Why am I posting this here? Because I am still not sure whether this is caused by my code or my hardware/drivers, and I think this question is difficult enough from a technical standpoint in a way that I will only get the "buy a new GC" answer in any other place.
I use alot of OpenGL, anything from framebuffers and shaders to 3d textures and buffertextures and texturearrays and what not.
My hardware has not suffered any damage as far as I'm aware, though it is an elitebook so it is prone to overheating.
MY GUESS
(When I mention RAM, I mean video RAM, on my graphics card)
I don't know a whole lot about OpenGL and I know even less about Graphics cards and their drivers, but I'll try to approach this and you can shoot my down at any point. The following is a disorganized train of thought, feel free to only read the bold parts.
All internet sources I can find on graphics artifacts tell me I have bad RAM caused by overheating.
However, if this is the case, then why is this "bad RAM" only accessed after consecutive runs, and never after a fresh start? Shouldn't my OS clean up all graphics memory when my application stops, resetting the state of my GC? If I have bad RAM, it seems that my graphics card cannot keep up with disposing my data and eventually accesses this piece of ram when everything else is "taken". And if the malfunctioning of the RAM is based on temperature, then why can I run my application once for 15 minutes, but not 4 times in the same period if the temperature stays the same?
Also, it it is truly bad ram, then why am I seeing part of my texture? Wouldn't this imply that the RAM worked fine at one part? (The blue blocks in the picture are part of a texture I use)
And more importantly: Why do I only seem to get artifacts with my application and nowhere else? Not one other application I have installed produces these artifacts anywhere!
This would suggest that it is not my hardware but me, I am doing something wrong! (Or OpenGL is doing something wrong on my laptop, since most games likely run on DirectX).
And now the last part, which makes the entire thing a miracle: My hardware is funded partly by my university, this means a friend of mine has identical hardware (truly identical 100%), and he does not get any artifacts running my code.
So... is it a driver bug then? My friend is running the same driver as me... I am totally lost here.
All I can conclude is that my RAM is broken, and somehow everybody manages to avoid the bad parts except for me and my application.
What I am trying to ask is this: How did I get these artifacts? and how are some applications able to avoid them? What is going on in the hardware/software?
PS: I understand this is quite unstructured and chaotic as a question, this is because I've been at this for a while and I've tried to include any piece of information I have found. I will appreciate ANY piece of information anyone might think is related to this subject, today or a year from now, and I will gladly restructure this post if any suggestions come up. I've searched alot about artifacts but a great deal of search results describe anomalies due to code, restricted to the application in question, which is of little help to me. This means I could have missed important sources, if you think I did, please link them.
Contrary to what this might look like, I am only asking for information and discussion, not for a direct solution, the best solution is obviously to buy a new graphics card.
Important source : Diagnosing video card problems
All internet sources I can find on graphics artifacts tell me I have bad RAM caused by overheating.
And they're most likely right. This is the typical kind of error seen with defective graphics RAM.
However, if this is the case, then why is this "bad RAM" only accessed after consecutive runs, and never after a fresh start?
Because after a fresh start only a smaller portion of the graphics RAM are actually used. Something like only the first 64MiB or so. That's a rather miniscule amount of memory, compared to the huge quantities of RAM present on modern graphics card (several 100MiB).
After your program did run for a little while it consumes some RAM, even more so, if it's creating and freeing a lot RAM. For performance reasons all allocations of graphics RAM must be contiguous. And to keep things simple and fast the driver will merely increment the base pointer for further allocations to the beginning of unused, contiguous RAM.
Shouldn't my OS clean up all graphics memory when my application stops, resetting the state of my GC?
Well, say the OS did reserve some new graphics memory while your program was running, even if the program terminates, the other allocations will stay. Or after your program terminates the internal state of the driver will make it hand out different portions of memory than before.
So not all of your graphics RAM is defective. Only later parts which will be accessed only with the first part of (nondefective) RAM being pre-allocated.
Technically it was possible to identify the defective parts of RAM and no longer use it. But there's no support (I know about) for it in the drivers.
All,
Given a code that you are not at all knowledgeable about in terms of the functionality and implementation, how would you go about finding the performance bottlenecks in that code? Please list any specific tools / standard approaches that you might be using.
I assume you have the source code, and that you can run it under a debugger, and that there is a "pause" button (or Ctrl-C, or Esc) with which you can simply stop it in its tracks.
I do that several times while it's making me wait, like 10 or 20, and each time study the call stack, and maybe some other state information, so I can give a verbal explanation of what it is doing and why.
That's the important thing - to know why it's doing what it's doing.
Typically what I see is that on, say, 20%, or 50%, or 90% of samples, it is doing something, and often that thing could be done more efficiently or not at all. So fixing that thing reduces execution time by (roughly) that percent.
The bigger a problem is, the quicker you see it.
In the limit, you can diagnose an infinite loop in 1 sample.
This gets a lot of flak from profiler-aficionados, but people who try it know it works very well. It's based on different assumptions.
If you're looking for the elephant in the room, you don't need to measure him.
Here's a more detailed explanation, and a list of common myths.
The next best thing would be a wall-time stack sampler that reports percent at the line or instruction level, such as Zoom or LTProf, but they still leave you puzzling out the why.
Good luck.
You should use a profiling too, depends on the platform:
.NET: Visual Studio performance tools, JetBrains dotTrace
Java: JProfiler
The above tools work very well for applications, but the features vary. For example, Visual Studio can summarize performance data based on tiers.
How to approach the problem is highly dependent on the type of the program, and the performance problem you're facing. But basically, you'll repeat the following cycle:
Record performance data (maybe change the settings for higher / lower granularity on recorded data)
Identify hot spots, where most of the application time is consumed
Maybe use reverse call tables to identify how the hot spot is invoked, and from where in the code
Try to refactor / optimize the hot spot
Start over, and check how much your optimization was effective.
It might take several iterations of the above cycle to get you to a point that you have acceptable performance.
Note that these tools provide many different features and ways to look at performance data, or record them. Provided that you don't have any knowledge of the internal structure of the application, you should start playing with different features and reports that the tools provide, so that you can pinpoint where to optimize.
Use differential analysis. Pick one part of the program and artificially slow it down (add a bunch of code that does nothing but waste time). Re-run your test and observe the results. Do this for a variety of aspects of your program. If adding the delays does not alter performance, then that aspect is not your bottleneck. The aspect that results in the largest perrformance hit might be the first place to look for bottlenecks.
This works even better if the severity of the delay code is adjustable while the program is running. You can increase and decrease the artificial delay and see how that affects the performance. If you encounter a test where the change in observed performance seems to follow the artificial delay linearly, then that aspect of the program might be your bottleneck.
This is just a poor man's way of doing it. The best method is probably to use a profiler. If you specify your language and platform, someone could probably recommend a good profiler.
Without having an idea on the kind of system you are working with, these pieces of gratuitous advice:
Try to build up knowledge on how the system scales: how are 10 times more users handled, how does it cope with 100 times more data, or with a 100 times slower network environment...
Find the proper 'probing' points in the system: a distributed system is, of course, harder to analyze than a desktop app.
Find proper technology to analyze the data received from the probes. Profilers do a great job visualizing bottleneck functions, but I can imagine they are of no help for your cloud service. Try to graphically visualize your data, your brain is much better at recognizing graphical patterns than numerical, let alone textual.
oh - find out what the expectations are! It's no use optimizing the boot time of your app if it's only booted three times a year.
I'd say the steps would be:
Identify the actual functionality that is slow, based on use of the system or interviewing users. That should narrow down the problem areas (and if nobody is complaining, maybe there's no problem.)
Run a code profiler (such as dotTrace / Compuware) and data layer profiler (e.g. SQL Profiler, NHibernate Profiler, depending on what you're using.) You should get some good results after a day or so of real use.
If you can't get a good idea of the problems from this, add some extra stopwatch code to the next live build that logs the number of milliseconds in each operation.
That should give you a pretty good picture of the multiple database queries that should be combined into one, or code that can be moved out of an inner loop or pre-calculated, etc.
We have an Java ERP type of application. Communication between server an client is via RMI. In peak hours there can be up to 250 users logged in and about 20 of them are working at the same time. This means that about 20 threads are live at any given time in peak hours.
The server can run for hours without any problems, but all of a sudden response times get higher and higher. Response times can be in minutes.
We are running on Windows 2008 R2 with Sun's JDK 1.6.0_16. We have been using perfmon and Process Explorer to see what is going on. The only thing that we find odd is that when server starts to work slow, the number of handles java.exe process has opened is around 3500. I'm not saying that this is the acual problem.
I'm just curious if there are some guidelines I should follow to be able to pinpoint the problem. What tools should I use? ....
Can you access to the log configuration of this application.
If you can, you should change the log level to "DEBUG". Tracing the DEBUG logs of a request could give you a usefull information about the contention point.
If you can't, profiler tools are can help you :
VisualVM (Free, and good product)
Eclipse TPTP (Free, but more complicated than VisualVM)
JProbe (not Free but very powerful. It is my favorite Java profiler, but it is expensive)
If the application has been developped with JMX control points, you can plug a JMX viewer to get informations...
If you want to stress the application to trigger the problem (if you want to verify whether it is a charge problem), you can use stress tools like JMeter
Sounds like the garbage collection cannot keep up and starts "halt-the-world" collecting for some reason.
Attach with jvisualvm in the JDK when starting and have a look at the collected data when the performance drops.
The problem you'r describing is quite typical but general as well. Causes can range from memory leaks, resource contention etcetera to bad GC policies and heap/PermGen-space allocation. To point out exact problems with your application, you need to profile it (I am aware of tools like Yourkit and JProfiler). If you profile your application wisely, only some application cycles would reveal the problems otherwise profiling isn't very easy itself.
In a similar situation, I have coded a simple profiling code myself. Basically I used a ThreadLocal that has a "StopWatch" (based on a LinkedHashMap) in it, and I then insert code like this into various points of the application: watch.time("OperationX");
then after the thread finishes a task, I'd call watch.logTime(); and the class would write a log that looks like this: [DEBUG] StopWatch time:Stuff=0, AnotherEvent=102, OperationX=150
After this I wrote a simple parser that generates CSV out from this log (per code path). The best thing you can do is to create a histogram (can be easily done using excel). Averages, medium and even mode can fool you.. I highly recommend to create a histogram.
Together with this histogram, you can create line graphs using average/medium/mode (which ever represents data best, you can determine this from the histogram).
This way, you can be 100% sure exactly what operation is taking time. If you can't determine the culprit, binary search is your friend (fine grain the events).
Might sound really primitive, but works. Also, if you make a library out of it, you can use it in any project. It's also cool because you can easily turn it on in production as well..
Aside from the GC that others have mentioned, Try taking thread dumps every 5-10 seconds for about 30 seconds during your slow down. There could be a case where DB calls, Web Service, or some other dependency becomes slow. If you take a look at the tread dumps you will be able to see threads which don't appear to move, and you could narrow your culprit that way.
From the GC stand point, do you monitor your CPU usage during these times? If the GC is running frequently you will see a jump in your overall CPU usage.
If only this was a Solaris box, prstat would be your friend.
For acute issues like this a quick jstack <pid> should quickly point out the problem area. Probably no need to get all fancy on it.
If I had to guess, I'd say Hotspot jumped in and tightly optimised some badly written code. Netbeans grinds to a halt where it uses a WeakHashMap with newly created objects to cache file data. When optimised, the entries can be removed from the map straight after being added. Obviously, if the cache is being relied upon, much file activity follows. You probably wont see the drive light up, because it'll all be cached by the OS.