I'm working on a 2D game on Android / Java.
During each frame, I use a fixed size Back Buffer bitmap (1024x768) to draw all the game's assets (background, sprites, ...).
Then at the end of the onDraw(), I draw this back buffer on the screen with the right size :
Rect Source = new Rect(0, 0, 1024, 768);
Rect Dest = new Rect(0, 0, m_ScreenWidth, m_ScreenHeight);
canvas.drawBitmap(BackBufferBitmap, Source, Dest, null);
The problem is that when I use this engine to just draw a simple 1024x768 image on the screen (the simplest I can do), the operation takes between 35 and 40 milliseconds on a LG G4 phone (i.e. approx 25fps). Is it possible to get a better fps with another way of managing 2D graphics?
I turn on hardware acceleration, thread's max priority.
I don't have this problem on phones with lower pixel count. I guess my problem is linked to the high number of pixels on a LG G4 (2560x1440).
Is it possible to do the drawing faster?
Or, otherwise, is it possible to just run my game on such high-definition devices with a lower definition (like we do on PC)?
EDIT : here is the full code :
1) my View
public class ElementaryView extends SurfaceView implements SurfaceHolder.Callback
{
private ElementaryThread m_Thread;
private SurfaceHolder m_Holder;
private Bitmap m_SimpleBitmap=null;
private Bitmap m_BackBuffer =null;
private Canvas m_BackBufferCanvas =null;
public ElementaryView(Context context)
{
super (context);
m_Holder =getHolder();
m_Holder.addCallback(this);
setFocusable(true);
}
#Override
public void surfaceCreated(SurfaceHolder holder)
{
m_Thread =new ElementaryThread(m_Holder,this);
m_Thread.setPriority(Thread.MAX_PRIORITY);
m_BackBuffer =Bitmap.createBitmap(1024,768,Config.ARGB_8888);
m_BackBufferCanvas =new Canvas(m_BackBuffer);
m_SimpleBitmap =BitmapFactory.decodeResource(this.getResources(), R.drawable.splashscreen);
//"splashscreen" is a 1024x768 jpg image
m_Thread.setRunning(true);
m_Thread.start();
}
#Override
protected void onDraw(Canvas canvas)
{
m_BackBufferCanvas.drawBitmap(m_SimpleBitmap, 0, 0, null);
Rect Source =new Rect(0,0,1024,768);
Rect Dest =new Rect(0,0,this.getWidth(),this.getHeight());
canvas.drawBitmap(m_BackBuffer,Source,Dest,null);
}
}
2) my thread :
public class ElementaryThread extends Thread
{
private SurfaceHolder m_SurfaceHolder;
private ElementaryView m_View;
private boolean m_Running;
public ElementaryThread(SurfaceHolder sh, ElementaryView view)
{
super();
m_SurfaceHolder = sh;
m_View = view;
}
public void setRunning(boolean r) { m_Running = r; }
#SuppressLint("WrongCall") #Override
public void run()
{
Canvas canvas;
while (m_Running)
{
canvas =null;
try
{
canvas =this.m_SurfaceHolder.lockCanvas();
this.m_View.onDraw(canvas);
}
finally
{
if (canvas!=null) m_SurfaceHolder.unlockCanvasAndPost(canvas);
}
}
}
}
And this gives a frame rate lower than 30fps on a LG G4. Yet, a lot of 2D games run with a better fps on my G4. Does anybody knows how they do that ?
By default, the LG G4 will always run at a res. of 2560x1440, which of course is going to make it difficult for the phone to render any graphics. This is also why the lesser res. phones will run much more smoothly. So, the only solution to this problem would be to change the resolution dependent on the number of pixels and processing power of the device.
First, for this (for games) is better to use SurfaceView
Second, show the method onDraw entirely. Most likely, you have a resource-intensive operation there. (scale etc.)
I have onDraw takes about 3ms with a similar task.
I'm not sure that you are properly working. What do you want? "this.m_View.onDraw(canvas);"
Remove
#Override
protected void onDraw(Canvas canvas)
From ElementaryView
And move code into Thread. Something like this:
#SuppressLint("WrongCall") #Override
public void run()
{
Canvas canvas;
while (m_Running)
{
canvas =null;
try
{
canvas =this.m_SurfaceHolder.lockCanvas();
Rect Source =new Rect(0,0,1024,768);
Rect Dest =new Rect(0,0,m_View.getWidth(),m_View.getHeight());
canvas.drawBitmap(m_SimpleBitmap, 0, 0, null);
canvas.drawBitmap(m_BackBuffer,Source,Dest,null);
}
finally
{
if (canvas!=null) m_SurfaceHolder.unlockCanvasAndPost(canvas);
}
}
}
}
Related
I have a basic program where stock images form the background for a larger program - but a number of the images have slightly different sizes compared to each other.
My initial code loads up the background image and tries to set the canvas size based on the image dimensions:
PImage bg;
void setup() {
bg = loadImage("102.jpg");
println(bg.width);
println(bg.height);
wWidth = bg.width;
wHeight = bg.height;
size(wWidth,wHeight);
}
void draw() {
background(bg);
}
I get 'IllegalStateException' error
bg.width and bg.height are 806 and 1229, when I include 806 and 1229 instead of wWidth and wHeight respectively, I get the output I want - would I need to declare the size() in a different way? Or would it be simpler to try and resize the jpg files via processing to the same size?
That would've worked in Processing 2, but things changed in Processing 3: you simply need to use settings().
Here's a minimal sketch that loads in image and changes the sketch size to the image dimensions:
PImage img;
void settings(){
img = loadImage("https://processing.org/img/processing-web.png");
println(img.width, img.height);
size(img.width, img.height);
}
void setup(){
image(img, 0, 0);
}
You're code would look something like:
PImage bg;
int wWidth;
int wHeight;
void settings(){
bg = loadImage("102.jpg");
println(bg.width);
println(bg.height);
wWidth = bg.width;
wHeight = bg.height;
size(wWidth, wHeight);
}
void setup() {
}
void draw() {
background(bg);
}
As slightly simpler version would be:
PImage bg;
void settings(){
bg = loadImage("102.jpg");
println(bg.width);
println(bg.height);
size(bg.width, bg.height);
}
void setup() {
}
void draw() {
background(bg);
}
The wWidth, wHeight variables might be redundant since Processing's width,height variables store the same data after size() has been called.
I'm currently working on a libgdx game and before I give it final touches I wanted to actually hear something from experienced users, that has been bothering me for a few days already.
If I want to support as many as possible devices, essentially I will be designing graphics for the biggest possible res ,which is then going to be scaled if needed, for smaller screens, right? How do I go about developing for a resolution that is even bigger than my laptop's(the 2015/16 gen phones). My laptop has a resolution of 1920x1080px and the S7 Samsung has 2k+ width.
Thank you!
I think what you are looking for is Viewports. You have to decide which strategy fits best your needs. For example a FitViewport always keeps the aspect ratio you define, which might lead to black bars on some devices.
When I personally develop with libgdx I place and size all objects relative to the screen width and height. This includes images, fonts, buttons, etc. This gives me a pretty consistent result across all devices because most devices today have a ratio 16:9 or something close to it. For developing an image larger than your screen size what's wrong with just using photoshop to create the image of the specified size?
Better you choose the screen with as 1280 and screen height as 800 and also use the fill viewPort . So you will be able to render your game in almost all the screens without the issue of stretching.
Viewport is the method which provided by the libgdx to solve this multi screen compatible issue . here i will post some sample code which you can use for the reference.
public class myGame extends ApplicationAdapter {
public OrthographicCamera camera;
public Viewport viewPort;
private SpriteBatch batch;
private BitmapFont myScoreFont;
private Texture texture;
public myGAme() {
}
#Override
public void create() {
myScoreFont = new BitmapFont(Gdx.files.internal(Constants.PATH_TO_MY_SCORE_FONT), true);
batch = new SpriteBatch();
float w = Gdx.graphics.getWidth();
float h = Gdx.graphics.getHeight();
texture = new Texture(Gdx.files.internal(Constants.PATH_TO_LEFT_BAR));
camera = new OrthographicCamera();
camera.position.set(0, 0, 0);
camera.update();
camera.setToOrtho(false, Constants.APP_WIDTH, Constants.APP_HEIGHT);
// Here is the viewport is setting up with the camera and the screen size
viewPort = new FillViewport(1280, 800, camera);
}
#Override
public void dispose() {
batch.dispose();
}
#Override
public void render() {
Gdx.gl.glClearColor(1, 1, 1, 1);
Gdx.gl.glClear(GL30.GL_COLOR_BUFFER_BIT);
float deltaTime = Gdx.graphics.getDeltaTime();
batch.setProjectionMatrix(camera.combined);
batch.begin();
batch.draw(myScorefont,"Score",0,0);
batch.end();
}
#Override
public void resize(int width, int height) {
// the game area will be resized as per the screen size of the device
viewPort.update(width, height);
}
#Override
public void pause() {
}
#Override
public void resume() {
}
}
So I was told to create a simple game in android using canvas only for a school project (meaning I can't use any extensions or game engines beside the ones that come with android), and I wanted to create a simple 2d game with a player walking around.
I did it like that:
public GameView(Context c) {
// TODO Auto-generated constructor stub
super(c);
this.c=c;
this.Sprite=BitmapFactory.decodeResource(getResources(), R.drawable.walk1);
this.Sprite=Bitmap.createScaledBitmap(Sprite, Sprite.getWidth()*2, Sprite.getHeight()*2, false);
sprite2=new Sprite("Spicy",Sprite);
this.requestFocus();
this.setFocusableInTouchMode(true);
animate1();
}
I created a view class, loaded a simple player sprite, created sprite class and an handler -
public void animate1(){
handlerAnimation100 = new Handler();
final Runnable r = new Runnable() {
public void run() {
invalidate();
handlerAnimation100.postDelayed(this, 1);
}
};
handlerAnimation100.postDelayed(r, 1);
}
that does invalidate every 0.001 seconds for the game time and animation.
#Override
protected void onDraw(Canvas canvas) {
super.onDraw(canvas);
sprite2.Draw(canvas);
}
#Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
sprite2.Update(keyCode);
invalidate();
return false;
}
on onDraw i called the draw function in the sprite class and onkeydown I sent the key pressed to the update function in the sprite class.
Now for the sprite class there isn't something special:
public class Sprite {
enum State
{
Walking
}
State mCurrentState = State.Walking;
int mDirection = 0;
int mSpeed = 0;
int mPreviousKeyboardState;
private String spriteName;
Bitmap sprite;
int SPRITE_SPEED = 5;
int MOVE_LEFT = -1;
int MOVE_RIGHT = 1;
private float mScale = 1.0f;
Point Position;
public Sprite(String name,Bitmap sprite) {
this.sprite=sprite;
this.spriteName=name;
Position=new Point(150,150);
}
public void Update(int keyboard)
{
int aCurrentKeyboardState = keyboard;
UpdateMovement(aCurrentKeyboardState);
mPreviousKeyboardState = aCurrentKeyboardState;
}
private void UpdateMovement(int aCurrentKeyboardState)
{
if (mCurrentState == State.Walking)
{
mSpeed = 0;
mDirection = 0;
if (aCurrentKeyboardState==KeyEvent.KEYCODE_A)
{
mSpeed = SPRITE_SPEED;
mDirection = MOVE_LEFT;
}
else if(aCurrentKeyboardState==KeyEvent.KEYCODE_D)
{
mSpeed = SPRITE_SPEED;
mDirection= MOVE_RIGHT;
}
Position.x += mDirection * mSpeed;
}
}
public void Draw(Canvas c)
{
c.drawBitmap(sprite, Position.x,Position.y, null);
}
}
I just change the position of the image and move it depending on the key pressed.
Now here is the problem:
Using handler and invalidate was the only option I was able to find to replace "Game time" that appears in game engines, and although it works, it works very choppily, if the speed of the player is high it looks like it jumps pixels and if its low the animation is blurry and very slow, it looks like the invalidate takes more time and it happens not every 0.001 seconds but 0.5 seconds or so.
Here is how it looks like:
Slow speed (blurry and very slow):
Faster Speed (choppy not smooth):
Is there a better way to do it, again with only using what android offers?
Threads! However the handler is not the root of your problem judging by how slow your animation is running and how choppy the fast one is, you are probably using a canvas tied into an ImageView? This is not very fast at all. You should look into using a SurfaceView (works with lower APIs but not the fastest) or into TextureView (super fast, I had to delay my thread cause the animation was just a blur). Both of these rely on canvas at their core.
There are lots of examples out there on in internets on how to code these and you can adapt them for your purposes. To give you a place to start you can look HERE at some samples I wrote.
I have a small problem. In carrying out the method paintComponent() during the animation I have to constantly update the variable bgImage. But it takes a lot of time, so that the animation slows down.
A block of code with the problem:
public class ProblemClass extends JComponent {
private static final int FRAME_FREQUENCY = 30;
private final Timer animationTimer;
public ProblemClass() {
this.animationTimer = new Timer(1000 / FRAME_FREQUENCY, new ActionListener() {
#Override
public void actionPerformed(ActionEvent e) {
repaint(); // When the animation started is often invoked repaint()
}
});
}
// Other code...
/**
* Start animation from another class
*/
public void startAnimation() {
this.animationTimer.start();
}
#Override
protected void paintComponent(Graphics g) {
// GraphicsUtils.gaussianBlur(...) it's a long-time operation
BufferedImage bgImage = GraphicsUtils.gaussianBlur(AnotherClass.getBgImage());
g2.drawImage(bgImage, 0, 0, null);
// Other code...
}
}
I read on the Internet that I need run long task in parallel thread (SwingWorker), but I have no idea how to do it in my case. How can I solve this problem?
P.S. Sorry for my bad English, it's not my first language.
The best you're going to do is having the image update outside of the paint method, and only redraw whenever a new image is ready. Take your existing code, and add a persistent reference to the image, which gets drawn onto the JComponent each paint method. Then have your animation timer do the Gaussian blur and update your image.
public class ProblemClass extends JComponent {
private static final int FRAME_FREQUENCY = 30;
private final Timer animationTimer;
//persistent reference to the image
private BufferedImage bgImage;
public ProblemClass() {
this.animationTimer = new Timer(1000 / FRAME_FREQUENCY, new ActionListener() {
#Override
public void actionPerformed(ActionEvent e) {
//Update the image on each call before repainting
bgImage = GraphicsUtils.gaussianBlur(AnotherClass.getBgImage());
repaint();
}
});
}
/**
* Start animation from another class
*/
public void startAnimation() {
this.animationTimer.start();
}
#Override
protected void paintComponent(Graphics g2) {
g2.drawImage(bgImage, 0, 0, null);
// Other code...
}
}
There is no generic way to solve this when you generate a new background image every time and then blur it. GaussianBlur is a slow operation, period.
If AnotherClass.getBgImage() delivers images from a predefined set of images, then apply the blur once to each image in the set, problem goes away.
If you create the image in AnotherClass.getBgImage() dynamically, then you may be able to speed it up by changing the image creation to create a blurred image from the start. Depending on what is done to create the image this may or may not be feasible.
If neither of the above works out, you need to investigate other options to produce the blurred image which are faster; there are simpler blurring methods that are generally faster but look somewhat similar to a gaussian.
You see it all boils down to getting rid of calling GaussianBlur repeatedly on the performance critical path.
You should extract logic from painter. Painters are called constrantly and should be executed very fast.
BufferedImage bgImage = GraphicsUtils.gaussianBlur(AnotherClass.getBgImage());
This line has to be executed every time? maybe you could use a field to store the image and the painter could just paitn the image, not applying each time a gaussianBlur.
Try this:
public class ProblemClass extends JComponent {
private static final int FRAME_FREQUENCY = 30;
private final Timer animationTimer;
private final BufferedImage bgImage;
public ProblemClass() {
bgImage = GraphicsUtils.gaussianBlur(AnotherClass.getBgImage());
this.animationTimer = new Timer(1000 / FRAME_FREQUENCY, new ActionListener() {
#Override
public void actionPerformed(ActionEvent e) {
repaint(); // When the animation started is often invoked repaint()
}
});
}
// Other code...
/**
* Start animation from another class
*/
public void startAnimation() {
this.animationTimer.start();
}
#Override
protected void paintComponent(Graphics g2) {
// GraphicsUtils.gaussianBlur(...) it's a long-time operation
g2.drawImage(bgImage, 0, 0, null);
// Other code...
}
}
extremely slow frame rates when using the openCV Java method in android for detecting circular shaped objects in images
Imgproc.HoughCircles(mGray, circles, Imgproc.CV_HOUGH_GRADIENT, 1, 50);
when i remove this method it runs fast, but after adding this method inside of this callback
public Mat onCameraFrame(final CvCameraViewFrame inputFrame) {
the frame rate slows to 1 to 2 frames per second, I don't understand why it gets so slow, i tried putting this method in a separate thread and it would not help, the only thing that worked is to use a counter and and an if statement to run the method every 10 frames.
in the OpenCV examples there is a sample project called face detection and it has both a native C++ and Java camera versions and they both are vary fast, how is it possible that when I use similar code I get this slow constipated action from OpenCV?
is there something i am doing wrong here? In the face detection project from openCV examples they take every frame and they don't launch a separate thread. how do I fix this problem and make my code run fast like the sample projects in OpenCV?
in a different project I am also having the same problem of slow frame rate, in this practice project where I am not using openCV, it is just the android Camera class only, in that I am taking the image from the onPreviewFrame(byte[] data, Camera camera) method and doing some light processing like converting the YUV format from the byte array into a bitmap and putting that into another view on the same screen as the camera view, and the result is vary slow frame rate.
EDIT: In some additional experimentation I added the Imgproc.HoughCircles() method to the OpenCV face Detection sample project. putting this method inside the onCameraFrame method of the java detector.
the result is the same as in my project, it became vary slow. so the HoughCircles method probably takes more processing power than the face detection method CascadeClassifier.detectMultiScale(), however that does not explain the fact I watched other circle detection projects on youTube and in their videos the frame rate is not slowed down. that is why I think there is something wrong with what I am doing.
here is a sample of the code I am using
public class CircleActivity extends Activity implements CvCameraViewListener2 {
Mat mRgba;
Mat mGray;
File mCascadeFile;
CascadeClassifier mJavaDetector;
CameraBridgeViewBase mOpenCvCameraView;
LinearLayout linearLayoutOne;
ImageView imageViewOne;
int counter = 0;
private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
#Override
public void onManagerConnected(int status) {
switch (status) {
case LoaderCallbackInterface.SUCCESS:
{
Log.i("OPENCV", "OpenCV loaded successfully");
mOpenCvCameraView.enableView();
} break;
default:
{
super.onManagerConnected(status);
} break;
}
}
};
/** Called when the activity is first created. */
#Override
public void onCreate(Bundle savedInstanceState) {
if (!OpenCVLoader.initDebug()) {
// Handle initialization error
}
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_coffee);
getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.fd_activity_surface_view);
mOpenCvCameraView.setCvCameraViewListener(this);
}
#Override
public void onPause()
{
super.onPause();
if (mOpenCvCameraView != null)
mOpenCvCameraView.disableView();
}
#Override
public void onResume()
{
super.onResume();
OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_3, this, mLoaderCallback);
}
public void onDestroy() {
super.onDestroy();
mOpenCvCameraView.disableView();
}
public void onCameraViewStarted(int width, int height) {
mGray = new Mat();
mRgba = new Mat();
}
public void onCameraViewStopped() {
mGray.release();
mRgba.release();
}
public Mat onCameraFrame(final CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
if(counter == 9) {
MatOfRect circles = new MatOfRect();
Imgproc.HoughCircles(mGray, circles, Imgproc.CV_HOUGH_GRADIENT, 1, 50);
// returns number of circular objects found
Log.e("circle check", "circles.cols() " + circles.cols());
}
counterAdder();
return mRgba;
} // end oncamera frame
public void counterAdder() {
if (counter > 10) {
counter = 0;
}
counter++;
}
}
Reducing resolution of camera frames might help
mOpenCvCameraView.setMaxFrameSize(640, 480);
From my brief experience, the running time for HoughCircles greatly depends on the image. A textured image with a lot of potential circles takes much longer than an image with a uniform background. Hope this helps.
I' ve faced this problem either.
I' ve tried to decrease the camera resolution with mOpenCvCameraView.setMaxFrameSize(1280, 720);
However it is still slow. I' ve been trying to work parallel with Threads, but it is still 3.5FPS.
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
//System.gc();
carrierMat = inputFrame.gray();
Thread thread = new Thread(new MultThread(carrierMat, this));
thread.start();
try {
thread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return carrierMat;
}
My MultThread class is just like this
public class MultThread implements Runnable {
private Mat source;
private Context context;
public MultThread(Mat source, Context context) {
this.source = source;
this.context = context;
}
#Override
public void run() {
//output = General.Threshold(source);
int x = General.MSERP(source);
Log.i("MtMTxtDtc:Main","x: " + x);
if (x > 10){
((Vibrator) context.getSystemService(Context.VIBRATOR_SERVICE)).vibrate(500);
}
}
}
You have to perform the Hough Circle transform in the background not in the main activity!
Otherwise your app response will be too slow and it may be killed by the operating system due to Application Not Responding (ANR) error.
You need to add this class to your main activity and you are good to go.
private class HoughCircleTransformTask
extends AsyncTask<Mat, Void, Integer> {
#Override
protected Boolean doInBackground(Mat mGray) {
MatOfRect circles = new MatOfRect();
Imgproc.HoughCircles(mGray, circles, Imgproc.CV_HOUGH_GRADIENT, 1, 50);
// returns number of circular objects found
// then display it from onPostExecute()
return circles.cols();
}
#Override
protected void onPostExecute(Integer circlesCols){
// This is only logging
// You can display it in a TextView as well in the main activity
Log.e("circle check", "circles.cols() " + circles.cols());
}
}
And just call it from onCameraFrame with one line of code only
public Mat onCameraFrame(final CvCameraViewFrame inputFrame) {
mRgba = inputFrame.rgba();
mGray = inputFrame.gray();
if(counter == 9) {
// call AsyncTask
new HoughCircleTransformTask().execute(mGray);
}
counterAdder();
return mRgba;
} // end oncamera frame