I am creating an android app which will be used to analyze golf shots. I am using cameraX api to get the frames into surfaceview and analyze callback to further processing.
Initially I am using Google Ml-kit object detection to detect the ball but video lags too much that most of the frames are skipped.
How can I use frames in realtime as there maybe 2,3 frames having the ball because the shot will be too fast?
I have tried to gather the frames as bitmaps into an ArrayList and then use that list for further processing but the number of frames vary for a 5 second video.
Should I record the video first for a constant time 5s and then extract frames from it?
Note: The above implementation is for starting because later I need to detect shot in realtime which I tried to detect the ball + detecting the specific sound but in vien.
Furthermore, I need to calculate framrate also which will later be used to calculate the speed of the ball.
Please suggest of any better implementation.
Thanks
Related
Is there a way to only play a specific range of frames in an Android VideoView?
I'm trying to avoid creating a separate clip if it's possible to reuse the same clip for multiple use cases.
In certain situations, I only want to play frames 1-30, for example.
_videoView.seekTo(0);
_videoView.start();
// Stop at frame 30
I'm using MediaExtractor/MediaCodec to decode a video and render it to a TextureView. As a template, I've used the code from: https://github.com/vecio/MediaCodecDemo/blob/master/src/io/vec/demo/mediacodec/DecodeActivity.java
I'd like to be able to playback the video at 2x speed. Luckily, the media encoding/decoding is fast enough so that I can accomplish this by letting the MediaCodec decode every frame, and then only render every other frame to the screen. However this doesn't feel like a great solution, particularly if you want to increase playback by an arbitrary value. For example, at 10x speeds, the Codec is not able to decode frames fast enough to playback every 10th frame at 30 fps.
So instead I'd like control playback by calling MediaExtractor.advance() multiple times to skip frames that don't need to be decoded. For example:
...
mDecoder.queueInputBuffer(inIndex, 0, sampleSize, mExtractor.getSampleTime(), 0);
for (i = 0; i < playbackSpeedIncrease; i++) {
mExtractor.advance();
}
...
With this code, in theory the extractor should only extract every nth frame, where n is defined by the variable 'playbackSpeedIncrease'. For example, if n = 5, this should advance past frames 1-4 and only extract frame 5.
However, this does not work in practice. When I run this code, the image rendered to the screen is distorted:
Does anyone know why this is? Any suggestions for better ways to playback videos at an arbitrary speed increase?
You can't generally do this with AVC video.
The encoded video has "key" (or "sync" or "I") frames that hold full images, but the frames between key frames hold "diffs" from the previous frames. You can find some articles at wikipedia about video encoding methods, e.g. this one. You're getting nasty video because you skipped a key frame and now the diffs are being computed against the wrong image.
If you've ever seen video fast-forward smoothly but fast-reverse clunkily, e.g. on a TiVo, this is why: the video decoder plays forward quickly, but in reverse it just plays the I-frames, holding them on screen long enough to get the desired rate. At "faster" forward/reverse it evens out because the device is just playing I-frames. You can do something similar by watching for the SAMPLE_FLAG_SYNC flag on the frames you get from MediaExtractor.
In general you're limited to either playing the video as fast as the device can decode it, or playing just the key frames. (If you know enough about the layout of a specific video in a specific encoding you may be able to do better, e.g. play the I and P but not the B, but I'm not sure that's even possible in AVC.)
The frequency of I frames is determined by the video encoder. It tends to be one or two per second, but if you get videos from different sources then you can expect the GOP size (group-of-pictures, i.e. how many frames there are between I-frames) to vary.
I'm trying to get the media player to play a specific range in a locally stored video. It doesn't seem to start at the specified time I tell it to.
Example: when I seek to 1000, it works. But when I seek to 1500, it goes to 2000 instead.
I also tried pausing seeking then starting on seek completion, it doesn't make any difference.
This is the code:
mediaPlayer.start();
mediaPlayer.seekTo(time);
Is this normal? Or am I using the media player the wrong way?
This is an encoding issue. Videos have keyframes (i-frames) that store more information than other frames. A non keyframe can be built only given the previous keyframe. Trying to display a non keyframe will show green spots and pixelated jittery screen.
Now, on some android devices there's no workaround implemented for this so you get this weird behavior. On a Nexus S for example seekTo() doesn't necessarily go to the specified frame. I tried other devices with the same android version and they seek just fine.
I am doing my final Year project on Speed Calculation using webcam. In this project we want to calculate speed of object by taking three sequential images whenever motion is detected. As given here: Raser Abwehr SpeedCam 2012, in this three line RED BLUE and GREEN are made and whenever any vehicle cross it, it takes one snap.
For this I have idea that suppose my camera resolution is 640*480 hence I can divide X-Axis in three parts of 210px each therefore I can have three rectangular screens of size (210*480). Now, I want that whenever any vehicle enters in Screen1 then it click a picture then it start second screen detector and when vehicle enters into second screen it takes second picture and at last it detect in third and click picture. Hence we have three picture and we can calculate Speed by process given here Calculating Speed using a Webcam
Presently, I am using JavaCV as Image Processing Library. It is just like running multiple instance of a single Java program to detect motion in different screen. Please suggest to me how i can do. Can Thread be useful here?
(More like a comment but it doesn't fit)
I'd suggest by starting try making it work by taking three pictures at a fixed interval (which you guess).
Then, if you want to address the issue of detecting speed of objects that are moving at quite different speeds, I'd just suggest by starting with taking as many pictures as possible once you detect any movement, for a sufficiently long time, and then figuring out afterwards which one you should use for the analysis.
I can see what you are trying to do but you should probably start with dumb things first. Just my two cents...
I'm working on an Android project. it's goal is to detect predefined movement gesture of the device. if the device rotates 45 degrees over an axis(X,Y or Z) and then rotates back to its first position( the first and second positions are not going to be accurate, I mean if it was rotated 50 degrees instead of 45, not important !!!)then the gesture has happened and the app should detect it.
I tried to do that using Accelerometer and Magnetic sensors of device to continually monitor the orientation of the device and detect gesture but the results weren't acceptable(explained here). any start point or idea please ?
It doesn't seem like anybody is going to provide a concrete and valuable answer. So let me try to do that.
First of all, even a bit primitive and straightforward approach allows to spot the fact you do not need to process all the data coming from sensors. Moreover humans are not that fast, so there is no need to proceed 10000 values per second in order to identify any specific move as well.
What you actually need is just to identify key points and make your decision. Does it sound like a tanget to you?
What I'm actually suggesting is to test your solution using an ordinary mouse and available gesture recognition framework. Because the actual idea is pretty much the same. So please check:
iGesture - Gesture Recognition Framework
Mouse Gestures
It such a way it might be easier to develop a proper solution.
Update
Let's imagine I'm holding my phone and I need to rotate it 90 degrees counterclockwise and then 180 degrees clockwise. I hope you will not expect me to do some complex 3D shapes in the air (it will break usability and frankly I do not want to loose my phone), so it is possible to say there might be a point we can track or we can easily simulate it.
Please see my other answer in order to see simple, but working solution of a similar problem: