I am using FFmpeg in my application to extract frames from a video, the frames will be added to a trim video view where you get an illustration as to what is happening in the video at a specific time within the video. So each frame needs to represent some time within the video.
I dont quite understand how FFmpeg is producing the frames. Here is my code:
"-i",
videoCroppedFile.getAbsolutePath(),
"-vf",
"fps=1/" + frameSeperation,
mediaStorageDir.getAbsolutePath() +
"/%d.jpg"
My app allows you to record a video at a max length of 20s. The number of frames extracted from the video depnds on how long the captured video is. frameSeperation is calculated doing the below code.
String time = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
long videoLength = Long.parseLong(time) / 1000;
double frameSeperationDouble = (double) videoLength;
// Divide by 11 because there is a maximum of 11 frames on trim video view
frameSeperationDouble /= 11;
frameSeperationDouble = Math.ceil(frameSeperationDouble);
int frameSeperation = (int) frameSeperationDouble;
Maybe the above logic is very bad, if there is a better way please can somebody tell me.
Anyway I run the code and below are a few test cases:
A video captured with a length of 6 seconds has 7 frames.
A video captured with a length of 2 seconds has 3 frames.
A video captured with a length of 10 seconds has 12 frames.
A video captured with a length of 15 seconds has 9 frames.
A video captured with a length of 20 seconds has 11 frames.
There is no consistency, and I find it hard to put timestamps against each frame because of this. I feel like my logic is wrong or im not understanding. Any help is much appreciated
Update 1
So I did what you said in comments:
final FFmpeg ffmpeg = FFmpeg.getInstance(mContext);
final File mediaStorageDir = new File(Environment.getExternalStorageDirectory()
+ "/Android/data/"
+ mContext.getPackageName()
+ "/vFrames");
if (!mediaStorageDir.exists()){
mediaStorageDir.mkdirs();
}
MediaMetadataRetriever retriever = new MediaMetadataRetriever();
retriever.setDataSource(mContext, Uri.fromFile(videoCroppedFile));
String time = retriever.extractMetadata(MediaMetadataRetriever.METADATA_KEY_DURATION);
long videoLength = Long.parseLong(time) / 1000;
double frameSeperationDouble = (double) videoLength / 8;
retriever.release();
final String cmd[] = {
"-i",
videoCroppedFile.getAbsolutePath(),
"-vf",
"fps=1/" + frameSeperationDouble,
"-vframes," + 8,
mediaStorageDir.getAbsolutePath() +
"/%d.jpg"
};
I also tried "-vframes=" + 8 at the same point where I put vFrames in cmd. It doesnt seem to work at all now no frames are being extracted from the video
This is the effective flow for fps=x, with the default rounding method near,
Generate intervals of 1/x seconds, e,g. for x=1/3, intervals are 0-3s, 3-6s, 6-9s... Within each interval, pick an earlier frame whose timestamp is nearest to the midpoint of that interval. This may lead to duplicate frames, when that nearest frame belongs to an earlier interval, and was already picked for that interval. Likely to happen if source is variable frame-rate.
In your case, I would suggest to avoid rounding or stripping fractional values, if possible. Or do it at the end.
Related
I am relatively new to programming, and to help myself I am making a personal project.
I am using Javafx to build drum machine, which allowing the user to program a sequence of beats.
I have constructed sets of rows which each will function as a programmable beat sequencer for each corresponding instrument, with each row consisting of 16 buttons. if that button is pressed, the button is activated, and it will produce the instrument's sound when the loop passes through that point.
for reference this piece of kit is similar to what i wish to construct :
https://images-na.ssl-images-amazon.com/images/I/81RctDCP38L._AC_SL1500_.jpg
Each button is assigned to a hashmap; the Key is an integer from 0-16, while the value is the button's characteristics itself.
The drum machine loops after 4 bars/ 16 buttons.
To trigger the event and to cause the instrument to play, the time (as a fraction of all buttons/16) will match the key of a button. once this occurs, the sound plays. the method to do this is below:
public void beatState(Button button, String filename) {
EventHandler handler = new EventHandler() {
#Override
public void handle(Event event) {
soundGeneration sound = new soundGeneration(filename);
// if the corresponding buttons key (on a range of 0-16) matches the time as a fraction (0-16)
// and if the button text is on (activated by clicking the pad)
if (button.equals(map.get(time.timeToFraction())) & button.getText().equals("On")) {
// plays the file
sound.play();
// when duration of animation is set lower the filename prints more frequently
// or sometimes not printed/sound played when the duration is higher
System.out.println(filename);
}
}
};
// as i increase the duration of the animation , the program becomes both slower but returns
// both the sound file and prints the filename more often
Timeline animationButton = new Timeline(new KeyFrame(Duration.millis(100), handler));
animationButton.setCycleCount(Timeline.INDEFINITE);
animationButton.play();
}
I'll next provide the time element:
public Integer timeToFraction(){
labelFormat();
new AnimationTimer() {
#Override
public void handle(long now) {
// time elapsed since program execution
double elapsedMillis = (System.currentTimeMillis() - startTime);
// converts to long
// multiplies the value to privide a fraction
long numerator = (long) ((elapsedMillis/1000)*8.33);
// value below denominates the time it takes to travel 4 beats at 125 beats per minute
double denominator = 1.92;
// converts below to show as a fraction
long denominatorToBeat =(long) Math.round(denominator * 8.3);
// if the elapsed time raises over 16
// resets numerator
if (numerator> denominatorToBeat) {
startTime = System.currentTimeMillis();
elapsedMillis = (System.currentTimeMillis() - startTime);
}
// converts from long to to allow for hashmap matching against button position
fractionTime = (int) numerator;
}
}.start();
return fractionTime;
}
When the values match up the beat plays, achieving my aim; however, it plays multiple times, and irregular in relation to the beat.
I assume that the animation timer repeat value in milliseconds is what causes this; I decrease it, there are more unwanted repeated sounds. I increase it and notes sometimes are not counted due to the animation passing over the value before it triggers.
I want the code to trigger the sound file when it passes over the button, more specifically when the Integer value of the hashmap the button corresponds to matches the time fraction.
I have spent hours researching and reading the documentation and for an easy problem it is becoming incredibly difficult to work around. Given that this project is seen in a multitude of portfolios and music development software i am sure there is a simple fix to my issue.
Thanks for taking the time.
Your approach in general is wrong, as you are abusing animation timers to do something not related to the GUI. You have to decouple the logic from the GUI.
I.e. The object which controls the timing and playing the sounds should have nothing to do with the buttons or any GUI. It should have some methods to set / clear a sound at particular interval, and then expose its state using properties, e.g. an ObjectProperty with some class describing the current tones being played.
The buttons should call the methods to set / clear tones, and you can add a change listener to the ObjectProperty to update the buttons appearance based on the state.
I´m making an Android game (just to learn some stuff). You collect objects and you get points for that. When player reaches for example 200 points, the objects start coming faster, the background changes etc...It is like a next level. But when you reach those 200 points, I would like to show an image or text on the screen (something like "2 level") for like 2 seconds but I don´t know how.
I tried to work with the timer but I failed.
I´ve got an "if" statement
if (score >= 200) {
frameLayout.setBackgroundResource(R.drawable.lvl2); // background change
// Make objects go faster
collect_obj1 = Math.round(screenWidth / 57F);
collect_obj2 = Math.round(screenWidth / 33F);
critical_obj = Math.round(screenWidth / 42F); // If you hit this one = Game Over
characterLvl1.setImageResource(R.drawable.characterLvl2); // character change
}
Try this:
long start_time = System.currentTimeMillis();
long current_time = System.currentTimeMillis();
long time_limit = 2000; // In milliseconds: this is the time you want to show the
// object(2 sec)
while(current_time - start_time) != 2000) {
ShowObject(); //Here is where you show the object.
current_time = System.currentTimeMillis();
}
HideObject(); //Hide the object after the while loop
It should work. And of course implement it however you want, because this is not the cleanest code ever. Or if you want something more efficient with handlers, etc. see this:
Link to another similar question on StackOverflow
Edit: Made it a bit better.
I have a generator which generates events for Flink CEP, code for which is given below. Basically, I am using Thread.sleep() and I have read somewhere that java can't sleep less than 1 millisecond even we use System.nanoTime(). Code for the generator is
public class RR_interval_Gen extends RichParallelSourceFunction<RRIntervalStreamEvent> {
Integer InputRate ; // events/second
Integer Sleeptime ;
Integer NumberOfEvents;
public RR_interval_Gen(Integer inputRate, Integer numberOfEvents ) {
this.InputRate = inputRate;
Sleeptime = 1000 / InputRate;
NumberOfEvents = numberOfEvents;
}
#Override
public void run(SourceContext<RRIntervalStreamEvent> sourceContext) throws Exception {
long currentTime;
Random random = new Random();
int RRInterval;
int Sensor_id;
for(int i = 1 ; i <= NumberOfEvents ; i++) {
Sensor_id = 2;
currentTime = System.currentTimeMillis();
// int randomNum = rand.nextInt((max - min) + 1) + min;
RRInterval = 10 + random.nextInt((20-10)+ 1);
RRIntervalStreamEvent stream = new RRIntervalStreamEvent(Sensor_id,currentTime,RRInterval);
synchronized (sourceContext.getCheckpointLock())
{
sourceContext.collect(stream);
}
Thread.sleep(Sleeptime);
}
}
#Override
public void cancel() {
}
}
I will specify my requirement here in simple words.
I want generator class to generate events, let's say an ECG stream at 1200 Hz. This generator will accept parameters like input rate and total time for which we have to generate the stream.
So far so good, the issue is that I need to send more than 1000 events / second. How can I do this by using generator function which is generating values U[10,20]?
Also please let me know if I am using wrong way to generate x number of events / second in the above below.
Sleeptime = 1000 / InputRate;
Thanks in advance
The least sleep time in Windows systems is ~ 10 ms and in Linux and Macintosh is 1 millisecond as mentioned here.
The granularity of sleep is generally bound by the thread scheduler's
interrupt period. In Linux, this interrupt period is generally 1ms in
recent kernels. In Windows, the scheduler's interrupt period is
normally around 10 or 15 milliseconds
Through my research, I learned that using the nano time sleep in java will not help as the issue in at OS level. If you want to send data at arrival rate > 1000 in a controlled way, then it can be done using Real-Time Operating Systems (RTOS), as they can sleep for less then a millisecond. Now, I have come up with another way of doing it, but in this solution, the interarrival times will not be constantly distributed.
Let's say you want arrival rate of 3000 events/ second, then you can create a for loop which iterates 3 times to send data in each iteration and then sleep for 1ms. So for the 3 tuples, the interarrival time will be close to one another, but the issue will be solved. This may be a stupid solution but it works.
Please let me know if there is some better solution to this.
When I use onSensorChanged() to test my cellphone's accelerometer, finding that it responses to fast. Almost every 1~3ms onSensorChanged() will be activated. I search for some other cellphone's information feeling that it is kind of weird, so I doubt that my code might be wrong. Here is part of my code:
public void onSensorChanged(SensorEvent se_a) { /* 取得x,y,z加速度值 */
xa = se_a.values[0];
ya = se_a.values[1];
za = se_a.values[2];
final String timeStamp_a = new SimpleDateFormat("HHmmssSSS",
Locale.UK).format(new Date());
String tmp_a = "0 " + timeStamp_a + " " + String.valueOf(xa)
+ " " + String.valueOf(ya) + " " + String.valueOf(za) + "\n";
And part of result is like:
0 160106203 9.5385 -0.6895301 1.1109096
0 160106204 9.500193 -0.5746084 1.1109096
0 160106206 9.576807 -0.5746084 1.1875241
0 160106207 9.461885 -0.6895301 1.3024458
My cellphone is LG G2. I set the accelerometer to SENSOR_DELAY_FASTEST. But using some app on google play to test my accelerometer, it shows that in SENSOR_DELAY_FASTEST the frequency is 120Hz, so it is very weird to find that onSensorChanged() response so fast(1~3ms). Where is my code can be wrong? Please help me!
If the update rate is to fast using SENSOR_DELAY_FASTEST you can set another flag which fit your needs.
See the docs for the different rates you can use:
The default data delay (SENSOR_DELAY_NORMAL) is suitable for
monitoring typical screen orientation changes and uses a delay of
200,000 microseconds. You can specify other data delays, such as SENSOR_DELAY_GAME (20,000 microsecond delay), SENSOR_DELAY_UI
(60,000 microsecond delay), or SENSOR_DELAY_FASTEST (0
microsecond!!!! delay). As of Android 3.0 (API Level 11) you can
also specify the delay as an absolute value (in microseconds).
Edit: Have a look at this method SensorManager.registerListener(SensorEventListener, Sensor, int) where you can specify your delay in ms if no flag supports your needs. This is available since API 9. If you want a specific frequenzy provide it as a parameter.
Hi,
I am developing a game with the help of LibGDX and using Box2d in it. The problem is that when run my game on hdpi or tablets it run fine but in case of ldpi and mdpi the box2d bodies are not acting accordingly.
I think, it is taking much more time to render on those phones. So, how can I optimize my game for ldpi and mdpi phones.The values I am passing in world.step isworldbox.step(Gdx.graphics.getDeltaTime(), 10, 2000);
Thanks.
It is bad idea to use frame rate as time step. Box2D manual says:
A variable time step produces variable results, which makes it difficult to debug. So don't tie the time step to your frame rate (unless you really, really have to).
Also, you use too big values for velocity and position iterations. Box2D manual says:
The suggested iteration count for Box2D is 8 for velocity and 3 for position.
Try fixed time step and recomended iteration count like this:
float mAccomulated = 0;
float mTimeStep = 1.0f / 60.0f;
int mVelocityIterations = 8;
int mPositionIterations = 3;
void updatePhysicWorld()
{
float elapsed = Gdx.graphics.getDeltaTime();
// take into account remainder from previous step
elapsed += mAccomulated;
// prevent growing up of 'elapsed' on slow system
if (elapsed > 0.1) elapsed = 0.1;
float acc = 0;
// use all awailable time
for (acc = mTimeStep; acc < elapsed; acc += mTimeStep)
{
mWorld->Step(mTimeStep, mVelocityIterations, mPositionIterations);
mWorld->ClearForces();
}
// remember not used time
mAccomulated = elapsed - (acc - mTimeStep);
}