I want to extract audio data to memory using vlcj (https://github.com/caprica/vlcj, version 4.2.0). I don't want to play the video on the same time, just extract the audio data, as fast as the performance allows.
Right now I'm using a workaround based on this: https://github.com/caprica/vlcj/blob/master/src/test/java/uk/co/caprica/vlcj/test/rip/RipAudioTest.java, i.e. output the data to a file first, and then read the file. While this solution is working, it's not optimal and it takes disk space.
Maybe the example above can be modified to direct the audio to a callback instead.
A second example is:
https://github.com/caprica/vlcj/blob/master/src/test/java/uk/co/caprica/vlcj/test/directaudio/DirectAudioPlayerTest.java
In that example, the audio data is extracted to memory, which is what I want. But it also plays the video in a window, which is not what I want. Maybe that example can be modified to turn off the video somehow, and make it run as fast as possible?
There's no perfect answer here, sadly.
Using the DirectAudioPlayerTest.java that you found already, you can change the media player factory creation to pass this parameter to prevent the video window being opened:
factory = new MediaPlayerFactory("--novideo");
You will receive the audio into a memory buffer at the rate at which VLC decodes it, so if you have a 5 minute video it will take 5 minutes to extract the audio - not ideal.
This is the only way with vlcj that you can grab the audio data in memory.
The RipAudioTest.java that you found, IIRC, extracts the audio as quickly as possible, which may be a lot faster than the normal playback speed - but here you can't grab the decoded audio directly into your Java application.
So the solution you already have, ripping the track to disk first, might actually be the best solution you can achieve with vlcj here (since it could be considerably quicker than using the direct audio player).
Related
I am currently delopping an application where the user can load a .mp3 file and enter a sequence of notes. The goal for the user is to match this sequence of notes with the song of the .mp3 file.
This requires the possibility to play the .mp3 file and the sequence of notes simultaneously. After some research I found out that either the Java Sound API or JFuge can do the job to produce a sequence of notes (MIDI). (The input given by the user). As stated here, JLayer can be used to play mp3 files in Java. (I could also transform the .mp3 to .wav and use another way to play the transformed .wav).
However, would it be possible to play this .mp3 and sequence of notes together without any problems, or should I first convert them to one single file?
The user should be able to play the .mp3 and his/her sequence of notes at any random timestamp simultaneously. Preferably without any delay so the user can easily adapt a note to match the pitch of the file. It seems that merging them together to one file, before playing them, would be too much overhead when the user is almost constantly changing a note and replaying to check if it matches the pitch.
Thanks in advance!
Java supports playback from multiple threads. All you need to do is run the .mp3 from one thread, and the midi-generated notes on another concurrently running thread.
There used to be a few Linux systems that could only handle output from one audio source at a time. I don't know if this is still an issue.
Another, much more elaborate possibility that would let you do live mixing and output to a single line would be to read the song file using AudioInputStream, convert the bytes to PCM on the fly (e.g., to floats ranging from -1 to 1) (or preload and store the audio as PCM), and then add this to PCM data coming from a do-it-yourself synth, and then convert this back to bytes and output via a SourceDataLine.
That is a lot of trouble and you probably don't want to go that route, but if you did, following is some info to help break down the various steps of one possible realization.
Loading .wav data and converting it into an internal PCM form can be seen in the open-source AudioCue (line 359 loadURL method). And here is an example (free download) of a real-time Java synth I made that runs via keystrokes. One of the voices is a simple organ, which outputs PCM audio data by just adding four sine waves at harmonic frequencies. Making other sounds is possible if you want to get into other forms of synthesis but gets more involved.
(IDK how to convert data coming from a MIDI-controlled synth, unless maybe a TargetDataLine can be identified, and data from it converted to PCM similar to the conversion used in reading from an AudioInputStream in the AudioCue source example.)
Given two PCM sources, the two can be mixed in real time using addition, converted to bytes and output via a single SourceDataLine (see line 1387 convertBufferToAudioBytes method). The SourceDataLine can be kept running indefinitely if you input zeros from the contributors when they are not playing. An SDL spends the vast majority of its time in a blocked state as audio data processing is much quicker than the rate it is consumed by the system, and so uses very little cpu.
This question already has answers here:
How to calculate a file size from URL in java
(7 answers)
Closed 6 years ago.
How can I obtain the file length from an HTTP URL?
I am creating a bot for an app and I have a music player for that bot.
When you type !stream http://example/audio.mp3 the bot connects to your voice channel and starts playing the song. I need to make the bot disconnect when the song has ended, and for that I need the length of that audio file from the website.
Edit: what I need is the length of the audio file preferably in seconds so I can determin after what ammount of time to stop the connection. Also the bot is for discord and I use an external api: https://github.com/DV8FromTheWorld/JDA I hope this makes things more clear.
Ok, so JDA. Thanks for the clarification.
I am not an expert in JDA, but found out the followings:
check this audio example: https://github.com/DV8FromTheWorld/JDA/blob/master/src/examples/java/AudioExample.java locate the usage of the player instance of FilePlayer
find the javdoc for FilePlayer: https://github.com/DV8FromTheWorld/JDA/blob/master/src/main/java/net/dv8tion/jda/audio/player/FilePlayer.java
notice that it has an isPlaying() method.
Now you can simply periodically check if the audio is still played by checking isPlaying periodically.
To dig out the length of the audio
You can see that FilePlayer extends Player.java.
check the source of Player java https://github.com/DV8FromTheWorld/JDA/blob/master/src/main/java/net/dv8tion/jda/audio/player/Player.java
you can see the usage of AudioInputStream in the setAudioSource method
get the javadoc of AudioInputStream https://docs.oracle.com/javase/7/docs/api/javax/sound/sampled/AudioInputStream.html
notice getFrameLength which returns the size of the stream. You can see in Player as the audio is actually getting resampled to 48KHz, so you know that 48000 frames mean 1 second. Multiply the result of getFrameLength by 48000 to obtain the length in seconds.
Note that it won't really help in your case, unless you're tracking the playback progress separately, so this solution is unnecessarily complicated.
Length of audio? In bytes? In seconds? there are plenty of good answers!
Stop when the download is over
You seem to access to a static mp3 file, which is loaded via http protocol. Now you don't need to do anything, since once all the data from that mp3 file has been received by your computer, the connection is over.
Check this: http://www.codejava.net/java-se/networking/use-httpurlconnection-to-download-file-from-an-http-url find the while loop which goes until there is data from the server. Then the while loop exits when there is no data.
So to put it short: once inputStream.read(buffer) returns -1, there is no more data, so receiving your mp3 is over.
Stop when the playback is over
It is a separate question if you want to detect when playback of the mp3 is over. For that, you need to post some of your code, and explain what mp3 decoder/sound playback libraries are you using.
For example Un4seen development's BASS framework offers both callbacks, reading out of the total time during decoding and reading the current time of the playback too. Doing the same with Java Audio API is not that trivial - since the API writes to the DMA buffers, and the last half a second plays after all the data has been sent.
There are a lot more frameworks out there to handle audio and mp3 decoding, therefore giving a clear answer to your question is not possible without further input :)
Im looking for a easy way to record video (including audio) into a circular buffer stored in RAM.
So I can leave the video recording it will keep the feed in RAM the last 2 minutes and then have the option to commit it to memory if required.
Unfortunately, I cannot see an easy way to do this. So far I have investigated using:
MediaRecorder - I could not see a way to store the output data in a
buffer. Only option is setOutputFile()
JavaCV FFmpegFrameRecorder - again the constructor for this requires
passing in a file.
android.hardware.Camera.PreviewCallback - this gives a byte array for
every frame which I could add to a buffer. However, this approach does not provide any audio data.
I feel like there must be a easy way to do this but so far I've not had much luck.
Any help with this would be very appreciated.
JavaCV offers cvCreateCameraCapture() to open an interface with the camera. This code demonstrates how to capture frames from the camera and display them on a window.
The interesting part resides in the while loop: every iteration of loop retrieves a single frame from the camera with cvQueryFrame(). Note that the data type returned by this function is IplImage, and this type is responsible for storing the image grabbed from the camera.
All you need to do is store the IplImage returned by cvQueryFrame() in your circular buffer.
I've been working on this problem for a while now. I'm working on a project where I recieve a stream of images which have been processed by OpenCV on a socket, and I need to display these feeds in MT4J. We are transferring the images inside of a Google Protocol Buffer message. In my current (and naive) approach, I grab images off of the socket and simply set them as a the texture in an overloaded draw method from MTRectangle. While this works, if I attempt to display more than 3 feeds at a time, the frame rate drops to an unacceptable rate (<1 fps), and it takes up ~70-80% of my CPU.
A co-worker managed to use GStreamer/Xuggler to display about 20 videos in MT4J, but was using static files instead of a real-time stream. Because of this, I attempted to find a tool in MT4J or Xuggler which would allow me to convert the images to a video stream on the fly, but everything seems to simply be converting a collection of images on the disk to a stream, or vice-versa, but not in real time.
So at this point I see two possible solutions. First, is there a more efficient way to set/draw textures in MT4J? Secondly, is there some tool that exists/some library in GStreamer or Xuggler which would do the image to video conversion in real time?
MT4J website for people who don't know what it is:
http://www.mt4j.org
Thanks in advance, and let me know if there is any information which I left out.
The goal is to get a simple 2d audio visualizer that is somewhat responsive to the music.
I've got the basics set up, where I have graphics that will respond to some data being fed in. Given a file, I load up an audioInputStream for playback (this works fine), and have that running in a thread. In another thread, I would like to extract byte data at a rate close to the playback (or perhaps faster, to allow for delay in processing that data). I then want to feed that to an FFT process, and feed the resulting data to my graphics object that will use it as a parameter for whatever the visualization is.
I have two questions for this process:
1) How can I get the byte data and process it at a rate that will match the normal playback of a file? Is using an audioInputStream the way to go here?
2) Once I do FFT, what's a good way to get usable data (ie: power spectrum? Somehow filtering out certain frequencies? etc..)
Some considerations about (2) using the FFT to extract "features".
You should calculate short-term FFTs for example 512 points, whenever there is enough CPU cycles free to do so. For a visualisation it is not necessary to preserve all information (i.e. work with overlapping windows), instead you could calculate a 100ms FFT 5 times per second.
Then you should calculate the logarithmic power spectrum in dB (decibel).
This gives you a pretty good impression about the detailed frequency content of your sound.
Depending on what you like to visualize you could for example combine some low frequency FFT lines (calculate the RMS) to get the "Bass" content of your sound and so on.
See this post for details.