Playing a wav file with TarsosDSP on Android - java

Problem: Wav file loads and is processed by AudioDispatcher, but no sound plays.
First, the permissions:
public void checkPermissions() {
if (PackageManager.PERMISSION_GRANTED != ContextCompat.checkSelfPermission(this.requireContext(), Manifest.permission.RECORD_AUDIO)) {
//When permission is not granted by user, show them message why this permission is needed.
if (ActivityCompat.shouldShowRequestPermissionRationale(this.requireActivity(), Manifest.permission.RECORD_AUDIO)) {
Toast.makeText(this.getContext(), "Please grant permissions to record audio", Toast.LENGTH_LONG).show();
//Give user option to still opt-in the permissions
}
ActivityCompat.requestPermissions(this.requireActivity(), new String[]{Manifest.permission.RECORD_AUDIO}, MY_PERMISSIONS_RECORD_AUDIO);
launchProfile();
}
//If permission is granted, then proceed
else if (ContextCompat.checkSelfPermission(this.requireContext(), Manifest.permission.RECORD_AUDIO) == PackageManager.PERMISSION_GRANTED) {
launchProfile();
}
}
Then the launchProfile() function:
public void launchProfile() {
AudioMethods.test(getActivity().getApplicationContext());
//Other fragments load after this that actually do things with the audio file, but
//I want to get this working before anything else runs.
}
Then the AudioMethods.test function:
public static void test(Context context){
String fileName = "audio-samples/samplefile.wav";
try{
releaseStaticDispatcher(dispatcher);
TarsosDSPAudioFormat tarsosDSPAudioFormat = new TarsosDSPAudioFormat(TarsosDSPAudioFormat.Encoding.PCM_SIGNED,
22050,
16, //based on the screenshot from Audacity, should this be 32?
1,
2,
22050,
ByteOrder.BIG_ENDIAN.equals(ByteOrder.nativeOrder()));
AssetManager assetManager = context.getAssets();
AssetFileDescriptor fileDescriptor = assetManager.openFd(fileName);
InputStream stream = fileDescriptor.createInputStream();
dispatcher = new AudioDispatcher(new UniversalAudioInputStream(stream, tarsosDSPAudioFormat),1024,512);
//Not playing sound for some reason...
final AudioProcessor playerProcessor = new AndroidAudioPlayer(tarsosDSPAudioFormat, 22050, AudioManager.STREAM_MUSIC);
dispatcher.addAudioProcessor(playerProcessor);
dispatcher.run();
Thread audioThread = new Thread(dispatcher, "Test Audio Thread");
audioThread.start();
} catch (Exception e) {
e.printStackTrace();
}
}
Console output. No errors, just the warning:
W/AudioTrack: Use of stream types is deprecated for operations other than volume control
See the documentation of AudioTrack() for what to use instead with android.media.AudioAttributes to qualify your playback use case
D/AudioTrack: stop(38): called with 12288 frames delivered
Because the AudioTrack is delivering frames, and there aren't any runtime errors, I'm assuming I'm just missing something dumb by either not having sufficient permissions or I've missed something in setting up my AndroidAudioPlayer. I got the 22050 number by opening the file in Audacity and looking at the stats there:
Any help is appreciated! Thanks :)

Okay, I figured this out.
I'll address my questions as the appeared originally:
TarsosDSPAudioFormat tarsosDSPAudioFormat = new TarsosDSPAudioFormat(TarsosDSPAudioFormat.Encoding.PCM_SIGNED,
22050,
16, //based on the screenshot from Audacity, should this be 32?
1,
2,
22050,
ByteOrder.BIG_ENDIAN.equals(ByteOrder.nativeOrder()));
ANS: No. Per the following TarsosDSP AndroidAudioPlayer header (copied below), I'm limited to 16:
/**
* Constructs a new AndroidAudioPlayer from an audio format, default buffer size and stream type.
*
* #param audioFormat The audio format of the stream that this AndroidAudioPlayer will process.
* This can only be 1 channel, PCM 16 bit.
* #param bufferSizeInSamples The requested buffer size in samples.
* #param streamType The type of audio stream that the internal AudioTrack should use. For
* example, {#link AudioManager#STREAM_MUSIC}.
* #throws IllegalArgumentException if audioFormat is not valid or if the requested buffer size is invalid.
* #see AudioTrack
*/
The following modifications needed to be made to the test() method (this worked for me):
public static void test(Context context){
String fileName = "audio-samples/samplefile.wav";
try{
releaseStaticDispatcher(dispatcher);
TarsosDSPAudioFormat tarsosDSPAudioFormat = new TarsosDSPAudioFormat(TarsosDSPAudioFormat.Encoding.PCM_SIGNED,
22050,
16,
1,
2,
22050,
ByteOrder.BIG_ENDIAN.equals(ByteOrder.nativeOrder()));
AssetManager assetManager = context.getAssets();
AssetFileDescriptor fileDescriptor = assetManager.openFd(fileName);
FileInputStream stream = fileDescriptor.createInputStream();
dispatcher = new AudioDispatcher(new UniversalAudioInputStream(stream, tarsosDSPAudioFormat),2048,1024); //2048 corresponds to the buffer size in samples, 1024 is the buffer overlap and should just be half of the 'buffer size in samples' number (so...1024)
AudioProcessor playerProcessor = new customAudioPlayer(tarsosDSPAudioFormat, 2048); //again, 2048 is the buffer size in samples
dispatcher.addAudioProcessor(playerProcessor);
dispatcher.run();
Thread audioThread = new Thread(dispatcher, "Test Audio Thread");
audioThread.start();
} catch (Exception e) {
e.printStackTrace();
}
}
You'll notice I now create a 'customAudioPlayer', which is, in reality copy-pasted straight from TarsosDSP AndroidAudioPlayer with two small adjustments:
I hardcoded the stream type in the AudioAttributes .Builder() method so am no longer passing them in.
I'm using the AudioTrack.Builder() method because using stream types for playback was deprecated. Admittedly, I'm not sure if this was the change that fixed it, or if it was the change to the buffer size (or both?).
/*
* Constructs a new AndroidAudioPlayer from an audio format, default buffer size and stream type.
*
* #param audioFormat The audio format of the stream that this AndroidAudioPlayer will process.
* This can only be 1 channel, PCM 16 bit.
* #param bufferSizeInSamples The requested buffer size in samples.
* #throws IllegalArgumentException if audioFormat is not valid or if the requested buffer size is invalid.
* #see AudioTrack
*/
public customAudioPlayer(TarsosDSPAudioFormat audioFormat, int bufferSizeInSamples) {
if (audioFormat.getChannels() != 1) {
throw new IllegalArgumentException("TarsosDSP only supports mono audio channel count: " + audioFormat.getChannels());
}
// The requested sample rate
int sampleRate = (int) audioFormat.getSampleRate();
//The buffer size in bytes is twice the buffer size expressed in samples if 16bit samples are used:
int bufferSizeInBytes = bufferSizeInSamples * audioFormat.getSampleSizeInBits()/8;
// From the Android API about getMinBufferSize():
// The total size (in bytes) of the internal buffer where audio data is read from for playback.
// If track's creation mode is MODE_STREAM, you can write data into this buffer in chunks less than or equal to this size,
// and it is typical to use chunks of 1/2 of the total size to permit double-buffering. If the track's creation mode is MODE_STATIC,
// this is the maximum length sample, or audio clip, that can be played by this instance. See getMinBufferSize(int, int, int) to determine
// the minimum required buffer size for the successful creation of an AudioTrack instance in streaming mode. Using values smaller
// than getMinBufferSize() will result in an initialization failure.
int minBufferSizeInBytes = AudioTrack.getMinBufferSize(sampleRate, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT);
if(minBufferSizeInBytes > bufferSizeInBytes){
throw new IllegalArgumentException("The buffer size should be at least " + (minBufferSizeInBytes/(audioFormat.getSampleSizeInBits()/8)) + " (samples) according to AudioTrack.getMinBufferSize().");
}
//http://developer.android.com/reference/android/media/AudioTrack.html#AudioTrack(int, int, int, int, int, int)
//audioTrack = new AudioTrack(streamType, sampleRate, AudioFormat.CHANNEL_OUT_MONO, AudioFormat.ENCODING_PCM_16BIT, bufferSizeInBytes,AudioTrack.MODE_STREAM);
try {
audioTrack = new AudioTrack.Builder()
.setAudioAttributes(new AudioAttributes.Builder()
.setUsage(AudioAttributes.USAGE_MEDIA)
.setContentType(AudioAttributes.CONTENT_TYPE_MUSIC)
.build())
.setAudioFormat(new AudioFormat.Builder()
.setEncoding(AudioFormat.ENCODING_PCM_16BIT)
.setSampleRate(sampleRate)
.setChannelMask(AudioFormat.CHANNEL_OUT_MONO)
.build())
.setBufferSizeInBytes(bufferSizeInBytes)
.build();
audioTrack.play();
} catch (Exception e) {
e.printStackTrace();
}
}
Also, on my device I noticed that the volume control rocker switches just control the ringer volume by default. I had to open an audio menu (three little dots once the ringer volume was 'active') to turn up the media volume.

Related

Unwanted downsampling : Java Sound

I have been trying to manually read a wav file in Java and read an array of bytes then write to an audio buffer for playback. I am receiving playback but it is heavily distorted. Java sound supports 16 bit sample rates but not 24-bit.
I went in to Logic 9 and exported a 24-bit audio file in to 16-bit and then used with my program. Originally, the 24-bit samples would produces white noise. Now I can hear my sample but very distorted and sounds like it has been bit crushed.
Can anyone help me to get a clean signal?
I am very new to audio programming but I am currently working on a basic Digital Audio Workstation.
import javax.sound.sampled.*;
import javax.sound.sampled.DataLine.Info;
import javax.swing.filechooser.FileNameExtensionFilter;
import java.io.*;
public class AudioData {
private String filepath;
private String filepath1;
private File file;
private byte [] fileContent;
private Mixer mixer;
private Mixer.Info[] mixInfos;
private AudioInputStream input;
private ByteArrayOutputStream byteoutput;
public static void main (String [] args) {
AudioData audiodata = new AudioData();
}
public AudioData () {
filepath = "/Users/ivaannagen/Documents/Samples/Engineering Samples - Obscure Techno Vol 3 (WAV)/ES_OT3_Kit03_Gmin_130bpm/ES_OT3_Kit03_FX_Fast_Snare_Riser_Gmin_130bpm.wav";
filepath1 = "/Users/ivaannagen/Documents/Samples/dawsampletest.wav";
file = new File (filepath1);
readAudio();
}
public void readAudio () {
mixInfos = AudioSystem.getMixerInfo();
mixer = AudioSystem.getMixer(mixInfos[0]);
AudioFormat format = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, 44100, 16, 2, 4, 44100, false);
// set up an audio format.
try {
DataLine.Info info = new DataLine.Info(SourceDataLine.class, format); // creates data line with class type and audio format.
SourceDataLine source = (SourceDataLine) AudioSystem.getLine(info);
System.out.println("Size of data line buffer: " + source.getBufferSize());
fileContent = new byte [source.getBufferSize() / 50];
byteoutput = new ByteArrayOutputStream();
input = AudioSystem.getAudioInputStream(file);
int readBytes = 0;
while ((readBytes = input.read(fileContent, 0, fileContent.length)) != -1) {
byteoutput.write(fileContent, 0, readBytes);
}
System.out.println("Size of audio buffer: " + fileContent.length);
//byteoutput.write(0);
// byteoutput.write(0);
System.out.println("Size of audio buffer: " + byteoutput.size());
source.open(format, source.getBufferSize()); // line must be open to be recognised by the mixer.
Line[] lines = mixer.getSourceLines();
System.out.println("mixer lines: " + lines.length);
// for(byte bytes: fileContent) {
// System.out.println(bytes);
// }
Thread playback = new Thread () {
public void run () {
// System.out.println((byteoutput.size() +2) % 4);
source.start(); // play (buffer originally empty)
source.write(byteoutput.toByteArray(), 0, byteoutput.size()); // write input bytes to output buffer
} // end run (to do).
}; // end thread action
playback.start(); // start thread
}
catch (LineUnavailableException lue) {
System.out.println(lue.getMessage());
}
catch (FileNotFoundException fnfe) {
System.out.println(fnfe.getMessage());
}
catch(IOException ioe) {
System.out.println(ioe.getMessage());
}
catch(UnsupportedAudioFileException uafe) {
System.out.println(uafe.getMessage());
}
}
}
Whether or not you can load and play a 24-bit file is system dependent, afaik.
I use Audacity for conversions. You should be able import your file into Audacity and export it as 16-bit, stereo, little-endian, 44100 fps, and then load that export with Java's AudioInputStream.
What you hear when playing from Audacity or from Java should be pretty much identical (adjusting for volume). If not, the most likely reason probably pertains to a mistake or overlook in the code, which is very easy to do.
The use of a ByteOutputStream in your code is superfluous. Read from the AudioInputStream into a fixed-size byte array (size being the buffer length, I recommend trying 8 or 16 * 1024 bytes as a first try) and then use the SourceDataLine write method to ship that array.
Following is code that works on my system for loading a playing a "CD Quality" wav called "a3.wav" that I have that is in the same directory as the Java class. You should be able to swap in your own 44100, 16-bit, stereo, little-endian wav file.
I've commented out an attempt to load and play a 24-bit wav file called "spoken8000_24.wav". That attempt gave me an IllegalArgumentException: No line matching interface SourceDataLine supporting format PCM_SIGNED 8000.0 Hz, 24 bit, stereo, 6 bytes/frame, little-endian is supported.
I have to admit, I'm unclear if my system doesn't provide the needed line or if I might have coded the format incorrectly! My OS can certainly play the file. So I'm thinking there is a distinction between what an OS can do and what a "Mixer" on a given system provides to Java.
As a get-around, I just always convert everything to "CD Quality" format, as that seems to be the most widely supported.
public class TriggerSound_SDL extends JFrame
{
public TriggerSound_SDL()
{
JButton button = new JButton("Play Sound");
button.addActionListener(e -> new Thread(() -> playBuzzer()).start());
getContentPane().add(button);
}
private void playBuzzer()
{
try
{
URL url;
url = getClass().getResource("a3.wav");
// url = getClass().getResource("spoken8000_24.wav");
AudioInputStream ais = AudioSystem.getAudioInputStream(url);
System.out.println(ais.getFormat());
AudioFormat audioFmt;
// "CD Quality" 44100 fps, 16-bit, stereo, little endian
audioFmt = new AudioFormat(
AudioFormat.Encoding.PCM_SIGNED,
44100, 16, 2, 4, 44100, false);
// 8000 fps, 32-bit, stereo
// audioFmt = new AudioFormat(
// AudioFormat.Encoding.PCM_SIGNED,
// 8000, 24, 2, 6, 8000, false);
Info info = new DataLine.Info(SourceDataLine.class,
audioFmt);
SourceDataLine sdl = (SourceDataLine)AudioSystem.getLine(info);
int bufferSize = 16 * 1024;
byte[] buffer = new byte[bufferSize];
sdl.open(audioFmt, bufferSize);
sdl.start();
int numBytesRead = 0;
while((numBytesRead = ais.read(buffer)) != -1)
{
sdl.write(buffer, 0, numBytesRead);
}
}
catch (IOException | UnsupportedAudioFileException
| LineUnavailableException ex)
{
ex.printStackTrace();
}
}
private static void createAndShowGUI()
{
JFrame frame = new TriggerSound_SDL();
frame.setDefaultCloseOperation(DISPOSE_ON_CLOSE);
frame.pack();
frame.setVisible(true);
}
public static void main(String[] args)
{
SwingUtilities.invokeLater(() -> createAndShowGUI());
}
}
This code, with some small tweaks should let you at least test the different formats.
EDIT:
I'm seeing where your goal is to make a DAW!
In that case, you will want to convert the bytes to PCM data. Can I suggest you borrow some code from AudioCue? I basically wrote it to be a Clip-substitute, and part of that involved making the PCM data available for manipulation. Some techniques for mixing, playing back at different frequencies, multithreading can be found in it.
Thanks for all the advice guys. I will be getting rid of the ByteOutputStream and just use the AudioInputStream, I now understand what I was doing was unnecessary!! Thanks for the advice all! I have indeed tried using AudioCue but it is not low level enough for what I want to do!
One more thing guys. Previously, I created a multitrack media player which is using the Clip class. To play all the audio tracks together, I was looping through a list of Clips and playing them. However, this means that all tracks may be playing a tiny amount after each other due to the processing of the loop. Also, Clip class created a new thread per audio. I do not wants 100 threads running on 100 tracks, I want one thread for my audio output. I am still trying to work out how to start all tracks at the same time without a loop....(im guessing AudioCue have nailed the concurrent cues).
Does anyone know the best way to play multiple audio tracks in to one output? Do I need to route/bus all my audio tracks in to one output and somehow write all data from audio files in to one output buffer then play this output in a thread?
Thanks!!

Get AudioInputStream of FloatBuffer

I have a callback that gets incoming audio data as FloatBuffer containing 1024 floats that gets called several times per second. But I need an AudioInputStream since my system only works with them.
Converting the floats into 16bit PCM isgned audio data is not a problem, but I cannot create an InputStream out of it. The AudioInputStream constructor only accepts data with known length, but I have a constant stream. The AudioSystem.getAudioInputStream throws an "java.io.IOException: mark/reset not supported" if I feed it with a PipedInputStream containing the audio data.
Any ideas?
Here's my current code:
Jack jack = Jack.getInstance();
JackClient client = jack.openClient("Test", EnumSet.noneOf(JackOptions.class), EnumSet.noneOf(JackStatus.class));
JackPort in = client.registerPort("in", JackPortType.AUDIO, EnumSet.of(JackPortFlags.JackPortIsInput));
PipedInputStream pin = new PipedInputStream(1024 * 1024 * 1024);
PipedOutputStream pout = new PipedOutputStream(pin);
client.setProcessCallback(new JackProcessCallback() {
public boolean process(JackClient client, int nframes) {
FloatBuffer inData = in.getFloatBuffer();
byte[] buffer = new byte[inData.capacity() * 2];
for (int i = 0; i < inData.capacity(); i++) {
int sample = Math.round(inData.get(i) * 32767);
buffer[i * 2] = (byte) sample;
buffer[i * 2 + 1] = (byte) (sample >> 8);
}
try {
pout.write(buffer, 0, buffer.length);
} catch (IOException e) {
e.printStackTrace();
}
return true;
}
});
client.activate();
client.transportStart();
Thread.sleep(10000);
client.transportStop();
client.close();
AudioInputStream audio = AudioSystem.getAudioInputStream(new BufferedInputStream(pin, 1024 * 1024 * 1024));
AudioSystem.write(audio, Type.WAVE, new File("test.wav"));
It uses the JnaJack library, but it doesn't really matter where the data comes from. The conversion to bytes is fine by the way: writing that data directly to a SourceDataLine will work correctly. But I need the data as
AudioInputStream.
AudioSystem.getAudioInputStream expects a stream which conforms to a supported AudioFileFormat, which means it must conform to a known type. From the documentation:
The stream must point to valid audio file data.
And also from that documentation:
The implementation of this method may require multiple parsers to examine the stream to determine whether they support it. These parsers must be able to mark the stream, read enough data to determine whether they support the stream, and reset the stream's read pointer to its original position. If the input stream does not support these operation, this method may fail with an IOException.
You can create your own AudioInputStream using the three-argument constructor. If the length is not known, it can specified as AudioSystem.NOT_SPECIFIED. Frustratingly, neither the constructor documentation nor the class documentation mentions this, but the other constructor’s documentation does.

Java SFXR Port - Trouble writing byte[] to WAV file

I'm using a Java port of the sound effect generator SFXR, which involves lots of arcane music code that I don't understand, being something of a novice when it comes to anything to do with audio. What I do know is that the code can reliably generate and play sounds within Java, using a SourceDataLine object.
The data that the SDL object uses is stored in a byte[]. However, simply writing this out to a file doesn't work (presumably because of the lack of a WAV header, or so I thought).
However, I downloaded this WAV read/write class: http://computermusicblog.com/blog/2008/08/29/reading-and-writing-wav-files-in-java/ which adds in header information when it writes a WAV file. Giving it the byte[] data from SFXR still produces files that can't be played by any music player I have.
I figure I must be missing something. Here's the relevant code when it plays the sound data:
public void play(int millis) throws Exception {
AudioFormat stereoFormat = getStereoAudioFormat();
SourceDataLine stereoSdl = AudioSystem.getSourceDataLine(stereoFormat);
if (!stereoSdl.isOpen()) {
try {
stereoSdl.open();
} catch (LineUnavailableException e) {
e.printStackTrace();
}
}
if (!stereoSdl.isRunning()) {
stereoSdl.start();
}
double seconds = millis / 1000.0;
int bufferSize = (int) (4 * 41000 * seconds);
byte[] target = new byte[bufferSize];
writeBytes(target);
stereoSdl.write(target, 0, target.length);
}
That's from the SFXR port. Here's the save() file from the WavIO class (there's a lot of other code in that class of course, I figured this might be worth posting in case someone wants to see exactly how the buffer data is being handled:
public boolean save()
{
try
{
DataOutputStream outFile = new DataOutputStream(new FileOutputStream(myPath));
// write the wav file per the wav file format
outFile.writeBytes("RIFF"); // 00 - RIFF
outFile.write(intToByteArray((int)myChunkSize), 0, 4); // 04 - how big is the rest of this file?
outFile.writeBytes("WAVE"); // 08 - WAVE
outFile.writeBytes("fmt "); // 12 - fmt
outFile.write(intToByteArray((int)mySubChunk1Size), 0, 4); // 16 - size of this chunk
outFile.write(shortToByteArray((short)myFormat), 0, 2); // 20 - what is the audio format? 1 for PCM = Pulse Code Modulation
outFile.write(shortToByteArray((short)myChannels), 0, 2); // 22 - mono or stereo? 1 or 2? (or 5 or ???)
outFile.write(intToByteArray((int)mySampleRate), 0, 4); // 24 - samples per second (numbers per second)
outFile.write(intToByteArray((int)myByteRate), 0, 4); // 28 - bytes per second
outFile.write(shortToByteArray((short)myBlockAlign), 0, 2); // 32 - # of bytes in one sample, for all channels
outFile.write(shortToByteArray((short)myBitsPerSample), 0, 2); // 34 - how many bits in a sample(number)? usually 16 or 24
outFile.writeBytes("data"); // 36 - data
outFile.write(intToByteArray((int)myDataSize), 0, 4); // 40 - how big is this data chunk
outFile.write(myData); // 44 - the actual data itself - just a long string of numbers
}
catch(Exception e)
{
System.out.println(e.getMessage());
return false;
}
return true;
}
All I know is, I've got a bunch of data, and I want it to end up in a playable audio file of some kind (at this point I'd take ANY format!). What's the best way for me to get this byte buffer into a playable file? Or is this byte[] not what I think it is?
I do not get much chance to play with the sound capabilities of Java so I'm using your question as a learning exercise (I hope you don't mind). The article that you referenced about Reading and Writing WAV Files in Java is very old in relation to Java history (1998). Also something about constructing the WAV header by hand didn't sit quite right with me (it seemed a little error prone). As Java is quite a mature language now I would expect library support for this kind of thing.
I was able to construct a WAV file from a byte array by hunting around the internet for sample code snippets. This is the code that I came up with (I expect it is sub-optimal but it seems to work):
// Generate bang noise data
// Sourced from http://www.rgagnon.com/javadetails/java-0632.html
public static byte[] bang() {
byte[] buf = new byte[8050];
Random r = new Random();
boolean silence = true;
for (int i = 0; i < 8000; i++) {
while (r.nextInt() % 10 != 0) {
buf[i] =
silence ? 0
: (byte) Math.abs(r.nextInt()
% (int) (1. + 63. * (1. + Math.cos(((double) i)
* Math.PI / 8000.))));
i++;
}
silence = !silence;
}
return buf;
}
private static void save(byte[] data, String filename) throws IOException, LineUnavailableException, UnsupportedAudioFileException {
InputStream byteArray = new ByteArrayInputStream(data);
AudioInputStream ais = new AudioInputStream(byteArray, getAudioFormat(), (long) data.length);
AudioSystem.write(ais, AudioFileFormat.Type.WAVE, new File(filename));
}
private static AudioFormat getAudioFormat() {
return new AudioFormat(
8000f, // sampleRate
8, // sampleSizeInBits
1, // channels
true, // signed
false); // bigEndian
}
public static void main(String[] args) throws Exception {
byte[] data = bang();
save(data, "test.wav");
}
I hope it helps.

Android AudioRecord class - process live mic audio quickly, set up callback function

I want to record audio from the mic and access it for possible playback in near real-time. I am unsure of how to use the Android AudioRecord class to record some mic audio and quickly access it.
For the AudioRecord class, the official site says 'the app polls the AudioRecord object in time', and 'the size of the buffer being filled determines the time-length of the recording before over-running unread data'. Later it's suggested that a larger buffer should be used when polling less frequently. They never actually show an example in code.
One example I've seen in a book uses the AudioRecord class to continuously read a buffer freshly populated with live mic audio, and then the app writes this data to an SD file. The pseudo-code looks something like -
set up AudioRecord object with buffer size and recording format info
set up a file and an output stream
myAudioRecord.startRecording();
while(isRecording)
{
// myBuffer is being filled with fresh audio
read audio data into myBuffer
send contents of myBuffer to SD file
}
myAudioRecord.stop();
How this code synchronizes its reading with the rate of recording is unclear - is the boolean "isRecording" sequenced on and off properly elsewhere? It seems this code could either read too frequently or too infrequently, depending on how long the reading and writing takes.
The site doc also says the AudioRecord class has a nested class named OnRecordPositionUpdateListener which is defined as an interface. The information suggests that somehow, you specify the period you want for being notified of the progress of the recording, and the name of your event handler, and a call is automatically made to your event handler at the specified frequency. I think the structure, in pseudo-code would be something like -
set target of period update message = myListener
set period to be about every 250 ms
other code
myListener()
{
if(record button was recently tapped)
handle message that another 250 ms of fresh audio is available
ie, read it and send it somewhere
)
I need to find some specific code which allows me to capture and process mic audio with a delay of less than about 500 ms. Android offers another class called MediaRecorder, but it doesn't support streaming, and I may want to stream live mic audio over a Wi-Fi network in near real-time. Where can I find some specific examples?
After experimenting lots with the notifications and a bunch of other techniques I settled on this code:
private class AudioIn extends Thread {
private boolean stopped = false;
private AudioIn() {
start();
}
#Override
public void run() {
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
AudioRecord recorder = null;
short[][] buffers = new short[256][160];
int ix = 0;
try { // ... initialise
int N = AudioRecord.getMinBufferSize(8000,AudioFormat.CHANNEL_IN_MONO,AudioFormat.ENCODING_PCM_16BIT);
recorder = new AudioRecord(AudioSource.MIC,
8000,
AudioFormat.CHANNEL_IN_MONO,
AudioFormat.ENCODING_PCM_16BIT,
N*10);
recorder.startRecording();
// ... loop
while(!stopped) {
short[] buffer = buffers[ix++ % buffers.length];
N = recorder.read(buffer,0,buffer.length);
//process is what you will do with the data...not defined here
process(buffer);
}
} catch(Throwable x) {
Log.w(TAG,"Error reading voice audio",x);
} finally {
close();
}
}
private void close() {
stopped = true;
}
}
So far it's working pretty robustly on the half a dozen Android phones I've tried it on.
I wonder if you could combine these answers in the following way...
Use setPositionNotificationPeriod(160) before the while loop. This should cause the callback to be called every time 160 frames are read. Instead of calling process(buffer) inside of the thread that's doing the read loop, call process(buffer) from the callback. Use a variable to keep track of the last read buffer so you process the right one. As it is now, you block on the read, then you're not reading while you're processing. I think it might be better to separate those two.
Here is the code you need to use the OnRecordPositionUpdateListener and Notification Period.
I noticed that in practice it does not send the notification consistently at the same exact time, I want, but it is close enough.
About detectAfterEvery:
The size of detectEvery needs to be large enough to hold just the amount of data you want. So for this example, we have a sample rate of 44100 Hz, that means we want 44100 samples per second. By setting the setPositionNotificationPeriod to be 44100, the code tells Android to callback after it has recorded 44100 samples, which is about every 1 second.
The complete code is here:
final int sampleRate = 44100;
int bufferSize =
AudioRecord.getMinBufferSize(sampleRate,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT);
//aim for 1 second
int detectAfterEvery = (int)((float)sampleRate * 1.0f);
if (detectAfterEvery > bufferSize)
{
Log.w(TAG, "Increasing buffer to hold enough samples " + detectAfterEvery + " was: " + bufferSize);
bufferSize = detectAfterEvery;
}
recorder =
new AudioRecord(AudioSource.MIC, sampleRate,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT, bufferSize);
recorder.setPositionNotificationPeriod(detectAfterEvery);
final short[] audioData = new short[bufferSize];
final int finalBufferSize = bufferSize;
OnRecordPositionUpdateListener positionUpdater = new OnRecordPositionUpdateListener()
{
#Override
public void onPeriodicNotification(AudioRecord recorder)
{
Date d = new Date();
//it should be every 1 second, but it is actually, "about every 1 second"
//like 1073, 919, 1001, 1185, 1204 milliseconds of time.
Log.d(TAG, "periodic notification " + d.toLocaleString() + " mili " + d.getTime());
recorder.read(audioData, 0, finalBufferSize);
//do something amazing with audio data
}
#Override
public void onMarkerReached(AudioRecord recorder)
{
Log.d(TAG, "marker reached");
}
};
recorder.setRecordPositionUpdateListener(positionUpdater);
Log.d(TAG, "start recording, bufferSize: " + bufferSize);
recorder.startRecording();
//remember to still have a read loop otherwise the listener won't trigger
while (continueRecording)
{
recorder.read(audioData, 0, bufferSize);
}
private int freq =8000;
private AudioRecord audioRecord = null;
private Thread Rthread = null;
private AudioManager audioManager=null;
private AudioTrack audioTrack=null;
byte[] buffer = new byte[freq];
//call this method at start button
protected void Start()
{
loopback();
}
protected void loopback() {
android.os.Process.setThreadPriority(android.os.Process.THREAD_PRIORITY_URGENT_AUDIO);
final int bufferSize = AudioRecord.getMinBufferSize(freq,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
AudioFormat.ENCODING_PCM_16BIT);
audioRecord = new AudioRecord(MediaRecorder.AudioSource.MIC, freq,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
MediaRecorder.AudioEncoder.AMR_NB, bufferSize);
audioTrack = new AudioTrack(AudioManager.ROUTE_HEADSET, freq,
AudioFormat.CHANNEL_CONFIGURATION_MONO,
MediaRecorder.AudioEncoder.AMR_NB, bufferSize,
AudioTrack.MODE_STREAM);
audioTrack.setPlaybackRate(freq);
final byte[] buffer = new byte[bufferSize];
audioRecord.startRecording();
Log.i(LOG_TAG, "Audio Recording started");
audioTrack.play();
Log.i(LOG_TAG, "Audio Playing started");
Rthread = new Thread(new Runnable() {
public void run() {
while (true) {
try {
audioRecord.read(buffer, 0, bufferSize);
audioTrack.write(buffer, 0, buffer.length);
} catch (Throwable t) {
Log.e("Error", "Read write failed");
t.printStackTrace();
}
}
}
});
Rthread.start();
}
It plays the recorded audio less than 100 ms delay.

Specifying start/stop time in millisecs when playing MP3 using JLayer

I need to play a part of an MP3 file in my java code. I wish to do this via a function which accepts the start and stop time in millisecs.
JLayer contains a class called AdvancedPlayer which has a method that accepts the start and stop position in frames:
/**
* Plays a range of MPEG audio frames
* #param start The first frame to play
* #param end The last frame to play
* #return true if the last frame was played, or false if there are more frames.
*/
public boolean play(final int start, final int end) throws JavaLayerException
{
boolean ret = true;
int offset = start;
while (offset-- > 0 && ret) ret = skipFrame();
return play(end - start);
}
According to this, a frame lasts 26millisecs. However I need a finer degree of control than this, i.e. I may wish to play from 40millisecs to 50millisecs.
How can I do this? Do I need to convert the MP3 to .wav first?
The solution I used in the end was to first write the code to play a part of a wave file (i.e. from xxx ms to xxx ms) as I also need support for this file format. Here's the code for that:
File soundFile = new File(this.audioFilePath);
AudioInputStream originalAudioInputStream = AudioSystem.getAudioInputStream(soundFile);
AudioFormat audioFormat = originalAudioInputStream.getFormat();
float startInBytes = (startTimeinMs / 1000 * audioFormat.getSampleRate() * audioFormat.getFrameSize());
float lengthInFrames = ((endTimeinMs - startTimeinMs) / 1000 * audioFormat.getSampleRate());
originalAudioInputStream.skip((long) startInBytes);
AudioInputStream partAudioInputStream = new AudioInputStream(originalAudioInputStream,
originalAudioInputStream.getFormat(), (long) lengthInFrames);
// code to actually play the audio input stream here
Once this was working I wrote this code to convert an MP3 to a temporary wave file (which I can then use with the above code) - this is using JLayer and MP3SPI. I did try simply performing the above directly on the converted audio stream without first writing out to a file but couldn't get it to work. I'm only using small MP3 files that convert/write out instantly so I'm happy with this solution.
File soundFile = new File(this.inputFilePath);
AudioInputStream mp3InputStream = AudioSystem.getAudioInputStream(soundFile);
AudioFormat baseFormat = mp3InputStream.getFormat();
AudioFormat decodedFormat = new AudioFormat(AudioFormat.Encoding.PCM_SIGNED, baseFormat.getSampleRate(), 16, baseFormat.getChannels(), baseFormat.getChannels() * 2, baseFormat.getSampleRate(), false);
AudioInputStream convertedAudioInputStream = AudioSystem.getAudioInputStream(decodedFormat, mp3InputStream);
File outputFile = new File(this.outputFilePath);
AudioSystem.write(convertedAudioInputStream, AudioFileFormat.Type.WAVE, outputFile);
If 26 milliseconds is the finest resolution you can achieve in an MP3 file, then you're out of luck. Converting it to WAV might work, but the source data (i.e. the MP3) stil has that basic resolution limit.
Out of curiosity, why do you want to play 10 milliseconds worth of audio?

Categories