I'm using this instructions for get a signal audio of a microphone:
while(!stopCapture){
int cnt = targetDataLine.read(tempBuffer, 0, tempBuffer.length);
if(cnt > 0){
byteArrayOutputStream.write(tempBuffer, 0, cnt);
}
}
byteArrayOutputStream.close();
byte audio[] = byteArrayOutputStream.toByteArray();
InputStream input = new ByteArrayInputStream(audio);
AudioInputStream ais = new AudioInputStream(input, audioFormat, audio.length / audioFormat.getFrameSize());
But these instructions wait the entire signal before elaborate it.
I need elaborate it in real time...is possible?
How can i do it?
thanks
Related
I'm trying to capture the sound of the PC. I have managed to capture the sound that enters the microphone through TargetDataLine, but I cannot find the way to capture the sound that comes out of the speakers.
I've been watching the mixer but I have not managed to capture the sound. I would like to know if someone has done it and if you can give me some clue as to where to start.
Although, your question is not really according to the "rules", here is a code snippet:
private byte[] record() throws LineUnavailableException {
AudioFormat format = AudioUtil.getAudioFormat(audioConf);
DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
// Checks if system supports the data line
if (!AudioSystem.isLineSupported(info)) {
LOGGER.error("Line not supported");
System.exit(0);
}
microphone = (TargetDataLine) AudioSystem.getLine(info);
microphone.open(format);
microphone.start();
LOGGER.info("Listening, tap enter to stop ...");
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
int numBytesRead;
byte[] data = new byte[microphone.getBufferSize() / 5];
// Begin audio capture.
microphone.start();
// Here, stopped is a global boolean set by another thread.
while (!stopped) {
// Read the next chunk of data from the TargetDataLine.
numBytesRead = microphone.read(data, 0, data.length);
// Save this chunk of data.
byteArrayOutputStream.write(data, 0, numBytesRead);
}
return byteArrayOutputStream.toByteArray();
}
Get more info from here:
https://www.programcreek.com/java-api-examples/?class=javax.sound.sampled.TargetDataLine&method=read
I would like to extract byte array from a given mp3 file in order to apply fast fourier transform on the latter. The performed FFT will give me some features for my pet-project musical -- recommendation system.
I have written the following code to extract the bytes from a given mp3 file:
public class TrackSample {
private static byte[] readBytesInPredefinedFormat(TargetDataLine format, InputStream inStream) throws IOException {
ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream();
byte[] buffer = new byte[8192];
int bytesRead;
while ((bytesRead = inStream.read(buffer)) > 0) {
int count = format.read(buffer, 0, buffer.length);
if (count > 0) {
byteArrayOutputStream.write(buffer, 0, count);
}
byteArrayOutputStream.write(buffer, 0, bytesRead);
}
byte[] bytes = byteArrayOutputStream.toByteArray();
byteArrayOutputStream.close();
inStream.close();
return bytes;
}
public static byte[] getTrackBytes(String pathToTrackSample) throws IOException, LineUnavailableException {
FileInputStream fileInputStream = new FileInputStream(pathToTrackSample);
final AudioFormat format = CurrentAudioFormat.getAudioFormat(); //Fill AudioFormat with the wanted settings
DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
TargetDataLine line = (TargetDataLine) AudioSystem.getLine(info);
line.open(format);
line.start();
return readBytesInPredefinedFormat(line, fileInputStream);
}
}
And the specified audio format is
public class CurrentAudioFormat {
public static AudioFormat getAudioFormat(){
float sampleRate = 44100;
int sampleSizeInBits = 8;
int channels = 1; //mono
boolean signed = true;
boolean bigEndian = true;
return new AudioFormat(sampleRate, sampleSizeInBits, channels, signed, bigEndian);
}
}
I tried to test this code on the following mp3 file:
File type ID: MPG3
Num Tracks: 1
----
Data format: 2 ch, 44100 Hz, '.mp3' (0x00000000) 0 bits/channel, 0 bytes/packet, 1152 frames/packet, 0 bytes/frame
no channel layout.
estimated duration: 104.176325 sec
audio bytes: 4167053
audio packets: 3988
bit rate: 320000 bits per second
packet size upper bound: 1052
maximum packet size: 1045
audio data file offset: 3169
optimized
audio 4591692 valid frames + 576 priming + 1908 remainder = 4594176
The system characteristics are:
processor: Intel core i5, 1.4 GHz;
RAM: DDR3, 4Gb
OS: Mac OS X El Captain
It took roughly 5 minutes to extract the byte array from this mp3 file.
What are the possible bottlenecks and how can I improve them?
To read the bytes you just need
while ((bytesRead = inStream.read(buffer)) > -1) {
byteArrayOutputStream.write(buffer, 0, bytesRead);
}
I dont know why you are reading twice.
To make sure that what you got is right try to resave it to a new audio file.
--
The standard way to read the audio file is
AudioInputStream audioInputStream=null;
try {
audioInputStream=AudioSystem.getAudioInputStream(new File(file));
}
catch(UnsupportedAudioFileException auf) { auf.printStackTrace(); }
then you pass this audioInputStream to your reading method.
I've created a byte array WebSocket that receives audio chunks in real time from the client's mic (navigator.getUserMedia). I'm already recording this stream to a WAV file in the server, after some time that the WebSocket stops to receive new byte arrays. The following code represents the current situation.
WebSocket
#OnMessage
public void message(byte[] b) throws IOException{
if(byteOutputStream == null) {
byteOutputStream = new ByteArrayOutputStream();
byteOutputStream.write(b);
} else {
byteOutputStream.write(b);
}
}
Thread that stores the WAV file
public void store(){
byte b[] = byteOutputStream.toByteArray();
try {
AudioFormat audioFormat = new AudioFormat(44100, 16, 1, true, true);
ByteArrayInputStream byteStream = new ByteArrayInputStream(b);
AudioInputStream audioStream = new AudioInputStream(byteStream, audioFormat, b.length);
DateTime date = new DateTime();
File file = new File("/tmp/"+date.getMillis()+ ".wav");
AudioSystem.write(audioStream, AudioFileFormat.Type.WAVE, file);
audioStream.close();
} catch (IOException e) {
e.printStackTrace();
}
}
But instead of record a WAV file, my goal with this WebSocket is to process audio in real time using YIN pitch detection algorithm implemented on TarsosDSP library. In other words, this is basically execute the PitchDetectorExample, but using the data from the WebSocket instead of the Default Audio Device (OS mic). The following code represents how PitchDetectorExample is currently initializing live audio processing using the mic line provided by the OS.
private void setNewMixer(Mixer mixer) throws LineUnavailableException, UnsupportedAudioFileException {
if(dispatcher!= null){
dispatcher.stop();
}
currentMixer = mixer;
float sampleRate = 44100;
int bufferSize = 1024;
int overlap = 0;
final AudioFormat format = new AudioFormat(sampleRate, 16, 1, true, true);
final DataLine.Info dataLineInfo = new DataLine.Info(TargetDataLine.class, format);
TargetDataLine line;
line = (TargetDataLine) mixer.getLine(dataLineInfo);
final int numberOfSamples = bufferSize;
line.open(format, numberOfSamples);
line.start();
final AudioInputStream stream = new AudioInputStream(line);
JVMAudioInputStream audioStream = new JVMAudioInputStream(stream);
// create a new dispatcher
dispatcher = new AudioDispatcher(audioStream, bufferSize, overlap);
// add a processor
dispatcher.addAudioProcessor(new PitchProcessor(algo, sampleRate, bufferSize, this));
new Thread(dispatcher,"Audio dispatching").start();
}
There is a way to deal with WebSocket data as a TargetDataLine, so it will be possible to hook it up with AudioDispatcher and PitchProcessor? Somehow, i need to send the byte arrays received from the WebSocket to the audio processing Thread.
Another ideas on how reach this objective are welcome. Thanks!
I'm not sure you need an audioDispatcher. If you know how the bytes are encoded (PCM, 16bits le mono?) then you can convert them to floating points real-time and feed them to the pitchdetector algorithm, in your websocket you can do something like this (and forget about the inputstreams and audiodispatcher):
int index;
byte[] buffer = new byte[2048];
float[] floatBuffer = new float[1024];
FastYin detector = new FastYin(44100,1024);
public void message(byte[] b){
for(int i = 0 ; i < b.length; i++){
buffer[index] = b[i];
index++
if(index==2048){
AudioFloatConverter converter = AudioFloatConverter.getConverter(new Format(16bits, little endian, mono,...));
//converts the byte buffer to float
converter.toFloatArray(buffer,floatBuffer);
float pitch = detector.getPitch(floatBuffer);
//here you have your pitch info that you can use
index = 0;
}
}
You do need to watch the number of bytes that have passed: since two bytes represent one float (if 16bits pcm encoding is used) you need to start on even bytes. The endianness and samplerate are also important.
Regards
Joren
I want to use java(httpurlconnection class) to download a pdf file,but have some thing puzzle,the core code is:
BufferedInputStream inputStream = new BufferedInputStream(httpConn.getInputStream());
OutputStream output = new FileOutputStream("F:/httpclient-tutorial.pdf");
byte[] infoBytes = new byte[BUFFER];
while(inputStream.read(infoBytes) != -1){
output.write(infoBytes, 0, BUFFER);
}
Question:when the BUFFER is 1024,get the pdf file is larger than real one;but the BUFFER is 8,then it's ok. I don't know why it is? Has anyone know about that?
You're unconditionally writing out the whole of the buffer on each iteration. You should only write out as many bytes as you've just read:
int bytesRead;
while((bytesRead = inputStream.read(infoBytes)) != -1){
output.write(infoBytes, 0, bytesRead);
}
I'm writing an application that records the screen and audio. While the screen recording works perfectly, I'm having difficulty in getting the raw audio using the JDK libraries. Here's the code:
try {
// Now, we're going to loop
long startTime = System.nanoTime();
System.out.println("Encoding Image.....");
while (!Thread.currentThread().isInterrupted()) {
// take the screen shot
BufferedImage screen = robot.createScreenCapture(screenBounds);
// convert to the right image type
BufferedImage bgrScreen = convertToType(screen,
BufferedImage.TYPE_3BYTE_BGR);
// encode the image
writer.encodeVideo(0, bgrScreen, System.nanoTime()
- startTime, TimeUnit.NANOSECONDS);
/* Need to get audio here and then encode using xuggler. Something like
WaveData wd = new WaveData();
TargetDataLine line;
AudioInputStream aus = new AudioInputStream(line);
short[] samples = getSourceSamples();
writer.encodeAudio(0, samples); */
if (timeCreation < 10) {
timeCreation = getGMTTime();
}
// sleep for framerate milliseconds
try {
Thread.sleep((long) (1000 / FRAME_RATE.getDouble()));
} catch (Exception ex) {
System.err.println("stopping....");
break;
}
}
// Finally we tell the writer to close and write the trailer if
// needed
} finally {
writer.close();
}
This page has some pseudo code like
while(haveMoreAudio())
{
short[] samples = getSourceSamples();
writer.encodeAudio(0, samples);
}
but what exactly should I do for getSourceSamples()?
Also, a bonus question - is it possible to choose from multiple microphones in this approach?
See also:
Xuggler encoding and muxing
Try this:
// Pick a format. Need 16 bits, the rest can be set to anything
// It is better to enumerate the formats that the system supports, because getLine() can error out with any particular format
AudioFormat audioFormat = new AudioFormat(44100.0F, 16, 2, true, false);
// Get default TargetDataLine with that format
DataLine.Info dataLineInfo = new DataLine.Info( TargetDataLine.class, audioFormat );
TargetDataLine line = (TargetDataLine) AudioSystem.getLine(dataLineInfo);
// Open and start capturing audio
line.open(audioFormat, line.getBufferSize());
line.start();
while (true) {
// read as raw bytes
byte[] audioBytes = new byte[ line.getBufferSize() / 2 ]; // best size?
int numBytesRead = 0;
numBytesRead = line.read(audioBytes, 0, audioBytes.length);
// convert to signed shorts representing samples
int numSamplesRead = numBytesRead / 2;
short[] audioSamples = new short[ numSamplesRead ];
if (format.isBigEndian()) {
for (int i = 0; i < numSamplesRead; i++) {
audioSamples[i] = (short)((audioBytes[2*i] << 8) | audioBytes[2*i + 1]);
}
}
else {
for (int i = 0; i < numSamplesRead; i++) {
audioSamples[i] = (short)((audioBytes[2*i + 1] << 8) | audioBytes[2*i]);
}
}
// use audioSamples in Xuggler etc
}
To pick a microphone, you'd probably have to do this:
Mixer.Info[] mixerInfo = AudioSystem.getMixerInfo();
// Look through and select a mixer here, different mixers should be different inputs
int selectedMixerIndex = 0;
Mixer mixer = AudioSystem.getMixer(mixerInfo[ selectedMixerIndex ]);
TargetDataLine line = (TargetDataLine) mixer.getLine(dataLineInfo);
I think it's possible that multiple microphones will show up in one mixer as different source data lines. In that case you'd have to open them and call dataLine.getControl(FloatControl.Type.MASTER_GAIN).setValue( volume ); to turn them on and off.
See:
WaveData.java
Sound wave from TargetDataLine
How to set volume of a SourceDataLine in Java