Before, I asked question about Get frequency wav audio using FFT and Complex class ,
There, I need to calculate FFT value from AudioRecord input --> from microphone , I somehow managed to get the FFT value...
Now I need to calculate FFT value from *.wav audio file that I saved before,
I saved the audio in 'raw' folder inside 'res' folder from my project
I still using the same FFT Class: http://www.cs.princeton.edu/introcs/97data/FFT.java
The complex class to go with it: http://introcs.cs.princeton.edu/java/97data/Complex.java.html
I use this method to read audio file from my raw foldern, then I call method calculateFFT to go with it
private static final int RECORDER_BPP = 16;
private static final int RECORDER_SAMPLERATE = 44100;
private static final int RECORDER_CHANNELS = AudioFormat.CHANNEL_IN_STEREO;
private static final int RECORDER_AUDIO_ENCODING = AudioFormat.ENCODING_PCM_16BIT;
private void asli(){
int counter = 0;
int data;
InputStream inputStream = getResources().openRawResource(R.raw.b1);
DataInputStream dataInputStream = new DataInputStream(inputStream);
List<Integer> content = new ArrayList<Integer>();
try {
while ((data = dataInputStream.read()) != -1) {
content.add(data);
counter++; }
} catch (IOException e) {
e.printStackTrace();}
int[] b = new int[content.size()];
int cont = 0;
byte[] audio = convertArray(b);
}
Method to convert to byte
public byte[] convertArray(int[] array) {
int minBufferSize = AudioTrack.getMinBufferSize(RECORDER_SAMPLERATE,RECORDER_CHANNELS,RECORDER_AUDIO_ENCODING);
AudioTrack audioTrack = new AudioTrack(AudioManager.STREAM_MUSIC,RECORDER_SAMPLERATE,RECORDER_CHANNELS,RECORDER_AUDIO_ENCODING,minBufferSize, AudioTrack.MODE_STREAM);
byte[] newarray = new byte[array.length];
for (int i = 0; i < array.length; i++) {
newarray[i] = (byte) ((array[i]) & 0xFF); }
absNormalizedSignal = calculateFFT(newarray);
return newarray;
}
And this is the CalculateFFT method
public double[] calculateFFT(byte[] signal)
{
final int mNumberOfFFTPoints =1024;
double mMaxFFTSample;
double temp;
Complex[] y;
Complex[] complexSignal = new Complex[mNumberOfFFTPoints];
double[] absSignal = new double[mNumberOfFFTPoints/2];
for(int i = 0; i < mNumberOfFFTPoints; i++){
temp = (double)((signal[2*i] & 0xFF) | (signal[2*i+1] << 8)) / 32768.0F;
complexSignal[i] = new Complex(temp,0.0);
}
y = FFT.fft(complexSignal);
mMaxFFTSample = 0.0;
mPeakPos = 0;
for(int i = 0; i < (mNumberOfFFTPoints/2); i++)
{
absSignal[i] = Math.sqrt(Math.pow(y[i].re(), 2) + Math.pow(y[i].im(), 2));
if(absSignal[i] > mMaxFFTSample)
{
mMaxFFTSample = absSignal[i];
mPeakPos = i;
}
}
return absSignal;
}
I used this CalculateFFT method too to process audio from AudioRecorder --> that one with microphone input before... I managed to get value from the AudioRecorder, but I failed to get value from my audio file... I'm not planning to play the audio.. I just need to process it with FFT.
Is there any wrong with my code ?? :o Seems like I fail at getting value from method Asli(); But I dont know which part is wrong..
Any help would be appreciated... :)
Thanks
I spent a better part of the morning coding a solution for this using bits and pieces of FFT java snippets I was finding... but then I stumbled upon this amazingly wondeful google code project that has a bunch of util classes for doing signal processing tasks on WAV and MP3 files alike.
https://github.com/Uriopass/audio-analysis
Formerly SVN export was on Google code here: https://storage.googleapis.com/google-code-archive-source/v2/code.google.com/audio-analysis/source-archive.zip
It now becomes INSANELY easy:
WaveDecoder decoder = new WaveDecoder(new FileInputStream(wavFile));
FFT fft = new FFT(1024, wavFileObj.getSampleRate());
Now you can use the fft object to do various calculations. They have a bunch of great examples, such as generating a List containing the spectral flux:
float[] samples = new float[1024];
float[] spectrum = new float[1024 / 2 + 1];
float[] lastSpectrum = new float[1024 / 2 + 1];
List<Float> spectralFlux = new ArrayList<Float>();
while (decoder.readSamples(samples) > 0) {
fft.forward(samples);
System.arraycopy(spectrum, 0, lastSpectrum, 0, spectrum.length);
System.arraycopy(fft.getSpectrum(), 0, spectrum, 0, spectrum.length);
float flux = 0;
for (int i = 0; i < spectrum.length; i++)
flux += (spectrum[i] - lastSpectrum[i]);
spectralFlux.add(flux);
}
My company needed a way for me to analyze some audio to see if some expected hold music had happened. So first I generated a WAV file for an example that did have the hold music. Then I captured some audio of one of thee examples that did not have the hold music. Now all that is left is to average up the spectral flux of the wav and I'm set.
Note: I could not have simply taken amplitudes... but the fourier transformation has frequencies that I could correctly use to make my comparison.
I love math.
Related
I have been working with a java program (developed by other people) for text-to-speech synthesis. The synthesis is done by concatenation of "di-phones". In the oroginal version, there was no signal processing. The diphones were just collected and concatenated together to produce the output. In order to improve the output, I tried to perform "phase matching" of the concatenating speech signals. The modification I've made is summarized here:
Audio data is collected from the AudioInputStream into a byte array.
Since the audio data is 16 bit, I converted the byte array to a short
array.
The "signal processing" is done on the short array.
To output the audio data, short array is again converted to byte array.
Here's the part of the code that I've changed in the existing program:
Audio Input
This segment is called for every diphone.
Original Version
audioInputStream = AudioSystem.getAudioInputStream(sound);
while ((cnt = audioInputStream.read(byteBuffer, 0, byteBuffer.length)) != -1) {
if (cnt > 0) {
byteArrayPlayStream.write(byteBuffer, 0, cnt);
}
}
My Version
// public varialbe declarations
byte byteSoundFile[]; // byteSoundFile will contain a whole word or the diphones of a whole word
short shortSoundFile[] = new short[5000000]; // sound contents are taken in a short[] array for signal processing
short shortBuffer[];
int pos = 0;
int previousPM = 0;
boolean isWord = false;
public static HashMap<String, Integer> peakMap1 = new HashMap<String, Integer>();
public static HashMap<String, Integer> peakMap2 = new HashMap<String, Integer>();
// code for receiving and processing audio data
if(pos == 0) {
// a new word is going to be processed.
// so reset the shortSoundFile array
Arrays.fill(shortSoundFile, (short)0);
}
audioInputStream = AudioSystem.getAudioInputStream(sound);
while ((cnt = audioInputStream.read(byteBuffer, 0, byteBuffer.length)) != -1) {
if (cnt > 0) {
byteArrayPlayStream.write(byteBuffer, 0, cnt);
}
}
byteSoundFile = byteArrayPlayStream.toByteArray();
int nSamples = byteSoundFile.length;
byteArrayPlayStream.reset();
if(nSamples > 80000) { // it is a word
pos = nSamples;
isWord = true;
}
else { // it is a diphone
// audio data is converted from byte to short, so nSamples is halved
nSamples /= 2;
// transfer byteSoundFile contents to shortBuffer using byte-to-short conversion
shortBuffer = new short[nSamples];
for(int i=0; i<nSamples; i++) {
shortBuffer[i] = (short)((short)(byteSoundFile[i<<1]) << 8 | (short)byteSoundFile[(i<<1)+1]);
}
/************************************/
/**** phase-matching starts here ****/
/************************************/
int pm1 = 0;
int pm2 = 0;
String soundStr = sound.toString();
if(soundStr.contains("\\") && soundStr.contains(".")) {
soundStr = soundStr.substring(soundStr.indexOf("\\")+1, soundStr.indexOf("."));
}
if(peakMap1.containsKey(soundStr)) {
// perform overlap and add
System.out.println("we are here");
pm1 = peakMap1.get(soundStr);
pm2 = peakMap2.get(soundStr);
/*
Idea:
If pm1 is located after more than one third of the samples,
then threre will be too much overlapping.
If pm2 is located before the two third of the samples,
then where will also be extra overlapping for the next diphone.
In both of the cases, we will not perform the peak-matching operation.
*/
int idx1 = (previousPM == 0) ? pos : previousPM - pm1;
if((idx1 < 0) || (pm1 > (nSamples/3))) {
idx1 = pos;
}
int idx2 = idx1 + nSamples - 1;
for(int i=idx1, j=0; i<=idx2; i++, j++) {
if(i < pos) {
shortSoundFile[i] = (short) ((shortSoundFile[i] >> 1) + (shortBuffer[j] >> 1));
}
else {
shortSoundFile[i] = shortBuffer[j];
}
}
previousPM = (pm2 < (nSamples/3)*2) ? 0 : idx1 + pm2;
pos = idx2 + 1;
}
else {
// no peak found. simply concatenate the audio data
for(int i=0; i<nSamples; i++) {
shortSoundFile[pos++] = shortBuffer[i];
}
previousPM = 0;
}
Audio Output
After collecting all the diphones of a word, this segment is called to play the audio output.
Original Version
byte audioData[] = byteArrayPlayStream.toByteArray();
... code for writing audioData to output steam
My Version
byte audioData[];
if(isWord) {
audioData = Arrays.copyOf(byteSoundFile, pos);
isWord = false;
}
else {
audioData = new byte[pos*2];
for(int i=0; i<pos; i++) {
audioData[(i<<1)] = (byte) (shortSoundFile[i] >>> 8);
audioData[(i<<1)+1] = (byte) (shortSoundFile[i]);
}
}
pos = 0;
... code for writing audioData to output steam
But after the modification has done, the output has become worse. There is a lot of noise in the output.
Here is a sample audio with modification: modified output
Here is a sample audio from the original version: original output
Now I'd appreciate it if anyone can point out the reason that generates the noise and how to remove it. Am I doing anything wrong in the code? I have tested my algorithm in Mablab and it worked fine.
The problem has been solved temporarily. It turns out that the conversion between byte array and short array is not necessary. The required signal processing operations can be performed directly on byte arrays.
I'd like to keep this question open in case someone finds the bug(s) in the given code.
I am writing an AI to play Mancala and this is my method in which the AI's calculations are done by examining the outcomes of all 6 possible moves. I use the array staticBoardState to restore boardState (which stores the information about all of the holes on the board) back to its original values after each examination of move outcomes, but staticBoardState seems to be changing in odd ways even though I believe that I do not change it. I am a beginner amateur coder, so please ask questions if my code does not make sense. This is my code:
public int getBotCalc(int boardState[]) {
int[] staticBoardState = boardState;
double[] movePoints = new double[6];
int initialScore = boardState[6];
int scorePoints;
int freeTurnPoints;
double bestMovePoints;
int bestMove;
for(int f = 0; f <= 5; f++) {
boardState = staticBoardState;
int botChoice = f;
int botHole = boardState[botChoice];
boardState[botChoice] = 0;
for(int g = 0; g < botHole; g++) {
botChoice++;
if(botChoice>12) {
botChoice = 0;
}
boardState[botChoice]++;
}
if(botChoice<=5&&boardState[botChoice]==1&&boardState[12-botChoice]>=1) {
boardState[6] += boardState[12 - botChoice] + 1;
boardState[botChoice] = 0;
boardState[12 - botChoice] = 0;
}
scorePoints = boardState[6] - initialScore;
if(botChoice==6) {
freeTurnPoints = 1;
} else {
freeTurnPoints = 0;
}
movePoints[f] = scorePoints + (1.5 * freeTurnPoints);
}
bestMovePoints = movePoints[0];
bestMove = 0;
for(int f = 1; f <= 5; f++) {
if(movePoints[f]>bestMovePoints) {
bestMovePoints = movePoints[f];
bestMove = f;
}
}
boardState = staticBoardState;
return bestMove;
}
Any help is greatly appreciated.
It looks like you're confusing value-type assignment with reference assignment. When you write
staticBoardState = boardState
what happens is that staticBoardState simply holds a reference to the array in memory that boardState is also already referring to. Not they both refer to the same array in memory, which is why staticBoardState is apparently being modified through the use of boardState. What you need to do to fix this is allocate staticBoardState as a new array and explicitly copy its contents, for example using a boardState.clone(), and perform similar copying each time you want to restore your boardState.
I have an original .wav file of frequency of 18kHz and 19kHz on the left and right channel.
I have another filtered .wav file that is filtering 18kHz - 20kHz using IIR Filter with bandpass filter.
How do I detect the differences between the two? As in, how do I check that the the audio being implemented with the filter is being filtered successfully?
I am using the library that I found https://github.com/ddf/Minim/blob/master/src/ddf/minim/effects/BandPass.java https://github.com/DASAR/Minim-Android/blob/master/src/ddf/minim/effects/IIRFilter.java
The following is the code relevant to the filtering.
float[][] deinterleaveData(float[] samples, int numChannels) {
// assert(samples.length() % numChannels == 0);
int numFrames = samples.length / numChannels;
float[][] result = new float[numChannels][];
for (int ch = 0; ch < numChannels; ch++) {
result[ch] = new float[numFrames];
for (int i = 0; i < numFrames; i++) {
result[ch][i] = samples[numChannels * i + ch];
}
}
return result;
}
float[] interleaveData(float[][] data) {
int numChannels = data.length;
int numFrames = data[0].length;
float[] result = new float[numFrames*numChannels];
for (int i = 0; i < numFrames; i++) {
for (int ch = 0; ch < numChannels; ch++) {
result[numChannels * i + ch] = data[ch][i];
}
}
return result;
}
/**
* Convert byte[] raw audio to 16 bit int format.
* #param rawdata
*/
private int[] byteToShort(byte[] rawdata) {
int[] converted = new int[rawdata.length / 2];
for (int i = 0; i < converted.length; i++) {
// Wave file data are stored in little-endian order
int lo = rawdata[2*i];
int hi = rawdata[2*i+1];
converted[i] = ((hi&0xFF)<<8) | (lo&0xFF);
}
return converted;
}
private float[] byteToFloat(byte[] audio) {
return shortToFloat(byteToShort(audio));
}
/**
* Convert int[] audio to 32 bit float format.
* From [-32768,32768] to [-1,1]
* #param audio
*/
private float[] shortToFloat(int[] audio) {
float[] converted = new float[audio.length];
for (int i = 0; i < converted.length; i++) {
// [-32768,32768] -> [-1,1]
converted[i] = audio[i] / 32768f; /* default range for Android PCM audio buffers) */
}
return converted;
}
private void writeAudioDataToFile() throws IOException {
int read = 0;
byte data[] = new byte[bufferSize];
String filename = getTempFilename();
FileOutputStream os = null;
FileOutputStream rs = null;
try {
os = new FileOutputStream(filename);
rs = new FileOutputStream(getFilename().split(".wav")[0] + ".txt");
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
if (null != os) {
BandPass bandpass = new BandPass(19000,2000,44100);
while (isRecording) {
// decode and deinterleave stereo 16-bit per sample data
float[][] signals = deinterleaveData(byteToFloat(data), 2);
// filter data samples, updating the buffers with the filtered samples.
bandpass.process(signals[0], signals[1]);
// recombine signals for playback
audioTrack.write(interleaveData(signals), 0, count, WRITE_NON_BLOCKING);
// audioTrack.write(data, 0, count);
read = recorder.read(data, 0, bufferSize);
if (AudioRecord.ERROR_INVALID_OPERATION != read) {
try {
os.write(data);
rs.write(data);
} catch (IOException e) {
e.printStackTrace();
}
}
}
try {
os.close();
rs.close();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Full code is here if necessary: http://pastebin.com/23aS2A2w
Do I find the peaks and valleys of the original .wav file and the filtered .wav file to detect the difference? If not, how would I detect?
Thanks for all responses and help. Appreciated it!
There are a couple of techniques you could use for measuring the frequency response of a filter.
The first would be to run the filter with pure sine waves at various frequencies and then measure the amplitude of the output. For example, generate a tone at 1kHz with a peak amplitude of 1.0 and run the filter. Then look at the output. If the output is half the amplitude then you can say the filter has 6dB of attenuation at 1kHz. If you do this for enough frequencies you can make a plot of the frequency response with frequency on the x axis and output level on the y axis. This is a pretty crude approach though and requires a lot of points to get fine detail. Also, after changing frequencies you would need to skip some of the output to avoid looking at transients.
Another approach is to measure the impulse response of the filter. You do this by inputing a signal which has a single 1.0 followed by the rest zeros. Taking an FFT of the impulse response will give you the frequency response. You need to be careful though to supply enough samples to the filter to allow the impulse to die out though. You can figure this out by looking at the output samples and determining when the samples decay down to zero.
Basically I have the following scenario:
inputByteBuffer (capacity in debug) => 1024
byte[] inByteBufferArray = inputByteBuffer.array();
inByteBufferArray.length => 1031 ????
the ByteBuffer array() method says it "Returns the byte array which this buffer is based on, if there is one." ref
This was working in Android API 18, did something change? There is a ByterBuffer arrayOffset() method, do I need that?
This is happening on a Nexus 6 device, but I don't think that should matter.
Thanks
private static int bufferSizeStatic = 1024;
private float[] Read(){
mAudioRecord.read(inputByteBuffer, bufferSize);
return readByteArray(inputByteBuffer.array(), inputByteBuffer.arrayOffset());
}
private float[] readByteArray(byte[] byteArray, int offSet) {
if (floatArray == null || floatArray.length!=(byteArray.length-offSet) / (numChannels*numBytePerFrame)){
floatArray = new float[(byteArray.length-offSet) / (numChannels*numBytePerFrame)];
}
if (numChannels == 1){
for (int i = 0; i < floatArray.length; i++){
short tempShort = (short) ((byteArray[2*i+1+offSet]<<8) + byteArray[2*i+offSet]);
floatArray[i] = (float) (tempShort / Math.pow(2,15));
}
} //TODO add stereo support
return floatArray;
}
I don't see why you care. Your code certainly shouldn't care. The only reason it does is because you're misusing the API. You shouldn't be getting data out of the buffer that isn't in it, and that is given by limit(), not by array().length(), or by capacity() either.
All you need to do is adjust the calling sequence and use the ByteBuffer directly:
private static int bufferSizeStatic = 1024;
private float[] Read(){
// I don't know why you need to pass `bufferSize` here: `inputByteBuffer` already knows its own limit
mAudioRecord.read(inputByteBuffer, bufferSize);
return readByteArray(inputByteBuffer);
}
private float[] readByteArray(ByteBuffer byteBuffer) {
if (floatArray == null || floatArray.length!=(byteArray.limit()-byteArray.position()) / (numChannels*numBytePerFrame)){
floatArray = new float[(byteArray.limit()-byteArray.position()) / (numChannels*numBytePerFrame)];
}
byteBuffer.flip();
if (numChannels == 1){
for (int i = 0; i < floatArray.length; i++){
short tempShort = (short) ((ByteBuffer.getShort(i*2));
floatArray[i] = (float) (tempShort / 32768.0);
}
} //TODO add stereo support
byteBuffer.compact();
return floatArray;
}
E&OE
You'll find this at least as efficient.
I found android code:
MemoryRef(int capacity) {
VMRuntime runtime = VMRuntime.getRuntime();
buffer = (byte[])runtime.newNonMovableArray(byte.class, capacity + 7);
allocatedAddress = runtime.addressOf(buffer);
// Offset is set to handle the alignment: http://b/16449607
offset = (int)(((allocatedAddress + 7) & ~(long)7) - allocatedAddress);
isAccessible = true;
}
Source in here:
https://android.googlesource.com/platform/libcore/+/fe0e5d3e460de87d24e0224ef10b089289355c47/ojluni/src/main/java/java/nio/DirectByteBuffer.java#60
I am trying to add a wave graph to my android app, that displays the wave form data for the currently playing audio file.
I am currently trying to write a method to build an arraylist with the wave file amplitudes (1 amplitude for every 100 millieseconds of audio length), however it takes ages (minutes) to finish running. It is extremely inefficient.
This is the code:
public ArrayList <Integer> buildAudioWaveData(Recording recording){
final Recording finalRecording = recording;
(new Thread(){
#Override
public void run(){
File recFile = finalRecording.getFile();
ArrayList <Integer> dataSeries = new ArrayList<Integer>();
try {
InputStream bis = new BufferedInputStream(new FileInputStream(recFile));
DataInputStream dis = new DataInputStream(bis);
long sampleRate = finalRecording.getSampleRate(new RandomAccessFile(recFile, "rw"));
long samplesPerDatum = sampleRate / 10; // One sample for every 100 ms.
long fileLengthInBytes = recFile.length();
long fileDataRemaining = fileLengthInBytes / 2; // 16 bit wave file = 2 bytes per sample.
int max = 0;
while(fileDataRemaining > 0){
if(fileDataRemaining > samplesPerDatum) {
for (int i = 0; i < samplesPerDatum; i++) {
short temp = dis.readShort();
if (temp > max) {
max = temp;
}
}
Log.i("temp", Integer.toString(max));
dataSeries.add(max);
max = 0;
}
fileDataRemaining -= samplesPerDatum;
}
int x = 0;
}catch(Exception e){
}
}
}).start();
return null;
}
Does anyone know of a more efficient way in which i can generate the array for my graph?
Thanks heaps in advance.
Corey B :)