Getting frequencies and magnitudes from audio sample using FFT - java

I am currently working on a music visualization LED strip display (similar to this) but got stuck at the process of extracting audio frequency / magnitude from the sample data.
The entire process takes up quite a few steps, and I am not sure at which point I am failing exactly, so please bear with me. I start with reading audio from a TargetDataLine configured with AudioFormat.
AudioSystem.getTargetDataLine(new AudioFormat(8000.0f, 16, 1, true, true));
Next, I read the data as described in this tutorial. Only difference is that I am using a smaller buffer, effectively reading 64 bytes at a time.
Here is an example data read in this way from a 500Hz test tone.
final byte[] buffer = { // 32 16-bit samples
30, 111, 43, -19, 50, -74, 49, -54,
41, 70, 26, 121, 7, -90, -13, -89,
-31, -118, -44, 17, -51, 75, -50, 60,
-42, -62, -27, -115, -8, 91, 12, 85,
30, 106, 43, -34, 50, -93, 49, -76,
41, 52, 26, 108, 7, -93, -13, -84,
-31, -104, -44, 38, -51, 98, -50, 84,
-42, -42, -27, -98, -8, 99, 12, 83
};
Next, I transform the sampled bytes into an array of double values in range [-1, 1]. I took inspiration from the aforementioned tutorial and this code.
final double[] samples = new double[buffer.length];
int sampleIndex = 0;
int byteIndex = 0;
while (byteIndex < buffer.length) {
int low = buffer[byteIndex];
++byteIndex;
int high = buffer[byteIndex];
++byteIndex;
int sampleIntValue = ((high << 8) + (low & 0x00ff));
double maxSampleValue = 32768;
samples[2 * sampleIndex] = ((double) sampleIntValue) / maxSampleValue; // Transforming to [-1, 1]
samples[(2 * sampleIndex) + 1] = 0; // Imaginary part
++sampleIndex;
}
Next, I perform a Fast Fourier Transform on the data using the edu.emory.mathcs:JTransforms:2.4 library.
final DoubleFFT_1D fft = new DoubleFFT_1D(samples.length / 2);
fft.complexForward(samples);
Then, I calculate frequencies and magnitudes as described in this code.
final float sampleRate = 8000f;
final Map<Double, Double> frequenciesToMagnitudes = IntStream.range(0, samples.length / 2)
.boxed()
.collect(Collectors.toMap(
index -> 2 * ((double) index / (double) samples.length) * sampleRate,
index -> Math.log10(
Math.sqrt(
Math.pow(samples[2 * index], 2) // real part
+ Math.pow(samples[(2 * index) + 1], 2) // imaginary part
)
)
));
I find the maximum magnitude so I can scale the displayed values accordingly.
final double maximumMagnitude = frequenciesToMagnitudes.values().stream().max(Double::compare).orElse(0d);
And finally, I display the results (taller block represents a brighter LED).
final char[] magnitudeDisplay = {'▁', '▂', '▃', '▄', '▅', '▆', '▇', '█'};
frequenciesToMagnitudes.entrySet().stream().sorted(Map.Entry.comparingByKey()).forEach(frequencyToMagnitude -> {
final double magnitudePercentage = frequencyToMagnitude.getValue() / maximumMagnitude;
final int characterToDisplayIndex = Math.min(magnitudeDisplay.length - 1, Math.max(0, (int) (magnitudePercentage * magnitudeDisplay.length)));
final char characterToDisplay = magnitudeDisplay[characterToDisplayIndex];
System.out.print(characterToDisplay);
});
System.out.println();
From all this, I would expect to see two distinct spikes (my frequency + alias region), but instead there is 8.
▁▁▅▁▁▁█▁▁▁▄▁▁▁▅▁▁▁▅▁▁▁▄▁▁▁█▁▁▁▅▁
The number of spikes changes depending on frequency of audio playback (sometimes it's two, sometimes it's four), and the lower the frequency the more spikes I see.
My question is: how do I extract frequency / magnitude pairs from audio data?

After a bit of digging I have found the OpenIMAJ library which does exactly what I need. Here are the differences in my code and their code.
Transforming from bytes to floating point is incorrect. All examples use big endian audio format, but take the bytes in the wrong order. It should be:
int high = buffer[byteIndex];
++byteIndex;
int low = buffer[byteIndex];
++byteIndex;
Size of FFT output is padded to next power of 2. This was not a problem since I always chose a buffer of size that was a power of 2, but in case if your buffer is not, this will be necessary.
final int fftSize = (int) Math.pow(2, 32 - Integer.numberOfLeadingZeros(numberOfSamples - 1));
The values are transformed to floating point a bit differently.
samples[2 * sampleIndex] = (float) sampleIntValue * Integer.MAX_VALUE / Short.MAX_VALUE;
Real parts are normalized.
for (int i = 0; i < samples.length; i += 2) samples[i] /= fftSize;
No logarithms are used for fetching magnitudes.
final Map<Float, Float> frequenciesToMagnitudes = IntStream.range(0, samples.length / 4)
.boxed()
.collect(Collectors.toMap(
index -> (((float) (index * 2)) * (sampleRate / (float) numberOfSamples)) / 2,
index -> {
final float realPart = samples[index * 2];
final float imaginaryPart = samples[index * 2 + 1];
return (float) Math.sqrt(realPart * realPart + imaginaryPart * imaginaryPart);
}
));
After all the changes, the output appears to be much more reasonable:
▁▁▂▁▁▁█▁▁▁▂▁▁▁▂▁
I am still not entirely sure if the frequency calculation is correct, though.

Related

Move element of array to specific position [duplicate]

This question already has answers here:
Java: moving items in array
(6 answers)
Closed 6 years ago.
How to move element of array to specific position on Android JAVA.
We have
int oldPosition, int newPosition
and some like
JSONObject[] tmp = new JSONObject[999];
array of JSONObjects
If you want to just move
tmp[newPosition]=tmp[oldPosition];
Swap
JSONObject jo= tmp[oldPosition];
tmp[oldPosition]=tmp[newPosition];
tmp[newPosition]=jo;
EDIT: There are other ways but you can go through this as well to get your result :)
Exercise : You can understand this logic and use JSONObject type and do necessary changes, Watch out for NullPointerExceptions , handle everything i am lazy to do them
let's say you have int array --> private int array[];
array = new int[]{10,20,30,40,50,60,70,80,90,100};
call this method if you want to swap elements,
swapNumbers(array,9,1);
.
public int[] swapNumbers(int [] arr, int possition1, int possition2){
int temp = arr[possition2];
arr[possition2] = arr[possition1];
arr[possition1] = temp;
System.out.println("array -->" + Arrays.toString(array));
return arr;
}
out put : array[10, 100, 30, 40, 50, 60, 70, 80, 90, 20]
But that doesn't satisfy you?
you need out put to be like this : array[10, 100, 20, 30, 40, 50, 60, 70, 80, 90]
you can use below method
resetUpMyArray(array, array[9],1);
System.out.println("array Finally changed-->" + Arrays.toString(array));
enjoy,
public int[] resetUpMyArray(int[] inputArray, int deleteMeValue,int addMePosition) {
List resultLinkedList = new LinkedList();
for (int itemValue : inputArray)
if (deleteMeValue == itemValue) {
System.out.println("Do not add this value"+itemValue);
} else {
System.out.println("Do add this value "+itemValue +"position-"+deleteMeValue);
resultLinkedList.add(itemValue);
}
System.out.println("array -as new L.L->" + resultLinkedList);
resultLinkedList.add(addMePosition,deleteMeValue);
System.out.println("array -as new L.L all set->" + resultLinkedList);
array = new int[resultLinkedList.size()];
for (int i = 0; i < resultLinkedList.size(); i++) {
array[i] = (int) resultLinkedList.get(i); // Watch out for NullPointerExceptions!
}
return array;
}
Here is two simple algorithm.
Switch values :
switch(array, from, to)
tmp = array[to]
array[to] = array[from]
array[from] = tmp
That will give something like
[10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
|--<---->--|
[10, 20, 60, 40, 50, 30, 70, 80, 90, 100]
This will simply store the values that will be replace at the index to to be place at index from
Move and shift values:
This one will move one values a shift the values next.
[10, 20, 30, 40, 50, 60, 70, 80, 90, 100]
|------------^
[10, 20, , 40, 50, 60, 30, 70, 80, 90, 100]
<-----------
[10, 20, 40, 50, 60, 30, 70, 80, 90, 100]
For this, the solution is quite the same, store the values in tmp but shift every value to fill the gap. This code will only work if from < to
tmp = array[to]
i = from
while(i < to)
array[i] = array[i+1]; --Shift the value
i = i + 1
array[to] = tmp;

mediacodec and opencv mismatch color type

I have 2 apps, 1 I implement on Matlab(c) using OpenCV and the second is on eclipse(android,java) using MediaCodec.
I want to Decode the first frame from a same mp4 video(I need more than 1 frame, I look at the first one only for debugging, that's why I chose MediaCodec and OpenCV).
In Matlab I do this:
currFrame = videoFReader.read;
% Convert to grayscale
%currGray = Convert_Movie_To_Gray(currFrame); % Works on a single frame as well
currGray = cv.cvtColor(currFrame,'BGR2GRAY');
while currFrame is the Frame[0] and currGray is the grayscale frame.
In Eclipse I do this:
GLES20.glReadPixels(0, 0, mWidth, mHeight, GLES20.GL_RGBA, GLES20.GL_UNSIGNED_BYTE,
mPixelBuf);
mPixelBuf.rewind();
int[] colors = mColorSwapBuf;
mPixelBuf.asIntBuffer().get(colors);
NativeClass.GrayIntArrayToMat(colors, mainMat.getNativeObjAddr());
while GrayIntArrayToMat is a native function that convert the frames to grayscale:
for(int i = 0 ; i< out->cols * out->rows; i++)
{
int col = inputarray[i];
int R,G,B;
B = (col >> 16) & 0xff;
G = (col >> 8) & 0xff;
R = col & 0xff;
inputarray[i] = (int)(((((R*0.3) + (G*0.59) + (B*0.11)))));
res[i] = inputarray[i];
}
My problem is that I don't get the same colors in both examples.
If I look at the first 10 columns of the first row, I'm getting this(grayscale frame):
[27, 27, 27, 27, 27, 27, 29, 30, 31, 32] - in Matlab
[27, 28, 27, 29, 28, 29, 30, 31, 31, 31] - in Eclipse
My reference is the Matlab, so I need to get the same or at least closer frame to his frame.
Thanks for your help!

Whole item distribution by ratio

The goal:
Say I have X workers in a fruit plantation. At the plantation they're growing apples, pears and grapes.
At the end of the day, the foreman grades each workers with a ratio. The total of all ratio's is 100. The ratio is there to determine how to distribute the fruit amongst the workers at the end of the day.
How do I distribute the fruit amongst all workers so that they each get their fair share (within a certain randomness to account for integer division). Only whole fruits are divided, so integer results. And all fruits must be given out.
I'm doing this with about 20 workers, so right now the ratio is in neighbourhood of 0.05 per worker.
What I've tried (pseudo-code):
for each worker:
if applesGiven < appleStock:
worker.give(ratio * applestock);
if pearsGiven < pearStock:
worker.give(ratio * pearStock);
if grapesGiven < grapeStock:
worker.give(ratio * grapeStock);
I would let the exact numer of [fruit] they where given be determined by a boolean Roundup that was initialized with a random boolean and switched after every fruit that was processed.
What I've tried (full-code):
public void balance() {
boolean roundUp = random.nextBoolean();
for (Employee e : employees) {
double ratio = e.getRatio();
if (applePlanned < appleNeeded) {
int apple;
if (roundUp) {
apple = (int) Math.ceil(ratio * appleNeeded);
} else {
apple = (int) Math.floor(ratio * appleNeeded);
}
e.setrapple(apple);
applePlanned += apple;
roundUp = !roundUp;
}
if (pearPlanned < pearNeeded) {
int pear;
if (roundUp) {
pear = (int) Math.ceil(ratio * pearNeeded);
} else {
pear = (int) Math.floor(ratio * pearNeeded);
}
e.setrpear(pear);
pearPlanned += pear;
roundUp = !roundUp;
}
if (grapePlanned < grapeNeeded) {
int grape;
if (roundUp) {
grape = (int) Math.ceil(ratio * grapeNeeded);
} else {
grape = (int) Math.floor(ratio * grapeNeeded);
}
e.setrgrape(grape);
grapePlanned += grape;
roundUp = !roundUp;
}
}
Problems I ran into:
Only about 3/4 of all items are distributed
When I have an even number of fruit, the boolean gets the same value at the start of each new person.
Thank you for looking into this!
Answers in java, python or pseudo-code please, that's what I can read.
Use double math, round down, then randomly hand out the remaining fruit weighted based on ratio. Note, you can make this a lot less ugly with object orientation and loops, but this is a start.
public void distribute(int apple, int pear, int grape) {
double total = apple + pear + grape;
double appleRatio = apple/total;
double pearRatio = pear/total;
double grapeRatio = grape/total;
// apple worker
int appleWorkerApple = (int) (appleRatio*apple);
int appleWorkerPear = (int) (appleRatio*pear);
int appleWorkerGrape = (int) (appleRatio*grape);
// pear worker
int pearWorkerApple = (int) (pearRatio*apple);
int pearWorkerPear = (int) (pearRatio*pear);
int pearWorkerGrape = (int) (pearRatio*grape);
// grape worker
int grapeWorkerApple = (int) (grapeRatio*apple);
int grapeWorkerPear = (int) (grapeRatio*pear);
int grapeWorkerGrape = (int) (grapeRatio*grape);
int appleRemain = apple - appleWorkerApple - pearWorkerApple - grapeWorkerApple;
int pearRemain = pear - appleWorkerApple - pearWorkerApple - grapeWorkerApple;
int grapeRemain = grape - appleWorkerApple - pearWorkerApple - grapeWorkerApple;
Random r = new Random();
while(appleRemain > 0 && pearRemain > 0 && grapeRemain > 0) {
double target = r.nextDouble();
switch(r.nextInt(3)) {
case 0:
if(appleRemain > 0) {
appleRemain--
if(target < appleRatio)
appleWorkerApple++;
else if (target < appleRatio + grapeRatio)
pearWorkerApple++;
else
grapeWorkerApple++;
}
break;
case 1:
if(grapeRemain > 0)
// etc.
}
}
}
It's not particularly useful 'cause I've overused Numpy but I'll share since it's relevant
import numpy
import random
# apple, bannana, grapes, guava, melon, pear
fruits = numpy.array([100, 150, 175, 200, 230, 247])
# Bill, Bob, Dan, Fred, Joe
ratios = numpy.array([21, 7, 32, 13, 27])
# Original fruit amount for each worker: 0
worker_fruits = numpy.zeros((5, 6), dtype=int)
worker_lucky = numpy.zeros((5, 6), dtype=float)
# For each worker with his ratio
for worker, lucky, ratio in zip(worker_fruits, worker_lucky, ratios):
# Give him fruits, storing partials as weighting
to_give = (ratio * fruits) / 100
lucky += to_give % 1
worker += to_give
# Calculate how much we have left over
spares = fruits - worker_fruits.sum(axis=0)
# Share it out in a weighted distribution
for fruit, lucky, numspare in zip(worker_fruits.transpose(), worker_lucky.transpose(), spares):
if numspare:
indexes = numpy.arange(len(fruit))
add_to = numpy.random.choice(indexes, replace=False, size=numspare, p=lucky/numspare)
fruit[add_to] += 1
# Our results!
worker_fruits
#>>> array([[21, 31, 36, 42, 49, 51],
#>>> [ 7, 11, 12, 14, 16, 18],
#>>> [32, 48, 56, 64, 74, 79],
#>>> [13, 19, 23, 26, 29, 32],
#>>> [27, 41, 48, 54, 62, 67]])
# Proof it's perfectly shared
fruits - worker_fruits.sum(axis=0)
#>>> array([0, 0, 0, 0, 0, 0])

Restricting decimal places in an array

Currently, the output of this array has a too large decimal place trail. How can I restrict this to say 2 decimal places? By this I mean the array 'percentage1'. I've seen methods to do it online, but I don't understand how I would implement those methods into the code as shown below.
int[] correct1 = {20, 20, 13, 15, 22, 18, 19, 21, 23, 25};
int[] incorrect1 = {2, 1, 5, 2, 2, 5, 8, 1, 0, 0};
double[] percentage1 = new double[correct1.length];
for(int a = 0; a < correct1.length; a++ ){
percentage1[a] = (((double)correct1[a] / (correct1[a] + incorrect1[a]))*100);
}
Any help would be very much appreciated. Thanks
Please try adding a DecimalFormat object.
Add this to the beginning of the loop, it declares the format you're looking for - 2 decimal places: DecimalFormat df = new DecimalFormat("#.##");
Format it using format, then convert it back into a double. The reason why you need to restore it back is that format returns a String.
percentage1[a] = Double.valueOf(df.format((((double)correct1[a] / (correct1[a] + incorrect1[a]))*100)));
See revised code below:
public static void main(String[] args) {
// TODO Auto-generated method stub
int[] correct1 = {20, 20, 13, 15, 22, 18, 19, 21, 23, 25};
int[] incorrect1 = {2, 1, 5, 2, 2, 5, 8, 1, 0, 0};
double[] percentage1 = new double[correct1.length];
DecimalFormat df = new DecimalFormat("#.##");
for(int a = 0; a < correct1.length; a++ ){
percentage1[a] = Double.valueOf(df.format((((double)correct1[a] / (correct1[a] + incorrect1[a]))*100)));
System.out.println(percentage1[a]);
}
}
Sample result:
90.91
95.24
72.22
88.24
91.67
78.26
70.37
95.45
100.0
100.0
You can't. Doubles don't have decimal places. They have binary places. If you want decimal places you have to use a decimal radix, i.e. a String created by DecimalFormat, or a BigDecimal.
Proof here.

My table is outputting incorrect values for the percentage

public class apples {
private static String[] level1 = new String[] { "A", "B", "I", "K", "N", "O", "P", "S", "T", "W" };
public static void main(String[] args) {
int[] scores1 = { 99, 80, 56, 88, 70, 35, 67, 60, 78, 56 };
int[] correct1 = {20, 20, 13, 15, 22, 18, 19, 21, 23, 25};
int[] incorrect1 = {2, 1, 5, 2, 2, 5, 8, 1, 0, 0};
double[] percentage1 = new double[correct1.length];
for(int a = 0; a < correct1.length; a++ ){
percentage1[a] = (double)((correct1[a] / (correct1[a] + incorrect1[a]))*100);
}
System.out.println("Character \t Correct \t Incorrect \t Percentage");
for(int counter = 0; counter<scores1.length;counter++){
System.out.println(level1[counter] + "\t\t " + correct1[counter] + "\t\t " + incorrect1[counter] + "\t\t " + percentage1[counter]);
}
}
}
This outputs a table with 4 headings. The character, correct and incorrect columns show as expected. However the percentage row is not working properly. For example, character 'A', correct 20 and incorrect 2 gives a percentage of 0.0. Any 'incorrect' value > 0 outputs a percentage value of 0, and any 'incorrect' value which = 0 gives a percentage value of 100 (which is correct)... Can someone please explain where I have gone wrong?
You are dealing with integers here, and for integer division, the result is truncated. You'll need to cast the original values to double instead, or multiply one part by 1.0 to get it as a double:
percentage1[a] = ((correct1[a]*1.0 / (correct1[a] + incorrect1[a]))*100);
percentage1[a] = (double)((correct1[a] / (correct1[a] + incorrect1[a]))*100);
The above code casts to a double after the calculation is competed.
To cast as part of the calculation, use:
percentage1[a] = (( ((double)correct1[a]) / (correct1[a] + incorrect1[a]))*100);
You calculations here
percentage1[a] = (double)((correct1[a] / (correct1[a] + incorrect1[a]))*100);
perform integer division (you just cast them afterwards to double). If you want them to return the actual floating point division result, you have to cast all operands to double before the calculation.
So the fastest option would be to change this:
double[] correct1 = {20, 20, 13, 15, 22, 18, 19, 21, 23, 25};
double[] incorrect1 = {2, 1, 5, 2, 2, 5, 8, 1, 0, 0};
Another would be to change the computation to something like this
percentage1[a] = (1.0 * correct1[a] / (correct1[a] + incorrect1[a]))*100;
or to simplify a little:
percentage1[a] = 100.0 * correct1[a] / (correct1[a] + incorrect1[a]);

Categories