We Keep Coding

Creating a pi(x) Table

Let pi(x) denote the number of primes <= x. For example, pi(100) = 25. I would like to create a table which stores values of pi(x) for all x <= L. I figured the quickest way would be to use the sieve of Eratosthenes. First I mark all primes, and then I use dynamic programming, summing the count of primes and increasing each time a new prime appears. This is implemented in the Java code below:
public static int [] piTableSimple (int L)
{
int sqrtl = (int) Math.sqrt(L);
int [] piTable = new int [L + 1];
Arrays.fill(piTable, 1);
piTable[0] = 0;
piTable[1] = 0;
for (int i = 2 ; i <= sqrtl ; i++)
if (piTable[i] == 1)
for (int j = i * i ; j <= L ; j += i)
piTable[j] = 0;
for (int i = 1 ; i < piTable.length ; i++)
piTable[i] += piTable[i - 1];
return piTable;
}
There are 2 problems with this implementation:
It uses large amounts of memory, as the space complexity is O(n)
Because Java arrays are "int"-indexed, the bound for L is 2^31 - 1
I can "cheat" a little though. Because for even values of x, pi(x) = pi(x - 1), enabling me to both reduce memory usage by a factor of 2, and increase the bound for L by a factor of 2 (Lmax <= 2^32).
This is implemented with a simple modification to the above code:
public static long [] piTableSmart (long L)
{
long sqrtl = (long) Math.sqrt(L);
long [] piTable = new long [(int) (L/2 + 1)];
Arrays.fill(piTable, 1);
piTable[0] = 0;
piTable[1] = 0;
piTable[2] = 1;
for (int i = 3 ; i <= sqrtl ; i += 2)
if (piTable[(i + 1) / 2] == 1)
{
long li = (long) i;
long inc = li * 2L;
for (long j = li * li ; j <= L ; j += inc)
piTable[(int) ((j + 1) / 2)] = 0;
}
piTable[2] = 2;
for (int i = 1 ; i < piTable.length ; i++)
piTable[i] += piTable[i - 1];
return piTable;
}
Note that the value of pi(2) = 1 is not directly represnted in this array. But this has simple workarounds and checks that solve it. This implementation comes with a small cost, that the actual value of pi(x) is not accessed in a straight-forward way, but rather to access the value of pi(x), one has to use
piTable[(x + 1) / 2]
And this works for both even and odd values of x of course. The latter completes creating a pi(x) table for x <= L = 10^9 in 10s on my rather slowish laptop.
I would like to further reduce the space required and also increase the bound for L for my purposes, without severly deteriorating performance (for example, the cost of slightly more arithmetic operations to access the value of pi(x) as in the latter code barely deteriorates performance). Can it be done in an efficient and smart way?

You should use a segmented Sieve of Eratosthenes, which reduces the memory requirement from O(n) to O(sqrt(n)). Here is an implementation.
Do you need to store all the pi? Here is a function that computes pi(x). It's reasonably quick up to 10**12.
If you find this useful, please upvote this answer and also the two linked answers.

Now that I understand better what you want to do, I can give a better answer.
The normal way to compute pi(x) starts with pre-computed tables arranged at intervals, then uses a segmented sieve to interpolate between the pre-computed points; the pre-computations may be done by sieving or by any of several other methods. Those tables get big, as you have pointed out. If you want to be able to compute pi(x) up to 1020, and you are willing to sieve a range up to 1012 each time someone calls your function, you will need a table with 108 64-bit integers, which will take nearly a gigabyte of space; calls to your function will take about half-a-minute each for the sieving, assuming a recent-vintage personal computer. Of course, you can choose where you want to be on the time/space trade-off curve by choosing how many pre-computed points you will have.
You are talking about computing pi(x) for x > 1024, which will take lots more space, or lots more time, or both. Lots. Recent projects that have computed huge values of pi(x), for values of x like 1024 or 1025, have taken months to compute.
You might want to look at Kim Walisch's primesieve program, which has a very fast segmented sieve. You might also look at the website of Tomás Oliveira e Silva, where you will find tables of pi(x) up to 1022.
Having said all that, what you want to do probably isn't feasible.

Calculating Least Common Multiple (how do I know when there isn't one?)

The code below was my first attempt at a LCM (lowest common multiple) calculator with a user interface (UI code not shown) written months ago. I know there are simpler ways to write this, but I'd like help understanding why sometimes THIS specific code is not finding a common multiple (with most number sets it works fine).
When a user inputs almost any number set, the app spits out the correct LCM. But when the number set 1,234 / 2,345 / 5,432 / 4,321 is used, the app initially was stopping when x hit 536,870,912. This was because the result of x * mult was a number that couldn't be held by the int primitive. After changing x to a double and casting result = (int) (mult * x), the code continues to function as expected but seems to increment x indefinitely.
public static void compare(){
result = 0;
int mult = 0;
double x = 1;
int[] nums = UserInterface.getNums();
// finds highest number in user-input set
for(int i = 0; i < nums.length; i ++){
if (nums[i] > mult) mult = nums[i];
}
// finds lowest common multiple
for(int i = 0; i < nums.length;){
if((mult * x) % nums[i] == 0){
result = (int) (mult * x);
i++;
}
else{
result = 0;
x++;
i = 0;
}
}
}

We know the LCM of your test set must be less than or equal to 67,920,681,416,560.
In java the int datatype has a max value of 2^31-1 = 2,147,483,647 so you are obviously going to get an overflow. You can change your code to use long throughout this has a max value of 2^64-1=18,446,744,073,709,551,615 so it should be sufficient for your calculation. If you need bigger values then look at the BigInteger class.
In javascript things are more complicated. All numbers are floating point so you loose accuracy. This probably mean the condition
if((mult * x) % nums[i] == 0)
is never satisfied so your loop never quits.
Your algorithm is very basic, there are much better algorithms out there, elclanrs has one above and see https://en.wikipedia.org/wiki/Least_common_multiple for some hints.
Also you should change the title of the question. As it stands it make no sense as any set of numbers must have a LCM.

What's wrong when my matrix is "not positive definite"?

i'm using the Jahmm java lib for classification. I want to do some testing so i generate some random data sets.
I create data sets in this format:
[val_1.1 val_1.2 val_1.3];[val_2.1 val_2.2 val_2.3]; [val_3.1 val_3.2 val_3.3] etc...
i use a random function so that
val_1.1 == val_1.2 == val_1.3
and
val_2.1 == val_2.2 == val_2.3
and so on.
When i call the following function with this dataset it throws an IllegalArgumentException
static double[][] decomposeCholesky(double[][] m)
{
if (!isSquare(m))
throw new IllegalArgumentException("Matrix is not square");
double[][] l = matrix(nbRows(m), nbColumns(m));
for (int j = 0; j < nbRows(m); j++)
{
double[] lj = l[j];
double d = 0.;
for (int k = 0; k < j; k++) {
double[] lk = l[k];
double s = 0.;
for (int i = 0; i < k; i++)
s += lk[i] * lj[i];
lj[k] = s = (m[j][k] - s) / l[k][k];
d = d + s * s;
}
if ((d = m[j][j] - d) <= 0.)
throw new IllegalArgumentException("Matrix is not positive " +
"defined");
l[j][j] = Math.sqrt(d);
for (int k = j+1; k < nbRows(m); k++)
l[j][k] = 0.;
}
return l;
}
so my sequence matrix is not "positive defined", but what does it mean? And what should do to my dataset to avoid it?
I am not good in math! Thanks in advance

I think the author of the code meant "positive definite." A matrix must be positive definite in order to be factored with a Cholesky decomposition. The formal definition is a square matrix A is positive definite if and only if, for all vectors x:
x'Ax > 0
All positive definite matrices are symmetric about the diagonal, and square, so a good start would be to use only square symmetric matrices in the test and see how that works. To absolutely ensure that a matrix is positive definite, you can test all its eigenvalues to see if they are each > 0. I don't know if JAHMM has methods for taking the eigenvalues of a matrix but if so, you could do that.

i do not know that library,
but i think you mean "positive definite" not "positive defined".
here's the thing: if you have a normal number you can easily tell if it's positive oder negative (or zero) by looking at the sign. definiteness is an extension of that idea into the world of matrices, where just looking at the sign doesn't work anymore because some entries might be positive and some might be negative.
there are many different definitions of definiteness (which can proven to be equal), you can find them neatly listed here:
http://en.wikipedia.org/wiki/Positive-definite_matrix#Characterizations
now, the problem is that when you choose your rows equally that doesn't guarantee
positive definiteness. in fact that 3x3 matrix will always be positive semidefinite and never positive definite.
i've looked around a bit; here are a few hints how to generate positive definite matrices:
https://ece.uwaterloo.ca/~dwharder/NumericalAnalysis/04LinearAlgebra/posdef/
( rand(n, n) + (n - 1)*eye( n ) )
so you generate a n x n matrix where all entries are random between 0 and 1,
then add the identity matrix multiplied with n-1, in your case that is [2,0,0];[0,2,0];...
hope that helps ...
p.s. i forgot your matrix has to be symmetric as well because you'd like to do a cholesky on it. but that's easy, just generate the matrix A as mentioned above, then choose B = 1/2 * ( A + A.transposed() );
This matrix B will still be positive definite and it'll be symmetric as well :)

Implementing exponential moving average in Java

I essentially have an array of values like this:
0.25, 0.24, 0.27, 0.26, 0.29, 0.34, 0.32, 0.36, 0.32, 0.28, 0.25, 0.24, 0.25
The above array is oversimplified, I'm collecting 1 value per millisecond in my real code and I need to process the output on an algorithm I wrote to find the closest peak before a point in time. My logic fails because in my example above, 0.36 is the real peak, but my algorithm would look backwards and see the very last number 0.25 as the peak, as there's a decrease to 0.24 before it.
The goal is to take these values and apply an algorithm to them which will "smooth" them out a bit so that I have more linear values. (ie: I'd like my results to be curvy, not jaggedy)
I've been told to apply an exponential moving average filter to my values. How can I do this? It's really hard for me to read mathematical equations, I deal much better with code.
How do I process values in my array, applying an exponential moving average calculation to even them out?
float[] mydata = ...
mySmoothedData = exponentialMovingAverage(mydata, 0.5);
float[] exponentialMovingAverage(float[] input, float alpha) {
// what do I do here?
return result;
}

To compute an exponential moving average, you need to keep some state around and you need a tuning parameter. This calls for a little class (assuming you're using Java 5 or later):
class ExponentialMovingAverage {
private double alpha;
private Double oldValue;
public ExponentialMovingAverage(double alpha) {
this.alpha = alpha;
}
public double average(double value) {
if (oldValue == null) {
oldValue = value;
return value;
}
double newValue = oldValue + alpha * (value - oldValue);
oldValue = newValue;
return newValue;
}
}
Instantiate with the decay parameter you want (may take tuning; should be between 0 and 1) and then use average(…) to filter.
When reading a page on some mathmatical recurrence, all you really need to know when turning it into code is that mathematicians like to write indexes into arrays and sequences with subscripts. (They've a few other notations as well, which doesn't help.) However, the EMA is pretty simple as you only need to remember one old value; no complicated state arrays required.

I am having a hard time understanding your questions, but I will try to answer anyway.
1) If your algorithm found 0.25 instead of 0.36, then it is wrong. It is wrong because it assumes a monotonic increase or decrease (that is "always going up" or "always going down"). Unless you average ALL your data, your data points---as you present them---are nonlinear. If you really want to find the maximum value between two points in time, then slice your array from t_min to t_max and find the max of that subarray.
2) Now, the concept of "moving averages" is very simple: imagine that I have the following list: [1.4, 1.5, 1.4, 1.5, 1.5]. I can "smooth it out" by taking the average of two numbers: [1.45, 1.45, 1.45, 1.5]. Notice that the first number is the average of 1.5 and 1.4 (second and first numbers); the second (new list) is the average of 1.4 and 1.5 (third and second old list); the third (new list) the average of 1.5 and 1.4 (fourth and third), and so on. I could have made it "period three" or "four", or "n". Notice how the data is much smoother. A good way to "see moving averages at work" is to go to Google Finance, select a stock (try Tesla Motors; pretty volatile (TSLA)) and click on "technicals" at the bottom of the chart. Select "Moving Average" with a given period, and "Exponential moving average" to compare their differences.
Exponential moving average is just another elaboration of this, but weights the "older" data less than the "new" data; this is a way to "bias" the smoothing toward the back. Please read the Wikipedia entry.
So, this is more a comment than an answer, but the little comment box was just to tiny. Good luck.

Take a look at this.
If your noise has zero average, consider also the use of a Kalman filter.

In a rolling manner.... i also use commons.apache math library
public LinkedList EMA(int dperiods, double alpha)
throws IOException {
String line;
int i = 0;
DescriptiveStatistics stats = new SynchronizedDescriptiveStatistics();
stats.setWindowSize(dperiods);
File f = new File("");
BufferedReader in = new BufferedReader(new FileReader(f));
LinkedList<Double> ema1 = new LinkedList<Double>();
// Compute some statistics
while ((line = in.readLine()) != null) {
double sum = 0;
double den = 0;
System.out.println("line: " + " " + line);
stats.addValue(Double.parseDouble(line.trim()));
i++;
if (i > dperiods)
for (int j = 0; j < dperiods; j++) {
double var = Math.pow((1 - alpha), j);
den += var;
sum += stats.getElement(j) * var;
System.out.println("elements:"+stats.getElement(j));
System.out.println("sum:"+sum);
}
else
for (int j = 0; j < i; j++) {
double var = Math.pow((1 - alpha), j);
den += var;
sum += stats.getElement(j) * var;
}
ema1.add(sum / den);
System.out.println("EMA: " + sum / den);
}
return ema1;
}

public class MovingAvarage {
public static void main(String[] args) {
double[] array = {1.2, 3.4, 4.5, 4.5, 4.5};
double St = 0D;
for(int i=0; i<array.length; i++) {
St = movingAvarage(St, array[i]);
}
System.out.println(St);
}
private static double movingAvarage(double St, double Yt) {
double alpha = 0.01, oneMinusAlpha = 0.99;
if(St <= 0D) {
St = Yt;
} else {
St = alpha*Yt + oneMinusAlpha*St;
}
return St;
}
}

If you're having trouble with the math, you could go with a simple moving average instead of exponential. So the output you get would be the last x terms divided by x. Untested pseudocode:
int data[] = getFilled();
int outdata[] = initializeme()
for (int y = 0; y < data.length; y++)
int sum = 0;
for (int x = y; x < y-5; x++)
sum+=data[x];
outdata[y] = sum / 5;
Note that you will need to handle the start and end parts of the data since clearly you can't average the last 5 terms when you are on your 2nd data point. Also, there are more efficient ways of calculating this moving average(sum = sum - oldest + newest), but this is to get the concept of what's happening across.

FFT library in android Sdk [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am working with android project.I need FFT algorithm to process the android accelerometer data.Is there FFT library available in android sdk?

You can use this class, which is fast enough for real time audio analysis
public class FFT {
int n, m;
// Lookup tables. Only need to recompute when size of FFT changes.
double[] cos;
double[] sin;
public FFT(int n) {
this.n = n;
this.m = (int) (Math.log(n) / Math.log(2));
// Make sure n is a power of 2
if (n != (1 << m))
throw new RuntimeException("FFT length must be power of 2");
// precompute tables
cos = new double[n / 2];
sin = new double[n / 2];
for (int i = 0; i < n / 2; i++) {
cos[i] = Math.cos(-2 * Math.PI * i / n);
sin[i] = Math.sin(-2 * Math.PI * i / n);
}
}
public void fft(double[] x, double[] y) {
int i, j, k, n1, n2, a;
double c, s, t1, t2;
// Bit-reverse
j = 0;
n2 = n / 2;
for (i = 1; i < n - 1; i++) {
n1 = n2;
while (j >= n1) {
j = j - n1;
n1 = n1 / 2;
}
j = j + n1;
if (i < j) {
t1 = x[i];
x[i] = x[j];
x[j] = t1;
t1 = y[i];
y[i] = y[j];
y[j] = t1;
}
}
// FFT
n1 = 0;
n2 = 1;
for (i = 0; i < m; i++) {
n1 = n2;
n2 = n2 + n2;
a = 0;
for (j = 0; j < n1; j++) {
c = cos[a];
s = sin[a];
a += 1 << (m - i - 1);
for (k = j; k < n; k = k + n2) {
t1 = c * x[k + n1] - s * y[k + n1];
t2 = s * x[k + n1] + c * y[k + n1];
x[k + n1] = x[k] - t1;
y[k + n1] = y[k] - t2;
x[k] = x[k] + t1;
y[k] = y[k] + t2;
}
}
}
}
}
Warning: this code appears to be derived from here, and has a GPLv2 license.

Using the class at: https://www.ee.columbia.edu/~ronw/code/MEAPsoft/doc/html/FFT_8java-source.html
Short explanation: call fft() providing x as you amplitude data, y as all-zeros array, after the function returns your first answer will be a[0]=x[0]^2+y[0]^2.
Complete explanation: FFT is complex transform, it takes N complex numbers and produces N complex numbers. So x[0] is the real part of the first number, y[0] is the complex part. This function computes in-place, so when the function returns x and y will have the real and complex parts of the transform.
One typical usage is to calculate the power spectrum of audio. Your audio samples only have real part, you your complex part is 0. To calculate the power spectrum you add the square of the real and complex parts P[0]=x[0]^2+y[0]^2.
Also it's important to notice that the Fourier transform, when applied over real numbers, result in symmetrical result (x[0]==x[x.lenth-1]). The data at x[x.length/2] have the data from frequency f=0Hz. x[0]==x[x.length-1] has the data for a frequency equals to have the sampling rate (eg if you sampling was 44000Hz than it means f[0] refeers to 22kHz).
Full procedure:
create array p[n] with 512 samples with zeros
Collect 1024 audio samples, write them on x
Set y[n]=0 for all n
calculate fft(x,y)
calculate p[n]+=x[n+512]^2+y[n+512]^2 for all n=0 to 512
to go 2 to take another batch (after 50 batches go to next step)
plot p
go to 1
Than adjust the fixed number for your taste.
The number 512 defines the sampling window, I won't explain it. Just avoid reducing it too much.
The number 1024 must be always the double of the last number.
The number 50 defines you update rate. If your sampling rate is 44000 samples per second you update rate will be: R=44000/1024/50 = 0.85 seconds.

kissfft is a decent enough library that compiles on android. It has a more versatile license than FFTW (even though FFTW is admittedly better).
You can find an android binding for kissfft in libgdx https://github.com/libgdx/libgdx/blob/0.9.9/extensions/gdx-audio/src/com/badlogic/gdx/audio/analysis/KissFFT.java
Or if you would like a pure Java based solution try jTransforms
https://sites.google.com/site/piotrwendykier/software/jtransforms

Use this class (the one that EricLarch's answer is derived from).
Usage Notes
This function replaces your inputs arrays with the FFT output.
Input
N = the number of data points (the size of your input array, must be a power of 2)
X = the real part of your data to be transformed
Y = the imaginary part of the data to be transformed
i.e. if your input is
(1+8i, 2+3j, 7-i, -10-3i)
N = 4
X = (1, 2, 7, -10)
Y = (8, 3, -1, -3)
Output
X = the real part of the FFT output
Y = the imaginary part of the FFT output
To get your classic FFT graph, you will want to calculate the magnitude of the real and imaginary parts.
Something like:
public double[] fftCalculator(double[] re, double[] im) {
if (re.length != im.length) return null;
FFT fft = new FFT(re.length);
fft.fft(re, im);
double[] fftMag = new double[re.length];
for (int i = 0; i < re.length; i++) {
fftMag[i] = Math.pow(re[i], 2) + Math.pow(im[i], 2);
}
return fftMag;
}
Also see this StackOverflow answer for how to get frequencies if your original input was magnitude vs. time.

Yes, there is the JTransforms that is maintained on github here and avaiable as a Maven plugin here.
Use with:
compile group: 'com.github.wendykierp', name: 'JTransforms', version: '3.1'
But with more recent, Gradle versions you need to use something like:
dependencies {
...
implementation 'com.github.wendykierp:JTransforms:3.1'
}

#J Wang
Your output magnitude seems better than the answer given on the thread you have linked however that is still magnitude squared ... the magnitude of a complex number
z = a + ib
is calculated as
|z|=sqrt(a^2+b^2)
the answer in the linked thread suggests that for pure real inputs the outputs
should be using a2 or a for the output because the values for
a_(i+N/2) = -a_(i),
with b_(i) = a_(i+N/2) meaning the complex part in their table is in the second
half of the output table.
i.e the second half of the output table for an input table of reals is the conjugate of the real ...
so z = a-ia giving a magnitude
|z|=sqrt(2a^2) = sqrt(2)a
so it is worth noting the scaling factors ...
I would recommend looking all this up in a book or on wiki to be sure.

Unfortunately the top answer only works for Array that its size is a power of 2, which is very limiting.
I used the Jtransforms library and it works perfectly, you can compare it to the function used by Matlab.
here is my code with comments referencing how matlab transforms any signal and gets the frequency amplitudes (https://la.mathworks.com/help/matlab/ref/fft.html)
first, add the following in the build.gradle (app)
implementation 'com.github.wendykierp:JTransforms:3.1'
and here it is the code for for transforming a simple sine wave, works like a charm
double Fs = 8000;
double T = 1/Fs;
int L = 1600;
double freq = 338;
double sinValue_re_im[] = new double[L*2]; // because FFT takes an array where its positions alternate between real and imaginary
for( int i = 0; i < L; i++)
{
sinValue_re_im[2*i] = Math.sin( 2*Math.PI*freq*(i * T) ); // real part
sinValue_re_im[2*i+1] = 0; //imaginary part
}
// matlab
// tf = fft(y1);
DoubleFFT_1D fft = new DoubleFFT_1D(L);
fft.complexForward(sinValue_re_im);
double[] tf = sinValue_re_im.clone();
// matlab
// P2 = abs(tf/L);
double[] P2 = new double[L];
for(int i=0; i<L; i++){
double re = tf[2*i]/L;
double im = tf[2*i+1]/L;
P2[i] = sqrt(re*re+im*im);
}
// P1 = P2(1:L/2+1);
double[] P1 = new double[L/2]; // single-sided: the second half of P2 has the same values as the first half
System.arraycopy(P2, 0, P1, 0, L/2);
// P1(2:end-1) = 2*P1(2:end-1);
System.arraycopy(P1, 1, P1, 1, L/2-2);
for(int i=1; i<P1.length-1; i++){
P1[i] = 2*P1[i];
}
// f = Fs*(0:(L/2))/L;
double[] f = new double[L/2 + 1];
for(int i=0; i<L/2+1;i++){
f[i] = Fs*((double) i)/L;
}