I am trying to create a logistic regression algorithm in java but when I calculate the logarithm of the likelihood it is always returning NaN. My method which calculates the logarithm looks like this :
//Calculate log likelihood on given data
private double getLogLikelihood(double cat, double[] x) {
return cat * Math.log(findProbability(x))
+ (1 - cat) * Math.log(1 - findProbability(x));
}
And the findProbability method is just take an instance from the dataset and returning the sigmoid funcion result which is between 0 and 1.
//Calculate the sum of w * x for each weight and attribute
//call the sigmoid function with that s
public double findProbability(double[] x){
double s = 0;
for(int i = 0; i < this.weights.length; i++){
if(i >= x.length) break;
s += this.weights[i] * x[i];
}
return sigmoid(s);
}
private double sigmoid(double s){
return 1 / (1 + Math.exp(-s));
}
Moreover, my starting weights are :
[-0.2982955509135178, -0.4984900460081106, -1.816880187922516, -2.7325608512266073, 0.12542715714800834, 0.1516078084483485, 0.27631147403449774, 0.1371611094778011, 0.16029832096058613, 0.3117065974657231, 0.04262385176091778, 0.1948263133838624, 0.10788353525185314, 0.770608588466501, 0.2697281907888033, 0.09920694325563077, 0.003224073601703939, 0.021573742410541247, 0.21528348692817675, 0.3275511757298476, -0.1500597314893408, -0.7221692528386277, -2.062544912370121, 1.4315146889363015, 0.2522133355419722, 0.23919315019065995, 0.3200037377021523, 0.059466770771758076, 0.04012493980772944, 0.2553236501265919]
Finally, an instance from my dataset is :[M,17.99,10.38,122.8,1001,0.1184,0.2776,0.3001,0.1471,0.2419,0.07871,1.095,0.9053,8.589,153.4,0.006399,0.04904,0.05373,0.01587,0.03003,0.006193,25.38,17.33,184.6,2019,0.1622,0.6656,0.7119,0.2654,0.4601,0.1189]
I tried to initialize the starting weightss with different random numbers but thats didnt solve the problem.
The arithematic is causing a rounding error leaving you with 1.
double b = 1 + Math.exp(-3522);
b will be equal to 1, because otherwise you will need too many sig figs. You'll have to approximate the value to keep the precision. 1/(1+s) ~= 1 - s; Which means you need to calculate log(1) and log(s).
edit: sorry, I made a mistake, it appears Math.exp(-3522) is evaluated as 0 after rounding. Ill leave this answer because Math.exp(-x) might be too small to add to 1, or it might just be zero.
NaN is a result of dividing by zero or calling Math.log on a non-positive number, so u should try and find where exactly this happens. I suggest debugging or adding code to print the values of which u take the logarithm/dividy by.
EDIT: it seems it is a rounding error: exp(-s) will return a result so small that added with 1 it will still remain 1. This causes the logarithm to return -Inf. I'd suggest u try and find a mathematical way to solve this by trying to perhaps to approximate the log of the exponential.
I found a solution to my problem so I post it here:
I added an overflow check:
private double sigmoid(double s){
if(s>20){
s=20;
}else if(s<-20){
s=-20;
}
double exp = Math.exp(s);
return exp/(1+exp);
}
Also changing 1/(1+Math.exp(s) to exp/(1+exp) proved to be more stable in small disturbances of inputs.
Related
I'm havinga Problem with a method, which takes a Polynom like f(x)=x²+1 and calculates possible zero points with the newton algorithm.
I have given requirements for specific variables so even if the naming is not good or a variable is not needed I have to use them :/
The Polynom I give my method as a parameter is a double-array: For f(x)=x²+1 it would be {1.0,0.0,1.0}
so its constructed like 1.0*x^0 + 0.0*x^1+1.0*x^2
For my Code:
x0 is the start value for the newton algorithm and eps is for the accuracy of the calculation
I followed my given Instructions and got the following code working:
public static double newton(double[] a, double x0, double eps) {
double z;
double xn;
double xa = x0;
double zaehler;
double nenner;
do {
zaehler = horner(a, xa);
nenner = horner(ableit(a), xa);
if(nenner == 0) {
return Double.POSITIVE_INFINITY;
}
xn = xa - (zaehler/nenner);
xa = xn;
} while((Math.abs(horner(a, xn))) >= eps);
z = xn;
return 0;
}
the method horner() calculates the y-Value of a given function for a given x-Value.
My Problem is if the Function doesn't has a zero-point like x²+1 and I start with x0=1 and eps=0.1 I get Infinity returned.
But If I start with x0=10 and eps=0.1for example I create an endless loop.
How can I deal with this or is this a general Problem with the Newton Algorithm?!
Is the only way to set a fixed maximum of Iterations?
The Code is working for Polynoms that have at least one zero-point!
The Newton–Raphson method requires the existence of a real root x such that f(x)=0. The function you use x^2+1 has no real roots, so your algorithm will not work in this case (nor in others where there is no root).
Since x^2+1 >= 1 for all real x this implies horner(a, xn) >= 1, so the loop
while((Math.abs(horner(a, xn))) >= eps)
will not terminate for eps < 1.
Maybe before starting to iterate, you should check the existence of a zero.
E.g. if the highest (according to the power of x) nonzero coefficient is odd then there will be a real zero.
Or extend your algorithm such that it previously tries to find some real aand b such that f(a)f(b) <= 0 (then between a and b there is a root).
I must calculate sin(x) with a Taylor's series, until the output has 6 decimal places. The argument is an angle. I didn't implement checking the decimal places, I'm just printing next values (to check if it's working), but after 10-20 iteration it shows infinities/NaN's.
What's wrong in my thinking?
public static void sin(double x){
double sin = 0;
int n=1;
while(1<2){
sin += (Math.pow(-1,n) / factorial(2*n+1)) * Math.pow(x, 2*n+1);
n++;
try {
Thread.sleep(50);
} catch (InterruptedException ex) {
}
// CHECKING THE PRECISION HERE LATER
System.out.println(sin);
}
}
the Equation:
Don't compute each term using factorials and powers! You will rapidly overflow.
Just realize that each next term is -term * x * x / ((n+1)*(n+2)) where n increases by 2 for each term:
double tolerance = 0.0000007; // or whatever limit you want
double sin = 0.;
int n = 1;
double term = x;
while ( Math.abs(term) > tolerance ) {
sin += term;
term *= -( (x/(n+1)) * (x/(n+2)) );
n+= 2;
}
To add on to the answer provided by #Xoce (and #FredK), remember that you are computing the McLaurin series (special case of Taylor about x = 0). While this will converge fairly quickly for values that are within about pi/2 of zero, you may not get convergence of the digits before the factorial explodes for values of x further than that.
My recommendation is to use the actual Taylor series about the closest value of sin(x) for which the exact value is known (i.e., the nearest multiple of pi/2, not just about zero. And definitely do the convergence check!
Problem:
NAN error is normally a really big number, something that can happend if you divide 2 numbers but the divisor is very small, or zero.
Solution
This happens because your factorial number is getting an overflow, and later at some point you are dividing by zero again
if your factorial is taken as argument an int, then change it by , for example, a BIgInterger object.
I am trying to create a recursive method to compute a series in java 1/i.
Here is my code:
public static double computeSeries(int n) {
if (n == 1)
return 1;
else
return (1/n) + computeSeries(n - 1);
}
where 'n' is passed through the main method. However it doesn't exactly work correctly. FE when I type in 3, it returns 2.0, where I calculated it to be 1.8, and if I use 2, it gives me 1.0
While you're going to work with decimals, you might at least want to have a double as input
Solution
public static double computeSeries(double n)
However, if you only want the method to have an int as input, you might want to change 1/n to 1.0/n this will result to an operation of type double instead of int
This is called Promotion
JLS §5.6.2
If either operand is of type double, the other is converted to double.
First part of your result (1/n) is truncated to an int:
return (1/n) + computeSeries(n - 1);
Force calculations to be done in a double type by changing 1 to 1.0 (then it's a double):
return (1.0/n) + computeSeries(n - 1);
1/n
You are calculating in integers. This is always ZERO unless n == 1
Your first division will be 0 in most of the cases, since you're calculating with integers.
Instead use one of
return (1.0 / n) + computeSeries(n - 1);
return (1 / (double) n) + computeSeries(n - 1);
public static double computeSeries(double n) {
Bonus: You should take care of n = 0 to prevent a java.lang.ArithmeticException.
As most others have said, you need to convert the values to double before performing operations. Also, and this may just be personal preference, you may not want to have n calls of computeSeries running before they can all be completed.
Edit: After thinking more, using the for loop as I did below, there is no need for an extra method to calculate each term in the series. You can simply do the following
public static double computeSeries(int n) {
double sum = 0.0;
for(int i=1; i<=n; i++){
sum += (1.0/(double)i);
}
return sum;
}
I would like to know how I would go about writing this summation equation in java. But, the trick is, I need the summation to be equal to an amount.
x= Total Loss Streak amount
sb= Starting Bet
m= multiplier
The whole equation will equal to the current amount of currency in one's account. The amount of times the summation can complete itself while adding up needs to be less than or equal to the amount of currency in ones account.
Fyi, this is for a dicebot that work's on peerbet.org and I want to be able to show the user how many times he can loose in a row without wasting all his money.
If this question is bad, please do not answer it and let me delete it. Also, it thought the middle part was code, so I had to put it as such or it wouldn't let me post.
Renaming sb to just b. This is just a sum of a geometric progression
In Java, you can write:
return b * (m * m - Math.pow(m, x + 1)) / (1 - m);
This will be considerably faster than using a loop, although you must check that m is not 1.
If you want to solve for x given a sum S then a rearrangement of the formula gives the following Java code:
double x = Math.log(m * m - S * (1 - m) / b) / log(m) - 1;
and truncate this result to get the integral value of x where the next integer bankrupts the player.
EDIT: apparently we are solving for x. still easily doable with a loop.
int sum = 0;
int x =2;
while(sum<=amount){
sum+=sb*(Math.pow(m,x));
}
return x;
A summation is really just an adding for loop right?
int sum = 0;
for(int i=2; i<x; i++){
sum+=sb*(Math.pow(m,i));
}
return sum;
I'm not entirely clear I'm reading your formula correctly: are you summing up integers from 2 to x on the left-hand side of the equals sign, and you want that sum to be equal to the term on the right-hand side?
In that case, we could do the following transformation:
(Note that the first step might not be what you had in mind.)
We can now easily solve this using the quadratic formula to get:
Assuming that we're calculating in the reals, note that the root is only defined for non-negative arguments. The result of taking that root yields a non-negative number and substracting that non-negative number from -1 would give something <= -1, i.e., a negative number. Dividing it by 2 won't make it positive, either, but we've assumed from the get-go that our x must be >= 2, or else the very first sum wouldn't make any sense.
Therefore we can disregard the - case of the +/- in the formula altogether. Hence:
This should be straight-forward to translate into Java code, but note that the result is likely not to be an integer, so you will have to round if you're looking for an upper bound.
I'm trying to write a method that takes in a base k and a value n to 2 decimal places, then computes the log base k of n without using any of Java's Math.log methods. Here's what I have so far:
public static double log(double k, double n) {
double value = 0.0;
for(double i = 1; i > .001; i /= 10) {
while(!(Math.pow(k, value) >= n )) {
value += i;
}
}
return value;
}
The problem comes up when I try computing log base 4 of 5.0625, which returns 2.0, but should return 1.5.
I have no idea why this isn't working. Any help is appreciated.
No this is not homework, it's part of a problem set that I'm trying to solve for fun.
You're adding the amount i once too ofter. Thus you'll quite soon reach a value larger than the actual value and the while loop will never be entered again.
Subtract i once from the value and you'll be fine:
for(double i = 1; i > .001; i /= 10) {
while(!(Math.pow(k, value) > n )) {
value += i;
}
value -= i;
}
Step through the code on paper:
Iteration: i=1 value = 0.0, calculated power = 1
Iteration: i=1 value = 1.0, calculated power = 4
Iteration: i=1 value = 2.0, calculated power = 16
Now at this point, your value is 2.0. But at no point in the code to you have a way to correct back in the other direction. You need to check for both overshoot and undershoot cases.
This loop
while(!(Math.pow(k, value) >= n )) {
value += i;
}
goes too far. It only stops after the correct value has been surpassed. So when calculating the ones place, 1 isn't enough, so it goes to 2.0, and all subsequent tests show that it is at least enough, so that's where it ends.
Calculating logs by hand, what fun! I suggest doing it out on paper, then stepping through your code with watch variables or outputting each variable at each step. Then check this method out and see if it lines up with what you're doing: Link
You could always look at:
https://stackoverflow.com/a/2073928/251767
It provides an algorithm which will compute a log of any number in any base. It's a response to a question about calculating logs with BigDecimal types, but it could be adapted, pretty easily, to any floating-point type.
Since it uses squaring and dividing by two, instead of using multiple calls to Math.pow(), it should converge pretty quickly and use less CPU resources.