I'm havinga Problem with a method, which takes a Polynom like f(x)=x²+1 and calculates possible zero points with the newton algorithm.
I have given requirements for specific variables so even if the naming is not good or a variable is not needed I have to use them :/
The Polynom I give my method as a parameter is a double-array: For f(x)=x²+1 it would be {1.0,0.0,1.0}
so its constructed like 1.0*x^0 + 0.0*x^1+1.0*x^2
For my Code:
x0 is the start value for the newton algorithm and eps is for the accuracy of the calculation
I followed my given Instructions and got the following code working:
public static double newton(double[] a, double x0, double eps) {
double z;
double xn;
double xa = x0;
double zaehler;
double nenner;
do {
zaehler = horner(a, xa);
nenner = horner(ableit(a), xa);
if(nenner == 0) {
return Double.POSITIVE_INFINITY;
}
xn = xa - (zaehler/nenner);
xa = xn;
} while((Math.abs(horner(a, xn))) >= eps);
z = xn;
return 0;
}
the method horner() calculates the y-Value of a given function for a given x-Value.
My Problem is if the Function doesn't has a zero-point like x²+1 and I start with x0=1 and eps=0.1 I get Infinity returned.
But If I start with x0=10 and eps=0.1for example I create an endless loop.
How can I deal with this or is this a general Problem with the Newton Algorithm?!
Is the only way to set a fixed maximum of Iterations?
The Code is working for Polynoms that have at least one zero-point!
The Newton–Raphson method requires the existence of a real root x such that f(x)=0. The function you use x^2+1 has no real roots, so your algorithm will not work in this case (nor in others where there is no root).
Since x^2+1 >= 1 for all real x this implies horner(a, xn) >= 1, so the loop
while((Math.abs(horner(a, xn))) >= eps)
will not terminate for eps < 1.
Maybe before starting to iterate, you should check the existence of a zero.
E.g. if the highest (according to the power of x) nonzero coefficient is odd then there will be a real zero.
Or extend your algorithm such that it previously tries to find some real aand b such that f(a)f(b) <= 0 (then between a and b there is a root).
Related
We are often taught that floating-point numbers should not be compared for exact equality. However, the following function, which returns the Golden Ratio when passed any positive number, does in fact compare doubles for equality and to my surprise it seems to always work:
public static double f(double x) {
double y;
while ((y = 1 + 1 / x) != x)
x = (x + y) / 2;
return x;
}
#Test
void test() {
assertEquals((1 + sqrt(5)) / 2, f(1.0)); // Passes!
}
I thought that maybe it works for some input arguments but not others. But even if I use JQwik's property testing, it still works!
#Property
void test2(#ForAll #Positive double x) {
assertEquals((1 + sqrt(5)) / 2, f(x)); // Always passes!
}
Can anyone tell me why I never come across a situation where the two floating-point numbers are different by a very small amount?
You were just lucky, in general you don't get exact equality. Try this for example:
public static void main(String[] args) {
var s = 0.0;
for (int i = 0; i < 10; i++) {
s += 0.1;
}
System.out.println(s == 1.0);
}
In your concrete example one would have to do a careful analysis to prove that your iteration always converges to the floating point number closest to phi. If sqrt also returns the closest floating point number to the exact root we would get exact equality.
... and to my surprise it seems to always work:
Not always.
When I tried f(-1/1.6180339887498949), the x and y values oscillated between two floating point values that differed in the last few bits #Steve Summit. Thus an infinite loop.
x:-0.61803398874989490 y:-0.61803398874989468 // Decimal notation
x:-0x1.3c6ef372fe950p-1 y:-0x1.3c6ef372fe94ep-1 // Hex notation
x:-0.61803398874989479 y:-0.6180339887498949
x:-0x1.3c6ef372fe94fp-1 y:-0x1.3c6ef372fe950p-1
x:-0.61803398874989490 y:-0.61803398874989468
x:-0x1.3c6ef372fe950p-1 y:-0x1.3c6ef372fe94ep-1
f(some_starting_x) generally converges to render an x, such that 1 + 1 / x is x again and so meeting the stopping condition.
Better routines can prove that if x is reasonably close, the while loop will eventually get close to the desired answer, yet even then, an oscillation, as shown above is possible. Thus using an iteration limit or close enough test is needed. Usually the 2 oscillation values, when close, they are massaged (e.g. averaged) to form the best answer. If not close, the looping simply failed to find a stable answer.
Can anyone tell me why I never come across a situation where the two floating-point numbers are different by a very small amount?
Inadequate testing.
Morale of the story:
Do not rely on only floating point equality, except in select cases.
f() was not a select case and deserved additional stopping code.
Ref: Two x with math property: x = 1 + 1/x:
x1 = 1.6180339887498948482045868343656...
x2 = -0.61803398874989484820458683436564...
Note x1*x2 == -1. x1 is the Golden_ratio φ.
I have to write a program in which I write a,b c,d (coefficients of equation 3 degree) and as a result I should get X1, X2, X3 (solutions of equation). I have to use Viete's formulas and BigDecimal for this, because my lecturer requires it from me.
I came to the conclusion that I have to solve the following system of equations:
x1+x2+x3=-b/a
x1*x2+x1*x3+x2*x3=c/a
x1*x2*x3=-d/a
I have no idea how I can do it in Java.
I tried to use the JAMA package, but I don't think I can use it to solve such a system of equations.
How can I do that?
If you want to find the roots of a cubic polynomial in Java you can do it easily using Newton-Raphson's method.
The algorithm -
1. Input: initial x, func(x), derivFunc(x)
Output: Root of Func()
2. Compute values of func(x) and derivFunc(x) for given initial x
3. Compute h: h = func(x) / derivFunc(x)
4. While h is greater than allowed error ε
- h = func(x) / derivFunc(x)
- x = x – h
Here is a demonstration for solving the cubic equation x^3-x^2+2
class XYZ {
static final double EPSILON = 0.001;
// An example function whose solution
// is determined using Bisection Method.
// The function is x^3 - x^2 + 2
static double func(double x)
{
return x * x * x - x * x + 2;
}
// Derivative of the above function
// which is 3*x^x - 2*x
static double derivFunc(double x)
{
return 3 * x * x - 2 * x;
}
// Function to find the root
static void newtonRaphson(double x)
{
double h = func(x) / derivFunc(x);
while (Math.abs(h) >= EPSILON)
{
h = func(x) / derivFunc(x);
// x(i+1) = x(i) - f(x) / f'(x)
x = x - h;
}
System.out.print("The value of the"
+ " root is : "
+ Math.round(x * 100.0) / 100.0);
}
// Driver code
public static void main (String[] args)
{
// Initial values assumed
double x0 = -20;
newtonRaphson(x0);
}
}
Output - The value of root is : -1.00
To do it your way you have to solve a system of non-linear equations which is harder but can be done using the Newton Raphson's Multivariate method. You might want to look it up. Also note that this is an approximate method and guesses the roots after you put an initial 'guess' of your own (in this case its -20)
The Newton (Raphson, Kantorovich) method for the Viete equations gives you the (Weierstrass-)Durand-Kerner method of simultaneous root approximation. However, in the completed method you will no longer see the Viete identities, they kind of cancel out. You will need complex numbers over the demanded real numbers data type.
If you go with the simple Newton method like in the other answer, then after computing the one real root you can split off the linear factor belonging to it via the Horner-Ruffini scheme and then solve the remaining quadratic equation directly. Then you only need to consider the possible complex nature of the roots in constructing the output strings, as the real and imaginary parts have easy direct formulas.
I am trying to create a logistic regression algorithm in java but when I calculate the logarithm of the likelihood it is always returning NaN. My method which calculates the logarithm looks like this :
//Calculate log likelihood on given data
private double getLogLikelihood(double cat, double[] x) {
return cat * Math.log(findProbability(x))
+ (1 - cat) * Math.log(1 - findProbability(x));
}
And the findProbability method is just take an instance from the dataset and returning the sigmoid funcion result which is between 0 and 1.
//Calculate the sum of w * x for each weight and attribute
//call the sigmoid function with that s
public double findProbability(double[] x){
double s = 0;
for(int i = 0; i < this.weights.length; i++){
if(i >= x.length) break;
s += this.weights[i] * x[i];
}
return sigmoid(s);
}
private double sigmoid(double s){
return 1 / (1 + Math.exp(-s));
}
Moreover, my starting weights are :
[-0.2982955509135178, -0.4984900460081106, -1.816880187922516, -2.7325608512266073, 0.12542715714800834, 0.1516078084483485, 0.27631147403449774, 0.1371611094778011, 0.16029832096058613, 0.3117065974657231, 0.04262385176091778, 0.1948263133838624, 0.10788353525185314, 0.770608588466501, 0.2697281907888033, 0.09920694325563077, 0.003224073601703939, 0.021573742410541247, 0.21528348692817675, 0.3275511757298476, -0.1500597314893408, -0.7221692528386277, -2.062544912370121, 1.4315146889363015, 0.2522133355419722, 0.23919315019065995, 0.3200037377021523, 0.059466770771758076, 0.04012493980772944, 0.2553236501265919]
Finally, an instance from my dataset is :[M,17.99,10.38,122.8,1001,0.1184,0.2776,0.3001,0.1471,0.2419,0.07871,1.095,0.9053,8.589,153.4,0.006399,0.04904,0.05373,0.01587,0.03003,0.006193,25.38,17.33,184.6,2019,0.1622,0.6656,0.7119,0.2654,0.4601,0.1189]
I tried to initialize the starting weightss with different random numbers but thats didnt solve the problem.
The arithematic is causing a rounding error leaving you with 1.
double b = 1 + Math.exp(-3522);
b will be equal to 1, because otherwise you will need too many sig figs. You'll have to approximate the value to keep the precision. 1/(1+s) ~= 1 - s; Which means you need to calculate log(1) and log(s).
edit: sorry, I made a mistake, it appears Math.exp(-3522) is evaluated as 0 after rounding. Ill leave this answer because Math.exp(-x) might be too small to add to 1, or it might just be zero.
NaN is a result of dividing by zero or calling Math.log on a non-positive number, so u should try and find where exactly this happens. I suggest debugging or adding code to print the values of which u take the logarithm/dividy by.
EDIT: it seems it is a rounding error: exp(-s) will return a result so small that added with 1 it will still remain 1. This causes the logarithm to return -Inf. I'd suggest u try and find a mathematical way to solve this by trying to perhaps to approximate the log of the exponential.
I found a solution to my problem so I post it here:
I added an overflow check:
private double sigmoid(double s){
if(s>20){
s=20;
}else if(s<-20){
s=-20;
}
double exp = Math.exp(s);
return exp/(1+exp);
}
Also changing 1/(1+Math.exp(s) to exp/(1+exp) proved to be more stable in small disturbances of inputs.
I am brand new to Java, second day! I want generate samples with normal distribution. I am using inverse transformation.
Basically, I want to find the inverse normal cumulative distribution, then find its inverse. And generate samples.
My questions is: Is there a built-in function for inverse normal cdf? Or do I have to hand code?
I have seen people refer to this on apache commons. Is this a built-in? Or do I have to download it?
If I have to do it myself, can you give me some tips? If I download, doesn't my prof also have to have the "package" or special file installed?
Thanks in advance!
Edit:Just found I can't use libraries, also heard there is simpler way converting normal using radian.
As it is mentioned here:
Apache Commons - Math has what you are looking for.
More specifically, check out the NormalDistrubitionImpl class.
And no your professor doesn't need to download stuff if you provide him with all the needed libraries.
UPDATE :
If you want to hand code it (I don't know the actual formula), you can check the following link:
http://home.online.no/~pjacklam/notes/invnorm/
There are 2 people who implemented it in java: http://home.online.no/~pjacklam/notes/invnorm/#Java
I had had the same problem and find its solution, the following code will give results for cumulative distribution function just like excel do:
private static double erf(double x)
{
//A&S formula 7.1.26
double a1 = 0.254829592;
double a2 = -0.284496736;
double a3 = 1.421413741;
double a4 = -1.453152027;
double a5 = 1.061405429;
double p = 0.3275911;
x = Math.abs(x);
double t = 1 / (1 + p * x);
//Direct calculation using formula 7.1.26 is absolutely correct
//But calculation of nth order polynomial takes O(n^2) operations
//return 1 - (a1 * t + a2 * t * t + a3 * t * t * t + a4 * t * t * t * t + a5 * t * t * t * t * t) * Math.Exp(-1 * x * x);
//Horner's method, takes O(n) operations for nth order polynomial
return 1 - ((((((a5 * t + a4) * t) + a3) * t + a2) * t) + a1) * t * Math.exp(-1 * x * x);
}
public static double NORMSDIST(double z)
{
double sign = 1;
if (z < 0) sign = -1;
double result=0.5 * (1.0 + sign * erf(Math.abs(z)/Math.sqrt(2)));
return result;
}
Mathematically, this is a hard problem, and there are a few solutions you might consider.
Dislcaimer: Mathematical jargon ahead.
As you probably already know, the normalcdf function is used to calculate probabilities of normal random variables. Because a normal distribution is continuous, the corresponding probability density function (normalpdf) does not itself give probabilities, (in contrast to discrete distributions like binomial or geometric distributions). Instead, the area under the curve gives the probability that the normal random variable falls within a range of values. So, the normalcdf function you seek is the area under a section of the normalpdf function.
Mathematically, finding the area under a continuous curve is a fundamental problem of calculus. The solution to this type of problem is called an integral and integrating a function over a range of numbers means finding the area under the curve and between the lowest value in the range to the highest.
In most circumstances, we could just integrate the pdf function to get the cdf function, then evaluate it wherever we want. The heart of the problem, and the reason that an algorithm in Java is not as simple as one might think, is that normalpdf function does not have a closed form integral- it's value cannot be calculated in any finite number of steps. So, values of the normalcdf function are particularly elusive.
There are two main classes of solutions for the problem.
1. Numerical Integration Techniques
Numerical integration techniques solve the problem by approximating the area under the curve geometrically. The area is divided into rectangles or other shapes of equal or varying widths, with the height of each being given by the pdf function. The sum of the areas of the rectangle is an approximation of the area under the curve, which is the corresponding probability. These technique can be used to compute values to arbitrary precision, but is more computationally expensive than class 2. Using better approximations (e.g. Simpson's rule) improves computation. A simple numeric integration method follows.
public static double normCDF(double z)
{ double LeftEndpoint = -100;
int nRectangles = 100000;
double runningSum = 0;
double x;
for(int n = 0; n < nRectangles; n++){
x = LeftEndpoint + n*(z-LeftEndpoint)/nRectangles;
runningSum += Math.pow(Math.sqrt(2*Math.PI),-1)*Math.exp(-Math.pow(x,2)/2)*(z-LeftEndpoint)/nRectangles;
}
System.out.println(runningSum);
return runningSum;
}
2. Analytic Techniques
Analytic techniques take advantage of the fact that while the normalpdf does not have a closed-form integral, the pdf can be "converted" to a sum called a Taylor series, then integrated term-by-term. Basically, it turns the pdf into a sum of infinitely many simple functions, then integrates each one analytically, then adds together all of the integrals. Since this is an analytic procedure, a programmer need only include the integral series in the program after computing the coefficients. The precision of the result just depends on how many terms of the sum you include in the calculation, and tends to approach accurate values much sooner than numerical integration techniques. For example, the solution by Mohammad Aldefrawy computes just five coefficients. Below is a method that includes the computation of coefficients, so you one could compute values to arbitrary precision (Actually, the normalcdf series isn't computed directly. Instead, the coefficients of the related error function are computed then converted by a linear transformation). However, since computation of the coefficients involves the factorial function, one experiences memory issues for substantially large numbers of coefficients. Thankfully, this method returns values with much higher precision in a fraction of the iterations required by methods in the previous class of solutions to yield similar results.
public static double normalCDF(double x){
System.out.println(0.5*(1+erf(x/Math.sqrt(2))));
return 0.5*(1+erf(x/Math.sqrt(2)));
}
public static double erf(double z)
{
int nTerms = 315;
double runningSum = 0;
for(int n = 0; n < nTerms; n++){
runningSum += Math.pow(-1,n)*Math.pow(z,2*n+1)/(factorial(n)*(2*n+1));
}
return (2/Math.sqrt(Math.PI))*runningSum;
}
static double factorial(int n){
if(n == 0) return 1;
if(n == 1) return 1;
return n*factorial(n-1);
}
Other functions
For the inverse function, since we used the error function in the normalCDF method, we can use the inverse error function in a similar way. Again, we obtain the coefficients of the inverse error function analytically, then compute them as needed in the method.
public static double invErf(double z)
{
int nTerms = 315;
double runningSum = 0;
double[] a = new double[nTerms + 1];
double[] c = new double[nTerms + 1];
c[0]=1;
for(int n = 1; n < nTerms; n++){
double runningSum2=0;
for (int k = 0; k <= n-1; k++){
runningSum2 += c[k]*c[n-1-k]/((k+1)*(2*k+1));
}
c[n] = runningSum2;
runningSum2 = 0;
}
for(int n = 0; n < nTerms; n++){
a[n] = c[n]/(2*n+1);
runningSum += a[n]*Math.pow((0.5)*Math.sqrt(Math.PI)*z,2*n+1);
}
return runningSum;
}
public static double invNorm(double A){
return (2/Math.sqrt(2))*invErf(2*A-1);
}
I don't have a method for the lognormal function, but you could obtain one using the same idea.
I never tried it but the guys from algo team were using Colt and they were happy with the results.
http://introcs.cs.princeton.edu/java/13flow/Sqrt.java.html:
public class Sqrt {
public static void main(String[] args) {
// read in the command-line argument
double c = Double.parseDouble(args[0]);
double epsilon = 1e-15; // relative error tolerance
double t = c; // estimate of the square root of c
// repeatedly apply Newton update step until desired precision is achieved
while (Math.abs(t - c/t) > epsilon*t) {
t = (c/t + t) / 2.0;
}
// print out the estimate of the square root of c
System.out.println(t);
}
}
The thing is..I understand perfectly well how the program works itself. The problem I have is with the equation f(x) = x^2 - c and how that relates to the code above. Like, why divide it by x so that x(x - c/x)? There seems to be a missing mathematical explanation when it comes to some of these examples. In other words, I'm looking for an explanation from a simple mathematical stand point, NOT coding as so much.
You are given c and you want to solve
t = sqrt(c)
or equivalently,
c = t^2
or then again,
c - t^2 = 0.
I'll call the above equation f(t) = 0 (no mention of c since it is a given constant).
Newton method iterates over trial values of t, which I'll label t_i, t_{i+1}, ....
The Taylor expansion to 1st order is:
f(t_i + dt_i) = f(t_i) + dt_i * f'(t_i) + ...
So if you don't quite have f(t_i) = 0, you add a dt_i such that
f(t_i + dt_i) nearly = 0 = f(t_i) + dt_i * f'(t_i) + ...
So dt_i = -f(t_i) / f'(t_i), i.e. f(t_i + -f(t_i) / f'(t_i)) is closer to zero than f(t_i).
If you do the derivatives for f(t) = c - t^2, you'll see that the equation in the code t_{i+1} = (c / t_i + t_i) / 2 is just the iterative formula t_{i+1} = t_i + dt_i with the dt_i estimated above.
This is iterative method, so it does not give an exact solution. You need to decide when you want to stop (sufficient precision), otherwise the algorithm would go on forever. That's why you check f(t_i) < threshold instead of the true f(t_i) = 0. In their case they chose a threshold = epsilon * t^2; I think the multiplication by t^2 was used because if you used a fixed constant as a threshold, you might run into numerical accuracy problems (i.e. if you are playing with trillions, you could never get an fixed accuracy of 10^{-10} due to the finite precision of floating point representation.)
Based on the code, the following has already been explained on the Javadoc Comment:
* Computes the square root of a nonnegative number c using
* Newton's method:
* - initialize t = c
* - replace t with the average of c/t and t
* - repeat until desired accuracy reached
Ok, I'll give it a bash (see inline comments):
public class Sqrt {
public static void main(String[] args) {
// read in the command-line argument (i.e. this is the value that we want
// square root from.)
double c = Double.parseDouble(args[0]);
// Since the the square root of non-squares are irrational, we need some
// error tolerance. In other words, if the answer is less than epsilon wrong
// we'll take it.
double epsilon = 1e-15; // relative error tolerance
// t is our first guess (c / 2.0 works well too - in fact it tends to be
// better.)
double t = c; // estimate of the square root of c
// repeatedly apply Newton update step until desired precision is achieved
// The condition here is rather elegant and optimized... to see why it works,
// simply break it up. The absolute is there to cater for negative values, but
// for c >= 0:
// | c - c/t | > epsilon * t
// t * ( t - c / t ) > epsilon
// tt - c = > epsilon)
while (Math.abs(t - c/t) > epsilon*t) {
// Improve the guess by applying Newton's genius :-)
// Take the original number, divide by the guess add t and take the
// average.
t = ( c / t + t) / 2.0;
}
// print out the estimate of the square root of c
System.out.println(t);
}
}
ejlab.net jelmar
I believe the above mentioned code is from R.Sedgewick's book 'Introduction to Programming in Java', page 62. What he tries to say in the book is that you can use f(x)=x^2-c as a special case to find the square root of any positive number. So how it works:
Newton's method states X(n+1)=X(n)-(F(X(n))/F'(X(n))). Assume that in F(X)=X^2-C, where C=2 since we are looking for square root of 2 (if you want to find square root of 36 then C=36 etc). Then the first derivative of the function F(X) is F'(X)=2X. Applying the Newton's method we get
X(n+1)=X(n)-((X^2-C)/(2X))
for X(0)=2 we get
n=1, X(1)=2-(2^2-2)/(2*2) > X(1)=1.5;
n=2 X(2)=1.5 -(1.5^2-2)/(2*1.5) > X(2)=1.41666667
and so on...