I want to get data from the database (MySQL) by JPA, I want it sorted by some column value.
So, what is the best practice, to:
Retrieve the data from the database as list of objects (JPA), then
sort it programmatically using some java APIs.
OR
Let the database sort it by using a sorting select query.
Thanks in advance
If you are retrieving a subset of all the database data, for example displaying 20 rows on screen out of 1000, it is better to sort on the database. This will be faster and easier and will allow you to retrieve one page of rows (20, 50, 100) at a time instead of all of them.
If your dataset is fairly small, sorting in your code may be more convenient if you want implement a complex sort. Usually this complex sort can be done in SQL but not as easily as in code.
The short of it is, the rule of thumb is sort via SQL, with some edge cases to the rule.
In general, you're better off using ORDER BY in your SQL query -- this way, if there is an applicable index, you may be getting your sorting "for free" (worst case, it will be the same amount of work as doing it in your code, but often it may be less work than that!).
I ran into this very same question, and decided that I should run a little benchmark to quantify the speed differences. The results surprised me. I would like to post my experience with this very sort of question.
As with a number of the other posters here, my thought was that the database layer would do the sort faster because they are supposedly tuned for this sort of thing. #Alex made a good point that if the database already has an index on the sort, then it will be faster. I wanted to answer the question which raw sorting is faster on non-indexed sorts. Note, I said faster, not simpler. I think in many cases letting the db do the work is simpler and less error prone.
My main assumption was that the sort would fit in main memory. Not all problems will fit here, but a good number do. For out of memory sorts, it may well be that databases shine here, though I did not test that. In the case of in memory sorts all of java/c/c++ outperformed mysql in my informal benchmark, if one could call it that.
I wish I had had more time to more thoroughly compare the database layer vs application layer, but alas other duties called. Still, I couldn't help but record this note for others who are traveling down this road.
As I started down this path I started to see more hurdles. Should I compare data transfer? How? Can I compare time to read db vs time to read a flat file in java? How to isolate the sort time vs data transfer time vs time to read the records? With these questions here was the methodology and timing numbers I came up with.
All times in ms unless otherwise posted
All sort routines were the defaults provided by the language (these are good enough for random sorted data)
All compilation was with a typical "release-profile" selected via netbeans with no customization unless otherwise posted
All tests for mysql used the following schema
mysql> CREATE TABLE test_1000000
(
pk bigint(11) NOT NULL,
float_value DOUBLE NULL,
bigint_value bigint(11) NULL,
PRIMARY KEY (pk )
) Engine MyISAM;
mysql> describe test_1000000;
+--------------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+--------------+------------+------+-----+---------+-------+
| pk | bigint(11) | NO | PRI | NULL | |
| float_value | double | YES | | NULL | |
| bigint_value | bigint(11) | YES | | NULL | |
+--------------+------------+------+-----+---------+-------+
First here is a little snippet to populate the DB. There may be easier ways, but this is what I did:
public static void BuildTable(Connection conn, String tableName, long iterations) {
Random ran = new Random();
Math.random();
try {
long epoch = System.currentTimeMillis();
for (long i = 0; i < iterations; i++) {
if (i % 100000 == 0) {
System.out.println(i + " next 100k");
}
PerformQuery(conn, tableName, i, ran.nextDouble(), ran.nextLong());
}
} catch (Exception e) {
logger.error("Caught General Exception Error from main " + e);
}
}
MYSQL Direct CLI results:
select * from test_10000000 order by bigint_value limit 10;
10 rows in set (2.32 sec)
These timings were somewhat difficult as the only info I had was the time reported after the execution of the command.
from mysql prompt for 10000000 elements it is roughly 2.1 to 2.4 either for sorting bigint_value or float_value
Java JDBC mysql call (similar performance to doing sort from mysql cli)
public static void SortDatabaseViaMysql(Connection conn, String tableName) {
try {
Statement stmt = conn.createStatement();
String cmd = "SELECT * FROM " + tableName + " order by float_value limit 100";
ResultSet rs = stmt.executeQuery(cmd);
} catch (Exception e) {
}
}
Five runs:
da=2379 ms
da=2361 ms
da=2443 ms
da=2453 ms
da=2362 ms
Java Sort Generating random numbers on fly (actually was slower than disk IO read). Assignment time is the time to generate random numbers and populate the array
Calling like
JavaSort(10,10000000);
Timing results:
assignment time 331 sort time 1139
assignment time 324 sort time 1037
assignment time 317 sort time 1028
assignment time 319 sort time 1026
assignment time 317 sort time 1018
assignment time 325 sort time 1025
assignment time 317 sort time 1024
assignment time 318 sort time 1054
assignment time 317 sort time 1024
assignment time 317 sort time 1017
These results were for reading a file of doubles in binary mode
assignment time 4661 sort time 1056
assignment time 4631 sort time 1024
assignment time 4733 sort time 1004
assignment time 4725 sort time 980
assignment time 4635 sort time 980
assignment time 4725 sort time 980
assignment time 4667 sort time 978
assignment time 4668 sort time 980
assignment time 4757 sort time 982
assignment time 4765 sort time 987
Doing a buffer transfer results in much faster runtimes
assignment time 77 sort time 1192
assignment time 59 sort time 1125
assignment time 55 sort time 999
assignment time 55 sort time 1000
assignment time 56 sort time 999
assignment time 54 sort time 1010
assignment time 55 sort time 999
assignment time 56 sort time 1000
assignment time 55 sort time 1002
assignment time 56 sort time 1002
C and C++ Timing results (see below for source)
Debug profile using qsort
assignment 0 seconds 110 milliseconds Time taken 2 seconds 340 milliseconds
assignment 0 seconds 90 milliseconds Time taken 2 seconds 340 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 330 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 340 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 330 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 340 milliseconds
assignment 0 seconds 90 milliseconds Time taken 2 seconds 340 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 330 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 340 milliseconds
assignment 0 seconds 100 milliseconds Time taken 2 seconds 330 milliseconds
Release profile using qsort
assignment 0 seconds 100 milliseconds Time taken 1 seconds 600 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 600 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 580 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 590 milliseconds
assignment 0 seconds 80 milliseconds Time taken 1 seconds 590 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 590 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 600 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 590 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 600 milliseconds
assignment 0 seconds 90 milliseconds Time taken 1 seconds 580 milliseconds
Release profile Using std::sort( a, a + ARRAY_SIZE );
assignment 0 seconds 100 milliseconds Time taken 0 seconds 880 milliseconds
assignment 0 seconds 90 milliseconds Time taken 0 seconds 870 milliseconds
assignment 0 seconds 90 milliseconds Time taken 0 seconds 890 milliseconds
assignment 0 seconds 120 milliseconds Time taken 0 seconds 890 milliseconds
assignment 0 seconds 90 milliseconds Time taken 0 seconds 890 milliseconds
assignment 0 seconds 90 milliseconds Time taken 0 seconds 880 milliseconds
assignment 0 seconds 90 milliseconds Time taken 0 seconds 900 milliseconds
assignment 0 seconds 90 milliseconds Time taken 0 seconds 890 milliseconds
assignment 0 seconds 100 milliseconds Time taken 0 seconds 890 milliseconds
assignment 0 seconds 150 milliseconds Time taken 0 seconds 870 milliseconds
Release profile Reading random data from file and using std::sort( a, a + ARRAY_SIZE )
assignment 0 seconds 50 milliseconds Time taken 0 seconds 880 milliseconds
assignment 0 seconds 40 milliseconds Time taken 0 seconds 880 milliseconds
assignment 0 seconds 50 milliseconds Time taken 0 seconds 880 milliseconds
assignment 0 seconds 50 milliseconds Time taken 0 seconds 880 milliseconds
assignment 0 seconds 40 milliseconds Time taken 0 seconds 880 milliseconds
Below is the source code used. Hopefully minimal bugs :)
Java source
Note that internal to JavaSort the runCode and writeFlag need to be adjusted depending on what you want to time. Also note that the memory allocation happens in the for loop (thus testing GC, but I did not see any appreciable difference moving the allocation outside the loop)
public static void JavaSort(int iterations, int numberElements) {
Random ran = new Random();
Math.random();
int runCode = 2;
boolean writeFlag = false;
for (int j = 0; j < iterations; j++) {
double[] a1 = new double[numberElements];
long timea = System.currentTimeMillis();
if (runCode == 0) {
for (int i = 0; i < numberElements; i++) {
a1[i] = ran.nextDouble();
}
}
else if (runCode == 1) {
//do disk io!!
try {
DataInputStream in = new DataInputStream(new FileInputStream("MyBinaryFile.txt"));
int i = 0;
//while (in.available() > 0) {
while (i < numberElements) { //this should be changed so that I always read in the size of array elements
a1[i++] = in.readDouble();
}
}
catch (Exception e) {
}
}
else if (runCode == 2) {
try {
FileInputStream stream = new FileInputStream("MyBinaryFile.txt");
FileChannel inChannel = stream.getChannel();
ByteBuffer buffer = inChannel.map(FileChannel.MapMode.READ_ONLY, 0, inChannel.size());
//int[] result = new int[500000];
buffer.order(ByteOrder.BIG_ENDIAN);
DoubleBuffer doubleBuffer = buffer.asDoubleBuffer();
doubleBuffer.get(a1);
}
catch (Exception e) {
}
}
if (writeFlag) {
try {
DataOutputStream out = new DataOutputStream(new FileOutputStream("MyBinaryFile.txt"));
for (int i = 0; i < numberElements; i++) {
out.writeDouble(a1[i]);
}
} catch (Exception e) {
}
}
long timeb = System.currentTimeMillis();
Arrays.sort(a1);
long timec = System.currentTimeMillis();
System.out.println("assignment time " + (timeb - timea) + " " + " sort time " + (timec - timeb));
//delete a1;
}
}
C/C++ source
#include <iostream>
#include <vector>
#include <algorithm>
#include <fstream>
#include <cstdlib>
#include <ctime>
#include <cstdio>
#include <math.h>
#include <stdio.h>
#include <time.h>
#include <stdlib.h>
#define ARRAY_SIZE 10000000
using namespace std;
int compa(const void * elem1, const void * elem2) {
double f = *((double*) elem1);
double s = *((double*) elem2);
if (f > s) return 1;
if (f < s) return -1;
return 0;
}
int compb (const void *a, const void *b) {
if (*(double **)a < *(double **)b) return -1;
if (*(double **)a > *(double **)b) return 1;
return 0;
}
void timing_testa(int iterations) {
clock_t start = clock(), diffa, diffb;
int msec;
bool writeFlag = false;
int runCode = 1;
for (int loopCounter = 0; loopCounter < iterations; loopCounter++) {
double *a = (double *) malloc(sizeof (double)*ARRAY_SIZE);
start = clock();
size_t bytes = sizeof (double)*ARRAY_SIZE;
if (runCode == 0) {
for (int i = 0; i < ARRAY_SIZE; i++) {
a[i] = rand() / (RAND_MAX + 1.0);
}
}
else if (runCode == 1) {
ifstream inlezen;
inlezen.open("test", ios::in | ios::binary);
inlezen.read(reinterpret_cast<char*> (&a[0]), bytes);
}
if (writeFlag) {
ofstream outf;
const char* pointer = reinterpret_cast<const char*>(&a[0]);
outf.open("test", ios::out | ios::binary);
outf.write(pointer, bytes);
outf.close();
}
diffa = clock() - start;
msec = diffa * 1000 / CLOCKS_PER_SEC;
printf("assignment %d seconds %d milliseconds\t", msec / 1000, msec % 1000);
start = clock();
//qsort(a, ARRAY_SIZE, sizeof (double), compa);
std::sort( a, a + ARRAY_SIZE );
//printf("%f %f %f\n",a[0],a[1000],a[ARRAY_SIZE-1]);
diffb = clock() - start;
msec = diffb * 1000 / CLOCKS_PER_SEC;
printf("Time taken %d seconds %d milliseconds\n", msec / 1000, msec % 1000);
free(a);
}
}
/*
*
*/
int main(int argc, char** argv) {
printf("hello world\n");
double *a = (double *) malloc(sizeof (double)*ARRAY_SIZE);
//srand(1);//change seed to fix it
srand(time(NULL));
timing_testa(5);
free(a);
return 0;
}
This is not completely on point, but I posted something recently that relates to database vs. application-side sorting. The article is about a .net technique, so most of it likely won't be interesting to you, but the basic principles remain:
Deferring sorting to the client side (e.g. jQuery, Dataset/Dataview sorting) may look tempting. And it actually is a viable option for paging, sorting and filtering, if (and only if):
1. the set of data is small, and
1. there is little concern about performance and scalability
From my experience, the systems that meet this kind of criteria are few and far between. Note that it’s not possible to mix and match sorting/paging in the application/database—if you ask the database for an unsorted 100 rows of data, then sort those rows on the application side, you’re likely not going to get the set of data you were expecting. This may seem obvious, but I’ve seen the mistake made enough times that I wanted to at least mention it.
It is much more efficient to sort and filter in the database for a number of reasons. For one thing, database engines are highly optimized for doing exactly the kind of work that sorting and filtering entail; this is what their underlying code was designed to do. But even barring that—even assuming you could write code that could match the kind of sorting, filtering and paging performance of a mature database engine—it’s still preferable to do this work in the database, for the simple reason that it’s more efficient to limit the amount of data that is transferred from the database to the application server.
So for example, if you have 10,000 rows before filtering, and your query pares that number down to 75, filtering on the client results in the data from all 10,000 rows being passed over the wire (and into your app server’s memory), where filtering on the database side would result in only the filtered 75 rows being moved between database and application. his can make a huge impact on performance and scalability.
The full post is here:
http://psandler.wordpress.com/2009/11/20/dynamic-search-objects-part-5sorting/
I'm almost positive that it will be faster to allow the Database to sort it. There's engineers who spend a lot of time perfecting and optimizing their search algorithms, whereas you'll have to implement your own sorting algorithm which might add a few more computations.
I would let the database do the sort, they are generally very good at that.
Let the database sort it. Then you can have paging with JPA easily without readin in the whole resultset.
Well, there is not really a straightforward way to answer this; it must be answered in the context.
Is your application (middle tier) is running in the same node as the database?
If yes, you do not have to worry about the latency between the database and middle tier. Then the question becomes: How big is the subset/resultset of your query? Remember that to sort this is middle tier, you will take a list/set of size N, and either write a custom comparator or use the default Collection comparator. Or, whatever. So at the outset, you are setback by the size N.
But if the answer is no, then you are hit by the latency involved in transferring your resultset from DB to middle tier. And then if you are performing pagination, which is the last thing you should do, you are throwing away 90-95% of that resultset after cutting the pages.
So the wasted bandwidth cannot be justified. Imagine doing this for every request, across your tenant organizations.
However way you look at it, this is bad design.
I would do this in the database, no matter what. Just because almost all applications today demand pagination; even if they don't sending massive resultsets over the wire to your client is a total waste; drags everybody down across all your tenants.
One interesting idea that I am toying with these days is to harness the power of HTML5, 2-way data binding in browser frameworks like Angular, and push some processing back to the browser. That way, you dont end up waiting in the line for someone else before you to finish. True distributed processing. But care must be taken in deciding what can be pushed and what not.
Depends on the context.
TL;DR
If you have the full data in your application server, do it in the application server.
If you have the full dataset that you need on the application server side already then it is better to do it on the application server side because those servers can scale horizontally. The most likely scenarios for this are:
the data set you're retrieving from the database is small
you cached the data on the application server side on startup
You're doing event sourcing and you're building up the data in the application server side anyway.
Don't do it on client side unless you can guarantee it won't impact the client devices.
Databases themselves may be optimized, but if you can pull burden away from them you can reduce your costs overall because scaling the databases up is more expensive than scaling up application servers.
Related
During the last months I created in Java some classes implementing data structures, more specifically Lists, Binary Search Trees and Binary Heaps. I decided to make a stress test by creating an Integer array of n values between 0 and 10*n, then sorting in various ways and measuring the times.
Initially it was just a curiosity. Obviously I expected my classes to costs far more than the normal Arrays.sort() method. However when I did the tests and compared my classes against each other I found some unexpected surprises.
This is the list of the tests, with details and comments.
1. A copy of the array is created, then the copy is sorted using the Arrays.sort() method, native of Java. The time evaluated is that of the Arrays.sort() method, i.e. the creation of the copy of the array doesn't count. It is the fastest way, as expected.
2. A list is created from the array, then the list is sorted using the Insertion Sort algorithm. The time evaluated is that of the sorting method, i.e. the creation of the list from the array doesn't count. Since the performance of Insertion Sort is not exactly great, this method costs around 50 times the array method.
3. A Binary Search Trees (BST from now on) is created from the array repeating the add() method. The BST is not a balanced tree like the AVL or the Red-Black, merely a normal BST like found on Wikipedia: every node has links to three other nodes (parent, leftChild and rightChild), incapsulates a value, etc... This method costs arount 500 times the list method, i.e. 25 000 times the array method.
4-5. Two Binary Heap (BH_1 and BH_2 from now on) are created from the array repeating the add() method, then they are converted into two (sorted) array repeating the extractMin() method. Both BH belongs to the same class and stores the value in a Vector<Integer>. Both BH costs around 2 times the BST and 50 000 times the array method. However there is a twist.
BH_2 creates the array using the method convertHeapToArray() of the Heap<Integer> interface. convertHeapToArray() calls the method extractMin() for n times, while extractMin() in turn calls the method heapify() one time.
This doesn't happen in BH_1, which uses the method convertHeapToArray_1(). Instead of calling extractMin(), my "new" method directly executes the code of extractMin() - and when extractMin() calls the method heapify(), BH_1 instead executed its code. In short, a copy-paste that avoids a few calls.
In theory BH_1 should always costs less than BH_2: same input data, same code, less method calling. However this is true only 73% of the times!
My questions are the following:
1. Why the Binary Search Trees sorting method (computational complexity n log(n), expected to be balanced if created by add() only) costs 500 times an Insertion Sort (computational complexity n2, is preferable if n is less than 23)
2. Why the Binary Heaps (same computational complexity of Binary Search Trees, specifically designed to sort quickly) costs 2 times a Binary Search Trees?
3. And, more baffling than ever, why making less calls is more costly than making more calls in one case out of 4?!
Code for convertHeapToArray():
public default T[] convertHeapToArray(){
T[] output = AArrays.createTarray(length(), min());
for(int i=0 ; i<length() ; i++)
output[i] = this.extractMin();
return output;
}
public T extractMin() {
T min = storer.get(indexOfMinimum);
AVector.swap(storer, indexOfMinimum, length);
AVector.swap(storer, indexOfMinimum, length);
length--;
heapify(indexOfMinimum);
return min;
}
Report (5000 tests, 100 random arrays each):
The array use a Comparator<Integer>.
A Comparator<Integer> executes a confront in 66083 nanoseconds.
The list use a Comparator<NodeList<Integer>>.
A Comparator<NodeList<Integer>> executes a confront in 85973 nanoseconds.
The BST, BH_1 and BH_2 use a Relationship<Integer>.
A Relationship<Integer> executes a confront in 107145 nanoseconds.
The total time for the array sorting is 239 717 392 nanoseconds.
The total time for the list sorting is 984 872 184 nanoseconds.
The total time for the BST sorting is 533 338 808 811 nanoseconds.
The total time for the BH_1 sorting is 1 055 836 440 689 nanoseconds.
The total time for the BH_2 sorting is 1 198 365 741 676 nanoseconds.
The medium time for the array sorting is 47 943 nanoseconds.
The medium time for the list sorting is 196 974 nanoseconds.
The medium time for the BST sorting is 106 667 761 nanoseconds.
The medium time for the BH_1 sorting is 211 167 288 nanoseconds.
The medium time for the BH_2 sorting is 239 673 148 nanoseconds.
The first method for the Binary Heap has been faster than the second for 3 634 times out of 5 000.
EDIT:
Re-reading what I wrote, I realized I wasn't exactly clear in my initial question. Allow me to rectify my mistake.
I am aware that there is a difference between the actual time executed by the program and the computational complexity. I never had any doubt on the computational complexity of the methods used: the data structures are simple, their code is pratically taken from Wikipedia . I was certain that the written code was not well performing. It wasn't written with performance to begin with.
My test is on the actual time of execution. The Arrays.sort() method was included as a reference parameter. Since I didn't wrote the code with performance in mind, I suspected before the test that the results would be more costly than expected. However my prediction on what exactly was costing more than it should were thrown into the garbage bin by the results.
For example I believed that less method calls would results in a lesser cost, but I was wrong. The whole difference between the two Binary Heaps was that the first executed the same code of the second, just with less method calls (minimal code for the BinaryHeap included below). I expected that the first Binary Heap would always have a lesser cost: this was proven wrong, and I don't know why.
Another thing that baffled me was the fact that the Binary Heap costed more than the Binary Search Tree. The array used is created randomly. While with this starting condition the height of a Binary Search Tree is expected to be around log(n) (chapter 12, paragraph 4 of Introduction to Algorithm, by Cormen, Leiserson, Rivest, Stein), I never heard of an algorithm using Binary Search Trees to sort arrays: I included it in my test only as a curiosity. And yet for the small number of elements (initially 100) the Binary Search Trees was found consistently quicker than the Binary Heap.
Why this happens? When a Binary Heap starts being more convenient? Was including the conversion Heap»Array a mistake?
A third unexpected result is the performance of the double-linked List. As with the Binary Search Trees, I included in the test as a curiosity. The sorting algorithm is a very basic Insertion Sort, which is faster only for a number of element far less than what I used, and the code of the entire class is most definitively not created for quickness. I supposed it would have revealed as the most slow of my classes, instead it was the quickest! And I still don't know why.
This is why I asked advice. I didn't included the code because between all the classes it is around the ~3000 lines of code, most of which is unused by the test. I would not have read 3k lines of code, nor I expected the random stackoverflow-er to do so! However I included the code for the BinaryHeap<T>, which is ~300 lines.
Code for BinaryHeap<T> :
public class BinaryHeap<T> implements Heap<T> {
private static final int indexOfMinimum = 1;
private Vector<T> storer = new Vector<T>(indexOfMinimum + 10);
private Relationship<T> relationship = null;
// The class Relationship<T> has a single method whose signature is
// public boolean confront(T a, T b);
// The reason I included it instead of a Comparator is that, in my code, the value null represents -∞
//
// The following code works.
//
// public class Relationship<T>{
// Comparator<T> c;
// public Relationship<T>(Comparator<T> c){ this.c = c; }
// public boolean confront(T a, T b){ return a==null ? true : c.compare(a,b); }
// }
// TODO | Constructors
public BinaryHeap(Relationship<T> relationship){
storer.add(null);
this.relationship = relationship;
}
// TODO | Methods of the interface Heap
public void add(T newData) {
storer.add(null);
updateValue(storer.size() - 1, newData);
}
public T extractMin() {
T min = storer.get(indexOfMinimum);
heapify(indexOfMinimum);
return min;
}
public void updateValue(int indexOfToBeUpgraded, T newValue) {
int i = indexOfToBeUpgraded;
T support;
if( i >= indexOfMinimum )
{
storer.set(i, newValue);
while( i > indexOfMinimum && ! relationship.confront(storer.get(i/2), storer.get(i)) )
{
support = storer.get(i);
storer.set(i, storer.get(i/2));
storer.set(i, support);
i = i/2;
}
}
}
private void heapify(int i){
int j = i;
int maximumIndexOfArray = storer.size();
while( j <= maximumIndexOfArray/2 ) // i.e. if H[j] isn't a leaf of the Tree
{
int indexOfLocalMinimum = relationship.confront(storer.get(j), storer.get(2*j))
? ( relationship.confront(storer.get( j), storer.get(2*j+1)) ? j : 2*j+1 )
: ( relationship.confront(storer.get(2*j), storer.get(2*j+1)) ? 2*j : 2*j+1 ) ;
if( j != indexOfLocalMinimum )
{
AVector.swap(storer, j, indexOfLocalMinimum);
j = indexOfLocalMinimum;
}
else j = maximumIndexOfArray;
}
}
public default T[] convertHeapToArray(){
T[] output = (T[]) Array.newInstance(min().getClass(), length());
for(int i=0 ; i<length() ; i++)
output[i] = this.extractMin();
return output;
}
// TODO | Second version of the method convertHeapToArray, found out not to be improved
public T[] convertHeapToArray_1(){
int length = length(), j;
T[] output = (T[]) Array.newInstance(min().getClass(), length());
for(int i=0 ; i<length ; i++)
{
// output[i] = this.extractMin();
output[i] = storer.get(indexOfMinimum);
// heapify(indexOfMinimum);
j = indexOfMinimum;
int maximumIndexOfArray = storer.size();
while( j <= maximumIndexOfArray/2 ) // i.e. if H[j] isn't a leaf of the Tree
{
int indexOfLocalMinimum = relationship.confront(storer.get(j), storer.get(2*j))
? ( relationship.confront(storer.get( j), storer.get(2*j+1)) ? j : 2*j+1 )
: ( relationship.confront(storer.get(2*j), storer.get(2*j+1)) ? 2*j : 2*j+1 ) ;
if( j != indexOfLocalMinimum )
{
AVector.swap(storer, j, indexOfLocalMinimum);
j = indexOfLocalMinimum;
}
else j = maximumIndexOfArray;
}
}
return output;
}
Computational complexity does not measure nanoseconds, or milliseconds, or anything of that sort. It measures how the running time of an algorithm varies as a function of the size of its input, and it says nothing about the efficiency of the code that me or you might write to implement the algorithm.
Now, when you write an actual implementation of an algorithm you are introducing overhead which depends on a number of factors that computational complexity theory just does not care about.
The performance of your code depends on your language of choice, your execution environment, programming choices that you make, how experienced you are at writing performing code and avoiding common performance pitfalls, etc.
Furthermore, when you test the performance of your code, a lot depends on how well you know how to do that in your execution environment. With bytecode-translated, just-in-time-compiled, garbage-collected languages like java, on systems with hundreds of simultaneously running processes, it is quite quirky.
So, the answer to your question is that your comparisons are unequal because a) you have not written well performing code, b) certain things cost a lot more than you might expect in java, and c) the system you are trying to benchmark is more chaotic and less closed than you think.
In order to do a test which is true to computational complexity theory you would have to measure the performance of your code theoretically. This means that you would need to count not nanoseconds, but theoretical operations. Those would be the number of node accesses and the number of node creations (in trees.)
However, computational complexity theory still holds, so even if you insist on counting time, if you performance-test your algorithms for several orders of magnitude larger Ns, (which might mean allowing them to run for years, not hundreds of nanoseconds,) you should eventually begin to see the differences predicted by theory, because on the long run, log N beats N, which in turn beats N log N, which in turn beats N2, despite the inequalities introduced by badly performing code.
Of course there is always a chance that the algorithms and data structures that you are performance-testing have completely different computational complexity characteristics than what what we think they have, due to implementation tricks. For example, the linked list that you are performance-testing could internally employ a hash-map as an aid to boost its performance, who knows. But we would only be able to judge that if you posted your code so that we can see exactly what is going on.
I'm doing a simulation of a Bank where I have one queue and three cashiers; the objective is to obtain statistics, for example the average wait time of the customers. Every cashier has a different attention time and a new client comes enters the bank every 1.5 minutes. The bank just runs 5 hours. Now, is there a way I could program it in real time (cause I think its the only way) and then some way fast foward the jvm in order to get the statistics sooner?
You cannot fast-forward the JVM, but you can program this to simulate in non real-time to get statistics soon. Play with variables that represents time in a small enough unit of time (for example in seconds) and do the simulation normally, increasing the time in one unit. For the queue you can use a deque containing the instants when every client entered the bank, and then iterate over time.
private static int TOTAL_TIME = 5*3600; // Hours to seconds
private static int TIME_BETWEEN_CLIENTS = 90; // In seconds
private static int CASHIERS = 3;
public static void main(String[] args) {
// Create cashiers and assign range of attentions time to each cashier, in seconds
// This is an example for cashiers with 3, 3~6 and 4.5~15 minutes
List<Cashier> cashiers = new ArrayList<Cashier>(CASHIERS);
cashiers.add(new Cashier(180, 180));
cashiers.add(new Cashier(180, 360));
cashiers.add(new Cashier(270, 900));
int time = 0; // Counting variable in seconds
int waitingTime = 0; // Save here all waiting time for all clients
int clients = 0; // Save here all clients
// Register here all available cashiers
ArrayList<Cashier> freeCashiers;
Deque<Integer> queue = new ArrayDeque<Integer>(); // Clients queue
// Iterate until bank closes and all clients have been attended
while (time < TOTAL_TIME || !queue.isEmpty()) {
// New client if the bank is not closed
if (time < TOTAL_TIME && time%TIME_BETWEEN_CLIENTS == 0) {
queue.add(time); // Register customer start waiting time
clients++;
}
// Check for free cashiers when someone is on queue
if (!queue.isEmpty()) {
freeCashiers = new ArrayList<Cashier>(CASHIERS);
for (Cashier c : cashiers) {
if (c.isFree(time))
freeCashiers.add(c);
}
if (!freeCashiers.isEmpty()) {
// Register spent time for the removed client
waitingTime += time - queue.removeFirst();
// Select a random cashier from all the available cashiers
Cashier randomAvailableCashier = freeCashiers.get(Cashier.RANDOM.nextInt(freeCashiers.size()));
// Register when the randomly selected cashier will be free again
randomAvailableCashier.attendNewClient(time);
}
}
time++; // Adds one second
}
// Calculate statistics
int avgWaitingTime = waitingTime/clients; // In seconds
System.out.println("Average waiting time on queue: " + formatTime(avgWaitingTime));
}
/**
* Formats a time in minutes and seconds
* #param time the time in seconds
* #return the formatted time
*/
private static String formatTime(int time) {
StringBuilder result = new StringBuilder();
if (time > 60) {
result.append(time/60).append(" minutes");
time %= 60;
if (time > 0)
result.append(" and ");
else
result.append(".");
}
if (time > 0)
result.append(time).append(" seconds.");
return result.toString();
}
And the Cashier class:
public class Cashier {
public static final Random RANDOM = new Random();
private int minAttentionTime, maxAttentionTime, endTime;
/**
* Constructs new Cashier with a range of possible attention time, in seconds
* #param minAttentionTime in seconds
* #param maxAttentionTime in seconds
*/
public Cashier(int minAttentionTime, int maxAttentionTime) {
this.minAttentionTime = minAttentionTime;
this.maxAttentionTime = maxAttentionTime;
endTime = 0;
}
/**
* Register end time with a random attention time in the range.
* #param currentTime the current time in seconds
*/
public void attendNewClient(int currentTime) {
endTime = currentTime + getRandomNumberInRange(minAttentionTime, maxAttentionTime);
}
/**
* Returns if this cashier is available
* #param currentTime the current time in seconds
* #return true if this cashier is free, false otherwise
*/
public boolean isFree(int currentTime) {
return currentTime >= endTime;
}
/**
* Returns a random number in range [min, max]
* #param min the minimum number, inclusive
* #param max the maximum number, inclusive
* #return a random number in range [min, max]
*/
private int getRandomNumberInRange(int min, int max) {
return RANDOM.nextInt(max - min + 1) + min;
}
public String toString() {
return String.valueOf(endTime);
}
}
With this implementation a new client will go to a random free cashier.
If you instead used a queue system, you could go very quickly. The queue would contain events that are coming up, and a timestamp of when they are due. The queue would be organized by event timestamp. Initially, you could seed this queue with a new customer every 1.5 minutes, and an event for the bank closing in 5 hours. Then you pop the first event off the queue, figure out what that triggers. For example, the first event of a customer entering causes you to increase the customer wait line count. A teller becoming free causes the wait line count to reduce and an agent-customer activity that last for some random time length. This causes a new event (another teller being free) to be placed in the queue. When a teller being free occurs, and there are no customers in line, you set an empty teller flag, which would be checked when you next handle a customer entering event. You could quickly run through an entire day within a few seconds with approach, and you can gather statistics on whatever you want at each event.
If you don't mind to use formulas here is my math. If some cashier has time T1 for processing one person then this cashier processes 1/T1 people per time unit. Number of people processed by N cashiers are added. Coming back from processing speed to processing time we get common time for processing one person by all cashiers:
Tcom = 1/(1/T1 + 1/T2 + ... + 1/Tn)
You compare this time with your 1.5 minutes and if Tcom <= 1.5 minutes then your queue will be mostly empty or consist of 1 person (and average waiting time will be around half of average of Ti). In case Tcom > 1.5 minutes your queue will grow constantly by one person each Tcom - 1.5 minutes. Queue length at the end of working day will be Lavg = 5 * 60 / (Tcom - 15) people (and half of it in average). Each person in head of the queue is removed from it each Tcom minutes. It means that average waiting time is going to be Tcom * Lavg / 2.
I know it's all very rough estimates but why not?
(You deleted your other almost identical question. Here's the long answer i wrote before knowing that. Good that you had this copy, so i didn't waste my time)
Here is the functional pseudo. I actually coded it, out of curiosity.
I use a computer language called APL, which is most likely unknown to you, hence pseudo.
You need to create a 2-dimensional table, ie. an array, with 6 columns. If you cannot create 2-dimensional arrays, you can alternatively (hopefully) create 1-dimensional arrays, ie. lists, where one list holds the values of one column in the table. The important thing is that you can index into arrays. If you use listst, you must extend the code a bit.
You will only work with integer values.
The solution is that you pre-create a table, which holds the customer arrival times in column 1. All other columns hold zeroes to start with. Then you loop through the table, updating the other columns, row-by-row, using a help variable for the cashiers.
Once you have looped through all rows in the table, you extract the desired results from it, by additional calculation.
For the cashiers actions, you need another 2-dimensional table, holding [nr of cashiers] rows and 3 columns. Similarly, you can replace it with 3 lists, one per table column. Columns 2 and 3 in this small table accumulate each cashiers tim ethey spent serving a customer, and time they spent waiting for customers.
You do not work with clock times, but with seconds since day start. For example 8am equals 28800 (ie. number of seconds elapsed since day start). To this value, you can easily add customer arrival times with 90 seconds increments (or any, arbitrary amount of seconds intervals). You pre-create customer arrival times for the entire 5-hour (or any hour) day.
// Create a table with 6 columns. Columns are
// 1: Customer arrival timestamp (first value is 28800) [seconds since midnight]
// 2: The timestamp the customer was attended [seconds since midnight]
// 3: The time a customer had to wait [seconds]
// 4: The time the customer spent with the cashier [seconds]
// 5: The timestamp the customer leaves the bank [seconds since midnight]
// 6: The cashier # that provided service, 1,2 or 3 in your case
//
// If the customer arrive at regular 90 s intervals, the table now looks like this
// (10 first rows only, but you must populate it up to bank closing time)
// which would need ca 200 rows. Call this table "Cust":
// 28800 0 0 0 0 0
// 28890 0 0 0 0 0
// 28980 0 0 0 0 0
// 29070 0 0 0 0 0
// 29160 0 0 0 0 0
// 29250 0 0 0 0 0
// 29340 0 0 0 0 0
// 29430 0 0 0 0 0
// 29520 0 0 0 0 0
// 29610 0 0 0 0 0
//
// Create a variable for the cashiers, 1 row per cashier. Columns are:
// 1. An "action timestamp", initially 28800 [seconds since midnight]
// 2. Accumulation of cashiers attention time [seconds]
// 3. Accumulation of cashiers wait/slack time [seconds]
//
// The table now looks like this
// (add more rows if you have more cashiers). Call this table "Cash":
// 28800 0 0
// 28800 0 0
// 28800 0 0
rows = [numers of rows in Cust]
i = 0
:While i < rows
// Note! This pseudocode uses 0-origin, ie. array[0] is the first element
// Find the row number of Cash with the _smallest_ value in it's column 1
row = [0, 1 or 2] // You commonly first find the smallest number, then compare it against each row. The match is the row with smallest number.
// Attention time for this customer (you said "range of"),
// we use 265 s now, for simplicity, but you can give time another value each time
time = 265
:If Cust[i;0]<Cash[row;0] // Customer has waited
Cust[i;1]=Cash[row;0] // Timestamp this Customer got service
Cust[i;3]=time // Time spent with cashier
Cust[i;4]=Cash[row;0]+time // Timestamp customer leaves the bank
Cust[i;5]=row // Which cashier provided the service
Cash[row;1]+=time // Increase accumulated attend time for cashier
Cash[row;0]+=time // Next timestamp this cashier is free
:Else // Cashier had free time (bank was empty until now)
Cash[row;2]+=Cust[i;0]-Cash[row;0] // Accumulate freetime for cashier
Cash[row;1]+=time // Accumulate attend time for cashier
Cust[i;1]=Cust[i;0] // There was no wait time for customer
Cust[i;3]=time // Time spent with cashier
Cust[i;4]=Cust[i;0]+time // Timestamp customer leaves the bank
Cust[i;5]=row // Which cashier provided the service
Cash[row;0]=Cust[i;0]+time // Next timestamp this cashier is free
:End
i+←1
:End
// Resolve customer wait times, equals [time attended] - [time arrived] and
// populate 3rd column of Cust (you must probably loop), row by row
Cust[n;2] = Cust[n;1] - Cust[n;0] // n = each row
// Cust now looks like this:
// 28800 28800 0 265 29065 0
// 28890 28890 0 265 29155 1
// 28980 28980 0 265 29245 2
// 29070 29070 0 265 29335 0
// 29160 29160 0 265 29425 1
// 29250 29250 0 265 29515 2
// 29340 29340 0 265 29605 0
// 29430 29430 0 265 29695 1
// 29520 29520 0 265 29785 2
// 29610 29610 0 265 29875 0
fnAvg = [function that calculates the average of a list of numbers]
// Extract the results
// Note: For example "Cust[;2]" means _all rows in column 2 of Cust_, ie. a list of numbers
// You don't need to create any lists though, just loop through them all and calculate/accumulate
'Nr of customers attended immediately: ',[number of zeroes in Cust[;2]]
'Nr of customers who waited: ',[number of non-zeroes in Cust[;2]]
'Wait times: ',[all numbers in Cust[;2] - this is a list of numbers]
'Average wait times: ',[avg Cust[;2]]
'Average wait times (only those who waited): ',[avg Cust[;2] elements that are non-zero]
'Total cashier attend times: ',Cash[;1]
'Total cashier free times: ',Cash[;2]
// And finally a verification calc (backwards calc, just for curiosity)
'Check: Total customer existance time: ',Cust[i-1;4]-Cust[0;0]
'Check: Cashier total times (should be close to above value): ',[sum of Cash[n;1 2]]
The result with the arguments above is (numbers are seconds unless otherwise said):
Nr of customers attended immediately: 10
Nr of customers who waited: 0
Wait times: 0 0 0 0 0 0 0 0 0 0
Average wait times: 0
Average wait times (only those who waited): 0
Total cashier attend times: 1060 795 795
Total cashier free times: 15 100 190
Check: Total customer existance time: 1075
Check: Cashier total times (should be close to previous row): 1075 895 985
If you have 20 customers arriving, and use random attend times between 100 and 480 seconds, you may get for example:
Nr of customers served immediately: 8
Nr of customers waited: 12
Wait times: 0 0 0 0 0 14 0 0 0 200 120 147 183 122 149 111 185 178 244 218
Average wait times: 93.55
Average wait times (only those who waited): 155.9166667
Total cashier attend times: 1820 1836 1819
Total cashier free times: 217 197 309
Check: Total customer existance time: 2128
Check: Cashier total times (should be close to previous row): 2037 2033 2128
with Cust (edited: was erratically Cash) table looking like this:
28800 28800 0 160 28960 0
28890 28890 0 433 29323 1
28980 28980 0 114 29094 2
29070 29070 0 194 29264 0
29160 29160 0 117 29277 2
29250 29264 14 149 29413 0
29340 29340 0 470 29810 2
29430 29430 0 390 29820 1
29520 29520 0 417 29937 0
29610 29810 200 253 30063 2
29700 29820 120 389 30209 1
29790 29937 147 155 30092 0
29880 30063 183 445 30508 2
29970 30092 122 169 30261 0
30060 30209 149 216 30425 1
30150 30261 111 403 30664 0
30240 30425 185 408 30833 1
30330 30508 178 220 30728 2
30420 30664 244 173 30837 0
30510 30728 218 200 30928 2
Try with populating the 1st column of Cust with different, more random customer arrivbal times, and for each iteration, the time with values that fit your real environment better.
This pseudo supports both "bank temporarily empty" and "long que all time", and also supports the "Once the bank closes after 5 hours, every client gets attended"-criterion (the remaining customers will be cleaned up").
Good luck! :-)
I have to do a little assignment at my university:
I have a server that runs 'n' independent services. All these services started at the same time in the past. And every service 'i' writes 'b[i]' lines to a log file on the server after a certain period of time 's[i]' in seconds. The input consist of 'l' the number of lines of the log file and 'n' the number of services. Then we have in the next 'n' lines for every service i: 's[i]' the period as mentioned and 'b[i]' the number of lines the services writes to the log file.
I have to compute from the number of lines in the log file, how long ago, in seconds, the programs all started running. Example:
input:
19 3
7 1
8 1
10 2
Output:
42
I have to use divide and conquer, but I can't even figure out how to split this in subproblems. Also I have to use this function, where ss is the array of the periods of the services and bs the number of lines which each services writes to the log file:
long linesAt(int t, int[] ss, int[] bs) {
long out = 0;
for (int i = 0; i < ss.length; i++) {
// floor operation
out += bs[i] * (long)(t/ss[i]);
}
return out;
ss and bs are basically arrays of the input, if we take the example they will look like this, where the row above is the index of the array:
ss:
0 1 2
7 8 10
bs:
0 1 2
1 1 2
It is easily seen that 42 should be the output
linesAt(42) = floor(42/7)*1+floor(42/8)*1+floor(42/10)*2 = 19
Now I have to write a function
int solve(long l, int[] ss, int[] bs)
I already wrote some pseudocode in brute force, but I can't figure out how to solve this with the divide and conquer paradigm, my pseudocode looks like this:
Solve(l, ss, bs)
out = 0
t = 0
while (out != l)
out = linesAt(t, ss, bs)
t++
end while
return t
I think I have to split l in some way, so to calculate the time for smaller lengths. But I don't really see how, because when you look at this it doesn't seem to be possible:
t out
0..6 0
7 1
8 2
9 2
10 4
11..13 4
14 5
15 5
16 6
17..19 6
20 8
...
40 18
42 19
Chantal.
Sounds like a classic binary search would fit the bill, with a prior step to obtain a suitable maximum. You start with some estimate of time 't' (say 100) and call linesAt to obtain the lines for that t. If the value returned is too small (i.e. smaller than l), you double 't' and try again, until the number of lines is too large.
At this point, your maximum is t and your minimum is t/2. You then repeatedly:
pick t as the point halfway between maximum and minimum
call linesAt(t,...) to obtain the number of lines
if you've found the target, stop.
if you have too many lines, adjust the maximum: maximum = t
if you have too few lines adjust the minimum: minimum = t
The above algorithm is a binary search - it splits the search space in half each iteration. Thus, it is an example of divide-and-conquer.
You are trying to solve an integer equation:
floor(n/7)*1+floor(n/8)*1+floor(n/10)*2 = 19
You can remove the floor function and solve for n and get a lower bound and upper bound, then search between these two bounds.
Solving the following equation:
(n/7)*1+(n/8)*1+(n/10)*2 = 19
n=19/(1/7+1/8+2/10)
Having found n, which range of value m0 will be such that floor (m0 / 7) = floor (n/7)?
floor (n/7) * 7 <= m0 <= (ceiling (n/7) * 7) - 1
In the same manner, calculate m1 and m2.
Take max (mi) as upperbound and min(mi) as lowerbound for i between 1 and 3 .
A binary search at this point will probably be an overkill.
this is the C program under Linux/GUN:
#include<stdio.h>
#include<sys/time.h>
#define Max 1024*1024
int main()
{
struct timeval start,end;
long int dis;
int i;
int m=0;
int a[Max];
gettimeofday(&start,NULL);
for(i=0;i<Max;i += 1){
a[Max] *= 3;
}
gettimeofday(&end,NULL);
dis = end.tv_usec - start.tv_usec;
printf("time1: %ld\n",dis);
gettimeofday(&start,NULL);
for(i=0;i<Max;i += 16){
a[Max] *= 3;
}
gettimeofday(&end,NULL);
dis = end.tv_usec - start.tv_usec;
printf("time2: %ld\n",dis);
return 0;
}
the output:
time1: 7074
time2: 234
it's a big distance
this Java program:
public class Cache1 {
public static void main(String[] args){
int a[] = new int[1024*1024*64];
long time1 = System.currentTimeMillis();
for(int i=0;i<a.length;i++){
a[i] *= 3;
}
long time2 = System.currentTimeMillis();
System.out.println(time2 - time1);
time1 = System.currentTimeMillis();
for(int i=0;i<a.length;i += 16){
a[i] *= 3;
}
time2 = System.currentTimeMillis();
System.out.println(time2 - time1);
}
}
the output:
92
82
it's nealy the same
with the CPU Cache. why they hava so much difference? the Cpu Cache is invalid in C programing?
I hope you realize that the difference in units of time in those tests is 10^3. C code is order of magnitude faster than Java code.
In C code there should be a[i] instead of a[Max].
As for cache: since you access only one memory location in your C code (which triggers undefined behavior) your C test is completely invalid.
And even if it were correct, your method is flawed. It is quite possible that the multiplication operations and even the whole loops were skipped completely by C copiler, since nothing depends on their outcome.
The result where first run takes long, and the second takes less time, is expected. Data has to be loaded to cache anyway, and that takes time. Once it is loaded, operations on that data take less time.
Java may either not use cache at all (not likely) or preload the whole array to cache even before the loops are executed. That would explain equal execution times.
You have three cache sizes, these are typically
L1: 32 KB (data), 4 clock cycles
L2: 256KB, 10-11 clock cycles
L3: 3-24 MB. 40 - 75 clock cycles.
Anything larger than this will not fit into the cache as if you just scroll through memory it will be like they are not there.
I suggest you write a test which empirically works out the CPU cache sizes as a good exercise to help you understand this. BTW You don't need to use *= to exercise the cache as this exercises the ALU. Perhaps there is a simpler operation you can use ;)
In the case of your Java code, most likely it is not compiled yet so you are seeing the speed of the interperator, not the memory accesses.
I suggest you run the test repeatedly on smaller memory sizes for at least 2 seconds and take the average.
Have a look at this question :
The code:
class test
{
public static void main(String abc[])
{
for( int k=1; k<=3; k++)
{
for( int N=1; N<=1_000_000_000; N=N*10)
{
long t1 = System.nanoTime();
int j=1;
for(int i=0; i<=N; i++)
j=j*i;
long t2 = System.nanoTime() - t1;
System.out.println("Time taken for "+ N + " : "+ t2);
}
}
}
}
The output of above code:
Time taken for 1 : 2160
Time taken for 10 : 1142
Time taken for 100 : 2651
Time taken for 1000 : 19453
Time taken for 10000 : 407754
Time taken for 100000 : 4648124
Time taken for 1000000 : 12859417
Time taken for 10000000 : 13706643
Time taken for 100000000 : 136928177
Time taken for 1000000000 : 1368847843
Time taken for 1 : 264
Time taken for 10 : 233
Time taken for 100 : 332
Time taken for 1000 : 1562
Time taken for 10000 : 17341
Time taken for 100000 : 136869
Time taken for 1000000 : 1366934
Time taken for 10000000 : 13689017
Time taken for 100000000 : 136887869
Time taken for 1000000000 : 1368178175
Time taken for 1 : 231
Time taken for 10 : 242
Time taken for 100 : 328
Time taken for 1000 : 1551
Time taken for 10000 : 13854
Time taken for 100000 : 136850
Time taken for 1000000 : 1366919
Time taken for 10000000 : 13692465
Time taken for 100000000 : 136833634
Time taken for 1000000000 : 1368862705
In the loop, even though the value of i starts from 0, indicating the product to be zero, there is no JIT Optimization. Why not ?
In the link provided above, I had previously put the for loop in a method call, which the JIT was optimizing. Is putting the statements in a method facilitating in the optimization process ?
In your previous question the JIT optimized away the complete code of the method start without any analysis as to what number happened to be present in the variables upon method return. This is because you chose to make your method void, giving the JIT a dead-easy clue that any values calculated will be discarded.
Contrasting your current example with the one from your previous question, there are no void methods called so naturally the optimization does not occur. Why there is not some other optimization which would help this completely different case is an unanswerable question. There is just no such optimization in the specefic JVM implementation, and the specific JVM invocation, with which you have tested your code.
The loop itself does get jitted (as observed by the slightly lower running times on second and third execution), however eliminating the entire loop is - afaik - only done when the method itself is executed multiple times, because only then the JIT has sufficient runtime information to be sure it can actually eliminate it without consequence.
If I change your code, the loop is eliminated on the third invocation:
public class LoopJit2 {
public static void main(String abc[]) {
for (int x = 0; x < 3; x++) {
loopMethod();
}
}
private static void loopMethod() {
for (int N = 1; N <= 1_000_000_000; N = N * 10) {
long t1 = System.nanoTime();
int j = 1;
for (int i = 0; i <= N; i++)
j = j * i;
long t2 = System.nanoTime() - t1;
System.out.println("Time taken for " + N + " : " + t2);
}
}
}
Time series:
Time taken for 1 : 1466
Time taken for 10 : 1467
Time taken for 100 : 2934
Time taken for 1000 : 20044
Time taken for 10000 : 201422
Time taken for 100000 : 1993200
Time taken for 1000000 : 4038223
Time taken for 10000000 : 11182357
Time taken for 100000000 : 111290192
Time taken for 1000000000 : 1038002176
Time taken for 1 : 1466
Time taken for 10 : 1467
Time taken for 100 : 2934
Time taken for 1000 : 20044
Time taken for 10000 : 10755
Time taken for 100000 : 124667
Time taken for 1000000 : 1010045
Time taken for 10000000 : 10201156
Time taken for 100000000 : 103184413
Time taken for 1000000000 : 1019723107
Time taken for 1 : 978
Time taken for 10 : 1467
Time taken for 100 : 1467
Time taken for 1000 : 1955
Time taken for 10000 : 978
Time taken for 100000 : 489
Time taken for 1000000 : 977
Time taken for 10000000 : 977
Time taken for 100000000 : 978
Time taken for 1000000000 : 978