I have a class which loosely implements the HTTP Ranged-GET protocol. Essentially, some code invokes this class, telling it to download a file. The below MyClass is responsible for downloading the file in even-sized chunks (until the last chunk, which may be variable-length), and sending it to another service. When invoked, it sets the file size based on the Content-Range instance-length from the 1st HTTP response.
The class uses an ExecutorService with a thread pool size of 1 to control threading.
Below is the relevant implementation, with some hand-waving over the functions handling the GETs and PUTs.
class MyClass implements Runnable {
private long start;
private long chunkSize;
private int chunkNumber;
private int fileSize = 0;
private static final int MAX_RETRIES = 3;
public static final ExecutorService ES = Executors.newSingleThreadExecutor();
public MyClass(long start, long chunkSize, int chunkNumber) {
this.start = start;
this.chunkSize = chunkSize;
this.chunkNumber = chunkNumber;
}
public void run() {
for (int i = 0; i < MAX_RETRIES; i++) {
long end = start + chunkSize - 1; // inclusive so subtract 1
// doHttpGet() is a private instance function...
// if fileSize == 0 (i.e. first chunk downloaded), this will set the fileSize
doHttpGet(start, end);
// doHttpPost() is a private instance function
// it builds the POST from the GET message, which I'm not bothering to show here
if (!doHttpPost()) {
continue;
} else {
submitNextChunk(this);
break;
}
}
}
// this is the function a client uses to invoke the class
public static void submitWork(long newStartByte, long chunkSize, int chunkNumber) {
MyClass mc = new MyClass(newStartByte, chunkSize, chunkNumber);
if (ES.submit(mc) == null) {
//log error
}
}
// PROBLEM AREA?!?!
private static void submitNextChunk(MyClass mc) {
mc.chunkNumber++;
mc.start += mc.chunkSize;
// LOGGER.debug("start=" + mc.start + "\n" + "fileSize=" + mc.fileSize)
if (mc.start < mc.fileSize) {
if (ES.submit(mc) == null) {
//log error
}
}
}
}
And here is a snippet of the code which invokes MyClass.
long chunkSize = //somecalculation
DownloadAction.submitWork(0L, chunkSize, 1));
This code has been working fine, for a long time. However, I'm now noticing a potentially non-deterministic behavior when the file-to-download is extremely small (e.g., < 50 bytes). What appears to be happening is that the submitNextChunk() function does not seem to evaluate mc.start < mc.fileSize correctly. For example, if we set the packetSize=100K, and use a 50 byte file, then what I see -- via Wireshark -- is continuous HTTP GET requests asking for bytes 0-99999, 100000-199000, and 200000-299000, ..., etc. (The code on the other end is also slightly broken, as it continues to give us the original 50 bytes, rather than an HTTP out-of-range error code... but that another story.)
My concern was that there is a subtle race condition:
If I put some logging in submitNextChunk() to print out start and fileSize, then I only see one log statement of start=100000 and fileSize=100, and the function evaluates the less-than expression correctly to false. This makes sense, since it would be the first and only time submitNextChunk() would be called, and since that expression evaluates false, the function aborts.
I am concerned that this is somehow a threading issue, since without the debug statements, that less-than expression clearly results to true, which should not be occurring.
Related
I am having issues with my code regarding exception in thread main. This is the error that is popping up
Exception in thread "main" java.lang.IllegalStateException: Attempt to create a stack whose capacity exceeds allowed maximum of 10000
at ResizeableArrayStack.checkCapacity(ResizeableArrayStack.java:74)
at ResizeableArrayStack.ensureCapacity(ResizeableArrayStack.java:82)
at ResizeableArrayStack.push(ResizeableArrayStack.java:28)
at ResizeableArrayStack.evaluatePostfix(ResizeableArrayStack.java:98)
at ResizeableArrayStack.main(ResizeableArrayStack.java:157)
This is my code
import java.util.*;
public class ResizeableArrayStack<T> implements StackInterface<T>
{
private T[] stack;
private int topIndex;
private boolean integrityOK = false;
private static final int DEFAULT_CAPACITY = 50;
private static final int MAX_CAPACITY = 100000;
public ResizeableArrayStack()
{
this(DEFAULT_CAPACITY);
}
public ResizeableArrayStack(int initialCapacity)
{
integrityOK = false;
checkCapacity(initialCapacity);
// The cast is safe because the new array contains null entries
#SuppressWarnings("unchecked")
T[] tempStack = (T[])new Object[initialCapacity];
stack = tempStack;
topIndex = -1;
integrityOK = true;
}
public void push(T newEntry) {
checkIntegrity();
ensureCapacity();
stack[topIndex + 1] = newEntry;
topIndex++;
}
private void checkCapacity(int capacity) {
if (capacity > MAX_CAPACITY) {
throw new IllegalStateException("Attempt to create a stack whose capacity exceeds allowed
maximum of " + MAX_CAPACITY);
}
} // end checkCapacity
private void ensureCapacity() {
if (topIndex >= stack.length - 1) {
// If array is full, double its size
int newLength = 2 * stack.length;
checkCapacity(newLength);
stack = Arrays.copyOf(stack, newLength);
}
} //end ensureCapacity
public static void main(String[] args) {
String input = "ab*ca-/de*+";
ResizeableArrayStack<String> astack = new ResizeableArrayStack<>(input.length());
int evaluation = astack.evaluatePostfix(input);
System.out.println("Evaluating Postfix Expressions");
System.out.println("The evaluation of the postfix expression is " + evaluation);
}
}
I'm pretty sure the issue is with how the capacity values are set and compared but I can't figure out why I am getting this error. I think the issues are within the constructors that involve capacity and main method. Please ignore the evaluatePostfix method in the main as the errors all say they come from the constructors and main. I can put the evaluatePostfix up if you think the problem is within it. I also deleted the methods that weren't brought up in the problem.
The code you posted is not a MCVE. It doesn't compile because it is missing methods including checkIntegrity and evaluatePostfix. Even after I work around the missing methods, the code you posted does not cause IllegalStateException to be thrown. At a guess, after looking through the code that you did post, as well as the stack trace, the culprit appears to be method ensureCapacity which contains the following line:
int newLength = 2 * stack.length;
The value assigned to newLength may be greater than MAX_CAPACITY.
After you assign a value to newLength, you call method checkCapacity which explicitly throws a IllegalStateException.
private void checkCapacity(int capacity) {
if (capacity > MAX_CAPACITY) {
throw new IllegalStateException("Attempt to create a stack whose capacity exceeds allowed maximum of " + MAX_CAPACITY);
}
} // end checkCapacity
But as I wrote earlier, in the code that you posted capacity is never greater than MAX_CAPACITY and hence the code in your question never throws IllegalStateException.
I recommend that you run your code through a debugger. If you are using an IDE, then it should give you the option of running your code in "debug" mode. If you don't know how to use the debugger of the IDE then you should learn because knowing how to debug code is an essential skill for a programmer and a debugger helps a lot when it comes to debugging your code.
I always believed that accessing instanced fields from the instanced method/function should give some performance increase as the data should be "more locally available" (I know this is probably dependent on the system and JVM) as opposed to accessing members from a static context. See the code that illustrates my point:
public class Lecture
{
public static void main(String[] args)
{
hold h1 = new hold();
long start1 = System.currentTimeMillis();
h1.accessOOPs();
long end1 = System.currentTimeMillis();
System.out.println("OOPS: "+ (end1 - start1));
hold h2 = new hold();
long start2 = System.currentTimeMillis();
hold.accessStatic(h2);
long end2 = System.currentTimeMillis();
System.out.println("Static (same class): "+ (end2 - start2));
hold h3 = new hold();
long start3 = System.currentTimeMillis();
accessStatic(h3);
long end3 = System.currentTimeMillis();
System.out.println("Static (different class): "+ (end3 - start3));
}
public static void accessStatic(hold h)
{
for (int i=0;i<h.vars.length;i++)
h.vars[i] = i;
for (int i : h.vars)
h.var1 += i;
for (int i: h.vars)
h.name += i;
}
}
class hold
{
int var1;
int vars[] = new int[10000];
String name;
public void accessOOPs()
{
for (int i=0;i<vars.length;i++)
vars[i] = i;
for (int i : vars)
var1 += i;
for (int i: vars)
name += i;
}
public static void accessStatic(hold h)
{
for (int i=0;i<h.vars.length;i++)
h.vars[i] = i;
for (int i : h.vars)
h.var1 += i;
for (int i: h.vars)
h.name += i;
}
}
In the code I have 3 timed examples where I access/modify attributes in a simple object. The first example calls an instance method in the object so theoretically all the attributes should be faster to access since they are in the same context as the method. The second one calls a static function in the object class and uses the dot operator to access the attributes each time. My assumption was that this would be slower. The third example repeats the same operation as the second one but this time does it within a separate class. I was much confused about the timings I received:
Example run 1:
OOPS: 135
Static (same class): 130
Static (different class): 120
Example run 2:
OOPS: 137
Procedural (same class): 135
Procedural (different class): 128
Consistently the OOPs method beat the Static method when the method was in the separate class but I don't understand why that when the static method was in the same class it would beat the instance method. It was only be a slight margin but it was very consistent. Is this only occurring because the static implementation is caching the reference to the object being accessed? If that was occurring then I would think it wouldn't matter that the static method was in a separate class or not, so I am completely confused. Can anyone answer why the static method are not significantly slower?
Your h3 variable is not passed to any call, and so the computations done on h2 in the 3rd call is different to the computations of h1/h2 earlier.
When doing timing tests you should only measure warmed up code (as JIT compiler changes performance somewhat). If you fix to use h3 and run the code of the main inside a loop you'll see there is very little difference between the runs, the last run I got is
OOPS: 56
Static (same class): 56
Static (different class): 50
You can see that the bytecode for accessOOPs and accessStatic is very similar structure using:
javap -v hold.class > hold.log
javap -v Lecture.class > lecture.log
IN JDK14 they are both 114 items long and different only with field/index positions, there are the differences shown in WINDIFF.EXE
I have two threads doing calculation on a common variable "n", one thread increase "n" each time, another decrease "n" each time, when I am not using volatile keyword on this variable, something I cannot understand happens, sb there please help explain, the snippet is like follow:
public class TwoThreads {
private static int n = 0;
private static int called = 0;
public static void main(String[] args) {
for (int i = 0; i < 1000; i++) {
n = 0;
called = 0;
TwoThreads two = new TwoThreads();
Inc inc = two.new Inc();
Dec dec = two.new Dec();
Thread t = new Thread(inc);
t.start();
t = new Thread(dec);
t.start();
while (called != 2) {
//System.out.println("----");
}
System.out.println(n);
}
}
private synchronized void inc() {
n++;
called++;
}
private synchronized void dec() {
n--;
called++;
}
class Inc implements Runnable {
#Override
public void run() {
inc();
}
}
class Dec implements Runnable {
#Override
public void run() {
dec();
}
}
}
1) What I am expecting is "n=0,called=2" after execution, but chances are the main thread can be blocked in the while loop;
2) But when I uncomment this line, the program when as expected:
//System.out.println("----");
3) I know I should use "volatile" on "called", but I cannot explain why the above happens;
4) "called" is "read and load" in working memory of specific thread, but why it's not "store and write" back into main thread after "long" while loop, if it's not, why a simple "print" line can make such a difference
You have synchronized writing of data (in inc and dec), but not reading of data (in main). BOTH should be synchronized to get predictable effects. Otherwise, chances are that main never "sees" the changes done by inc and dec.
You don't know where exactly called++ will be executed, your main thread will continue to born new threads which will make mutual exclusion, I mean only one thread can make called++ in each time because methods are synchronized, and you don't know each exactly thread will be it. May be two times will performed n++ or n--, you don't know this, may be ten times will performed n++ while main thread reach your condition.
and try to read about data race
while (called != 2) {
//System.out.println("----");
}
//.. place for data race, n can be changed
System.out.println(n);
You need to synchronize access to called here:
while (called != 2) {
//System.out.println("----");
}
I sugest to add getCalled method
private synchronized int getCalled() {
return called;
}
and replace called != 2 with getCalled() != 2
If you interested in why this problem occure you can read about visibility in context of java memory model.
I have a thread, in Java, putting the first 100 Fibonacci numbers in an array. How do I get the numbers back from the thread. Is there an interrupt, handling, exception, implements, extends? I have been adding things and the trial and error is not getting me anywhere to understanding.
import java.util.Scanner;
import java.io.*;
import java.lang.Thread; //don't know if this is needed
public class FibThread extends Thread{
public FibThread (){
super();
}
public void run(int inputNum){
System.out.println(inputNum);
long[] fibArray = new long[inputNum];
fibArray[0]=0;
fibArray[1]=1;
fibArray[2]=1;
for(int i = 3; i<inputNum; i++){
fibArray[i]= fibArray[i-1] + fibArray[i-2];
// }
//System.out.println( );
// for(int j = 0; j<= inputNum; j++){
int output = (int) fibArray[i];
System.out.println(output);
}
}
public static void main(String[] args){
Scanner keyboard = new Scanner(System.in);
FibThread threadOne;
int inputNum, itter, output;
System.out.println("Please enter the number of Fibonacci numbers to be generated: ");
itter = keyboard.nextInt();
//inputNum = itter;
threadOne = new FibThread();
threadOne.start();
// for(int j = 0; j<= inputNum; j++){
// int output = (int) fibArray[j];
// System.out.println(output);
}
}
If you have a "task" that returns a value, make it a Callable.
If you want the callable to run in a background thread, then instead of handling the creation and execution of threads yourself, it's generally better to abstract this through an ExecutorService. A caller can interact with the service by passing in a Callable, and getting back a Future that will be populated with the value when the calculation has completed.
To modify your example, renaming FibThread to FibCalc:
public class FibCalc implements Callable<Integer> {
// We need some way to pass in the initial input - must be through the
// constructor and we'll store it here
private final inputNum;
public FibCalc(int inputNum) {
this.inputNum = inputNum;
}
public int call() {
// The same as your run() method from before, except at the end:
...
return output;
}
}
// And now for your main() method
public static void main(String[] args) throws Exception {
// As before up to:
...
itter = keyboard.nextInt();
// Create a very simple executor that just runs everything in a single separate thread
ExecutorService exec = Executors.newSingleThreadExecutor();
// Create the calculation to be run (passing the input through the constructor)
FibCalc calc = new FibCalc(itter);
// Send this to the executor service, which will start running it in a background thread
// while giving us back the Future that will hold the result
Future<Integer> fibResult = exec.submit(fibCalc);
// Get the result - this will block until it's available
int result = fibResult.get();
// Now we can do whatever we want with the result
System.out.println("We got: " + result);
}
If you absolutely have to create a Thread object yourself (due to artificial constraints on a homework question, or something like that - I can't see why one would realistically do this in reality), then the approach has to be different. You can't return a value because run() must return void due to the interface. So my approach here would be to store the result in a local variable of the FibThread class, and then add a method to that class (e.g. public int getResult()) which returned that variable.
(If you're doing it this way, bear in mind that you'll have to handle the concurrency issues (i.e. letting the caller know the result is ready) yourself. A naive approach, where the main method starts the thread and then immediately calls getResult(), means that it will almost certainly get an "empty" result before the calculation has finished. A crude solution to this problem would be calling join() on the spawned thread, to wait for it to finish before accessing the result.)
Basically I have a logging statement inside of a method. Whenever the method is called, I want to easily call the name of the method, and append it to the logging statement. This is done to minimize the number of hard coded lines in my application. Is this possible to do it cheaply?
No, not cheaply. The expensive and only way of doing it is with a StackTrace method call. Best to just hardcode it.
If you only need method name and line number, you can do it reasonably fast. In my micro benchmark you can do about 120000 calls per second:
public class A {
private static final int N = 1000000;
public static void main(String[] args) {
A a = new A();
long l = System.currentTimeMillis();
for (int i = 0; i < N; i++) {
a.test(0);
a.test(1);
a.test(2);
}
double time = (System.currentTimeMillis() - l) / 1000d;
System.err.println(time + " seconds");
System.err.println(N / time + " calls/second");
}
String test(int i) {
StackTraceElement ste = Thread.currentThread().getStackTrace()[i];
return ste.getMethodName() + ":" + ste.getLineNumber();
}
}
Also see this answer.
Why reinvent the wheel, java.util.logging.Logger uses
/**
* Gets the StackTraceElement of the first class that is not this class.
* That should be the initial caller of a logging method.
* #return caller of the initial logging method or null if unknown.
*/
private StackTraceElement getCallerStackFrame()
{
Throwable t = new Throwable();
StackTraceElement[] stackTrace = t.getStackTrace();
int index = 0;
// skip to stackentries until this class
while(index < stackTrace.length
&& !stackTrace[index].getClassName().equals(getClass().getName()))
index++;
// skip the stackentries of this class
while(index < stackTrace.length
&& stackTrace[index].getClassName().equals(getClass().getName()))
index++;
return index < stackTrace.length ? stackTrace[index] : null;
}
I doubt you will get it any faster than this.
[Edit] To get the calling methods name use,
StackTraceElement caller = getCallerStackFrame();
String name = caller != null ? caller.getMethodName() : "<unknown>"
did you consider writing sensible log messages instead? I mean, messages that just tell reader of log the "story" about how code runs? messages that really differ for different methods?
log4j: some.package.Class1: method1: checkpoint #1
log4j: some.package.Class1: method1: checkpoint #2
log4j: some.package.Class2: method2: checkpoint #1
log4j: some.package.Class2: method2: checkpoint #2
Been banging my head on log like above just a week or two ago (method names were hardcoded). Didn't feel it noticeably more helpful than no logging at all.