We Keep Coding

How to re-use value in different functions?

I am programming a Study in MotiveWave, a program used for (day)trading. The study is its own class. (info about MotiveWave's SDK found here: https://www.motivewave.com/sdk/javadoc/overview-summary.html)
public class L_V2 extends com.motivewave.platform.sdk.study.Study
My study uses 2 different timeframes: the 1 hour and the 4 hour bars. Both are calculated in a different function. Otherwise formulated: both use a different dataseries, as shown in the code below.
I have two values, being calculated on the 4 hour timeframe, called 'ma9' and 'ma11' that I would like to use in an 'if'-statement on the 1 hour timeframe.
This is the code for the 4 hour timeframe. It simply calculates 2 moving averages
#Override
protected void calculateValues(DataContext ctx)
{
int maPeriodTF2 = getSettings().getInteger(MA_PERIOD_TF2);
int ma2PeriodTF2 = getSettings().getInteger(MA2_PERIOD_TF2);
//Object maInput = getSettings().getInput(MA_INPUT, Enums.BarInput.CLOSE);
BarSize barSizeTF2 = getSettings().getBarSize(MA_BARSIZE_TF2);
DataSeries series2 = ctx.getDataSeries(barSizeTF2);
StudyHeader header = getHeader();
boolean updates = getSettings().isBarUpdates() || (header != null && header.requiresBarUpdates());
// Calculate Moving Average for the Secondary Data Series
for(int i = 1; i < series2.size(); i++) {
if (series2.isComplete(i)) continue;
if (!updates && !series2.isBarComplete(i)) continue;
// MA TF2
Double ma9 = series2.ma(getSettings().getMAMethod(MA_METHOD_TF2), i, maPeriodTF2, getSettings().getInput(MA_INPUT_TF2));
Double ma11 = series2.ma(getSettings().getMAMethod(MA2_METHOD_TF2), i, ma2PeriodTF2, getSettings().getInput(MA2_INPUT_TF2));
series2.setDouble(i, Values.MA9_H4, ma9);
series2.setDouble(i, Values.MA11_H4, ma11);
}
// Invoke the parent method to run the "calculate" method below for the primary (chart) data series
super.calculateValues(ctx);
I would now like to use those 2 values, 'ma9' and 'ma11' in another function, on the 1 hour timeframe:
#Override
protected void calculate(int index, DataContext ctx)
DataSeries series=ctx.getDataSeries();
if (ma9 < ma11 && other conditions)
{ctx.signal(index, Signals.YOU_SHOULD_BUY, "This would be my buying signal", series.getClose(index));
}
How can I export the ma9 and the ma11 so they become 'global' and I can re-use them in this other function ?

Basically, the idea is to store somewhere the values or just pass them appropriately after being computed.
There is a java pattern based on singleton that allow you to store/retrieve values inside a class (using a collection : HashMap). Any values could be added,retried in any classes based on predefined (key,value) using the construction Singelton.getInstance() with HashMap standard operation (put, get).
Maybe this example could be useful.
import java.util.Hashtable;
class Singleton extends Hashtable<String, Object> {
private static final long serialVersionUID = 1L;
private static Singleton one_instance = null;
private Singleton() {
};
public static Singleton getInstance() {
one_instance = (one_instance == null) ? new Singleton() : one_instance;
return one_instance;
}
}
import java.util.Random;
public class Reuse {
public static void main(String[] args) {
Reuse r = new Reuse();
Compute c = r.new Compute();
Singleton.getInstance().put("r1", c.getRandom());
Singleton.getInstance().put("r2", c.getRandom());
Singleton.getInstance().put("n", c.getName());
System.out.println(Singleton.getInstance().get("r1"));//print random_number_1
System.out.println(Singleton.getInstance().get("r2"));//print random_number_2
System.out.println(Singleton.getInstance().get("n"));// print name (value for key n)
}
class Compute
{
public Double getRandom()
{
return new Random().nextDouble();
}
public String getName()
{
return "name";
}
}
}

Is there a way to repeat a instance variable call multiple times in java

I am trying to repeat an instant variable call N times in java as follows:
System.out.println(ClassName.InstantVarialble);
System.out.println(ClassName.IntantVariable * N);
//where N = 3, should preform the following:
System.out.println(ClassName.IntantVariable.IntantVariable.IntantVariable);
Is this even possible? if not, is there a built in java function that can do what im asking for or can i get ideas on how to code a method that could repeat the instant call?
im working on Doubly Linked List that doesnt return to the null position here is an image of what im trying to accomplish doesnt return to null position. its similar to what they do in this video https://youtu.be/BspFdzVvYe8?list=PLPf65fTMT69iUS9A43IyaB1Rz3BBOiPpa

It's possible with reflection, though I don't recommend it. I'm not sure why you would want to do this: the fields would have to be named identically.
class A {
public static int field = 99;
}
class B {
public static A field = new A();
}
class C {
public static B field = new B();
}
public class Blah
{
public static String foo(Class<?> clazz, String fieldName, int n)
throws NoSuchFieldException, IllegalAccessException
{
for (int i = 0; i < n - 1; ++i)
{
clazz = clazz.getField(fieldName).get(null).getClass();
}
return clazz.getField(fieldName).get(null).toString();
}
public static void main(String... args)
throws NoSuchFieldException, IllegalAccessException
{
System.out.println(foo(C.class, "field", 1));
System.out.println(foo(C.class, "field", 2));
System.out.println(foo(C.class, "field", 3));
}
}
Run it here. Sample output:
B#65e579dcA#61baa89499
You will need to allow access if want to be able to do this for private fields.

Java multi-thread scalability issue

more updates
As is explained in the selected answer, the problem is in JVM's garbage collection algorithm.
JVM uses card marking algorithm to keep track of modified references in object fields. For each reference assignment to a field, it marks an associated bit in the card to be true -- this causes a false-sharing hence blocks scaling. The details are well described in this article: https://blogs.oracle.com/dave/entry/false_sharing_induced_by_card
The option -XX:+UseCondCardMark (in Java 1.7u40 and up) mitigates the problem, and makes it scale almost perfectly.
updates
I found out (hinted from Park Eung-ju) that assigning an object into a field variable makes the difference. If I remove the assignment, it scales perfectly.
I think probably it has something to do with Java memory model -- such as, an object reference must point to a valid address before it gets visible, but I am not completely sure. Both double and Object reference (likely) have 8 bytes size on 64 bit machine, so it seems to me that assigning a double value and an Object reference should be the same in terms of synchronization.
Anyone has a reasonable explanation?
Here I have a weird Java multi-threading scalability problem.
My code simply iterates over an array (using the visitor pattern) to compute simple floating-point operations and assign the result to another array. There is no data dependency, nor synchronization, so it should scale linearly (2x faster with 2 threads, 4x faster with 4 threads).
When primitive (double) array is used, it scales very well. When object type (e.g. String) array is used, it doesn't scale at all (even though the value of the String array is not used at all...)
Here's the entire source code:
import java.util.ArrayList;
import java.util.Arrays;
import java.util.concurrent.CyclicBarrier;
class Table1 {
public static final int SIZE1=200000000;
public static final boolean OBJ_PARAM;
static {
String type=System.getProperty("arg.type");
if ("double".equalsIgnoreCase(type)) {
System.out.println("Using primitive (double) type arg");
OBJ_PARAM = false;
} else {
System.out.println("Using object type arg");
OBJ_PARAM = true;
}
}
byte[] filled;
int[] ivals;
String[] strs;
Table1(int size) {
filled = new byte[size];
ivals = new int[size];
strs = new String[size];
Arrays.fill(filled, (byte)1);
Arrays.fill(ivals, 42);
Arrays.fill(strs, "Strs");
}
public boolean iterate_range(int from, int to, MyVisitor v) {
for (int i=from; i<to; i++) {
if (filled[i]==1) {
// XXX: Here we are passing double or String argument
if (OBJ_PARAM) v.visit_obj(i, strs[i]);
else v.visit(i, ivals[i]);
}
}
return true;
}
}
class HeadTable {
byte[] filled;
double[] dvals;
boolean isEmpty;
HeadTable(int size) {
filled = new byte[size];
dvals = new double[size];
Arrays.fill(filled, (byte)0);
isEmpty = true;
}
public boolean contains(int i, double d) {
if (filled[i]==0) return false;
if (dvals[i]==d) return true;
return false;
}
public boolean contains(int i) {
if (filled[i]==0) return false;
return true;
}
public double groupby(int i) {
assert filled[i]==1;
return dvals[i];
}
public boolean insert(int i, double d) {
if (filled[i]==1 && contains(i,d)) return false;
if (isEmpty) isEmpty=false;
filled[i]=1;
dvals[i] = d;
return true;
}
public boolean update(int i, double d) {
assert filled[i]==1;
dvals[i]=d;
return true;
}
}
class MyVisitor {
public static final int NUM=128;
int[] range = new int[2];
Table1 table1;
HeadTable head;
double diff=0;
int i;
int iv;
String sv;
MyVisitor(Table1 _table1, HeadTable _head, int id) {
table1 = _table1;
head = _head;
int elems=Table1.SIZE1/NUM;
range[0] = elems*id;
range[1] = elems*(id+1);
}
public void run() {
table1.iterate_range(range[0], range[1], this);
}
//YYY 1: with double argument, this function is called
public boolean visit(int _i, int _v) {
i = _i;
iv = _v;
insertDiff();
return true;
}
//YYY 2: with String argument, this function is called
public boolean visit_obj(int _i, Object _v) {
i = _i;
iv = 42;
sv = (String)_v;
insertDiff();
return true;
}
public boolean insertDiff() {
if (!head.contains(i)) {
head.insert(i, diff);
return true;
}
double old = head.groupby(i);
double newval=Math.min(old, diff);
head.update(i, newval);
head.insert(i, diff);
return true;
}
}
public class ParTest1 {
public static int THREAD_NUM=4;
public static void main(String[] args) throws Exception {
if (args.length>0) {
THREAD_NUM = Integer.parseInt(args[0]);
System.out.println("Setting THREAD_NUM:"+THREAD_NUM);
}
Table1 table1 = new Table1(Table1.SIZE1);
HeadTable head = new HeadTable(Table1.SIZE1);
MyVisitor[] visitors = new MyVisitor[MyVisitor.NUM];
for (int i=0; i<visitors.length; i++) {
visitors[i] = new MyVisitor(table1, head, i);
}
int taskPerThread = visitors.length / THREAD_NUM;
MyThread[] threads = new MyThread[THREAD_NUM];
CyclicBarrier barrier = new CyclicBarrier(THREAD_NUM+1);
for (int i=0; i<THREAD_NUM; i++) {
threads[i] = new MyThread(barrier);
for (int j=taskPerThread*i; j<taskPerThread*(i+1); j++) {
if (j>=visitors.length) break;
threads[i].addVisitors(visitors[j]);
}
}
Runtime r=Runtime.getRuntime();
System.out.println("Force running gc");
r.gc(); // running GC here (excluding GC effect)
System.out.println("Running gc done");
// not measuring 1st run (excluding JIT compilation effect)
for (int i=0; i<THREAD_NUM; i++) {
threads[i].start();
}
barrier.await();
for (int i=0; i<10; i++) {
MyThread.start = true;
long s=System.currentTimeMillis();
barrier.await();
long e=System.currentTimeMillis();
System.out.println("Iter "+i+" Exec time:"+(e-s)/1000.0+"s");
}
}
}
class MyThread extends Thread {
static volatile boolean start=true;
static int tid=0;
int id=0;
ArrayList<MyVisitor> tasks;
CyclicBarrier barrier;
public MyThread(CyclicBarrier _barrier) {
super("MyThread"+(tid++));
barrier = _barrier;
id=tid;
tasks = new ArrayList(256);
}
void addVisitors(MyVisitor v) {
tasks.add(v);
}
public void run() {
while (true) {
while (!start) { ; }
for (int i=0; i<tasks.size(); i++) {
MyVisitor v=tasks.get(i);
v.run();
}
start = false;
try { barrier.await();}
catch (InterruptedException e) { break; }
catch (Exception e) { throw new RuntimeException(e); }
}
}
}
The Java code can be compiled with no dependency, and you can run it with the following command:
java -Darg.type=double -server ParTest1 2
You pass the number of worker threads as an argument (the above uses 2 threads).
After setting up the arrays (that is excluded from the measured time), it does a same operation for 10 times, printing out the execution time at each iteration.
With the above option, it uses double array, and it scales very well with 1,2,4 threads (i.e. the execution time reduces to 1/2, and 1/4), but
java -Darg.type=Object -server ParTest1 2
With this option, it uses Object (String) array, and it doesn't scale at all!
I measured the GC time, but it was insignificant (and I also forced running GC before measuring times). I have tested with Java 6 (updates 43) and Java 7 (updates 51), but it's the same.
The code has comments with XXX and YYY describing the difference when arg.type=double or arg.type=Object option is used.
Can you figure out what is going on with the String-type argument passing here?

HotSpot VM generates following assemblies for reference type putfield bytecode.
mov ref, OFFSET_OF_THE_FIELD(this) <- this puts the new value for field.
mov this, REGISTER_A
shr 0x9, REGISTER_A
movabs OFFSET_X, REGISTER_B
mov %r12b, (REGISTER_A, REGISTER_B, 1)
putfield operation is completed in 1 instruction.
but there are more instructions following.
They are "Card Marking" instructions. (http://www.ibm.com/developerworks/library/j-jtp11253/)
Writing reference field to every objects in a card (512 bytes), will store a value in a same memory address.
And I guess, store to same memory address from multiple cores mess up with cache and pipelines.
just add
byte[] garbage = new byte[600];
to MyVisitor definition.
then every MyVisitor instances will be spaced enough not to share card marking bit, you will see the program scales.

This is not a complete answer but may provide a hint for you.
I have changed your code
Table1(int size) {
filled = new byte[size];
ivals = new int[size];
strs = new String[size];
Arrays.fill(filled, (byte)1);
Arrays.fill(ivals, 42);
Arrays.fill(strs, "Strs");
}
to
Table1(int size) {
filled = new byte[size];
ivals = new int[size];
strs = new String[size];
Arrays.fill(filled, (byte)1);
Arrays.fill(ivals, 42);
Arrays.fill(strs, new String("Strs"));
}
after this change, the running time with 4 threads with object type array reduced.

According to http://docs.oracle.com/javase/specs/jls/se7/html/jls-17.html#jls-17.7
For the purposes of the Java programming language memory model, a single write to a non-volatile long or double value is treated as two separate writes: one to each 32-bit half. This can result in a situation where a thread sees the first 32 bits of a 64-bit value from one write, and the second 32 bits from another write.
Writes and reads of volatile long and double values are always atomic.
Writes to and reads of references are always atomic, regardless of whether they are implemented as 32-bit or 64-bit values.
Assigning references are always atomic,
and double is not atomic except when it is defined as volatile.
The problem is sv can be seen by other threads and its assignment is atomic.
Therefore, wrapping visitor's member variables (i, iv, sv) using ThreadLocal will solve the problem.

"sv = (String)_v;" makes the difference. I also confirmed that the type casting is not the factor. Just accessing _v can't make the difference. Assigning some value to sv field makes the difference. But I can't explain why.

2 dimensional array changing during serialisation

I'm serialising and deserialising a large two dimensional array of objects. Each object contains instructions to creating a BufferedImage - this is done to get around BufferedImage not being directly serializable itself.
The class being serialised is:
public final class MapTile extends TransientImage
{
private static final long serialVersionUID = 0;
private transient BufferedImage f;
transient BufferedImage b;
int along;
int down;
boolean flip = false;
int rot = 0;
public MapTile(World w, int a, int d)
{
// f = w.getMapTiles();
along = a;
down = d;
assignImage();
}
public MapTile(World w, int a, int d, int r, boolean fl)
{
// f = w.getMapTiles();
along = a;
down = d;
rot = r;
flip = fl;
assignImage();
}
public int getA()
{
return along;
}
public int getD()
{
return down;
}
#Override
public void assignImage()
{
if (f == null)
{
f = World.mapTiles;
}
b = f.getSubimage(along, down, World.squareSize, World.squareSize);
if (rot != 0)
{
b = SmallMap.rotateImage(b, rot);
}
if (flip)
{
b = SmallMap.flipImage(b);
}
super.setImage(b);
f.flush();
b.flush();
f = null;
b = null;
}
}
which extends:
public abstract class TransientImage implements Serializable
{
private transient BufferedImage image;
public BufferedImage getImage()
{
return image;
}
public void setImage(BufferedImage i)
{
image = i;
}
public abstract void assignImage();
private void readObject(ObjectInputStream in) throws IOException, ClassNotFoundException
{
in.defaultReadObject();
assignImage();
}
}
This will ultimately be part of a map - usually it is created randomly but certain areas must be the same each time, hence serialising them and reading the array back in. As I will never need to save the image during normal usage I am putting in the write code:
try (ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("verticalroad.necro")))
{
//out.writeObject(mapArray);
//}
//catch (IOException e) {
//}
in the class that creates the map, the read code:
try{
FileInputStream door = new FileInputStream(new File(f.getPath()+ "//verticalroad.necro"));
ObjectInputStream reader = new ObjectInputStream(door);
homeTiles = (MapTile[][]) reader.readObject();
}
catch (IOException | ClassNotFoundException e)
{
System.out.println("Thrown an error" + e.getMessage());
}
in the initialising class and commenting in and out as needed.
However. Each time I run the program the contents of the two dimensional array (mapArray in write, homeTiles in read) is different. Not only different from the one I (thought) I wrote, but also different each time the program is opened.
As can be seen, I'm printing out the toString to System.out which reveals further oddities. As its just a standard array, the toString isn't 100% helpful but it seems to cycle between several distinct values. However, even when the toStringg gives the same value, the contents of the array as displayed are not the same.
An example of a toString is hometiles:[[Lriseofthenecromancer.MapTile;#7681720a Looking at the documentation for Array.toString (here) it seems to be badly formed, lacking a trailing ]. I'm not sure if this is a clue to the issue or if its simply that the array is very large (several thousand objects) and its an issue of display space (I'm using NetBeans).
Any insight as to why this is changing would be appreciated. My working assumption is that its serializing the array but not the contents. But I have no idea a) if that's the case and b)if it is, what to do about it.
EDIT: Looking into this a bit further, it seems that instance variables aren't being set immediately. Printing them out directly after the call to setImage() has them all at zero, printing them from the calling class has them where they should be.

The underlying problem was that I'm an idiot. The specific expression of this in this particular case was that I forgot that subclasses couldn't inherit private methods. As such, the assignImage call wasn't being made and the image wasn't being set up.
Sorry for wasting the time of anyone who looked at this. I feel quite embarrassed.

Javolution ByteBuffer question

I have the following implementation with Javolution:
public class RunScan extends Struct
{
public final Signed32 numOfClusters = new Signed32();
public final ClusterData[] clusters;
public final Signed32 numOfRecons = new Signed32();
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons)
{
this.numOfClusters.set(numOfClusters);
this.numOfRecons.set(numOfRecons);
clusters = array(new ClusterData[numOfClusters]);
recons = array(new ReconData[numOfRecons]);
}
}
public class ClusterData extends Struct
{
public final UTF8String scanType = new UTF8String(CommInterfaceFieldConstants.SCAN_TYPE_SIZE);
public final UTF8String patientId = new UTF8String(CommInterfaceFieldConstants.PATIENT_ID_SIZE);
.
.
.
}
public class ReconData extends Struct
{
public final UTF8String patientId = new UTF8String(CommInterfaceFieldConstants.PATIENT_ID_SIZE);
public final UTF8String scanSeriesId = new UTF8String(CommInterfaceFieldConstants.SCAN_SERIES_ID_SIZE);
.
.
.
}
In our communication class, before we put data onto socket, we need to get the bytes[] of the RunScan object but we get BufferUnderflowException in the line with "//<<<<<<<":
private byte[] getCmdBytes(Struct scCmd)
{
ByteBuffer cmdBuffer = scCmd.getByteBuffer();
int cmdSize = scCmd.size();
byte[] cmdBytes = new byte[cmdSize];
if (cmdBuffer.hasArray())
{
int offset = cmdBuffer.arrayOffset() + scCmd.getByteBufferPosition();
System.arraycopy(cmdBuffer.array(), offset, cmdBytes, 0, cmdSize);
}
else
{
String msg = "\n\ncmdBufferRemaining=" + cmdBuffer.remaining() + ", cmdBytesSize=" + cmdBytes.length + "\n\n";
System.out.println(msg);
cmdBuffer.position(scCmd.getByteBufferPosition());
cmdBuffer.get(cmdBytes); //<<<<<<<<<< underFlowException
}
return cmdBytes;
}
This method works in other cases. The exception happens because this line,
ByteBuffer cmdBuffer = scCmd.getByteBuffer();
only returns a 8 bytes (from the remaining() method) ByteBuffer of the RunScan object which are those two Signed32 fields, I think. But this line,
int cmdSize = scCmd.size();
returns a right length of the RunScan object which includes the size of those two arrays.
If I create those two array at the time I declare them (not "new" them in the constructor) with hard coded length, it works fine without any exception.
Anybody can help me figure out what's wrong with our implementation?

I ran into a similar situation with my code. Generally, with the current Struct object, you cannot have a variable length array defined in the same struct as the member that contains the number of elements in the array.
Try something like this:
public class RunScanHeader extends Struct
{
public final Signed32 numOfClusters = new Signed32();
public final Signed32 numOfRecons = new Signed32();
}
public class RunScanBody extends Struct
{
public final ClusterData[] clusters;
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons)
{
clusters = array(new ClusterData[numOfClusters]);
recons = array(new ReconData[numOfRecons]);
}
}
You'll then need a two phase approach to read and write, first read/write the header data, then read/write the body data.
Sorry I don't have more details at this time, if you can't solve this, let me know and I'll dig back through my code.

The initialization order is important has it defines the position of each field. Either your initialization is done when the field is declared (most common case). Or if you do it in the constructor you have to remember that the constructor is called after the member initialization. Here is an example with initialization done in the constructor:
public class RunScan extends Struct {
public final Signed32 numOfClusters;
public final ClusterData[] clusters;
public final Signed32 numOfRecons;
public final ReconData[] recons ;
public RunScan (int numOfClusters, int numOfRecons) {
// Initialization done in the constructor for all members
// Order is important, it should match the declarative order to ensure proper positioning.
this.numOfClusters = new Signed32();
this.clusters = array(new ClusterData[numOfClusters]);
this.numOfRecons = new Signed32();
this.recons = array(new ReconData[numOfRecons]);
// Only after all the members have been initialized the set method can be used.
this.numOfClusters.set(numOfClusters);
this.numOfRecons.set(numOfRecons);
}
}

get() will move the position of the ByteBuffer.
scCmd.getByteBuffer().slice().get(dest) might solve your issue with moving the position and unintended side effects.
scCmd.getByteBuffer().duplicate().get(dest) might also solve your issue if slice() produces the wrong picture of the origin buffer.
Additionally, it appears as though scCmd.getByteBuffer() creates a redundant reference and you are calling the source and child reference in the same method.
If scCmd.getByteBuffer() is already passing you a slice(), your redundant access to these methods is certainly going to do something other than what you planned.