classifying a dataset in java - java

Hi I am trying to implement my own classifier in a java .Here is what I have gotten so far:
import weka.core.*;
public class RandomProbability extends Classifier {
Instances data;
public RandomProbability ()
{
/*DataSource d = new DataSource("C:\\Program Files\\Weka-3-6\\data\\labor.arff");
data=((Object) d).getSourceData();*/
DataSource source = null;
try {
source = new DataSource("C:\\Program Files\\Weka-3-6\\data\\labor.arff");
} catch (Exception e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
Instances instances = source.getDataSet();
//instances.setClassIndex(instances.numAttributes() - 1);
// Print header and instances.
System.out.println("\nDataset:\n");
System.out.println(instances);
Now the problem is I cant get it to classify the data in the dataset (as good or bad).
I need help in trying to access a single instance in this code.

You could use instanceOf operator as below:
int countA = 0, countB=0;
double pred;
for (int i=0;i<57;i++) {
pred = classifyInstance(instances.instance(i));
System.out.println("===== Classified instance =====");
System.out.println("Class predicted:" + instances.classAttribute().value((int) pred));
if (instances.classAttribute().value((int) pred).toString().equals("bad")) {
countB++;
} else {
countA++;
}
}

package dm;
//import javax.activation.DataSource;
import weka.core.converters.ConverterUtils.DataSource;
import weka.classifiers.*;
import weka.core.*;
import org.jfree.data.*;
import weka.core.*;
public class RandomProbability extends Classifier {
Instances data;
public RandomProbability ()
{
/*DataSource d = new DataSource("C:\\Program Files\\Weka-3-6\\data\\labor.arff");
data=((Object) d).getSourceData();*/
DataSource source = null;
try {
source = new DataSource("C:\\Program Files\\Weka-3-6\\data\\labor.arff");
} catch (Exception e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
try {
Instances instances = source.getDataSet();
instances.setClassIndex(instances.numAttributes() - 1);
// Print header and instances.
System.out.println("\nDataset:\n");
System.out.println(instances);
int total=instances.numInstances();
System.out.println("total"+total);
Attribute attr= instances.attribute(16);
System.out.println("attr"+attr);
// checking class
int countA = 0, countB=0;
double pred;
for (int i=0;i<57;i++)
{
pred=0;
pred = classifyInstance(instances.instance(i));
System.out.println("===== Classified instance =====");
System.out.println("Class predicted:" + instances.classAttribute().value((int) pred));
if (instances.classAttribute().value((int) pred).toString().equals("bad"))
{
countB++;
}
else {
countA++;
}
}
System.out.println("good instances"+countA);
System.out.println("bad instances"+ countB);
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public void buildClassifier (Instances data)
{
}
public double classifyInstance (Instance inst)
{
return 0;
}
}

Related

java multithreading synchronized block not working

I am not able to get why I am getting exception with both : class level lock as well as with object level lock in below code :
It seems object level locking should work here as we are changing and accessing hm(Object) value using different threads, but still we are getting exception(java.util.ConcurrentModificationException).
I tried with all three locking commented in the code.
I know using Hashtable or ConcurrentHashMap we can resolve this problem, but I want to know the concept which is missing using HashMap.
import java.util.HashMap;
class A{
StringBuilder str = new StringBuilder("ABCD");
StringBuilder exception = new StringBuilder("");
HashMap hm = new HashMap();
public void change() {
//synchronized(A.class) {
//synchronized (this){
synchronized (hm){
(this.str).append(Thread.currentThread().getName().toString());
System.out.println(Thread.currentThread().getName()+"::::::"+str);
hm.put(Thread.currentThread(), Thread.currentThread());
}
}
public void impact() {
//synchronized(A.class) {
//synchronized(this) {
synchronized(hm) {
System.out.println(Thread.currentThread()+"...Inside impact :::"+hm.get(Thread.currentThread()));
}
}
public void print() {
System.out.println("Inside print :::"+str);
System.out.println("Inside print :::exception--"+exception);
}
}
class B extends Thread{
A a;
B(A a){
this.a=a;
}
public void run() {
try {
System.out.println("Inside B run::"+a.hm);
a.change();
a.impact();
}
catch(Exception e){
StringWriter sw = new StringWriter();
e.printStackTrace(new PrintWriter(sw));
System.out.println(sw.toString());
a.exception.append(sw.toString());
try {
sw.close();
} catch (IOException e1) {
e1.printStackTrace();
}
}
}
}
class C extends Thread{
A a;
C(A a){
this.a=a;
}
public void run() {
try {
System.out.println("Inside C run::"+a.hm);
a.change();
a.impact();
}
catch(Exception e){
StringWriter sw = new StringWriter();
e.printStackTrace(new PrintWriter(sw));
System.out.println(sw.toString());
a.exception.append(sw.toString());
try {
sw.close();
} catch (IOException e1) {
e1.printStackTrace();
}
}
}
}
public class multiTest {
public static void main(String[] args) {
// TODO Auto-generated method stub
A a = new A();
for(int i=0;i<=100;i++) {
B b = new B(a);
C c = new C(a);
b.start();
c.start();
}
try {
Thread.currentThread().sleep(5000);
}catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
a.print();
}
}
The problem is on this line:
System.out.println("Inside B run::"+a.hm);
There is sneaky implicit invocation of a.hm.toString() here, and that does sneaky iteration of the map's entries; but you aren't synchronizing on anything, so you don't have exclusive access to the hashmap.
Put it in a synchronized block:
synchronized (a.hm) {
System.out.println("Inside B run::"+a.hm);
}
(And make hm final; and don't use raw types).

Constructor.newInstance with Object[] argument

I have been programming java a long time and for some reason I am unable to figure out why this gives me an argument type mismatch. I created a very simple JUnit test anyone can run. Just copy and paste the code below. Any help would be much appreciated. Thanks!
import java.lang.reflect.Constructor;
import java.lang.reflect.InvocationTargetException;
import org.junit.Test;
public class TestObjectArrayConstructor {
#Test
public void testLabelValueObjectArrayConstructor(){
Constructor constructor = null;
try {
constructor = LabelValue.class.getConstructor(
new Class[]{Object[].class});
} catch (NoSuchMethodException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (SecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Object[] array = new Object[]{"Doll"};
Object labelValue = null;
try {
labelValue = constructor.newInstance(array);
} catch (InstantiationException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalArgumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvocationTargetException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("don");
}
}
public class LabelValue {
private String label;
private String value;
public LabelValue(){
}
public LabelValue(Object[] array)
{
if(array != null && array.length > 0)
{
this.label = (String)array[0];
this.value = (String)array[0];
}
}
public String getLabel() {
return label;
}
public void setLabel(String label) {
this.label = label;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
}
try this (not tested) :
constructor.newInstance(new Object[]{array});
because newInstance expects an array of Object (the different args), and your case is ambiguous, as your param is an array of Object.
When you want to create a newInstance from a constructorthen you need to pass your actual parameters, in your case a single argument array as new Object[]:
replace
labelValue = constructor.newInstance(array);
with
labelValue = constructor.newInstance(new Object[]{array});

Is it possible to know the newly added symbol in a HashSet?

I have this piece of code inside my application which runs continuously .
When ever a symbol is added , this below Thread gets fired up and executes two different tasks ( currently the task is represented as sys out for simplicity )
For the first time everything runs fine , but from the second time , the task is being repeated for all the symbols present inside the allSymbolsSet .
The issue i am facing here is that i want to run the task only for the new symbol added . (For example if the allSymbolsSet consists of 3 symbols initially and when a new symbol is added to it , it runs that task for all the 4 symbols , whereas i want it to execute it only for the newly added symbol )
This is my code
package com;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.PriorityBlockingQueue;
public class TaskerThread extends Thread {
private PriorityBlockingQueue<String> priorityBlocking = new PriorityBlockingQueue<String>();
private Set<String> allSymbolsSet = new HashSet<String>();
public void addSymbols(String str) {
if (str != null) {
priorityBlocking.add(str);
}
}
public void run() {
while (true) {
try {
boolean added = false;
while (priorityBlocking.peek() != null) {
added = true;
String symbol = priorityBlocking.poll();
allSymbolsSet.add(symbol);
try {
System.out.println("Symbol From priorityBlocking"+ " " + symbol);
} catch (Exception e) {
e.printStackTrace();
}
}
Iterator<String> ite = allSymbolsSet.iterator();
if (added) {
while (ite.hasNext()) {
String symbol = ite.next();
if (symbol != null && symbol.trim().length() > 0) {
try {
System.out.println("Symbol From allSymbolsSet"+ " " + symbol);
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
Thread.sleep(2000);
} catch (Exception e) {
e.printStackTrace();
}
}
}
public static void main(String args[]) {
try {
TaskerThread qT = new TaskerThread();
qT.start();
qT.addSymbols("SymbolTest");
Thread.sleep(110);
qT.addSymbols("Symbo2222222");
} catch (Exception e) {
e.printStackTrace();
}
}
}
add() method returns false if the Object being added was ignored because it was already present
A simple solution would be to have two hashsets - set1, holding all symbols, set2 containing newly added symbols. Add new symbols to set2, in your thread's run, when the execution is complete, add new symbol to set1 and remove it from set2. How about that?
Well, of course it runs for all elements in the set, you are iterating over them!
package com;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;
import java.util.concurrent.PriorityBlockingQueue;
public class TaskerThread extends Thread {
private final PriorityBlockingQueue<String> priorityBlocking = new PriorityBlockingQueue<String>();
private final Set<String> allSymbolsSet = new Collections.synchronizedSet(new HashSet<String>());
public void addSymbols(String str) {
if (str != null) {
priorityBlocking.add(str);
}
}
public void run() {
while (true) {
try {
while (true) {
final String symbol = priorityBlocking.take();
if (allSymbolsSet.add(symbol)) {
doSomething(symbol); // do whatever you want with the symbol
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public static void main(String args[]) {
try {
TaskerThread qT = new TaskerThread();
qT.start();
qT.addSymbols("SymbolTest");
} catch (Exception e) {
e.printStackTrace();
}
}
}
This should do what you were looking for. Take better care of possible exceptions, namely InterruptedException.

Why these errors while invoking main through reflection

Here is a custom executor that executes an application by searching for its main in the loaded classes loaded by a custom loader. There is a problem when I try to execute a program using my executor.
The source code of the program being executed can be located on the following link
http://agile.csc.ncsu.edu/SEMaterials/realestate/code/RealEstate.zip
package executorOfLoaderClasses;
import java.io.File;
import java.lang.reflect.Field;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.util.ArrayList;
import customClassLoader.ClassLoaderOfExtClass;
/**
* #author Sanyam
*
* sanyamgoyal007#gmail.com
*/
public class ClassExecutor{
private ClassLoaderOfExtClass classLoader;
private byte[][] ArrayOfClasses;
private String[] ArrayOfBinaryNames;
#SuppressWarnings("rawtypes")
private ArrayList<Class> loadedClasses;
private ArrayList<String> loadedClasesNames;
private Object[] parameters;
#SuppressWarnings("rawtypes")
public ClassExecutor() {
classLoader = new ClassLoaderOfExtClass();
new ArrayList<Class>();
loadedClasses = new ArrayList<Class>();
loadedClasesNames = new ArrayList<String>();
}
#SuppressWarnings("unchecked")
public void execute(File[] file, String[] binaryPaths) {
Object[] actuals = { new String[] { "" } };
Method m = null;
try {
Field classesx=ClassLoaderOfExtClass.class.getDeclaredField("classes");
classesx.setAccessible(true);
} catch (SecurityException e1) {
e1.printStackTrace();
} catch (NoSuchFieldException e1) {
e1.printStackTrace();
}
/*for (int i = 0; i < file.length; i++) {
for (int j = 0; j < file.length; j++) {
try {
#SuppressWarnings("rawtypes")
Class c = classLoader.loadClassCustom(file[i], binaryPaths[i]);
//Fied classex=classLoader.getResource("classes");
}catch(Exception e){
}
}
}
Class<?>[]classesxx= getLoadedClasses(classLoader);
System.out.println("Loaded classes have size "+ classesxx.length);*/
for (int i = 0; i < file.length; i++) {
try {
#SuppressWarnings("rawtypes")
Class c = classLoader.loadClassCustom(file[i], binaryPaths[i]);
try {
if (c.getMethod("main", new Class[] { String[].class }) != null) {
m = c.getMethod("main", new Class[] { String[].class });
} else {
System.out.println("This class does not contain main");
continue;
}
} catch (NoSuchMethodException e) {
// System.out.println("Main not found!!!");
// System.out.println("M here");
// e.printStackTrace(); // not printing stack trace
} catch (SecurityException e) {
e.printStackTrace();
}
} catch (ClassNotFoundException e) {
System.out.println("No such class definition exist!!");
// TODO Auto-generated catch block
// e.printStackTrace();
}
}
try {
m.invoke(null, actuals);
// CallStack.print();
} catch (IllegalArgumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvocationTargetException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
#SuppressWarnings({ })
public void execute(ArrayList<byte[]> stuffedFiles,
ArrayList<String> binaryPaths) {
convertToArray(stuffedFiles, binaryPaths);
loadAllClasses(ArrayOfClasses, ArrayOfBinaryNames);
Thread myThread = new MyThread();
myThread.start();
/*Object[] actuals = { new String[] { "" } };
Method m = null;
* Method[] m1= new Method[10]; for (Class c : loadedClasses) {
* m1=c.getMethods(); } for(Method m2: m1){
* System.out.println(m2.getName()); }
System.out.println(loadedClasses.size());
for (Class c : loadedClasses) {
* System.out.println(c.toString());
* System.out.println(c.getConstructors());
// for (int i = 1; i < file.size(); i++) {
* for(Method meth : c.getMethods()){ meth.setAccessible(true);
*
* }
try {
if (c.getMethod("main", new Class[] { String[].class }) != null) {
m = c.getMethod("main", new Class[] { String[].class });
break;
} else {
// System.out.println("This class does not contain main");
continue;
}
} catch (NoSuchMethodException e) {
System.out.println("Program does not contain main");
} catch (SecurityException e) {
e.printStackTrace();
}
}
try {
if(parameters==null){
m.invoke(null, actuals);
}
else{
try {
System.out.println("It Fails Here");
m.invoke(null, parameters);
} catch (Exception e) {
System.out.println("Illegal arguments");
}
}
// CallStack.print();
} catch (IllegalArgumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvocationTargetException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}*/
// remove till here
/*TraceParser pr = new TraceParser();
pr.traceCollector();
pr.traceStruct();
ArrayList<SingleTraceStructure> parsedExpressions = pr
.getTracedObjects();
AllStackTraceValidator validator = new AllStackTraceValidator(
parsedExpressions);
finalObjects = validator.getTraceObjects();
for(SingleTraceStructure ob : finalObjects){
validatedTraceObjects.add(ob);
}
TraceObjectsMinimizer tracerObj = new TraceObjectsMinimizer();
tracerObj.sortObjects(finalObjects);*/
/*
* for(SingleTraceStructure obj : finalObjects){
* System.out.println(obj.getCalledBy());
* System.out.println(obj.getClassName()+":"+obj.getMethodName()+":"+
* obj.getCallSequenceNumbr()+obj.getCalledBy()); }
*/
}
private void convertToArray(ArrayList<byte[]> classes,
ArrayList<String> binaryPaths) {
/* = new byte[classes.size()][]; */
ArrayOfClasses = new byte[classes.size()][];
ArrayOfBinaryNames = new String[binaryPaths.size()];
int i = 0;
for (byte[] tempClass : classes) {
ArrayOfClasses[i] = tempClass;
i++;
}
int j = 0;
for (String name : binaryPaths) {
ArrayOfBinaryNames[j] = name;
j++;
}
}
#SuppressWarnings("rawtypes")
public void loadAllClasses(byte[][] classes, String[] names) {
for (int i = 0; i < classes.length; i++) {
System.out.println("Round ----->" + i);
Class c = null;
for (int j = 0; j < classes.length; j++) {
if (classes[j] != null) {
try {
c = classLoader.loadClassCustom(classes[j], names[j]);
} catch (ClassNotFoundException e) {
// TODO Auto-generated catch block
System.out.println("unsucessful");
e.printStackTrace();
}
if (c != null) {
System.out.println("loading successfull");
loadedClasses.add(c);
loadedClasesNames.add(names[j]);
classes[j] = null;
names[j] = null;
} else {
// move on
}
} else {
// do nothing
}
}
}
}
#SuppressWarnings("rawtypes")
public ArrayList<Class> getLoadedClasses() {
return loadedClasses;
}
public void parametersToMain(ArrayList<String> strs){
if(strs!=null){
String[] obj = new String[strs.size()];
int i=0;
for(String str : strs){
obj[i]= (String)str;
i++;
}
parameters=obj;
}
else{
parameters=null;
}
}
// return loaded classes of a loader
public static Class<?>[] getLoadedClasses(final ClassLoader loader){
final Class<?>[] classes = getLoadedClasses(loader);
return classes;
}
public class MyThread extends Thread{
Object[] actuals = { new String[] { "" }};
public void run(){
Method m = null;
/*
* Method[] m1= new Method[10]; for (Class c : loadedClasses) {
* m1=c.getMethods(); } for(Method m2: m1){
* System.out.println(m2.getName()); }
*/
/* System.out.println(loadedClasses.size()); */
for (Class<?> c : loadedClasses) {
/*
* System.out.println(c.toString());
* System.out.println(c.getConstructors());
*/
// for (int i = 1; i < file.size(); i++) {
/*
* for(Method meth : c.getMethods()){ meth.setAccessible(true);
*
* }
*/
try {
if (c.getMethod("main", new Class[] { String[].class }) != null) {
m = c.getMethod("main", new Class[] { String[].class });
break;
} else {
// System.out.println("This class does not contain main");
continue;
}
} catch (NoSuchMethodException e) {
System.out.println("Program does not contain main");
} catch (SecurityException e) {
e.printStackTrace();
}
}
try {
if(parameters==null){
//System.out.println("Invoker of" + actuals[1]);
m.invoke(null, actuals);
}
else{
try {
System.out.println("It Fails Here");
m.invoke(null, parameters);
} catch (Exception e) {
System.out.println("Illegal arguments");
}
}
// CallStack.print();
} catch (IllegalArgumentException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IllegalAccessException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (InvocationTargetException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
The problem with my executor is while executing the realestate program genetraes the following error
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at executorOfLoaderClasses.ClassExecutor$MyThread.run(ClassExecutor.java:365)
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
at edu.ncsu.realestate.gui.Main.main(Main.java:39)
... 5 more
I realized that while invoking the main I am passing a parameter to the main which is an object of string "" however realestate program does not take any arguments by default ,so for the time being I changed the argument passed to the method to null just to check whether it works fine now.
and again an error was generated as shown below
java.lang.IllegalArgumentException: wrong number of arguments
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at executorOfLoaderClasses.ClassExecutor$MyThread.run(ClassExecutor.java:366)
Why is this happening the realestate program as mentioned in the http link does not take any arguments by default.
Is this way of execution buggy to other possible input programs ??
The realestate program I mentioned is working absolutely fine when run in eclipse as a Java application
The actual bug in the code from the question
The Main method you invoke expects either zero or two arguments, but you pass one.
public static void main(String[] args) {
GameMaster master = GameMaster.instance();
MainWindow window = new MainWindow();
GameBoard gameBoard = null;
if(args.length > 0) {
if(args[0].equals("test")) {
master.setTestMode(true);
}
try {
Class c = Class.forName(args[1]); // <-- this is l. 39
gameBoard = (GameBoard)c.newInstance();
}
You should write
Object[] actuals = { new String[] { }};
or
Object[] actuals = { new String[0] };
in order to pass not a single empty argument, but no argument at all.
General information about argument types
If main is the startup method of a Java application, it always has signature
public static void main(String[] args)
For this reason, from the java perspective it will always be a method expecting a single argument, which is an array of strings. So on the command line, you use a variable number of arguments, but within the Java application, it is always a single array. Some developers might declare the arguments as String... args, which means that you might pass a variable number of strings in the Java application, but that is only syntactic sugar which the compiler will translate into an array creation. For reflection, you will always have to pass the array yourself, as there is no automatic array creation for variadic methods there. So always pass a single argument of type String[] via reflection.
Method.invoke takes an Object... for the arguments, so you can either pass an Object[] containing all the Java arguments, or you can pass each argument separately and have the compiler construct an Object[] from those. So it is up to you whether you pass the String[] array directly to invoke or wrap it in an Object[] as you did with your actuals.

What is the best way to write to a file in a parallel thread in Java?

I have a program that performs lots of calculations and reports them to a file frequently. I know that frequent write operations can slow a program down a lot, so to avoid it I'd like to have a second thread dedicated to the writing operations.
Right now I'm doing it with this class I wrote (the impatient can skip to the end of the question):
public class ParallelWriter implements Runnable {
private File file;
private BlockingQueue<Item> q;
private int indentation;
public ParallelWriter( File f ){
file = f;
q = new LinkedBlockingQueue<Item>();
indentation = 0;
}
public ParallelWriter append( CharSequence str ){
try {
CharSeqItem item = new CharSeqItem();
item.content = str;
item.type = ItemType.CHARSEQ;
q.put(item);
return this;
} catch (InterruptedException ex) {
throw new RuntimeException( ex );
}
}
public ParallelWriter newLine(){
try {
Item item = new Item();
item.type = ItemType.NEWLINE;
q.put(item);
return this;
} catch (InterruptedException ex) {
throw new RuntimeException( ex );
}
}
public void setIndent(int indentation) {
try{
IndentCommand item = new IndentCommand();
item.type = ItemType.INDENT;
item.indent = indentation;
q.put(item);
} catch (InterruptedException ex) {
throw new RuntimeException( ex );
}
}
public void end(){
try {
Item item = new Item();
item.type = ItemType.POISON;
q.put(item);
} catch (InterruptedException ex) {
throw new RuntimeException( ex );
}
}
public void run() {
BufferedWriter out = null;
Item item = null;
try{
out = new BufferedWriter( new FileWriter( file ) );
while( (item = q.take()).type != ItemType.POISON ){
switch( item.type ){
case NEWLINE:
out.newLine();
for( int i = 0; i < indentation; i++ )
out.append(" ");
break;
case INDENT:
indentation = ((IndentCommand)item).indent;
break;
case CHARSEQ:
out.append( ((CharSeqItem)item).content );
}
}
} catch (InterruptedException ex){
throw new RuntimeException( ex );
} catch (IOException ex) {
throw new RuntimeException( ex );
} finally {
if( out != null ) try {
out.close();
} catch (IOException ex) {
throw new RuntimeException( ex );
}
}
}
private enum ItemType {
CHARSEQ, NEWLINE, INDENT, POISON;
}
private static class Item {
ItemType type;
}
private static class CharSeqItem extends Item {
CharSequence content;
}
private static class IndentCommand extends Item {
int indent;
}
}
And then I use it by doing:
ParallelWriter w = new ParallelWriter( myFile );
new Thread(w).start();
/// Lots of
w.append(" things ").newLine();
w.setIndent(2);
w.newLine().append(" more things ");
/// and finally
w.end();
While this works perfectly well, I'm wondering:
Is there a better way to accomplish this?
Your basic approach looks fine. I would structure the code as follows:
import java.io.BufferedWriter;
import java.io.File;
import java.io.IOException;
import java.io.Writer;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
import java.util.concurrent.TimeUnit;
public interface FileWriter {
FileWriter append(CharSequence seq);
FileWriter indent(int indent);
void close();
}
class AsyncFileWriter implements FileWriter, Runnable {
private final File file;
private final Writer out;
private final BlockingQueue<Item> queue = new LinkedBlockingQueue<Item>();
private volatile boolean started = false;
private volatile boolean stopped = false;
public AsyncFileWriter(File file) throws IOException {
this.file = file;
this.out = new BufferedWriter(new java.io.FileWriter(file));
}
public FileWriter append(CharSequence seq) {
if (!started) {
throw new IllegalStateException("open() call expected before append()");
}
try {
queue.put(new CharSeqItem(seq));
} catch (InterruptedException ignored) {
}
return this;
}
public FileWriter indent(int indent) {
if (!started) {
throw new IllegalStateException("open() call expected before append()");
}
try {
queue.put(new IndentItem(indent));
} catch (InterruptedException ignored) {
}
return this;
}
public void open() {
this.started = true;
new Thread(this).start();
}
public void run() {
while (!stopped) {
try {
Item item = queue.poll(100, TimeUnit.MICROSECONDS);
if (item != null) {
try {
item.write(out);
} catch (IOException logme) {
}
}
} catch (InterruptedException e) {
}
}
try {
out.close();
} catch (IOException ignore) {
}
}
public void close() {
this.stopped = true;
}
private static interface Item {
void write(Writer out) throws IOException;
}
private static class CharSeqItem implements Item {
private final CharSequence sequence;
public CharSeqItem(CharSequence sequence) {
this.sequence = sequence;
}
public void write(Writer out) throws IOException {
out.append(sequence);
}
}
private static class IndentItem implements Item {
private final int indent;
public IndentItem(int indent) {
this.indent = indent;
}
public void write(Writer out) throws IOException {
for (int i = 0; i < indent; i++) {
out.append(" ");
}
}
}
}
If you do not want to write in a separate thread (maybe in a test?), you can have an implementation of FileWriter which calls append on the Writer in the caller thread.
One good way to exchange data with a single consumer thread is to use an Exchanger.
You could use a StringBuilder or ByteBuffer as the buffer to exchange with the background thread. The latency incurred can be around 1 micro-second, doesn't involve creating any objects and which is lower using a BlockingQueue.
From the example which I think is worth repeating here.
class FillAndEmpty {
Exchanger<DataBuffer> exchanger = new Exchanger<DataBuffer>();
DataBuffer initialEmptyBuffer = ... a made-up type
DataBuffer initialFullBuffer = ...
class FillingLoop implements Runnable {
public void run() {
DataBuffer currentBuffer = initialEmptyBuffer;
try {
while (currentBuffer != null) {
addToBuffer(currentBuffer);
if (currentBuffer.isFull())
currentBuffer = exchanger.exchange(currentBuffer);
}
} catch (InterruptedException ex) { ... handle ... }
}
}
class EmptyingLoop implements Runnable {
public void run() {
DataBuffer currentBuffer = initialFullBuffer;
try {
while (currentBuffer != null) {
takeFromBuffer(currentBuffer);
if (currentBuffer.isEmpty())
currentBuffer = exchanger.exchange(currentBuffer);
}
} catch (InterruptedException ex) { ... handle ...}
}
}
void start() {
new Thread(new FillingLoop()).start();
new Thread(new EmptyingLoop()).start();
}
}
Using a LinkedBlockingQueue is a pretty good idea. Not sure I like some of the style of the code... but the principle seems sound.
I would maybe add a capacity to the LinkedBlockingQueue equal to a certain % of your total memory.. say 10,000 items.. this way if your writing is going too slow, your worker threads won't keep adding more work until the heap is blown.
I know that frequent write operations
can slow a program down a lot
Probably not as much as you think, provided you use buffering.

Categories