I'm beginner to Java and as my homework I'm supposed to implement concurrency to genetic algorithm solution for Travelling Salesman Problem posted here. Our goal is to make chromosome evaluation performed by threads. So my guess is I have to rewrite this part of code to be multithreaded:
// Gets the best tour in the population
public Tour getFittest() {
Tour fittest = tours[0];
// Loop through individuals to find fittest
for (int i = 1; i < populationSize(); i++) {
if (fittest.getFitness() <= getTour(i).getFitness()) {
fittest = getTour(i);
}
}
return fittest;
}
// Gets population size
public int populationSize() {
return tours.length;
}
Originaly I intended on manually splitting the Array beetwen threads but I believe it;s not the best solution to the problem. So I made some research and everyone suggest to use either parallel streams or ExecutorService. However I had trouble applying both of this solutions even thought I tried to emulate examples posted in other threads. So my questions are: how exactly do I implement them in this case and which one is faster?
Edit: Sorry, I forget to post solution I've tried. Here it is:
public Tour getFittest() {
Tour fittest = tours[0];
synchronized (fittest) {
final ExecutorService executor = Executors.newFixedThreadPool(4);
final List<Future<?>> futures = new ArrayList<>();
for (int i = 1; i < populationSize(); i++) {
Future<?> future = executor.submit((Runnable) () -> {
if (fittest.getFitness() <= getTour(i).getFitness()) {
fittest = getTour(i);
}
});
futures.add(future);
}
try {
for (Future<?> future : futures) {
future.get();
}
}catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return fittest;
}
public int populationSize() {
return tours.length;
}
However when trying to run it I receive "Local variable fittest defined in an enclosing scope must be final or effectively final" error at line:
fittest = getTour(i);
And I have no clue why it's happening or how can I fix it as adding final keyword while initializing it does not fix it. Other than that I have some doubts about using synchronized keyword in this solution. I believe that to achieve true multithreading I need to make use on it due to resource being shared by various threads. Am I right? Sadly I didn't saved my attemp at using streams but I have trouble understanding how it works at all.
Edit2: I managed to "fix" my solution by adding two workarounds. Currently my code looks like that:
public Tour getFittest() {
Tour fittest = tours[0];
synchronized (fittest) {
final ExecutorService executor = Executors.newFixedThreadPool(4);
final List<Future<?>> futures = new ArrayList<>();
for (int i = 1; i < populationSize(); i++) {
final Integer innerI = new Integer(i);
Future<?> future = executor.submit((Runnable) () -> {
if (fittest.getFitness() <= getTour(innerI).getFitness()) {
setFitness(innerI, fittest);
}
}
);
futures.add(future);
}
try {
for (Future<?> future : futures) {
future.get();
}
}catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
return fittest;
}
public int populationSize() {
return tours.length;
}
public Tour setFitness (int i, Tour fittest) {
fittest = getTour(i);
return fittest;
}
That said, while it's compiling, there are two problems. Memory usage keeps rising every second the program runs, maxing out my 16GB of RAM in like ten seconds while variable 'fittest' does not change at all. So I guess I'm still doing something wrong.
Here is my steams implementation:
private static Tour getFittest(Tour[] tours){
List<Map.Entry<Tour,Double>> lengths = new ArrayList<>();
Arrays.stream(tours).parallel().forEach(t->lengths.add(new AbstractMap.SimpleEntry<Tour,Double>(t,t.getLength())));
return Collections.min(lengths,Comparator.comparingDouble(Map.Entry::getValue)).getKey();
}
Upon further looking can be 1liner kinda depending on your definition
private static Tour getFittest(Tour[] tours) {
return Arrays.stream(tours).parallel().map(t -> new AbstractMap.SimpleEntry<Tour, Double>(t, t.getLength()))
.min(Comparator.comparingDouble(Map.Entry::getValue)).get().getKey();
}
also after further looking they use .getFitness() which is reciprocal of length. if you use that then use .max() as the filter.
actually even better after review
return Arrays.stream(tours).parallel()
.min(Comparator.comparingDouble(Tour::getLength)).get();
Related
I'm trying to split a list of objects within smaller sublist and to process them separately on different threads. So I have following code:
List<Instance> instances = xmlInstance.readInstancesFromXml();
List<Future<List<Instance>>> futureList = new ArrayList<>();
int nThreads = 4;
ExecutorService executor = Executors.newFixedThreadPool(nThreads);
final List<List<Instance>> instancesPerThread = split(instances, nThreads);
for (List<Instance> instancesThread : instancesPerThread) {
if (instancesThread.isEmpty()) {
break;
}
Callable<List<Instance>> callable = new MyCallable(instancesThread);
Future<List<Instance>> submit = executor.submit(callable);
futureList.add(submit);
}
instances.clear();
for (Future<List<Instance>> future : futureList) {
try {
final List<Instance> instancesFromFuture = future.get();
instances.addAll(instancesFromFuture);
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
}
executor.shutdown();
try {
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException ie) {
ie.printStackTrace();
}
And the MyCallable class :
public class MyCallable implements Callable<List<Instance>> {
private List<Instance> instances;
public MyCallable (List<Instance> instances) {
this.instances = Collections.synchronizedList(instances);
}
#Override
public List<Instance> call() throws Exception {
for (Instance instance : instances) {
//process each object and changing some fields;
}
return instances;
}
}
Split method(It's split an given list in given number of list, also trying to have almost same size on each sublist) :
public static List<List<Instance>> split(List<Instance> list, int nrOfThreads) {
List<List<Instance>> parts = new ArrayList<>();
final int nrOfItems = list.size();
int minItemsPerThread = nrOfItems / nrOfThreads;
int maxItemsPerThread = minItemsPerThread + 1;
int threadsWithMaxItems = nrOfItems - nrOfThreads * minItemsPerThread;
int start = 0;
for (int i = 0; i < nrOfThreads; i++) {
int itemsCount = (i < threadsWithMaxItems ? maxItemsPerThread : minItemsPerThread);
int end = start + itemsCount;
parts.add(list.subList(start, end));
start = end;
}
return parts;
}
So, when I'm trying to execute it, I'm getting java.util.ConcurrentModificationException on this line for (Instance instance : instances) {, can somebody give any ideas why it's happening?
public MyCallable (List<Instance> instances) {
this.instances = Collections.synchronizedList(instances);
}
Using synchronizedList like this doesn't help you in the way you think it might.
It's only useful to wrap a list in a synchronizedList at the time you create it (e.g. Collections.synchronizedList(new ArrayList<>()). Otherwise, the underlying list is directly accessible, and thus accessible in an unsynchronized way.
Additionally, synchronizedList only synchronizes for the duration of individual method calls, not for the whole time while you are iterating over it.
The easiest fix here is to take a copy of the list in the constructor:
this.instances = new ArrayList<>(instances);
Then, nobody else has access to that list, so they can't change it while you are iterating it.
This is different to taking a copy of the list in the call method, because the copy is done in a single-threaded part of the code: no other thread can be modifying it while you are taking that copy, so you won't get the ConcurrentModificationException (you can get a CME in single-threaded code, but not using this copy constructor). Doing the copy in the call method means the list is iterated, in exactly the same way as with the for loop you already have.
I have the following code :
import java.util.ArrayList;
public class main {
final static Object lock= new Object();
public static void main(String[] args) {
for (int i = 0; i < 100000; i++) {
System.out.println("-------");
finish finished = new finish(false);
ArrayList<Boolean> arr = new ArrayList<>();
Thread t1 = new Thread(() -> {
System.out.println(Thread.currentThread().getId() + " Is setting");
finished.setFinished(true);
});
t1.start();
synchronized (lock){
if (finished.isFinished == false) {
System.out.println(Thread.currentThread().getId() + " Is adding");
arr.add(new Boolean(finished.isFinished));
} else {
System.out.println("Done");
}
}
System.out.println("The length of array is " + arr.size());
if (arr.size() > 0) {
System.out.println("The val of array is " + arr.get(0));
}
}
}
}
class finish {
public boolean isFinished = false;
public finish(boolean finished) {
this.isFinished = finished;
}
public void setFinished(boolean finished) {
this.isFinished = finished;
}
}
I am expecting to get the following output :
The length of array is 1
The val of array is false
or
Done
It is the case most of the times.
But sometimes the output is :
The length of array is 1
The val of array is true
It means, that there was a context switch in the critical section.
I have tried to synchronized the code on an object, but it did not help.
It is a classical synchronization problem, I but was not able to solve it.
Maybe I should use atomic objects, but I have no idea how would they help in this case.
Or perhaps I am to harsh with java, and I should not test it in a for loop? I am running this example on a Linux OS.
I think my synchronization does not make sense. But I don't know how to solve it.
It's not at all clear what your example code is trying to do, but if you want to wait for the thread to finish, use t1.join() and deal with the InterruptedException that this throws. Also, finished should be an AtomicBoolean if you're going to use it in more than one thread.
But all in all, the code is pretty problematic, and doesn't reflect whatever real-life scenario you're trying to deal with.
In this code you are reading the same variable twice, which allows for the possibility it can be changed. The simplest solution is to read it just once.
boolean isFinished = finished.isFinished;
if (isFinished) {
System.out.println("Done");
} else {
System.out.println(t1 + " Is adding");
arr.add(isFinished);
}
I am implementing an application using concurrent hash maps. It is required that one thread adds data into the CHM, while there is another thread that copies the values currently in the CHM and erases it using the clear() method. When I run it, after the clear() method is executed, the CHM always remains empty, though the other thread continues adding data to CHM.
Could someone tell me why it is so and help me find the solution.
This is the method that adds data to the CHM. This method is called from within a thread.
import java.util.concurrent.ConcurrentMap;
import java.util.concurrent.ConcurrentHashMap;
public static ConcurrentMap<String, String> updateJobList = new ConcurrentHashMap<String, String>(8, 0.9f, 6);
public void setUpdateQuery(String ticker, String query)
throws RemoteException {
dataBaseText = "streamming";
int n = 0;
try {
updateJobList.putIfAbsent(ticker, query);
}
catch(Exception e)
{e.printStackTrace();}
........................
}
Another thread calls the track_allocation method every minute.
public void track_allocation()
{
class Track_Thread implements Runnable {
String[] track;
Track_Thread(String[] s)
{
track = s;
}
public void run()
{
}
public void run(String[] s)
{
MonitoringForm.txtInforamtion.append(Thread.currentThread()+"has started runnning");
String query = "";
track = getMaxBenefit(track);
track = quickSort(track, 0, track.length-1);
for(int x=0;x<track.length;x++)
{
query = track[x].split(",")[0];
try
{
DatabaseConnection.insertQuery(query);
}
catch(Exception e)
{
e.printStackTrace();
}
}
}
}
joblist = updateJobList.values();
MonitoringForm.txtInforamtion.append("\nSize of the joblist is:"+joblist.size());
int n = joblist.size()/6;
String[][] jobs = new String[6][n+6];
MonitoringForm.txtInforamtion.append("number of threads:"+n);
int i = 0;
if(n>0)
{
MonitoringForm.txtInforamtion.append("\nSize of the joblist is:"+joblist.size());
synchronized(this)
{
updateJobList.clear();
}
Thread[] threads = new Thread[6];
Iterator it = joblist.iterator();
int k = 0;
for(int j=0;j<6; j++)
{
for(k = 0; k<n; k++)
{
jobs[j][k] = it.next().toString();
MonitoringForm.txtInforamtion.append("\n\ninserted into queue:\n"+jobs[j][k]+"\n");
}
if(it.hasNext() && j == 5)
{
while(it.hasNext())
{
jobs[j][++k] = it.next().toString();
}
}
threads[j] = new Thread(new Track_Thread(jobs[j]));
threads[j].start();
}
}
}
I can see a glaring mistake. This is the implementation of your Track_Thread classes run method.
public void run()
{
}
So, when you do this:
threads[j] = new Thread(new Track_Thread(jobs[j]));
threads[j].start();
..... the thread starts, and then immediately ends, having done absolutely nothing. Your run(String[]) method is never called!
In addition, your approach of iterating the map and then clearing it while other threads are simultaneously adding is likely to lead to entries occasionally being removed from the map without the iteration actually seeing them.
While I have your attention, you have a lot of style errors in your code:
The indentation is a mess.
You have named your class incorrectly: it is NOT a thread, and that identifier ignores the Java identifier rule.
Your use of white-space in statements is inconsistent.
These things make your code hard to read ... and to be frank, they put me off trying to really understand it.
In my program I essentially have a method similar to:
for (int x=0; x<numberofimagesinmyfolder; x++){
for(int y=0; y<numberofimagesinmyfolder; y++){
compare(imagex,imagey);
if(match==true){
System.out.println("image x matches image y");
}
}
}
So basically I have a folder of images and I compare all combinations of images...so compare image 1 against all images then image 2...and so on. My problem is when searching to see what images match, it takes a long time. I am trying to multithread this process. Does anyone have any idea of how to do this?
Instead of comparing the images every time, hash the images, save the hash, and then compare the hashes of each pair of messages. Since a hash is far smaller you can fit more into memory and cache, which should significantly speed up comparisons.
There is probably a better way to do the search for equality as well, but one option would be to stick all the hashes into an array then sort them by hash value. Then iterate over the list looking for adjacent entries that are equal. This should be O(n*log(n)) instead of O(n^2) like your current version.
inner loop should start at y=x+1 to take advantage of symmetry.
load all images into memory first. don't do all compares from disk.
Use a Java ExecutorService (basically a thread pool). Queue tasks for all index combinations. Let threads pull index combinations out of a task queue and execute comparisons.
Here is some general code to do the multi threading:
public static class CompareTask implements Runnable {
CountDownLatch completion;
Object imgA;
Object imgB;
public CompareTask(CountDownLatch completion, Object imgA, Object imgB) {
this.completion = completion;
this.imgA = imgA;
this.imgB = imgB;
}
#Override
public void run() {
// TODO: Do computation...
try {
System.out.println("Thread simulating task start.");
Thread.sleep(500);
System.out.println("Thread simulating task done.");
} catch (InterruptedException e) {
e.printStackTrace();
}
completion.countDown();
}
}
public static void main(String[] args) throws Exception {
Object[] images = new Object[10];
ExecutorService es = Executors.newFixedThreadPool(5);
CountDownLatch completion = new CountDownLatch(images.length * (images.length - 1) / 2);
for (int i = 0; i < images.length; i++) {
for (int j = i + 1; j < images.length; j++) {
es.submit(new CompareTask(completion, images[i], images[j]));
}
}
System.out.println("Submitted tasks. Waiting...");
completion.await();
System.out.println("Done");
es.shutdown();
}
I'm new to using threads and just trying to figure things out. My end game is to have a list of URLs, my program will take one URL from the list at a time and perform an action using that URL. There'll be a lot of URLs and this list may possibly be added to while some threads are using the same list.
To start experimenting and learning I'm using a simple ArrayList filled with numbers and am using a threaded pool to get the URLs. Here's my code:
public static void main(String[] args) {
for (int i = 0; i < 200; i++){
test.add(i);
}
SlothTest runner = new SlothTest();
Thread alpha = new Thread(runner);
Thread beta = new Thread(runner);
ExecutorService tasker = Executors.newFixedThreadPool(10);
while (!listEmpty()){
tasker.submit(new SlothTest());
}
tasker.shutdown();
System.out.println("Complete...");
}
#Override
public void run() {
getLink();
try {
Thread.sleep(20);
} catch (InterruptedException e) {
}
}
private synchronized String getLink(){
link = Thread.currentThread().getName() + " printed " + test.indexOf(test.size()-1);
test.remove(test.size()-1);
System.out.println(link);
return link;
}
private synchronized static boolean listEmpty(){
if (test.size() > 0){
return false;
} else {
return true;
}
}
I'm running into some concurrency issues while running the program and getting some -1's for my output. I'm not sure why this is happening and I know my above code is rough but I'm really in the learning stage a multi-threaded apps. Can anyone help me first off with fixing my concurrency issue and then if you can give me any pointers about my above code that would also be great
One problem is that
while (!listEmpty()){
tasker.submit(new SlothTest());
}
is not atomic. So listEmpty might return false, but become true by the time you reach the next statement.
Another one is that you synchronize on two different monitors:
private synchronized String getLink(){ //synchronized on this
private synchronized static boolean listEmpty(){//synchronized on this.class
Have you considered using a BlockingQueue instead of a list, which has useful methods for what you are trying to achieve.
Try using a ConcurrentLinkedQueue for your list of URLs. This is a good implementation often used in producer-consumer examples, similar to yours (although you don't have an active 'producer', per-se).
You're not globally synchronizing. By using synchronized methods you are locking the current instance, which is different for each task. You should use a global lock instead:
final static Object globalLock = new Object();
private String getLink() {
synchronized (globalLock) {
link = Thread.currentThread().getName() + " printed " + test.indexOf(test.size()-1);
test.remove(test.size()-1);
}
System.out.println(link);
return link;
}
private boolean listEmpty(){
synchronized (globalLock) {
if (test.size() > 0){
return false;
} else {
return true;
}
}
}