Please, i have some problems with my web application.
I can't paste my code here ( too big and i have the difficulty to reproduce the error ) but, this is my issus.
I have a object that contains a collection. I use BlockquingQueue to share this objet betwen some thread. the second kind of thread is a servlet.
When i put my objet in the queue, the collection is not empty and i can display thier element.
But, when i take the same element, the collection size is not null, but it don't have elements.
NB: I don't have problems to get the objet in queue. My problems it which their attribute of type Collection. It show me a strange behavoir.
a big part of a code:
public class HttpCollectionConsumer extends JCasAnnotator_ImplBase{
private static BlockingQueue<Answer> queue = new LinkedBlockingQueue<>();
private static boolean hasNext = true;
public void initialize(UimaContext context) throws ResourceInitializationException{
super.initialize(context);
}
#Override
public void process(JCas jcas) throws AnalysisEngineProcessException {
// TODO Auto-generated method stub
edu.cmu.lti.oaqa.type.input.Question q = TypeUtil.getQuestion(jcas);
System.out.println("get Text " + q.getText());
Question question = new Question(q.getId() , q.getText());
Focus focus = TypeUtil.getFocus(jcas);
Collection<LexicalAnswerType> types = TypeUtil.getLexicalAnswerTypes(jcas);
Answer a = new Answer();
a.setQuestion(question);
a.setFocus(focus);
a.setTypes(types);
try {
System.out.println("identifiant : ( " + a + " ) types " + a.getTypes().iterator().next());
System.out.println("the answer type is not empty : " + a.getTypes().iterator().hasNext());
synchronized(this){
queue.put(a);
Thread.sleep(1000);
}
System.out.println("putting finished " );
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public static synchronized void put(Answer question) throws InterruptedException{
System.out.println("new answer : " + question);
queue.offer(question);
}
public synchronized static Answer take() throws InterruptedException{
Answer a = queue.take();
Thread.sleep(2000);
System.out.println(" someone takess ( " + a + " ) , remaining: " + queue.size());
System.out.println("the answer type is not empty : " + a.getTypes().iterator().hasNext());
return a;
}
public synchronized static BlockingQueue<Answer> getQueue(){
return queue;
}
public synchronized static void stop(){
hasNext = false;
}
}
Someone can know why ?
You can only get a item from a Queue once, First Object In First Out. If you want to access a object more than once or directly you should use a List and transform that into a Collection.
Also, if you share your object between Threads only one of the Threads will be able to access the mentioned object, since after being processed by one it won't be in the Queue anymore.
Related
I am automating e-commerce website. I am using JUNIT-Selenium framework.
Their are two files i am working with, first is the "TestCase.java" where my test steps are mentioned, aslo to start automation i run this file and second file is "TestMain.java" which has validation methods which will used by First file to verify and input the data in UI (mostly using If ..else validation).
First file consist of Automation initiation code, which uses Hashmap for reading the excel, extent report initiation and flush and use methods from testMain.java for input of data and validation through if... else statement.
TestCase.java looks like this:
public class TestCase extends AppTest {
private StringBuffer verificationErrors = new StringBuffer();
#Override
#Before
public void setUp() throws Exception {
super.setUp();
preparation = new Prep();
application = new AppToTest();
user = new Environment();
}
#Test
public void testLAP_Creamix() throws Exception {
try {
launchMainApplication();
Test_frMain Test_frMainPage = new Test_frMain(tool, test, user, application);
HashMap<String, ArrayList<String>> win = CreamixWindowsDataset.main();
SortedSet<String> keys = new TreeSet<>(win.keySet());
ExtentHtmlReporter htmlReporter = new ExtentHtmlReporter("Test_Report.html");
ExtentReports extent = new ExtentReports();
extent.attachReporter(htmlReporter);
ExtentTest test1 = extent.createTest("Creamix test");
for (String i : keys) {
System.out.println("########### Test = " + win.get(i).get(0) + " ###########");
Lapeyre_frMainPage.EnterTaille(win.get(i).get(1));
Lapeyre_frMainPage.SelectCONFIGURATION(win.get(i).get(2));
Lapeyre_frMainPage.SelectPLANVASQUE(win.get(i).get(3));
Lapeyre_frMainPage.SelectCOULEUR(win.get(i).get(4));
Lapeyre_frMainPage.SelectPOIGNEES(win.get(i).get(5));
Lapeyre_frMainPage.SelectTYPE_DE_MEUBLE(win.get(i).get(6));
Lapeyre_frMainPage.SelectCHOISISSEZ(win.get(i).get(7));
Lapeyre_frMainPage.VerifyREFERENCE(win.get(i).get(8));(FROM HERE Validation Starts)
Lapeyre_frMainPage.VerifyQUANTITY(win.get(i).get(9));
Lapeyre_frMainPage.VerifyREFERENCETwo(win.get(i).get(10));
Lapeyre_frMainPage.VerifyQUANTITYTwo(win.get(i).get(11));
Lapeyre_frMainPage.VerifyREFERENCEThree(win.get(i).get(12));
Lapeyre_frMainPage.VerifyQUANTITYThree(win.get(i).get(13));
Lapeyre_frMainPage.VerifyREFERENCEFour(win.get(i).get(14));
Lapeyre_frMainPage.VerifyQUANTITYFour(win.get(i).get(15));
Lapeyre_frMainPage.VerifyREFERENCEFive(win.get(i).get(16));
Lapeyre_frMainPage.VerifyQUANTITYFive(win.get(i).get(17));
Lapeyre_frMainPage.VerifyREFERENCESix(win.get(i).get(18));
Lapeyre_frMainPage.VerifyQUANTITYSix(win.get(i).get(19));
Lapeyre_frMainPage.VerifyREFERENCESeven(win.get(i).get(20));
Lapeyre_frMainPage.VerifyQUANTITYSeven(win.get(i).get(21));
Lapeyre_frMainPage.VerifyPanierPrice(win.get(i).get(22));
Lapeyre_frMainPage.VerifyECO_PARTPrice(win.get(i).get(23));
Lapeyre_frMainPage.ClickCREAMIXReinit();(Reset button to test next scenario)
test1.pass("Scenario " + win.get(i).get(0) + " is passed");
System.out.println("########### Test End ##############");
extent.flush();----------(Extent report over)
}
test.setResult("pass");
} catch (AlreadyRunException e) {
} catch (Exception e) {
verificationErrors.append(e.getMessage());
throw e;
}
}
#Override
#After
public void tearDown() throws Exception {
super.tearDown();
}
}
(Please note one loop is one scenario where im customizing and validating price of the product and then clicking reset button to next scenario for doing same)
And,
"TestMain.java" from where i am using methods to validate
one of the method is shown below
public void VerifyREFERENCE(String REF_1) throws Exception {
System.out.println("Verifying reference article");
if (REF_1.equals("SKIP")) {
System.out.println("SKIPPED");
} else {
WebElement referenceOne = tool.searchUsingXPath("//tbody//tr[1]//td//div[2]");
String Ref1 = referenceOne.getText().trim();
System.out.println("ref 1 is " + Ref1);
if (Ref1.equals("Ref. de l'article : " + REF_1)) {
System.out.println("Reference 1 is correct");
} else {
System.out.println("Reference 1 is incorrect");
}
}
}
I am using extent report in TestCase.java(Please check above code) to report my scenarios, but the problem is It shows all test case as PASS and if any failure occurs it doesn't report(it terminates).
Reason being i have not used assertions anywhere, BUT HOW CAN I APPLY SUCH ASSERTIONS IN THIS FRAMEWORK
TO SUMMARIZE:
1- I need to add price validation check in report
2- i tried of using this line in TestCase.java "assertEquals("Verify REFERENCE 1", win.get(i).get(8), Lapeyre_frMainPage.GetREFERENCE());" but i cant use assertion in TestCase.java(it wont allow me).
3- Please show me alternative way to report PASS and FAIL for such frameworks, where in extent report i can able to show price mismatch between excel and UI.
You can only expect assertEquals() to work if your TestCase class extends junit test, or if you implement the method yourself in your code.
In any case, you seem to be trying to use ExtentReports. I am not experienced with that library, but according to the javadoc for ExtentTest, it appears that you are expected to call .pass() or .fail() yourself based on the outcome of your test.
In your TestCase, I believe you want to try to maintain a boolean to track if the test has passed or not.
The first step would be to modify VerifyREFERENCE() to return a boolean indicating if it passed or failed, instead of being void.
public boolean VerifyREFERENCE(String REF_1) throws Exception {
System.out.println("Verifying reference article");
if (REF_1.equals("SKIP")) {
System.out.println("SKIPPED");
} else {
WebElement referenceOne = tool.searchUsingXPath("//tbody//tr[1]//td//div[2]");
String Ref1 = referenceOne.getText().trim();
System.out.println("ref 1 is " + Ref1);
if (Ref1.equals("Ref. de l'article : " + REF_1)) {
System.out.println("Reference 1 is correct");
return true;
} else {
System.out.println("Reference 1 is incorrect");
return false;
}
}
}
Then, in your TestCase, initialise a boolean to true just before the loop. Inside the loop, perform a logical AND (&&) with the return value of each VerifyREFERENCE() call. Finally, after the loop finishes, test the value of the boolean, and pass or fail the ExtentTest as appropriate:
ExtentTest test1 = extent.createTest("Creamix test");
boolean passed = true;
for (String i : keys) {
System.out.println("########### Test = " + win.get(i).get(0) + " ###########");
Lapeyre_frMainPage.EnterTaille(win.get(i).get(1));
....
Lapeyre_frMainPage.SelectCHOISISSEZ(win.get(i).get(7));
passed = passed && Lapeyre_frMainPage.VerifyREFERENCE(win.get(i).get(8));(FROM HERE Validation Starts)
passed = passed && Lapeyre_frMainPage.VerifyQUANTITY(win.get(i).get(9));
...
passed = passed && Lapeyre_frMainPage.VerifyECO_PARTPrice(win.get(i).get(23));
Lapeyre_frMainPage.ClickCREAMIXReinit();(Reset button to test next scenario)
if(passed) {
test1.pass("Scenario " + win.get(i).get(0) + " is passed");
} else {
test1.fail("Scenario " + win.get(i).get(0) + " is failed");
}
System.out.println("########### Test End ##############");
extent.flush();----------(Extent report over)
}
I am facing an issue when i try to query the queue using createquery api to fetch the queue element.
I am getting an error at the while statement stating the error below as
errorjava.lang.illegalstateexception :unread block data
i dont know why i am getting this error. I can able to use the fetchcount() api to get the count of workitem in the queue but the hasnext() api is not working nor next().
Is there any reason why this statement is not getting executed. is this related to any java issue. Can any one help
The code is
VWSession session = new VWSession();
session.setBootstrapCEURI(Ceuri);
session.logon(cename, fnPassword, connectionPoint);
VWQueue queue = session.getQueue(queue));
int queryFlag = VWQueue.QUERY_NO_OPTIONS;
int fetchType = VWFetchType.FETCH_TYPE_STEP_ELEMENT;
VWQueueQuery queueQuery = queue.createQuery(null,null, null,queryFlag, null, null, fetchType);
while (queueQuery.hasNext()) {
queueElement = (VWStepElement) queueQuery.next();
}
In you main (calling) method, do this :
VWSession vwsession = new VWSession();
vwsession.setBootstrapCEURI("http://servername:9080/wsi/FNCEWS40MTOM/");
vwsession.logon("userid", "password", "ConnPTName");
IteratePEWorkItems queueTest = new IteratePEWorkItems();
queueTest.testQueueElements(vwsession);
Later on create below metioned helper method:
public void testQueueElements(VWSession vwsession) {
System.out.println("Inside getListOfWorkitems: : ");
VWRoster roster = vwsession.getRoster("DefaultRoster");
int fetchType = VWFetchType.FETCH_TYPE_STEP_ELEMENT;
int queryFlags = VWQueue.QUERY_READ_UNWRITABLE;
try {
dispatchWorkItems(roster, fetchType, queryFlags, vwsession);
} catch (Exception exception) {
log.error(exception.getMessage());
}
}
public void dispatchWorkItems(VWRoster roster, int fetchType, int queryFlags, VWSession vwsession) {
String filter = "SLA_Date>=:A";
// get value and replace with 1234567890 as shown in process administrator
Object[] subVars = { 1234567890 };
VWRosterQuery rosterQuery = roster.createQuery(null, null, null,
VWRoster.QUERY_MIN_VALUES_INCLUSIVE | VWRoster.QUERY_MAX_VALUES_INCLUSIVE, filter, subVars,
VWFetchType.FETCH_TYPE_WORKOBJECT);
int i = 0;
// Iterate work items here...
while (rosterQuery.hasNext() == true) {
VWWorkObject workObject = (VWWorkObject) rosterQuery.next();
try {
i++;
System.out.println(" Subject: " + workObject.getFieldValue("F_Subject") + " Count: " + i);
} catch (Exception exception) {
exception.printStackTrace();
log.error(exception);
}
}
}
Try it and share the output.
I want to wait until my processes finish before I return my speechlet response, otherwise it seems to cut my process off and thus, not complete it, I actually believe it may freeze the process, but thats not my desire.
How do I go about waiting?
#Override
public SpeechletResponse onIntent(SpeechletRequestEnvelope<IntentRequest> requestEnvelope) {
IntentRequest request = requestEnvelope.getRequest();
Intent intent = request.getIntent();
String intentName = (intent != null) ? intent.getName() : null;
if (intentName == null) return null;
switch (intentName) {
case IntentTitle.NEW_TICKET:
switch (request.getDialogState()) {
case STARTED:
return Response.getDialogueResponse(intent, true);
case IN_PROGRESS:
return Response.getDialogueResponse(intent, false);
case COMPLETED:
String numberString = intent.getSlot(SlotTitle.ID).getValue();
if (!NumberUtils.isCreatable(numberString)) return Response.ERROR;
Member member = Info.GUILD.getMemberById(numberString);
User sender = UserDB.getUser(member);
CommissionTicket commissionTicket = new CommissionTicket(sender);
commissionTicket.create();
//wait until processes finish before continuing
return Response.NEW_TICKED_CREATED;
}
}
return null;
}
UPDATE:
Had a look at the CloudWatch logs, and well, pretty much what I expected was happening is happening... have a look at the times for these logs (I ran them 3 different times, so 3 different times are logged, but apart of the same lambda session)
public void create() {
System.out.println("log2");
GuildController guildController = Info.GUILD.getController();
guildController.createTextChannel(ticketType.name().toLowerCase() + "-" + creator.getName() + "-" + id.value()).queue(channel -> {
System.out.println("log3");
ChannelManager channelManager = channel.getManager();
GuildManager guildManager = channelManager.getGuild().getManager();
List<Member> members = guildManager.getGuild().getMembers();
List<Member> admins = new ArrayList<>();
for (Member member : members) {
for (Role role : member.getRoles()) {
if (!role.getName().equalsIgnoreCase(Info.ADMIN_STRING)) continue;
admins.add(member);
}
}
System.out.println("log4");
for (Member member : members) {
if (member.equals(creator.getMember())) continue;
channel.createPermissionOverride(member).setDeny(Permission.MESSAGE_READ).queue();
}
System.out.println("log5");
for (Member admin : admins) {
if (admin.equals(creator.getMember())) continue;
channel.createPermissionOverride(admin).setAllow(Permission.MESSAGE_READ).queue();
}
System.out.println("log6");
BotMessage botMessage = new BotMessage();
botMessage
.setTitle("New Ticket! User: " + creator.getName())
.setColour(Color.CYAN)
.setDescription("Please enter your full request here! \n" +
"Make sure to let us know whether you are looking for a quote/timeframe,\n" +
"or have a budget in mind, and we will work around you!\n\n" +
"A sales representative will be with you as soon as possible!")
.send((TextChannel) channel);
System.out.println("log7");
this.textChannel = (TextChannel) channel;
TicketDB.addTicket(this);
System.out.println("log8");
}
);
Logs:
https://gyazo.com/0ad2baa8d1438be8364dd1112159c4f4
https://gyazo.com/e197f33335046afe3c9f8f1ace267d30
UPDATE
Implemented the Future class, worked, but still a bit buggy.
It now completely creates the ticket, which is great, however, when I go to send the same call again, it for some reason sends a message in the same channel before preceding to create the next ticket.
So, to simulate...
Function call through AWS Lambda
Creates ticket completely
Function call through AWS Lambda again
Sends a message or two in the previous tickets channel
Creates new ticket completely
https://gyazo.com/dc6e4391f4964f41a73f1c3be92190f9
#Override
public SpeechletResponse onIntent(SpeechletRequestEnvelope<IntentRequest> requestEnvelope) {
IntentRequest request = requestEnvelope.getRequest();
Intent intent = request.getIntent();
String intentName = (intent != null) ? intent.getName() : null;
if (intentName == null) return null;
switch (intentName) {
case IntentTitle.NEW_TICKET:
switch (request.getDialogState()) {
case STARTED:
return Response.getDialogueResponse(intent, true);
case IN_PROGRESS:
return Response.getDialogueResponse(intent, false);
case COMPLETED:
String numberString = intent.getSlot(SlotTitle.ID).getValue();
if (!NumberUtils.isCreatable(numberString)) return Response.ERROR;
Member member = Info.GUILD.getMemberById(numberString);
User sender = UserDB.getUser(member);
System.out.println("log1");
Future<Ticket> commissionTicket = new CommissionTicket(sender).create();
try {
commissionTicket.get(10000, TimeUnit.MILLISECONDS);
} catch (Exception e) {
e.printStackTrace();
}
//wait until processes finish before continuing
return Response.NEW_TICKED_CREATED;
}
}
return null;
}
In the Ticket class:
public Future<Ticket> create() {
System.out.println("log2");
GuildController guildController = Info.GUILD.getController();
RequestFuture<Channel> channelRequestFuture = guildController.createTextChannel(ticketType.name().toLowerCase() + "-" + creator.getName() + "-" + id.value()).submit();
try {
Channel channel = channelRequestFuture.get(10000, TimeUnit.MILLISECONDS);
System.out.println("log3");
ChannelManager channelManager = channel.getManager();
GuildManager guildManager = channelManager.getGuild().getManager();
List<Member> members = guildManager.getGuild().getMembers();
List<Member> admins = new ArrayList<>();
for (Member member : members) {
for (Role role : member.getRoles()) {
if (!role.getName().equalsIgnoreCase(Info.ADMIN_STRING)) continue;
admins.add(member);
}
}
System.out.println("log4");
for (Member member : members) {
if (member.equals(creator.getMember())) continue;
channel.createPermissionOverride(member).setDeny(Permission.MESSAGE_READ).queue();
}
System.out.println("log5");
for (Member admin : admins) {
if (admin.equals(creator.getMember())) continue;
channel.createPermissionOverride(admin).setAllow(Permission.MESSAGE_READ).queue();
}
System.out.println("log6");
BotMessage botMessage = new BotMessage();
botMessage
.setTitle("New Ticket! User: " + creator.getName())
.setColour(Color.CYAN)
.setDescription("Please enter your full request here! \n" +
"Make sure to let us know whether you are looking for a quote/timeframe,\n" +
"or have a budget in mind, and we will work around you!\n\n" +
"A sales representative will be with you as soon as possible!")
.send((TextChannel) channel);
System.out.println("log7");
this.textChannel = (TextChannel) channel;
TicketDB.addTicket(this);
System.out.println("log8");
Future<Ticket> future = ConcurrentUtils.constantFuture(this);
return future;
} catch (Exception e) {
e.printStackTrace();
}
if (!userIsInTicket(creator)) users.add(creator);
return null;
}
I'm not sure what's going on inside commissionTicket.create(); (I assume it's your code, not part of some library, and that it's running asynchronously somehow), but one solution would be to have that method return a Future<> object and then wait on it to finish. Something along the lines of:
Future<CommissionTicket> commissionTicketFuture = CommissionTicket.create(sender);
commissionTicketFuture.get(SOME_TIMEOUT, TimeUnit.MILLIS);
return Response.NEW_TICKET_CREATED;
I'm having multiple threads running in my threadPool Each thread reads a huge file and returns the data from this file in a List.
Code looks like :
class Writer{
ArrayList finalListWhereDataWillBeWritten = new Array<Integer>()
for(query q : allQueries){ //all the read queries to read file
threadPool.submit(new GetDataFromFile(fileName,filePath));
}//all the read queries have been submitted.
}
Now I know that following section of code will occur some where in my code but I don't know where to place it.
Because if I place it just after submit() in for loop it'll not add it because each file is very huge and may not have completed its processing.
synchronized(finalListWhereDataWillBeWritten){
//process the data obtained from single file and add it to target list
finalListWhereDataWillBeWritten.addAll(dataFromSingleThread);
}
So can anyone please tell me that where do I place this chunk of code and what other things I need to make sure of so that Critical Section Problem donot occur.
class GetDataFromFile implements Runnable<List<Integer>>{
private String fileName;
private String filePath;
public List<Integer> run(){
//code for streaming the file fileName
return dataObtainedFromThisFile;
}
}
And do i need to use wait() / notifyAll() methods in my code given that I'm only reading data from files parallely in threads and placing them in a shared List
Instead of reinventing the wheel you should simply implement Callable<List<Integer>> and submit it to the JDK's standard Executor Service. Then, as the futures complete, you collect the results into the list.
final ExecutorService threadPool =
Executors.newFixedThreadPool(Runtime.getRuntime().availableProcessors());
final List<Future<List<Integer>>> futures = new ArrayList<>();
for(query q : allQueries) {
futures.add(threadPool.submit(new GetDataFromFile(fileName, filePath)));
}
for (Future<List<Integer>> f : futures) {
finalListWhereDataWillBeWritten.addAll(f.get());
}
And this is all assuming you are below Java 8. With Java 8 you would of course use a parallel stream:
final List<Integer> finalListWhereDataWillBeWritten =
allQueries.parallelStream()
.flatMap(q -> getDataFromFile(q.fileName, q.filePath))
.collect(toList());
UPDATE Please consider the answer provided by Marko which is far better
If you want to ensure that your threads all complete before you work on your list, do the following:
import java.util.List;
import java.util.Vector;
public class ThreadWork {
public static void main(String[] args) {
int count = 5;
Thread[] threads = new ListThread[count];
List<String> masterList = new Vector<String>();
for(int index = 0; index < count; index++) {
threads[index] = new ListThread(masterList, "Thread " + (index + 1));
threads[index].start();
}
while(isOperationRunning(threads)) {
// do nothing
}
System.out.println("Done!! Print Your List ...");
for(String item : masterList){
System.out.println("[" + item + "]");
}
}
private static boolean isOperationRunning(Thread[] threads) {
boolean running = false;
for(Thread thread : threads) {
if(thread.isAlive()) {
running = true;
break;
}
}
return running;
}
}
class ListThread extends Thread {
private static String items[] = { "A", "B", "C", "D"};
private List<String> list;
private String name;
public ListThread(List<String> masterList, String threadName) {
list = masterList;
name = threadName;
}
public void run() {
for(int i = 0; i < items.length;++i) {
randomWait();
String data = "Thread [" + name + "][" + items[i] + "]";
System.out.println( data );
list.add( data );
}
}
private void randomWait() {
try {
Thread.currentThread();
Thread.sleep((long)(3000 * Math.random()));
}
catch (InterruptedException x) {}
}
}
I have a list of different URLs (about 10) from which I need content. I have made a program with which I am getting the content of 1 URL but I am unable to do it with multiple URLs.
I've studied lots of tutorials on threads in Java but I'm unable to find an answer.
In my case, URLs are like www.example1.com, www.example2.com, www.example3.com, www.example4.com.
I want to make thread for each URL and run it at the same time.
public class HtmlParser {
public static int searchedPageCount = 0,
skippedPageCount = 0,
productCount = 0;
public static void main(String[] args) {
List<String> URLs = new LinkedList<String>();
long t1 = System.currentTimeMillis();
URLs.add("www.example.com");
int i = 0;
for (ListIterator iterator = URLs.listIterator(); i < URLs.size();) {
i++;
System.out.println("While loop");
List<String> nextLevelURLs = processURL(URLs.get(iterator
.nextIndex()));
for (String URL : nextLevelURLs) {
if (!URLs.contains(URL)) {
System.out.println(URL);
iterator.add(new String(URL));
}
}
System.out.println(URLs.size());
}
System.out.println("Total products found: " + productCount);
System.out.println("Total searched page: " + searchedPageCount);
System.out.println("Total skipped page: " + skippedPageCount);
long t2 = System.currentTimeMillis();
System.out.println("Total time taken: " + (t2 - t1) / 60000);
}
public static List<String> processURL(String URL) {
List<String> nextLevelURLs = new ArrayList<String>();
try {
searchedPageCount++;
// System.out.println("Current URL: " + URL);
Elements products = Jsoup.connect(URL).timeout(60000).get()
.select("div.product");
for (Element product : products) {
System.out.println(product.select(" a > h2").text());
System.out.println(product.select(" a > h3").text());
System.out.println(product.select(".product > a").attr("href"));
System.out
.println(product.select(".image a > img").attr("src"));
System.out.println(product.select(".price").text());
System.out.println();
productCount++;
}
// System.out.println("Total products found until now: " +
// productCount);
Elements links = Jsoup.connect(URL).timeout(60000).get()
.select("a[href]");
for (Element link : links) {
URL = link.attr("href");
if (URL.startsWith("http://www.example.com/")) {
// System.out.println("URLs added.");
nextLevelURLs.add(URL);
} else {
skippedPageCount++;
// System.out.println("URL skipped: " + URL);
}
}
} catch (Exception e) {
e.printStackTrace();
}
return nextLevelURLs;
}
}
Unfortunately, there is no way to start two threads at the same time.
Let me explain better: first of all, the sequence thread1.Start(); and thread2.Start(); is executed with thread1 first and, after that, thread2. It means only that thread thread1 is scheduled before thread 2, not actually started. The two methods take fractions of second each one, so the fact that they are in sequence cannot be seen by a human observer.
More, Java threads are scheduled, ie. assigned to be eventually executed. Even if you have a multi-core CPU, you are not sure that 1) the threads run in parallel (other system processes may interfere) and 2) the threads both start just after the Start() method is called.
but you can run multiple threads in this way..
new Thread(thread1).start();
new Thread(thread2).start();
basically create a class that implements Runnable, put the code that deals with one url in this code. In your main class for each URL, construct a class with the information that is needs (E.g. URL) and then run run
Plenty of sites that teach how to do multi-threaded java
First of all, the code you pasted looks like bad because it is orienting a simple process. You need to turn it into OO form and then extends the Thread (or Runnable) such like:
public class URLProcessor extends Thread {
private String url;
public URLProcessor(String url) {
this.url = url;
}
#Override
public void run() {
//your business logic to parse the site with "this.url" here
}
}
And then use the main entrance to load multiple ones by using:
public static void main(String[] args) {
List<String> allmyurls = null;//get multiple urls from somewhere
for (String url : allmyurls) {
URLProcessor p = new URLProcessor(url);
p.start();
}
}