I am working on a simple crawler. I can parse google using jsoup, no problem. But sometimes crawling some word gives error.I like the jsoup not to stop but ignore the error and keep giving me the output. I tried like following but it gives the same result as the google result is always same. Please give me a solution.
String GOOGLE_SEARCH_URL = "https://www.google.com/search";
String searchURL = GOOGLE_SEARCH_URL + "?q=" + word + "&num=" + 4;
for(int m =0; m <4;m++){
try {
Document doc = Jsoup.connect(searchURL).userAgent("Mozilla/5.0").get();
Elements results = doc.select("h3.r > a");
..........
.........
}
catch (HttpStatusException e) {
continue;
}
}
Result:
conn built
Success
Enter a word
far
http://www.merriam-webster.com/dictionary/far
Count of [far] is : 71
http://www.farmanager.com/
Count of [far] is : 7
.....repeat
If I don't use try catch the error is:
http://www.thefreedictionary.com/far
Exception in thread "main" org.jsoup.HttpStatusException: HTTP error fetching URL. Status=403, URL=http://www.thefreedictionary.com/_/403.aspx?p
..........
to be honest, I don't like to use for loop but is there any way just ignore some site and keep giving me the results?
Related
The below code is throwing ‘stale element reference: element is not attached to the page document’
How to handle error message and show the result as a pass
Action- The below code is adding one image(which can be one or more) which inside another image
public void click_on_the_Add_to_collection_button_displaying_down_below_the_assert() {
List<WebElement> all_colection = driver.findElements(By.xpath("//li//button[#class='icon-button ' and #title='Add To Collection']"));
int collection_size = all_colection.size();
System.out.println("the collection size is " + collection_size);
Random ran = new Random();
all_colection.get(collection_size - 1).click();
crate_new_colection.sendKeys("newcollection#1");
}
Wrap with try..catch like shown below:
public void click_on_the_Add_to_collection_button_displaying_down_below_the_assert() {
try{
List<WebElement> all_colection = driver.findElements(By.xpath("//li//button[#class='icon-button ' and #title='Add To Collection']"));
int collection_size = all_colection.size();
System.out.println("the collection size is " + collection_size);
Random ran = new Random();
all_colection.get(collection_size - 1).click();
crate_new_colection.sendKeys("newcollection#1");
}catch (StaleElementReferenceException e){
System.err.println("Skipping the exception. Do not let it get down stack.");
}
}
P.S. - I kept your snake case unchanged but please do not use Python-like code style in Java.
I am trying to develop and application to read and write to RF tags. Reading is flawless, but I'm having issues with writing. Specifically the error "GetStatus Write RFID_API_UNKNOWN_ERROR data(x)- Field can Only Take Word values"
I have tried reverse-engineering the Zebra RFID API Mobile by obtaining the .apk and decoding it, but the code is obfuscated and I am not able to decypher why that application's Write works and mine doesn't.
I see the error in the https://www.ptsmobile.com/rfd8500/rfd8500-rfid-developer-guide.pdf at page 185, but I have no idea what's causing it.
I've tried forcefully changing the writeData to Hex, before I realized that the API does that on its own, I've tried changing the Length of the writeData as well, but it just gets a null value. I'm so lost.
public boolean WriteTag(String sourceEPC, long Password, MEMORY_BANK memory_bank, String targetData, int offset) {
Log.d(TAG, "WriteTag " + targetData);
try {
TagData tagData = null;
String tagId = sourceEPC;
TagAccess tagAccess = new TagAccess();
tagAccess.getClass();
TagAccess.WriteAccessParams writeAccessParams = tagAccess.new WriteAccessParams();
String writeData = targetData; //write data in string
writeAccessParams.setAccessPassword(Password);
writeAccessParams.setMemoryBank(MEMORY_BANK.MEMORY_BANK_USER);
writeAccessParams.setOffset(offset); // start writing from word offset 0
writeAccessParams.setWriteData(writeData);
// set retries in case of partial write happens
writeAccessParams.setWriteRetries(3);
// data length in words
System.out.println("length: " + writeData.length()/4);
System.out.println("length: " + writeData.length());
writeAccessParams.setWriteDataLength(writeData.length()/4);
// 5th parameter bPrefilter flag is true which means API will apply pre filter internally
// 6th parameter should be true in case of changing EPC ID it self i.e. source and target both is EPC
boolean useTIDfilter = memory_bank == MEMORY_BANK.MEMORY_BANK_EPC;
reader.Actions.TagAccess.writeWait(tagId, writeAccessParams, null, tagData, true, useTIDfilter);
} catch (InvalidUsageException e) {
System.out.println("INVALID USAGE EXCEPTION: " + e.getInfo());
e.printStackTrace();
return false;
} catch (OperationFailureException e) {
//System.out.println("OPERATION FAILURE EXCEPTION");
System.out.println("OPERATION FAILURE EXCEPTION: " + e.getResults().toString());
e.printStackTrace();
return false;
}
return true;
}
With
Password being 00
sourceEPC being the Tag ID obtained after reading
Memory Bank being MEMORY_BANK.MEMORY_BANK_USER
target data being "8426017056458"
offset being 0
It just keeps giving me "GetStatus Write RFID_API_UNKNOWN_ERROR data(x)- Field can Only Take Word values" and I have no idea why this is the case, nor I know what a "Word value" is, and i've searched for it. This is all under the "OperationFailureException", as well. Any help would be appreciated, as there's almost no resources online for this kind of thing.
Even this question is a bit older, I had the same problem so as far as I know this should be the answer.
Your target data "8426017056458" length is 13 and at writeAccessParams.setWriteDataLength(writeData.length()/4)
you are devide it with four. Now if you are trying to write the target data it is longer than the determined WriteDataLength. And this throws the Error.
One 'word' is 4 Hex => 16 Bits long. So your Data have to be filled up first and convert it to Hex.
I am getting the below mentioned error in my catch block when i print using e.printStackTrace();This only occurs for few records when i read it from the DB2 database. So basically only few records throw the error rest work fine.
I have almost 1000 line of code and i am not sure were do i start from in order to understand/debug this error ? IS there any hint that i can get from the error message i should use to solve it ?
from the method getDataFromEMPHCForEmployeeDetails()----- com.ibm.db2.jcc.am.SqlException: [jcc][t4][1065][12306][3.63.123] Caught java.io.CharConversionException. See attached Throwable for details. ERRORCODE=-4220, SQLSTATE=null
at com.ibm.db2.jcc.am.fd.a(fd.java:663)
at com.ibm.db2.jcc.am.fd.a(fd.java:60)
at com.ibm.db2.jcc.am.fd.a(fd.java:112)
at com.ibm.db2.jcc.am.ic.a(ic.java:2820)
at com.ibm.db2.jcc.am.ic.p(ic.java:521)
at com.ibm.db2.jcc.am.ic.N(ic.java:1558)
at com.ibm.db2.jcc.am.vn.e(vn.java:1147)
at com.ibm.db2.jcc.am.vn.getString(vn.java:1122)
at com.ibm.db2.jcc.am.vn.getString(vn.java:1698)
at CreateChart.getDataFromEMPHCForEmployeeDetails(CreateChart.java:330)
at CreateChart.iterateDirectReportNamesFromArrayList(CreateChart.java:594)
at CreateChart.getDataFromEMPHCForDirectReport(CreateChart.java:295)
at CreateChart.iterateSecondLineManagerNamesFromArrayList(CreateChart.java:562)
at CreateChart.getDataFromEMPHCForSecondLine(CreateChart.java:251)
at CreateChart.iterateThirdLineManagerNamesFromArrayList(CreateChart.java:533)
at CreateChart.getDataFromEMPHCForThirdLine(CreateChart.java:208)
at CreateChart.iterateDirectorNamesFromArrayList(CreateChart.java:506)
at CreateChart.getDataFromEMPHCForDirector(CreateChart.java:168)
at CreateChart.iterateVPNamesFromArrayList(CreateChart.java:472)
at CreateChart.getDataFromEMPHCForVp(CreateChart.java:126)
at CreateChart.iterateFuncVPNamesFromArrayList(CreateChart.java:434)
at CreateChart.getDataFromEMPHCForFuncVp(CreateChart.java:95)
at CreateChart.main(CreateChart.java:613)
Caused by: java.nio.charset.MalformedInputException: Input length = 186
at com.ibm.db2.jcc.am.t.a(t.java:19)
at com.ibm.db2.jcc.am.ic.a(ic.java:2816)
... 19 more
Caused by: sun.io.MalformedInputException
at sun.io.ByteToCharUTF8.convert(Unknown Source)
at com.ibm.db2.jcc.am.t.a(t.java:16)
... 20 more
.
public void getDataFromEMPHCForEmployeeDetails(String funcvp_name11, String vp_name11, String director_name11, String thirdline_name11, String secondline_name11, String directreport_name11){
String myQuery11;
if(directreport_name11 == ""){directreport_name11 = " AND DIRECT_REPORT IS NULL";}
else{directreport_name11 =" AND DIRECT_REPORT ='"+ directreport_name11.replace("'", "''")+"'";}
myQuery11 = "SELECT DISTINCT(SERIAL_NUM) FROM DB2INST1.EMP_HC WHERE "+funcvp_name11+" "+vp_name11+" "+director_name11+" "+thirdline_name11+" "+ secondline_name11+""+directreport_name11;
arraylistofEmployeeDetails = new ArrayList<String>();
try {
resultSet = st.executeQuery(myQuery11);
while (resultSet.next())
{
String SERIAL_NUM = resultSet.getString("SERIAL_NUM");
if(SERIAL_NUM == null){SERIAL_NUM="";}
if (SERIAL_NUM.length() > 40){SERIAL_NUM = "Too Long";}
arraylistofEmployeeDetails.add(SERIAL_NUM);
System.out.println(SERIAL_NUM+"\n");
}
if(! (arraylistofEmployeeDetails.isEmpty())){
writeEmployeeDetailsToJsonFile(arraylistofEmployeeDetails);
}
} catch (SQLException e) {
e.printStackTrace();
StringWriter wtr = new StringWriter();
PrintWriter pwtr = new PrintWriter(wtr);
e.printStackTrace(pwtr);
String stackTrace = wtr.toString();
error_logs.append("from getDataFromEMPHCForEmployeeDetails()----- "+stackTrace+"\n");
}
}
I had similar problem with CharConversionException on DB2 jdbc driver with 1.8 Oracle Java with same error code, once updated to a never version it worked fine.
I downloaded db2jcc_license_cu and db2jcc.jar from these links.
Maybe it is worth of trying.
Try adding a JVM parameter -Ddb2.jcc.charsetDecoderEncoder=3, it solved the issue for me.
After setting the parameter db2.jcc.charsetDecoderEncoder=3, an exception will not be thrown when a non-UTF8 character is encountered but rather it will be substituted by its equivalent Unicode replacement character.
See the IBM Support article for more details https://www.ibm.com/support/pages/sqlexception-message-caught-javaiocharconversionexception-and-errorcode-4220
Nearly 1 month my program worked fine, but today im getting the error above when starting it.
The error comes up in this line:
sounds.put(key, (Clip)AudioSystem.getLine(new Line.Info(Clip.class)));
I have no idea why.
After i read the accepted answer here, i reinstalled eclipse, deleted metadata and so on, but the error still comes up.
The code is to 100% right, but any way ill post the code near the error:
protected SoundArchive(String soundArchive, boolean debugFrame){
f = new File(this.getClass().getResource("../" + soundArchive).toString().substring(5)).listFiles();
sounds = new HashMap<String, Clip>();
try{
for(int i = 0; i < f.length; i++){
String key = f[i].toString().split("\\\\")[f[i].toString().split("\\\\").length - 1].split("\\.")[0];
sounds.put(key, (Clip)AudioSystem.getLine(new Line.Info(Clip.class)));
sounds.get(key).open(AudioSystem.getAudioInputStream(f[i]));
}
}
catch(Exception e){
e.printStackTrace(System.out);
}
if(debugFrame) debugFrame();
}
So, does anyone know what i have to do now? I've my back to the wall....
Xml Template:
#(product: models.Product, productComponents: List[models.ProductComponent])
<product>
<name>#product.name</name>
<tomcatLocation>#product.tomcatLocation</tomcatLocation>
<productComponents>
#for(productComponent <- productComponents) {
<component>
<name>#productComponent.name</name>
<packageName>#productComponent.packageName</packageName>
<buildPath>#productComponent.buildPath</buildPath>
<packageFormat>#productComponent.packageFormat</packageFormat>
<versionAction>#productComponent.versionAction</versionAction>
<versionFilePath>#productComponent.versionFilePath</versionFilePath>
<versionCommand>#productComponent.versionCommand</versionCommand>
</component>
}
</productComponents>
</product>
Controller Action:
public static Result upgradeProduct(String serverId, String productId) throws InterruptedException
{
models.Server server = models.Server.retrieveById(Long.parseLong(serverId));
models.Product product = models.Product.retrieveById(Long.parseLong(productId));
List<models.ProductComponent> productComponents =
models.ProductComponent.retrieveByProductId(Long.parseLong(productId));
Xml renderedXmlPage = upgradeService.render(product, productComponents);
Promise<WS.Response> upgradeStatus =
WS.url("http://" + server.hostIp + ":8085/upgradeProduct").setHeader("Content-Type", "text/xml").post(
renderedXmlPage.toString());
String testMessage = upgradeStatus.get().getBody();
return TODO;
}
Web Service:
public static Result upgradeProduct()
{
return async( WS.url(MAIN_URL).get().map(
new Function<WS.Response, Result>() {
public Result apply(WS.Response response)
{
try
{
Document doc = response.asXml();
return ok("I WORKED");
}
catch(RuntimeException e)
{
return ok("Error" + e.getMessage()+ "\n");
}
}
})
);
}
When I run this action, The following runtime exception is thrown
"Errorjava.lang.RuntimeException: org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 1; Premature end of file.".
The this exception is being thrown at "Document doc = response.asXml();". I don't know why this is occurring? any help is much appreciated thanks!
Stack Trace:
play.libs.WS$Response.asXml(WS.java:332)
controllers.Service$1.apply(Service.java:40)
controllers.Service$1.apply(Service.java:35)
play.libs.F$Promise$2.apply(F.java:111)
play.api.libs.concurrent.STMPromise$$anonfun$map$1$$anonfun$apply$9.apply(Promise.scala:185)
scala.util.control.Exception$Catch$$anonfun$either$1.apply(Exception.scala:110)
scala.util.control.Exception$Catch$$anonfun$either$1.apply(Exception.scala:110)
scala.util.control.Exception$Catch.apply(Exception.scala:88)
scala.util.control.Exception$Catch.either(Exception.scala:110)
play.api.libs.concurrent.STMPromise.redeem(Promise.scala:166)
play.api.libs.concurrent.STMPromise$$anonfun$map$1.apply(Promise.scala:185)
play.api.libs.concurrent.STMPromise$$anonfun$map$1.apply(Promise.scala:184)
play.api.libs.concurrent.STMPromise$PromiseInvoker$$anonfun$receive$1.apply(Promise.scala:10 4)
play.api.libs.concurrent.STMPromise$PromiseInvoker$$anonfun$receive$1.apply(Promise.scala:10 3)
akka.actor.Actor$class.apply(Actor.scala:290)
play.api.libs.concurrent.STMPromise$PromiseInvoker.apply(Promise.scala:101)
akka.actor.ActorCell.invoke(ActorCell.scala:617)
akka.dispatch.Mailbox.processMailbox(Mailbox.scala:179)
akka.dispatch.Mailbox.run(Mailbox.scala:161)
akka.dispatch.ForkJoinExecutorConfigurator$MailboxExecutionTask.exec(AbstractDispatcher.scala:505)
akka.jsr166y.ForkJoinTask.doExec(ForkJoinTask.java:259)
akka.jsr166y.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:997)
akka.jsr166y.ForkJoinPool.runWorker(ForkJoinPool.java:1495)
akka.jsr166y.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:104)
Correct me if I'm wrong, but it doesn't look like you are returning your xml document from the service!? Instead you return "I WORKED", which of cause won't parse as xml, and the parser will throw an exception.
In the Service you are calling, via WS, an external URL (MAIN_URL) and using the response on line:
Document doc = response.asXml();
The error means that the service you are calling (MAIN_URL) is not returning XML.
By the way, your controller has TODO as return value, I'm not sure if that's what you want in there as it seems that you have working code in the controller.
Other than the obvious case of not having any XML input to begin with, sometimes this error also gets thrown the second time your code tries to read a javax.xml.transform.Source or a javax.xml.transform.sax.SAXSource. The first time everything runs OK, but the second time the bytes have been consumed and you get that exception. At least that how I've encountered it, in which case I simply create the Source anew - I don't know applicable this is in your particular case.