We Keep Coding

Jsoup not connecting to webpage in Android Studio

I am working on a project right now where I use jsoup in a class with the function retrieveMedia in order to return an ArrayList filled with data from the webpage. I run it in a thread since you shouldn't be connecting to URLs from the main thread. I run it and join it. However, it doesn't work (I tested the same code in Eclipse separate from Android Studio and it worked fine). It seems that no matter what I do I can't get jsoup to connect to the webpage. Below is my class MediaRetriever.
public class MediaRetreiever {
public ArrayList<Media> retrieveMedia() {
ArrayList<Media> mediaOutput = new ArrayList<Media>(); //Store each scraped post
Thread downloadThread = new Thread(new Runnable() {
public void run() {
Document doc = null;
try {
doc = Jsoup.connect(<Website Im connecting to>).timeout(20000).get();
} catch (IOException e) {
System.out.println("Failed to connect to webpage.");
mediaOutput.add(new Media("Failed to connect", "oops", "", "oh well"));
return;
}
Elements mediaFeed = doc.getElementById("main").getElementsByClass("node");
for (Element e : mediaFeed) {
String title, author, imageUrl, content;
title=e.getElementsByClass("title").text().trim();
author=e.getElementsByClass("content").tagName("p").select("em").text().trim();
content=e.getElementsByClass("content").text().replace(author,"").trim();
Media media = new Media(title, author, "", content);
mediaOutput.add(media);
}
}
});
downloadThread.start();
try {
downloadThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return mediaOutput;
}
}
Running this class's method from another class and it doesn't ever connect. Any ideas?

Since you say that the problem persists only in Android, it looks like that you should add the user agent string to your request - first get the user agent string of a browser that displays correctly the site, and then add it to the request:
doc = Jsoup.connect(<Website Im connecting to>)
.userAgent("your-user-agent-string")
.timeout(20000).get();
And as a sidenote - if you are catching exception, don't print your own error message - print the original message, it may be very useful.

How to get the data for restricted site using JSOUP or any programming language

This code works fine for unrestricted sites. what are the changes to be done for restricted websites. Here is the restricted link.
http://www.manta.com/c/mrl70cq/geico
public class Scrape1 {
public static void main(String[] args) throws SQLException, IOException {
Document doc = null;
try {
doc = Jsoup.connect("http://www.manta.com/c/mrl70cq/geico").ignoreHttpErrors(true).get();
} catch (IOException e) {
e.printStackTrace();
}
String text = doc.html();
System.out.println(text);
}
}
Here is my error :
when i am running the above code with any URL it getting the whole data from the page. When i am using the following URL manta.com/c/mrl70cq/geico i am getting the following error: You're a power user moving through this website with super-human speed.You've disabled JavaScript in your web browser.A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running
Any suggestions. Thanks in Advance.

What am I doing wrong with my Dom4j Code?

Hey guys I am working on a load class for my project that loads and pulls a info from a xml files. I have been following a few guides online but I am running into the error java.lang.NoClassDefFoundError: org/jaxen/JaxenException. I know the code is finding the save file because I can print out the name. But when I try to pull out the info I get that error. Snippet of code is below if there is no error in there let me know and Ill post more.
public void LoadProjects()
{
try
{
Files.walk(Paths.get("D:/workspace/Project Program/Projects/")).forEach(filePath ->
{
if(Files.isRegularFile(filePath))
{
System.out.println("Testing");
try
{
SAXReader reader = new SAXReader();
Document document = reader.read(filePath.toFile());
System.out.println(document.getName());
Node node = document.selectSingleNode("///Project/Info");
//String name = node.valueOf("#Name");
//String projNum = node.valueOf("#ProjectNumber");
//node = document.selectSingleNode("//Project/Dates");
//String dueBy = node.valueOf("#DueBy");
//CButton temp = new CButton(name, projNum, dueBy);
//Console.console.AddToList(temp);
}
catch (Exception e)
{
e.printStackTrace();
}
}
});
}
catch(Exception e)
{
e.printStackTrace();
}
}

Did you add the jaxen jar to your servers lib?
Your code seems correct so far but please add as much information as possible from the beginning.
Ff you are using maven:
<dependency>
<groupId>jaxen</groupId>
<artifactId>jaxen</artifactId>
<version>1.1.1</version>
</dependency>

Using Twitter4j to get favorited count for a particular tweet

I am new to the twitter4j api, is there any way of getting no of favorite count for a particular tweet using twitter4j. I am using api version 3.0.3 for Twitter4j.
In documentation the method getFavoriteCount() is present but the same method gives compilation error in code. Should I use different version of jar file?
public static void main(String[] args) {
ConfigurationBuilder cb= new ConfigurationBuilder();
cb.setDebugEnabled(true);
cb.setOAuthConsumerKey("**************************");
cb.setOAuthConsumerSecret("**************************");
cb.setOAuthAccessToken("*******************************");
cb.setOAuthAccessTokenSecret("*****************************");
Twitter twitter = new TwitterFactory(cb.build()).getInstance();
User user = null;
try {
user = twitter.verifyCredentials();
} catch (TwitterException e1) {
e1.printStackTrace();
}
String[] test = new String[]{"teststr"};
ResponseList<User> users;
try {
users = twitter.lookupUsers(test);
for (User user1 : users) {
if (user1.getStatus() != null)
{
Paging paging = new Paging(1, 40);
ResponseList<Status> statusess = twitter.getUserTimeline(user1.getName(),paging);
for (Status status3 : statusess)
{
long retweetCount = status3.getRetweetCount();
long favoriteCount = status3.getFavoriteCount();/** this line gives compilation error saying the method getFavoriteCount() is undefined for the type Status**/
}
}
}
} catch (TwitterException e) {
e.printStackTrace();
}
}

Unfortunately, it seems like the JavaDocs on the Twitter4J site linked next to the 3.0.3 release are not correct. The actual docs are here - you can see that getFavoriteCount() isn't present in these.
It looks like Status#getFavoriteCount() was only introduced in 3.0.4. So yes, you need to upgrade from 3.0.3 in order to use this method.
Although be aware that version 3.0.4 looks like it is under active development presently.

Although you have already found and marked an answer, let me provide an alternate to switching the Twitter4J version.
When using a method similar to what you are using, I had searched a lot and finally found the solution in an entirely unrelated post. Here is how you can get the Favorite count of tweets:
NOTE: This is what I use in an Android app of mine.
First, in your ConfigurationBuilder cb instance, add this one line:
cb.setJSONStoreEnabled(true);
This will return all results in a JSON format. Strangely, getting a JSON result it provides you that value.
Now, to fetch the Tweets:
try {
Paging paging = new Paging(initPagingOffset, 200);
statuses = twitter.getHomeTimeline(paging);
String strTweets = DataObjectFactory.getRawJSON(statuses);
JSONArray JATweets = new JSONArray(strTweets);
for (int i = 0; i < JATweets.length(); i++) {
JSONObject JOTweets = JATweets.getJSONObject(i);
..... // PARSE ANY OTHER DATA YOU MIGHT NEED FOR DISPLAYING THE TWEETS
String FAV_COUNT = JOTweet.getString("favorite_count");
}
} catch (TwitterException e) {
e.printStackTrace();
} catch (JSONException e) {
e.printStackTrace();
}

Using the content handler API (JSR 211) to open applications

I want to be able to launch native and J2ME applications through my application using the content handler API (JSR 211) on a Nokia 6212.
At the moment, I am unable to do so, as it always states that there is "No Content Handler Found" and throws a javax.microedition.content.ContentHandlerException.
At the moment, I am trying to get the phone to launch its browser and go to a certain website, just to test that I can use the framework. I have tried many different Invocation objects:
//throw exceptions
new Invocation("http://www.somesite.com/index.html",
"application/internet-shortcut");
new Invocation("http://www.google.co.uk","text/html");
// a long shot, I know
new Invocation("http://www.somesite.com/text.txt","text/plain");
// massive long shot
new Invocation("http://www.google.co.uk","application/browser");
//appears to download the link and content (and definitely does in the Nokia
// emulator) and then throws an exception
new Invocation("http://www.google.co.uk");
new Invocation("http://www.somesite.com/index.html");
Below is the code that I have been using, please bear in mind the parameters often changed to generate the different Invocation objects.
/*
* Invokes an application using the Content Handler API
*/
public void doInvoke(String url, String mime, String payload){
Registry register = Registry.getRegistry(this.getClass().getName());
Invocation invoke = new Invocation(url, mime, null, false,
ContentHandler.ACTION_OPEN);
boolean mustQuit = false;
try {
mustQuit = register.invoke(invoke);
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (ContentHandlerException e) {
e.printStackTrace();
} catch (SecurityException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
if(mustQuit){
this.quit();
}
}

Try this:
Registry register = Registry.getRegistry(this.getClass().getName());
You must call Registry.getRegistry for the MIDlet inheritor. Just use your MIDlet for getting the class name.