Jsoup not connecting to webpage in Android Studio - java

I am working on a project right now where I use jsoup in a class with the function retrieveMedia in order to return an ArrayList filled with data from the webpage. I run it in a thread since you shouldn't be connecting to URLs from the main thread. I run it and join it. However, it doesn't work (I tested the same code in Eclipse separate from Android Studio and it worked fine). It seems that no matter what I do I can't get jsoup to connect to the webpage. Below is my class MediaRetriever.
public class MediaRetreiever {
public ArrayList<Media> retrieveMedia() {
ArrayList<Media> mediaOutput = new ArrayList<Media>(); //Store each scraped post
Thread downloadThread = new Thread(new Runnable() {
public void run() {
Document doc = null;
try {
doc = Jsoup.connect(<Website Im connecting to>).timeout(20000).get();
} catch (IOException e) {
System.out.println("Failed to connect to webpage.");
mediaOutput.add(new Media("Failed to connect", "oops", "", "oh well"));
return;
}
Elements mediaFeed = doc.getElementById("main").getElementsByClass("node");
for (Element e : mediaFeed) {
String title, author, imageUrl, content;
title=e.getElementsByClass("title").text().trim();
author=e.getElementsByClass("content").tagName("p").select("em").text().trim();
content=e.getElementsByClass("content").text().replace(author,"").trim();
Media media = new Media(title, author, "", content);
mediaOutput.add(media);
}
}
});
downloadThread.start();
try {
downloadThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
return mediaOutput;
}
}
Running this class's method from another class and it doesn't ever connect. Any ideas?

Since you say that the problem persists only in Android, it looks like that you should add the user agent string to your request - first get the user agent string of a browser that displays correctly the site, and then add it to the request:
doc = Jsoup.connect(<Website Im connecting to>)
.userAgent("your-user-agent-string")
.timeout(20000).get();
And as a sidenote - if you are catching exception, don't print your own error message - print the original message, it may be very useful.

Related

WordAPI ApiException caught: Not Found

I'm following [this][1] documentation to connect to api, I want to build a JavaFX app that you can enter a word and it's retrieves it from api and I wanted to test the feature before displaying contents on GUI. Howerer I got this exception that the words is not found while I tested endpoint in their website and that word had definitions. I'm pretty new to using API perhaps I missed something that isn't shown in the documentation?
Here is my code:
public void testDetailsWords(String word,String detail) {
//String word = "lovely"; // Word
//String detail = "definitions"; // Detail
TimeZone.setDefault(TimeZone.getTimeZone("UTC"));
apiClient = Configuration.getDefaultApiClient();
apiClient.setBasePath("https://www.wordsapi.com/");
// configure authentications
Authentication auth;
auth = apiClient.getAuthentication("Default");
((ApiKeyAuth) auth).setApiKey(Apikey);
try {
WordsApi wordsApi = new WordsApi();
DetailsResponse response = wordsApi.details(word, detail);
System.out.println(response);
} catch (ApiException e) {
System.out.printf("ApiException caught: %s\n", e.getMessage());
}
}
Here is my Controller code :
#FXML
private void handleDefinitions(ActionEvent event) throws IOException {
String word =definitionsfield.getText();
String detail= "definitions";
apiCall.testDetailsWords(word,detail);
}
I would be grateful for the help.
[1]: http://restunited.com/docs/6vc24wq3ojpq
It's seems that tutorial that I tried following is very misleading which is very dissapoing, but I've learned my lesson now .
I decided to use Unirest to get request and it's worked perfectly.
If you stumble upon the link it's better not to follow that tutorial

How do I create an Alfresco site programmatically from a repository webscript?

I've implemented an Alfresco repository webscript (in Java) to programmatically create a new site.
I notice that there's a SiteService interface which I thought could be used to do this -
SiteInfo site = siteService.createSite("site-dashboard", "mySite",
"mySite", "", SiteVisibility.PUBLIC);
However, this results in the creation of a non-functional site, and although it's visible within the Alfresco Share dashboard, I'm not able to use it.
I then came across this code sample, which is doing exactly what I want. BUT the code includes a section to do authentication, involving sending the user's login and password details to a dologin web service. Don't really want to do this.
But as the user has already logged in via Alfresco Share, they should already be authenticated.
If I call the create-site webscript from my code, as shown in the example (without the initial call to dologin), I'm getting a 401 (unauthorised) return code.
So my question is, how do I tell the create-site webscript about my authentication?
I read about using an authentication ticket here. Is this ticket stored in the session, and if so, how do I access it within my Java code? If I could get the ticket, then this would be sufficient to invoke the create-site webscript.
Update: I've added the alf_ticket parameter as suggested by the comment, but I'm still getting a 401 response.
My current code is:
public NodeRef createServiceChange(String serviceChangeName) {
HttpClient client = new HttpClient();
String ticket = authService.getCurrentTicket();
PostMethod createSitePost = new PostMethod("http://localhost:8081/share/service/modules/create-site");
JSONObject siteObject = new JSONObject();
try {
siteObject.put("shortName", serviceChangeName);
siteObject.put("visiblity", "Public");
siteObject.put("sitePreset", "site-dashboard");
siteObject.put("title", serviceChangeName);
siteObject.put("description", serviceChangeName);
siteObject.put("alf_ticket", ticket);
createSitePost.setRequestHeader("Content-Type", "application/json");
createSitePost.setRequestHeader("Accept", "application/json");
createSitePost.setRequestEntity(new StringRequestEntity(siteObject.toString(), "application/json", "UTF-8"));
int status = client.executeMethod(createSitePost);
System.out.println("create a site script status :: " + status);
if (status == HttpStatus.SC_OK) {
System.out.println("Site created OK");
}
else{
System.out.println("There is error in site creation");
}
} catch (JSONException err) {
err.printStackTrace();
} catch (UnsupportedEncodingException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (HttpException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}
So I've managed to successfully create a site, programmatically, and here's what I did:
First, forget about writing a repository (platform) webscript. Creation of sites in Alfresco is done by invoking a Share module, so you'll need to implement either a page, or custom menu item to create a site. I was also getting a lot of problems with authentication, but if you log in to the system via Alfresco Share, and in your Javascript, use the provided Alfresco Ajax request, then authentication shouldn't be a problem.
Here are the components I used:-
Create a Share page to create your site. In the Freemarker template (.ftl) add a form to collect the site details.
Attach a button on the form to the following Javascript function. Note that I cobbled this together from various code fragments on the web, so it could use some cleaning up. But it basically works for me -
function create_site()
{
var sc_form = document.forms.namedItem('sc_form');
var name = sc_form.elements.namedItem('name').value;
var url = Alfresco.constants.URL_CONTEXT + "service/modules/create-site";
Alfresco.util.Ajax.request({
method : Alfresco.util.Ajax.POST,
url : url,
dataObj: {
sitePreset: "site-dashboard",
visibility: "PUBLIC",
title: name,
shortName: name,
description: name
},
requestContentType: Alfresco.util.Ajax.JSON,
successCallback:
{
fn: function(res){
alert("success");
alert(res.responseText);
},
scope: this
},
failureCallback:
{
fn: function(response)
{
Alfresco.util.PopupManager.displayPrompt(
{
title: Alfresco.util.message("message.failure", this.name),
text: "search failed"
});
},
scope: this
}
});
}

Google Drive for Android SDK Doesn't List Files

I've got a really odd problem with the Google Drive Android SDK. I've been using it for several months now, and until last week it performed perfectly. However, there is now a really odd error, which doesn't occur all the time but does 9 out of 10 times.
I'm trying to list the user's files and folders stored in a particular Google Drive folder. When I'm trying to use the method Drive.files().list().execute(), 9 out of 10 times literally nothing happens. The method just hangs, and even if I leave it for an hour, it just remains doing... nothing.
The code I'm using is below - all of this being run within the doInBackground of an AsyncTask. I've checked credentials - they are all fine, as is the app's certificate's SHA1 hash. No exceptions are thrown. Google searches have yielded nothing. Here is the particular bit of code that's bothering me:
try {
GoogleAccountCredential credential = GoogleAccountCredential.usingOAuth2(
SettingsActivity.this, Arrays.asList(DriveScopes.DRIVE));
if (googleAccountName != null && googleAccountName.length() > 0) {
credential.setSelectedAccountName(googleAccountName);
Drive service = new Drive.Builder(AndroidHttp.newCompatibleTransport(),
new GsonFactory(), credential).build();
service.files().list().execute(); // Google Drive fails here
} else {
// ...
}
} catch (final UserRecoverableAuthIOException e) {
// Authorisation Needed
runOnUiThread(new Runnable() {
#Override
public void run() {
try {
startActivityForResult(e.getIntent(), REQUEST_AUTHORISE_GDRIVE);
} catch (Exception e) {
Log.e("SettingsActivity: Google Drive", "Unable to add Google Drive account due to Exception after trying to show the Google Drive authroise request intent, as the UserRecoverableIOException was originally thrown. Error message:\n" + e.getMessage());
}
}
});
Log.d("SettingsActivity: Google Drive", "UserRecoverableAuthIOException when trying to add Google Drive account. This is normal if this is the first time the user has tried to use Google Drive. Error message:\n" + e.getMessage());
return;
} catch (Exception e) {
Log.e("SettingsActivity: Google Drive", "Unable to add Google Drive account. Error message:\n" + e.getMessage());
return;
}
I'm using Drive API v2. Thanks everyone!
Edit
Having played around a bit more, it turns out this isn't for just listing files. Trying to interact with any file on Google Drive behaves the same way - deleting, downloading, creating... Anything! I have also noticed that putting the device in aeroplane mode so it has not internet access makes no difference either: Google Drive doesn't throw an exception, or even return, it just freezes the thread it's on.
I've updated to the very latest Drive API lib but that hasn't helped. I remembered that the error happened soon after I added the JSch SSH library to the project, so I removed that, but it made no difference. Removing and re-adding the Drive API v2 has made no difference either, and nor has cleaning the project.
Edit 2
I've found something which may be significant. On the Google Developer console, I had some Drive errors recorded as follows:
TOP ERRORS:
Requests % Requests Methods Error codes
18 38.30% drive.files.list 400
14 29.79% drive.files.insert 500
11 23.40% drive.files.update 500
4 8.51% drive.files.get 400
Do you reckon these are the errors? How could I fix them? Thanks
This is my code and it's work
new AsyncTask<Void, Void, List<File>>() {
#Override
protected List<File> doInBackground(Void... params) {
List<File> result = new ArrayList<File>();
try {
com.google.api.services.drive.Drive.Files.List list = service.files().list();
list.setQ("'" + sourcePath + "' in parents");
FileList fileList = list.execute();
result = fileList.getItems();
if(result != null) {
return result;
}
} catch (UserRecoverableAuthIOException e) {
startActivityForResult(e.getIntent(), REQUEST_AUTHORIZATION);
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
protected void onPostExecute(List<File> result) {
//This is List file from Google Drive
};
}.execute();
I've come up with a solution which does work, and thought I'd post it so others could see it if they happen to come across the problem.
Luckily, I had backed up all of the previous versions of the app. So I restored the whole project to how it was two weeks ago, copied and pasted all changes from the newer version which had been made since then, and it worked. I don't see why this should work, since the end result is the same project, but it does!
Google Drive List Files
This might help you.. Try to display it in ListView u will see all fetched folders
public void if_db_updated(Drive service)
{
try {
Files.List request = service.files().list().setQ("mimeType = 'application/vnd.google-apps.folder'");
FileList files = request.execute();
for(File file : files.getItems())
{
String title = file.getTitle();
showToast(title);
}
} catch (UserRecoverableAuthIOException e) {
startActivityForResult(e.getIntent(), REQUEST_AUTHORIZATION);
} catch (IOException e) {
e.printStackTrace();
}
}
public void showToast(final String toast) {
runOnUiThread(new Runnable() {
#Override
public void run() {
Toast.makeText(getApplicationContext(), toast, Toast.LENGTH_SHORT).show();
}
});

Block videos in android webview - Alternative to shouldInterceptRequest()

I have created a web browser using webview in android. My aim is to control the content of the webview before it is loaded. Whenever the content of the webview makes a request to any domain server, it has to pass through shoulInterceptRequest(). If the url is pointing to any video uploading sites(youtube.com, vimeo.com), I can change it to some Access Denied url so that video will not be loaded.
#Override
public WebResourceResponse shouldInterceptRequest(final WebView view, String url) {
try {
if (access.permission(url)) {
return super.shouldInterceptRequest(view, url);
}
} catch (Exception e) {
e.printStackTrace();
}
return getResponseData();
}
private WebResourceResponse getResponseData() {
try {
String str = "Access Denied";
InputStream data = new ByteArrayInputStream(str.getBytes("UTF-8"));
return new WebResourceResponse("text/css", "UTF-8", data);
} catch (IOException e) {
return null;
}
}
But shoulInterceptRequest() is availabe from API 11. I NEED it to work from API 8.
Is there any alternative way to implement it ? I need to block the url if it is potinting to any video uploading sites BEFORE LOADING ANY DATA.
How about using the http://developer.android.com/reference/android/webkit/WebViewClient.html#shouldOverrideUrlLoading(android.webkit.WebView, java.lang.String) event?
You can block the url and then call http://developer.android.com/reference/android/webkit/WebView.html#loadUrl(java.lang.String) to show anything you want (and even run arbitrary javascript with the "javascript:do_something()" notation)

Using the content handler API (JSR 211) to open applications

I want to be able to launch native and J2ME applications through my application using the content handler API (JSR 211) on a Nokia 6212.
At the moment, I am unable to do so, as it always states that there is "No Content Handler Found" and throws a javax.microedition.content.ContentHandlerException.
At the moment, I am trying to get the phone to launch its browser and go to a certain website, just to test that I can use the framework. I have tried many different Invocation objects:
//throw exceptions
new Invocation("http://www.somesite.com/index.html",
"application/internet-shortcut");
new Invocation("http://www.google.co.uk","text/html");
// a long shot, I know
new Invocation("http://www.somesite.com/text.txt","text/plain");
// massive long shot
new Invocation("http://www.google.co.uk","application/browser");
//appears to download the link and content (and definitely does in the Nokia
// emulator) and then throws an exception
new Invocation("http://www.google.co.uk");
new Invocation("http://www.somesite.com/index.html");
Below is the code that I have been using, please bear in mind the parameters often changed to generate the different Invocation objects.
/*
* Invokes an application using the Content Handler API
*/
public void doInvoke(String url, String mime, String payload){
Registry register = Registry.getRegistry(this.getClass().getName());
Invocation invoke = new Invocation(url, mime, null, false,
ContentHandler.ACTION_OPEN);
boolean mustQuit = false;
try {
mustQuit = register.invoke(invoke);
} catch (IllegalArgumentException e) {
e.printStackTrace();
} catch (ContentHandlerException e) {
e.printStackTrace();
} catch (SecurityException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
if(mustQuit){
this.quit();
}
}
Try this:
Registry register = Registry.getRegistry(this.getClass().getName());
You must call Registry.getRegistry for the MIDlet inheritor. Just use your MIDlet for getting the class name.

Categories