I had problems with jsoup, because I have written the code for parsing some information from the web site in Java and working perfectly.
But I copy the code in Android (encapsulate it in the asyncTask) but the document is different from the doc Java parsing with jsoup.connect().
Why?
Some code lines are:
Document doc = null;
try {
doc=Jsoup.connect("myurl").timeout(10000).get();
} catch (IOException e) {
e.printStackTrace();
}
Element body = doc.body();
Element figlio = body.child(0);
Elements span_elements = figlio.getElementsByTag("span");
I posted here complete code in java and android.
JAVA
public class MainClass {
public static void main(String[] args){
String ProductName = "";
String Description = "";
String LongDescription = "";
String Category = "";
Document doc = null;
try {
doc=Jsoup.connect("http://eandata.com/lookup/9788820333584/").timeout(10000).get();
} catch (IOException e) {
e.printStackTrace();
}
Element body = doc.body();
Element figlio = body.child(0);
Elements span_elements = figlio.getElementsByTag("span");
for(Element p : span_elements) {
if((p.id().compareTo("")) == 0 || p.id() == null) {
continue;
}
else if(p.id().compareTo("upc_prod_product_o") == 0) {
ProductName = p.text();
continue;
}
else if(p.id().compareTo("upc_prod_description_o") == 0) {
Description = p.text();
continue;
}
else if(p.id().compareTo("upc_prod_cat_path_o") == 0) {
Category = p.text();
continue;
}
else if(p.id().compareTo("upc_prod_url_o") == 0) {
continue;
}
else if(p.id().compareTo("upc_prod_long_desc_o") == 0) {
LongDescription = p.text();
continue;
}
}
System.out.println(ProductName);
System.out.println(Description);
System.out.println(Category);
System.out.println(LongDescription);
This is instead code ANDROID (i have included the INTERNET PERMISSION in AndroidManifest)
ANDROID
public class MainActivity extends Activity {
//Campi necessari per il Parser HTML
String ProductName = "";
String Description = "";
String LongDescription = "";
String Category = "";
#Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
HttpHTML task3 = new HttpHTML();
task3.execute();
}
public class HttpHTML extends AsyncTask<Void,Void,Void> {
#Override
protected void onPreExecute() {
}
#Override
protected Void doInBackground(Void...params) {
Document doc = null;
try {
doc=Jsoup.connect("http://eandata.com/lookup/9788820333584/").timeout(10000).get();
} catch (IOException e) {
e.printStackTrace();
}
//Accedo all'elemento <body> del documento
Element body = doc.body();
System.out.println(body.text());
//Prendo l'elemento figlio del body
Element figlio = body.child(0);
System.out.println(figlio.text());
Elements span_elements = figlio.getElementsByTag("span");
for(Element p : span_elements) {
if((p.id().compareTo("")) == 0 || p.id() == null) {
continue;
}
else if(p.id().compareTo("upc_prod_product_o") == 0) {
ProductName = p.text();
continue;
}
else if(p.id().compareTo("upc_prod_description_o") == 0) {
Description = p.text();
continue;
}
else if(p.id().compareTo("upc_prod_cat_path_o") == 0) {
Category = p.text();
continue;
}
else if(p.id().compareTo("upc_prod_url_o") == 0) {
continue;
}
else if(p.id().compareTo("upc_prod_long_desc_o") == 0) {
LongDescription = p.text();
continue;
}
}
System.out.println(ProductName);
System.out.println(Description);
System.out.println(Category);
System.out.println(LongDescription);
return null;
}
#Override
protected void onProgressUpdate(Void... values) {
}
#Override
protected void onPostExecute(Void result) {
}
}
}
Without knowing the URL you're hitting, this is just a guess, but I would bet $5 I'm right: the server is sending back different HTML based on your user-agent string, and because you're not explicitly setting it, it's defaulting. And the default between Android and Java is different. The server is trying to be helpful and is giving you mobile optimized HTML for Android.
Make sure you specify a user-agent when building your request. See the Connection.userAgent() docs for details. I normally set it to my current browser.
Very interesting problem. If you look at the website the interesting part of the information is loaded dynamically. Jsoup is not supposed to parse this part. I don't understand why it is work differently on android. But it is not important. I found the url where the interesting information loaded from.
Try parsing this one. The added benefit is that it is returned with smaller dataset, it use smaller memory and could be quicker on android.
http://eandata.com/lookup.php?extra=x&code=9788820333584&mode=prod&show=&force_amazon=&ajax=1
Related
I am currently taking each column based on query and modifying variables based on the current position of the cursor. I was wondering if it would be possible to cut down the size of the code by doing something like this where a different function call would be made based on the column within the cursor that is currently being referenced:
do {
Ticket ticket = new Ticket();
for(int i = 0; i < cursor.getColumnCount(); i++)
{
if (cursor.getString(0) != null) {
/*Where the array contains a list of function calls*/
ticket.arrayList(i);
}
}while(cursor.moveToNext());
Below is the code I currently have. From what I know there isn't anything in Java that works like this, but I'm trying to cut down on the number of lines here as I will eventually have close to one hundred columns that will be pulled into the cursor.
public List<Ticket> getTickets(Context context, SQLiteDatabase db)
{
List<Ticket> ticketInfo = new ArrayList<>();
String selectQuery = "SELECT * FROM " + TABLE_TICKET;
Cursor cursor = null;
try {
cursor = db.rawQuery(selectQuery, null);
if (cursor != null) {
try {
if (cursor.moveToFirst()) {
do {
Ticket ticket = new Ticket();
//Set the ticket number
if (cursor.getString(0) != null) {
ticket.setTicketNr(Integer.parseInt(cursor.getString(0)));
}
//Set the ticket id
if (cursor.getString(1) != null) {
ticket.setTicketId(Integer.parseInt(cursor.getString(1)));
}
//
if (cursor.getString(2) != null) {
ticket.setServiceName(cursor.getString(2));
}
//
if (cursor.getString(3) != null) {
ticket.setServiceHouseNr(Integer.parseInt(cursor.getString(3)));
}
//
if (cursor.getString(4) != null) {
ticket.setServiceDirectional(cursor.getString(4));
}
//
if (cursor.getString(5) != null) {
ticket.setServiceStreetName(cursor.getString(5));
}
//
if (cursor.getString(6) != null) {
ticket.setServiceCommunityName(cursor.getString(6));
}
//
if (cursor.getString(7) != null) {
ticket.setServiceState(cursor.getString(7));
}
//
if (cursor.getString(8) != null) {
ticket.setServiceZip1(Integer.parseInt(cursor.getString(8)));
}
//
if (cursor.getString(9) != null) {
ticket.setServiceZip2(Integer.parseInt(cursor.getString(9)));
}
//
if (cursor.getString(10) != null) {
ticket.setTroubleReported(cursor.getString(10));
}
// Adding exercise to list
if (ticket != null) {
ticketInfo.add(ticket);
}
} while (cursor.moveToNext());
} else {
//No results from query
Toast.makeText(context.getApplicationContext(), "No tickets found", Toast.LENGTH_LONG).show();
}
} finally {
if (cursor != null && !cursor.isClosed()) {
cursor.close();
}
}
}
}
catch(SQLiteException exception)//If exception is found
{
Log.d(TAG, "Error", exception);
//Display exception
Toast.makeText(context.getApplicationContext(), exception.toString(), Toast.LENGTH_LONG).show();
}
return ticketInfo;
}
Thank you for any insights into this.
I think this would do it. Just advance the cursor and pass it into the Ticket constructor. You may want to add some error checking.
public class Ticket {
private static class Field {
int intValue;
String stringValue;
final Class type;
Field(Class fieldType){
type = fieldType;
}
void set(String value){
if(type.equals(String.class)){
stringValue = value;
}
else {
intValue = Integer.parseInt(value);
}
}
}
private List<Field> fields = new ArrayList<>();
private Field addField(Field field){
fields.add(field);
return field;
}
// This solution relies on adding fields in the order they'll be retrieved in the cursor.
// Other options are possible such as a map by column index.
private Field ticketNumber = addField(new Field(Integer.class));
private Field serviceName = addField(new Field(String.class));
public Ticket(Cursor cursor){
for(int i=0; i < fields.size(); i++){
Field f = fields.get(i);
f.set(cursor.getString(i));
}
}
}
public int getTicketNumber(){
return ticketNumber.intValue;
}
// Don't know if you need setters
public void setTicketNumber(int value){
ticketNumber.intValue = value;
}
// etc for remaining fields
I would also consider using an ORM to make this stuff easier, rather than dealing with cursors.
I have dug into the Android sources and found that under the hood, each time an Audio route event occurs, an AudioRoutesInfo object is based to the internal updateAudioRoutes method in MediaRouter:
void updateAudioRoutes(AudioRoutesInfo newRoutes) {
if (newRoutes.mMainType != mCurAudioRoutesInfo.mMainType) {
mCurAudioRoutesInfo.mMainType = newRoutes.mMainType;
int name;
if ((newRoutes.mMainType&AudioRoutesInfo.MAIN_HEADPHONES) != 0
|| (newRoutes.mMainType&AudioRoutesInfo.MAIN_HEADSET) != 0) {
name = com.android.internal.R.string.default_audio_route_name_headphones;
} else if ((newRoutes.mMainType&AudioRoutesInfo.MAIN_DOCK_SPEAKERS) != 0) {
name = com.android.internal.R.string.default_audio_route_name_dock_speakers;
} else if ((newRoutes.mMainType&AudioRoutesInfo.MAIN_HDMI) != 0) {
name = com.android.internal.R.string.default_media_route_name_hdmi;
} else {
name = com.android.internal.R.string.default_audio_route_name;
}
sStatic.mDefaultAudioVideo.mNameResId = name;
dispatchRouteChanged(sStatic.mDefaultAudioVideo);
}
final int mainType = mCurAudioRoutesInfo.mMainType;
boolean a2dpEnabled;
try {
a2dpEnabled = mAudioService.isBluetoothA2dpOn();
} catch (RemoteException e) {
Log.e(TAG, "Error querying Bluetooth A2DP state", e);
a2dpEnabled = false;
}
if (!TextUtils.equals(newRoutes.mBluetoothName, mCurAudioRoutesInfo.mBluetoothName)) {
mCurAudioRoutesInfo.mBluetoothName = newRoutes.mBluetoothName;
if (mCurAudioRoutesInfo.mBluetoothName != null) {
if (sStatic.mBluetoothA2dpRoute == null) {
final RouteInfo info = new RouteInfo(sStatic.mSystemCategory);
info.mName = mCurAudioRoutesInfo.mBluetoothName;
info.mDescription = sStatic.mResources.getText(
com.android.internal.R.string.bluetooth_a2dp_audio_route_name);
info.mSupportedTypes = ROUTE_TYPE_LIVE_AUDIO;
sStatic.mBluetoothA2dpRoute = info;
addRouteStatic(sStatic.mBluetoothA2dpRoute);
} else {
sStatic.mBluetoothA2dpRoute.mName = mCurAudioRoutesInfo.mBluetoothName;
dispatchRouteChanged(sStatic.mBluetoothA2dpRoute);
}
} else if (sStatic.mBluetoothA2dpRoute != null) {
removeRouteStatic(sStatic.mBluetoothA2dpRoute);
sStatic.mBluetoothA2dpRoute = null;
}
}
if (mBluetoothA2dpRoute != null) {
if (mainType != AudioRoutesInfo.MAIN_SPEAKER &&
mSelectedRoute == mBluetoothA2dpRoute && !a2dpEnabled) {
selectRouteStatic(ROUTE_TYPE_LIVE_AUDIO, mDefaultAudioVideo, false);
} else if ((mSelectedRoute == mDefaultAudioVideo || mSelectedRoute == null) &&
a2dpEnabled) {
selectRouteStatic(ROUTE_TYPE_LIVE_AUDIO, mBluetoothA2dpRoute, false);
}
}
}
Unfortunately, the only thing I have found that is exposed about the device type in the MediaRouter callbacks, is the internal string resource name of the device (e.g. Phone or Headphones). However, you can see that under the hood, this AudioRoutesInfo object has references to whether the device was a headphone, HDMI etc.
Has anyone found a solution to get at this information? The best way I have found is to use the internal resource names, which is pretty ugly. God, if they would just provide the AudioRoutesInfo object all this information could be accessed without having to rely on a resource hack.
I am new to Android and I want to play online radio (example link http://www.bbc.co.uk/radio/listen/live/r1_aaclca.pls) using ParsePls. I have PlsParse code but after this I don't know how to play using the player in Android.
PlsParser.java
public PlsParser(File file) throws FileNotFoundException {
this.reader = new BufferedReader(new FileReader(file), 1024);
}
#Override
public List<String> getUrls() {
LinkedList<String> urls = new LinkedList<String>();
while (true) {
try {
String line = reader.readLine();
if (line == null) {
break;
}
String url = parseLine(line);
if (url != null && !url.equals("")) {
urls.add(url);
}
} catch (IOException e) {
e.printStackTrace();
}
}
return urls;
}
private String parseLine(String line) {
if (line == null) {
return null;
}
String trimmed = line.trim();
if (trimmed.indexOf("http") >= 0) {
return trimmed.substring(trimmed.indexOf("http"));
}
return "";
}
}
and
PlaylistParser.java
public interface PlaylistParser {
public List<String> getUrls();
}
Basically I need to play online fm in Android. How can I do this?
I got some solution without Parse pls,
Just Enter URL of pls file in browser(example link yp.shoutcast.com/sbin/tunein-station.pls?id=213352),
you will get download "tunein-station.pls",
Just open the downloaded file with notepad,
you can see
[playlist]
numberofentries=2
File1=http://208.115.222.205:9938
Title1=(#1 - 1/100) RADIO CITY TAMIL
Length1=-1
File2=http://208.115.222.205:9948
Title2=(#2 - 67/300) RADIO CITY TAMIL
Length2=-1
Version=2
Here File1 URL(shoutcast online radio url)
Now using this URL i coded with mediaplayer.
I am trying to get the absolute URL in my managed bean's action listener. I have used:
HttpServletRequest#getRequestURL() // returning http://localhost:7101/POSM/pages/catalog-edit
HttpServetRequest#getQueryString() // returning _adf.ctrl-state=gfjk46nd7_9
But the actual URL is: http://localhost:7101/POSM/pages/catalog-edit?_adf.ctrl-state=gfjk46nd7_9&articleReference=HEN00067&_afrLoop=343543687406787. I don't know why the parameter artcileReference get omitted.
Is there any method which can give me the whole URL at once? How can I get the whole URL with all query string?
You can reconstruct your URL manually by using ServletRequest#getParameterNames() and ServletRequest#getParameter() both available with the HttpServletRequest instance.
Here is a sample code I've used in the past for this exact purpose :
private String getURL()
{
Enumeration<String> lParameters;
String sParameter;
StringBuilder sbURL = new StringBuilder();
Object oRequest = FacesContext.getCurrentInstance().getExternalContext().getRequest();
try
{
if(oRequest instanceof HttpServletRequest)
{
sbURL.append(((HttpServletRequest)oRequest).getRequestURL().toString());
lParameters = ((HttpServletRequest)oRequest).getParameterNames();
if(lParameters.hasMoreElements())
{
if(!sbURL.toString().contains("?"))
{
sbURL.append("?");
}
else
{
sbURL.append("&");
}
}
while(lParameters.hasMoreElements())
{
sParameter = lParameters.nextElement();
sbURL.append(sParameter);
sbURL.append("=");
sbURL.append(URLEncoder.encode(((HttpServletRequest)oRequest).getParameter(sParameter),"UTF-8"));
if(lParameters.hasMoreElements())
{
sbURL.append("&");
}
}
}
}
catch(Exception e)
{
// Do nothing
}
return sbURL.toString();
}
Here I came up with my solution, taking idea of the answer given by Alexandre, considering that HttpServletRequest#getParameterValues() method:
protected String getCurrentURL() throws UnsupportedEncodingException {
Enumeration parameters = getServletRequest().getParameterNames();
StringBuffer urlBuffer = new StringBuffer();
urlBuffer.append(getServletRequest().getRequestURL().toString());
if(parameters.hasMoreElements()) {
if(!urlBuffer.toString().contains("?")) {
urlBuffer.append("?");
} else {
urlBuffer.append("&");
}
}
while(parameters.hasMoreElements()) {
String parameter = (String)parameters.nextElement();
String[] parameterValues = getServletRequest().getParameterValues(parameter);
if(!CollectionUtils.sizeIsEmpty(parameterValues)) {
for(int i = 0; i < parameterValues.length; i++) {
String value = parameterValues[i];
if(StringUtils.isNotBlank(value)) {
urlBuffer.append(parameter);
urlBuffer.append("=");
urlBuffer.append(URLEncoder.encode(value, "UTF-8"));
if((i + 1) != parameterValues.length) {
urlBuffer.append("&");
}
}
}
}
if(parameters.hasMoreElements()) {
urlBuffer.append("&");
}
}
return urlBuffer.toString();
}
I wanted to harvest some data on specific apps in
Google play marketplace.
I have used this unofficial API:
https://code.google.com/p/android-market-api/
Here is my code, that basically gets list of apps' names
and try to fetch other data on each app:
public void printAllAppsData(ArrayList<AppHarvestedData> dataWithAppsNamesOnly)
{
MarketSession session = new MarketSession();
session.login("[myGamil]","[myPassword]");
session.getContext().setAndroidId("dead00beef");
final ArrayList<AppHarvestedData> finalResults = new ArrayList<AppHarvestedData>();
for (AppHarvestedData r : dataWithAppsNamesOnly)
{
String query = r.name;
AppsRequest appsRequest = AppsRequest.newBuilder()
.setQuery(query)
.setStartIndex(0).setEntriesCount(10)
//.setCategoryId("SOCIAL")
.setWithExtendedInfo(true)
.build();
session.append(appsRequest, new Callback<AppsResponse>() {
#Override
public void onResult(ResponseContext context, AppsResponse response) {
List<App> apps = response.getAppList();
for (App app : apps) {
AppHarvestedData r = new AppHarvestedData();
r.title = app.getTitle();
r.description = app.getExtendedInfo().getDescription();
String tmp = app.getExtendedInfo().getDownloadsCountText();
tmp = tmp.replace('<',' ').replace('>',' ');
int indexOf = tmp.indexOf("-");
tmp = (indexOf == -1) ? tmp : tmp.substring(0, indexOf);
r.downloads = tmp.trim();
r.rating = app.getRating();
r.version = app.getVersion();
r.userRatingCount = String.valueOf(app.getRatingsCount());
finalResults.add(r);
}
}
});
session.flush();
}
for(AppHarvestedData res : finalResults)
{
System.out.println(res.toString());
}
}
}
Should I realyy call session.flush(); at this point?
all my quesries return empty collection as a result,
even though I see there are some names as input.
It works fine when I send only one hard coded app name as a query.
session.flush() close you session
you should open session for each query. pay attention the user can be locked for few minutes so you should have many users to to those queries.
if you have the AppId you should use that query:
String query = r.name;
AppsRequest appsRequest = AppsRequest.newBuilder()
.setAppId("com.example.android")
.setWithExtendedInfo(true)
.build();
session.append(appsRequest, new Callback<AppsResponse>() {
#Override
public void onResult(ResponseContext context, AppsResponse response) {
List<App> apps = response.getAppList();
for (App app : apps) {
AppHarvestedData r = new AppHarvestedData();
r.title = app.getTitle();
r.description = app.getExtendedInfo().getDescription();
String tmp = app.getExtendedInfo().getDownloadsCountText();
tmp = tmp.replace('<',' ').replace('>',' ');
int indexOf = tmp.indexOf("-");
tmp = (indexOf == -1) ? tmp : tmp.substring(0, indexOf);
r.downloads = tmp.trim();
r.rating = app.getRating();
r.version = app.getVersion();
r.userRatingCount = String.valueOf(app.getRatingsCount());
finalResults.add(r);
}
}
});
session.flush();
}
if you want to download also screenshoot or images you should call this query:
GetImageRequest? imgReq; imgReq = GetImageRequest?.newBuilder().setAppId(appId).setImageUsage(AppImageUsage?.SCREENSHOT).setImageId("0").build();
session.append(imgReq, new Callback<GetImageResponse>() {
#Override public void onResult(ResponseContext? context, GetImageResponse? response) {
Log.d(tag, "------------------> Images response <----------------"); if(response != null) {
try {
//imageDownloader.download(image, holder.image, holder.imageLoader);
Log.d(tag, "Finished downloading screenshot 1...");
} catch(Exception ex) {
ex.printStackTrace();
}
} else {
Log.e(tag, "Response was null");
} Log.d(tag, "------------> End of Images response <------------");
}
});