I'm currently working on a project that involves analyzing weather information from various airports and weather stations. What I need to do is display the basic weather info from a url (example: http://w1.weather.gov/xml/current_obs/KFHB.xml). Where I need to display temperature, visibility, etc. in some basic format. I was thinking an array list.
Can anyone tell me how to do this? How do I display the elements from a webpage onto a java application? What classes (methods, or other procedures) should I look into? Are there any good templates out there for this? Any help would be appreciated. Thanks.
You can read through the contents of the url like reading a file doing the following:
URL url = new URL(link);
BufferedReader in = new BufferedReader(new InputStreamReader(url.openStream()));
You can also capture the RSS feed from that weather station. Read this article if you want an an example as how to parse a RSS feed:
http://www.vogella.com/articles/RSSFeed/article.html
Related
I was learning to make quiz app from online and it is going well. I was wondering instead of reading json from assets , it will be wise to read from online such that question can be added or changes accordingly and user don't have to update app.
Here is the JSON Structure.
{"questions" : [{"category":"general","question": "Grand Central Terminal, Park Avenue, New York is the world's", "choices": ["largest railway station","highest railway station","longest railway station","None of the above"], "correctAnswer":0},
{"category":"science","question": "Entomology is the science that studies", "choices": ["Behavior of human beings","Insects","The origin and history of technical and scientific terms","the formation of rocks"], "correctAnswer":1},
{"category":"science", "question":"What is known as the 'master gland' of the human body?", "choices":["Thyroid gland","Pituitary gland","Pineal gland","Pancreas"],"correctAnswer":1}
]}
and the code to read from assets is
private String loadJSONFromAsset() {
String json = null;
try {
InputStream is = mContext.getAssets().open("questionsJSON.json");
int size = is.available();
byte[] buffer = new byte[size];
is.read(buffer);
is.close();
json = new String(buffer, "UTF-8");
} catch (IOException ex) {
ex.printStackTrace();
return null;
}
return json;
}
I would like to show progress loading dialog when next question loads and any help will be largely appreciated. Thanks in advance.
Best option will be using REST APIs, Get data from Server/Database, which can be edited anytime from anywhere
You can learn to use Node js, it is not hard and it is based on JavaScript.
For getting JSON from APIs you can use Retrofit
Learning and implementing these things will be a bit hard if you are beginner but it will be the best option for long run
hope this helped!
Maybe consider using two different threads (or Runnables), one thread for downloading the JSON content and the other thread for displaying the GUI. For example, take a look at this: Stackoverflow Post
The solution involved making a Runnable that would first start downloading the data from the online website and then update the current progress onto the GUI thread as it is downloading. He uses the BufferedInputStream class so he can use a while loop to read the data in, update the number of bytes downloaded, get the current progress, and then display the results. I suppose you can do something similar here by using a while loop, and then checking if the download is finished. If so, you can close the display.
I am writing a crawler in java that examines an IMDB movie page and extracts some info like name, year etc. User writes (or copy/pastes) the link of the tittle and my program should do the rest.
After examining html sources of several (imdb) pages and browsing on how crawlers work I managed to write a code.
The info I get (for example title) is in my mother tongue. If there is no info in my mother tongue I get the original title. What I want is to get the title in a specific language of my choosing.
I'm fairly new to this so correct me if I'm wrong but I get the results in my mother tongue because imdb "sees" that I'm from Serbia and than customizes the results for me. So basically I need to tell it somehow that I prefer results in English? Is that possible (i imagine it is) and how do I do it?
edit:
Program crawls like this: it gets the url path in String, converts it to url, reads all of the source with bufferedreader and inspects what it gets. I'm not sure if that is the right way to do it but it's working (minus the language problem)
code:
public static Info crawlUrl(String urlPath) throws IOException{
Info info = new Info();
//
URL url = new URL(urlPath);
URLConnection uc = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
uc.getInputStream(), "UTF-8"));
String inputLine;
while ((inputLine = in.readLine()) != null){
if(inputLine.contains("<title>")) System.out.println(inputLine);
}
in.close();
//
return info;
}
this code goes trough a page and prints the main title on console.
You don't need to crawl IMDB, you can use the dumps they provide: http://www.imdb.com/interfaces
There's also a parser for the data they provide: https://code.google.com/p/imdbdumpimport/ it's not perfect but maybe it will help you (you can expect spending some effort to make it work).
An alternative parser: https://github.com/dedeler/imdb-data-parser
EDIT You're saying you want to crawl IMDB anyway for learning purposes. So you'll probably have to go with http://en.wikipedia.org/wiki/Content_negotiation as suggested in the other answer:
uc.setRequestProperty("Accept-Language", "de; q=1.0, en; q=0.5");
Try to look at the request headers used by your crawler, mine is containing Accept-Language:fr-FR,fr;q=0.8,en-US;q=0.6,en;q=0.4 so I get the title in French.
EDIT :
I checked with ModifyHeaders add-on on Google Chrome and the value en-US is getting me the English title for the movie =)
www.rgrfm.be/rgrsite/maxradio/android.php
www.rgrfm.be/rgrsite/maxradio/onair.txt
The track information of the music being played is contained in onair.txt. android.php is a php script I wrote.
I need to display the track information in my Android application. I do not want do download it to disk but keep it in memory. I don't know if the php script is useless because it would create additional overhead. So it's probably better to simply parse onair.txt
InputStream is = new URL("http://www.rgrfm.be/rgrsite/maxradio/onair.txt").openStream();
I am stuck with this. Has anyone got time to help me?
As described, php script seems useless. Since, you can directly read the text file. So, first read it as text, then parse it.
URL url = new URL("http://www.rgrfm.be/rgrsite/maxradio/onair.txt");
String text = readAsText(url)
parse(text);
String readAsText(URL url) {
// read the url as text here.
}
void parse(String text) {
}
I am trying to parse the images that are display in this link http://lawncare.ncsu.edu/RSSFeed.aspx and display it in an android device. Right now, I am only able to parse the text associated with the images. Can anyone suggest any ideas on how to go about parsing these images? Preferably not using JSoup because I am already half way down the code.
Try looking at this question over here at : How to parse XML using the SAX parser
specifically the answer given by damiean and his solution.
This is a part of my code where i am parsing the rss feed :
Bundle b = new Bundle();
b.putString("title", feed.getItem(position).getTitle());
b.putString("description", feed.getItem(position).getDescription());
b.putString("content", feed.getItem(position).getContent());
b.putString("link", feed.getItem(position).getLink());
b.putString("pubdate", feed.getItem(position).getPubDate().toString());
What i have done is that the "field" contains both the description(text) and the image . I passed the whole "content" field contents into a WebView and it displays the image as well as the text correctly.
I have a problem once again where I cant find the source code because its hidden or something... When my java program indexes the page it finds everything but the info i need... I assume its hidden for a reason but is there anyway around this?
Its just a bunch of tr/td tags that show up in firebug but dont show up when viewing the page source or when i do below
URL url = new URL("my url");
URLConnection yc = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(yc.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null) {
I really have no idea how to attempt to get the info that i need...
The reason for this behavior is because probably those tags are dynamically injected into the DOM using javascript and are not part of the initial HTML which is what you can fetch with an URLConnection. They might even be created using AJAX. You will need a javascript interpreter on your server if you want to fetch those.
If they don't show up in the page source, they're likely being added dynamically by Javascript code. There's no way to get them from your server-side script short of including a javascript interpreter, which is rather high-overhead.
The information in the tags is presumably coming from somewhere, though. Why not track that down and grab it straight from there?
Try Using Jsoup.
Document doc = doc=Jsoup.parse("http:\\",10000);
System.out.print(doc.toString());
Assuming that the issue is that the "missing" content is being injected using javascript, the following SO Question is pertinent:
What's a good tool to screen-scrape with Javascript support?