I am currently working on a project for my portfolio. I having a little trouble trying to find the right solution to this problem mainly because I have never tried it before.
I am using a free API service that I found online. I have created the database to match all the information, and not I just need to download the information and parse into my application.
I have parsed data from the API (JSON) and into my database. A couple of suggestions that I have found is reading 10 records at a time, but I want to try reading everything at once and then updating accordingly (let us say every 24 hours).
The APi I am using is the a free Game of Thrones API and below is the list I want of about the URL is formed to access each part data as I move through it.
https://anapioficeandfire.com/api/characters/
https://anapioficeandfire.com/api/books/
https://anapioficeandfire.com/api/houses/
At the end of the each of these URLS is a number that indicates the record number that I am trying to get. I have done this before while get information from a single page, and the page contained multiple JSON object. This time I need to move through multiple pages to get the single object on that page.
To give you an idea of the steps that I am looking for :
Go the page
Download the Information
Move on the next page
Break when there when I have reached the end.
Related
I've some problems to understand the functionality of RSS in my special situation.
I've a REST service (written in Java using Spring) which reads some information from a database and dynamically generates the RSS page. The pubdate element of each item is filled by the current date.
The service is reachable under an URL like "http://intern.system.com/rest/api/rss".
I took that URL and include it into a wiki page (the wiki is in this scenario the RSS reader).
The background of this workflow is the following: The database is filld with events or todo's for the next few days. Each event has a title, a description and a date. Until now this information is picked up by hand and transferred to show on a wiki page.
My goal is to automate this process. I want to generate a RSS feed of the events or todo's for the current day (that makes my REST service) and automatically show it on a wiki page.
Is this a good way to do? Does the RSS is shown the whole day (or only by the first call) and for all visitors? F.e. one person enters the page at 8 o'clock, another at 9. Both should see the same information for the day. I think the REST service is called twice in that case. Is this a problem?
This is not problem but as we are developers, we should chase best practices on what we are building. So your case reminds me caching your rest service. If your data is daily updated, you can use caching property which is mostly preferred for static resources. You can add #Cacheable("rssCache") annotation to your component methods, after the first call, the result will be cached.
Is it possible to send extra data attached to a http response via Java or Php?
My Website is a homework-platform: One User enters homeworks into a database, and all users can then see the homeworks on the website. The current load is very inefficient, as the browser makes two requests for eveything to load: One for the index file and one for the homeworks. For the homeworks request the client also sends settings of the user to the server, based on which the returned homeworks are generated by a Php script.
Now, I wonder, if it is possible, to combine those two requests into one? Is it maybe possible to detect the http request with Java or Php on the server, read the cookies (where the settings are saved), then get the homeworks from the database and send the data attached to the http response to the client? Or, even better, firstly only return the index file and as soon as possible and the homework data afterwards as a second response, because the client needs some time to parse the Html & build the DOM-tree when it can't show the homeworks anyway.
While browsing the web I stumbled across terms like "Server-side rendering" and "SPDY", but I don't know if those are the right starting points.
Any help is highly appreciated, as I'm personally very interested in a solution and it would greatly improve the load time of my website.
A simple solution to your problem is to initialize your data in the index file.
You would create a javascript object, and embed it right into the html, rendered by your server. You could place this object in the global namespace (such as under window.initData), so that it can be accessed by the code in your script.
<scipt>
window.initData = {
someVariable: 23,
}; // you could use json_encode if you use php, or Jackson if you use java
</script>
However, it is not a huge problem if your data is fetched in a separate server request. Especially when it takes more time to retrieve the data from the database/web services, you can provide better user experience by first fetching the static content very quickly and displaying a spinner while the (slower) data is being loaded.
In my GWT application I am retrieving the XML data from a REST server. I am using Piriti XML parser https://code.google.com/p/piriti/wiki/Xml for deserializing the object and display in a table. As long as we are returning upto 1000 records everything is fine but with the big result it just hang and gives user message to stop the java script running in the back. Could someone please help me to find the best way to handle big data in GWT OR more precisely the best approach to parse big XML file in GWT.
Thanks a lot for all your suggestions.
The problem is that parsing a big XML document slows down the browser. And you need enough memory to hold the whole DOM plus your mapped objects in memory. The only solution is to avoid such situation. You have to adapt your REST service to be able to send only small chunks of data to the browser. So if you already have a paged table you only retrieve the data for the first page at the beginning. If the user wants to change the page you do another REST call to retrieve the data for the next page.
If you cannot change the the REST service itself you can create another server side service (on a server controlled by you) as a proxy. At first access you call the original REST service, store the XML at your own server and allow the client to retrieve only parts of that XML.
Heres what i want to do. Im quite a beginner with this so maybe a lame question, But, I want to implement gui application in java wich gets data from sports livescore pages
e.g
http://www.futbol24.com/Live/
http://livescore.com/
and parse it (somehow) in my app...and then i will be able to store it in for example jtable ,save full time results in database,playing sounds after goal is scored and so on
What is the best way to do this ?
It would be almost impossible to parse an HTML document from a live web page and get specific information from it. If you did manage to work out exactly where in the document the data is, the page structure could change at any time. The scores might not even be in the HTML - they could be fetched by Javascript in the page.
I suggest you find an RSS feed of the information you want. Then you'll only have a nice, small piece of XML to parse. That's what it's for.
I daily visit this link to find my lectures at school. Every time I have to scroll down the list to find my own class, and then post it so I can view the result. Is there any way i could make a direct link to the preferred content? I'm looking to create a simple webview app in Android showing individual form categories.
EDIT : Really any method for converting the aspx info into another format would do the trick. Prefferably a direc link to each form item. But if I can convert every single item to a .xml file or anything else I could work with it. But I have to make it automated.
You can capture the outgoing request and write a simple application to POST the data back to the page. The WebClient class is useful for this.
Looking at the request in Chrome's developer tools, I see that the form posts back to itself and then redirects to the result page. Presumably, you should POST the form data to the initial page, which will then cause it to perform the redirect.
The form contains a large amount of ViewState data which may or may not need to be included in the request to make it work.
A completely different approach would be to find a browser extension, such as a macro recorder, which emulate your actions. This plugin (haven't tried it myself) appears to do exactly that.