I have a PHP page that grabs a variable via GET and then pulls in some information from a database based on that variable. Once finished with the server-side stuff, there is some javascript that runs and takes the data supplied and creates a .png image using a 3rd party API and then saves that image to my server using an AJAX POST call to another PHP page.
That all works fine, but what I'd like to do now is automate some calls to that PHP page. Namely, say I have 100 such variables to go through, I want to, preferably in Java with a for loop, call that PHP page with each variable in turn.
The problem is that client-side javascript. It won't execute with the simple URLConnection in Java. It seems like I need some sort of browser replicator or some way to have java act like it's calling the PHP in a browser?
Alternatively, I could make do with having a third PHP page act in place of the Java as the controller, but I'm faced with the same problem of getting the javascript to execute.
Am I missing something easy? Is this set up not possible? I'd really prefer to do it in Java if possible to fold it into other code I already have running.
Let me try to add some more specifics without bogging it down too much. There's a PHP file getData.php that takes in an ID number via GET. So I call it like ./getData.php?id=someId
That PHP file takes the ID, goes to my DB and retrieves some data and pastes it into the HTML source. Then once the page is finished, I have some javascript within getData.php that retrieves that data, formats it into a DataTable and passes it off to Google Visualization API in order to make a SVG chart.
Then I have more JS that runs that takes that SVG object, turns it into a Canvas object, grabs the base64 image data from it and finally POSTs to saveTo.php with the following array:
{'id' : id, 'data' : imgData}
saveTo.php simply takes in that POST data, creates a file on my server based on id and pastes the imgData into it. The end result is that I can pass in an ID to getData.php and end up with saved image of a Visualization chart that I want made based on data in my DB tied to that ID.
That all works by hand. But I have ~1,000 of these IDs and I'd like to have it so this whole process is run each morning so that I can have updated images based on yesterday's data.
I should mention that I did try using the 3rd party toolkit HtmlUnit (http://htmlunit.sourceforge.net/) but just keep getting these errors:
com.gargoylesoftware.htmlunit.IncorrectnessListenerImpl notify
WARNING: Obsolete content type encountered: 'text/javascript'.
Some more searching around and hitting upon the right keywords to try finally led me to Selenium, which is exactly what I was looking for.
http://docs.seleniumhq.org/projects/webdriver/
Related
Is it possible to send extra data attached to a http response via Java or Php?
My Website is a homework-platform: One User enters homeworks into a database, and all users can then see the homeworks on the website. The current load is very inefficient, as the browser makes two requests for eveything to load: One for the index file and one for the homeworks. For the homeworks request the client also sends settings of the user to the server, based on which the returned homeworks are generated by a Php script.
Now, I wonder, if it is possible, to combine those two requests into one? Is it maybe possible to detect the http request with Java or Php on the server, read the cookies (where the settings are saved), then get the homeworks from the database and send the data attached to the http response to the client? Or, even better, firstly only return the index file and as soon as possible and the homework data afterwards as a second response, because the client needs some time to parse the Html & build the DOM-tree when it can't show the homeworks anyway.
While browsing the web I stumbled across terms like "Server-side rendering" and "SPDY", but I don't know if those are the right starting points.
Any help is highly appreciated, as I'm personally very interested in a solution and it would greatly improve the load time of my website.
A simple solution to your problem is to initialize your data in the index file.
You would create a javascript object, and embed it right into the html, rendered by your server. You could place this object in the global namespace (such as under window.initData), so that it can be accessed by the code in your script.
<scipt>
window.initData = {
someVariable: 23,
}; // you could use json_encode if you use php, or Jackson if you use java
</script>
However, it is not a huge problem if your data is fetched in a separate server request. Especially when it takes more time to retrieve the data from the database/web services, you can provide better user experience by first fetching the static content very quickly and displaying a spinner while the (slower) data is being loaded.
I have a jsp page that calls a servlet when a drop down list with a bunch of different form names. If the user chooses to generate all forms the servlet then calls a java file that contains a loop. that runs each report generation one at a time. The whole process for this takes about 7 minutes as these are very large reports.
My question is, with keeping the same process (jsp -> servlet -> java code), can I get the progress status using ajax? What I want to do is find out what iteration the loop is on, find out if the function that writes the data out to excel (which is the format of all forms) has started, and write this information to the screen for the user to see so they are not guessing what is happening at any given time.
Additionally, we currently print out all this information to the console in eclipse so that for testing we can atleast know it is working. I am not sure if that matters.
Edit: I can provide code if needed but really the jsp contains a drop down list, the servlet takes that list and gets the value of the option selected (aka report1, report2, report 20...) and sends that value to the java code as a string. The java fucntion that is called is ReportAutomator.start(String reportNum){for loop here}. The for loop calls various functions to generate a report object which is forwarded onto a new java file that does more processing and then again forwarded to a final java file that unpacks the object and writes all the information to excel.
I am not really able to provide the actual code but I can try and make a dummy example if necessary. Sorry, and thank you for the help.
Determining the progress of a particular task is a surprisingly complicated thing once you get into the details however there is a jquery with ajax to update progress bar.
For a web app i recommend AJAX route it is suitable to indicate to the user the something is happening. Just have a spinner of some sort made visible when the page is submitted, and then hidden again when it is rendered
In my GWT application I am retrieving the XML data from a REST server. I am using Piriti XML parser https://code.google.com/p/piriti/wiki/Xml for deserializing the object and display in a table. As long as we are returning upto 1000 records everything is fine but with the big result it just hang and gives user message to stop the java script running in the back. Could someone please help me to find the best way to handle big data in GWT OR more precisely the best approach to parse big XML file in GWT.
Thanks a lot for all your suggestions.
The problem is that parsing a big XML document slows down the browser. And you need enough memory to hold the whole DOM plus your mapped objects in memory. The only solution is to avoid such situation. You have to adapt your REST service to be able to send only small chunks of data to the browser. So if you already have a paged table you only retrieve the data for the first page at the beginning. If the user wants to change the page you do another REST call to retrieve the data for the next page.
If you cannot change the the REST service itself you can create another server side service (on a server controlled by you) as a proxy. At first access you call the original REST service, store the XML at your own server and allow the client to retrieve only parts of that XML.
Im trying to make a simple webpage which obtains football league table data
http://www.skysports.com/football/league/0,19540,11660,00.html
For example i want to read in the points column and divide it by the number of games played to get an average points per game column that i will print onto my webpage.
How can i do this online?
Im quite experienced at doing this with offline programmes such as C/Matlab but i dont know where to start with it online.
Thanks
I wouldn't suggest to do it client side (on browser). It will be easier to scrap on server side (using java for example) following the steps:
Grab the content of the webpage (skysports)
Use existing html markup with regex to locate the desired content part.
Strip/split html markup with regex to get records (tr) and fields (td).
Cast values and do your math.
Use results to generate your version of html or json or whatever.
Serve the generated content to your client.
In general scrapping is easy but not guaranteed for tomorrow as source html markup may change at any time (and without warning).
I can provide a basic sample in C# if you want. (Sorry I haven't "java" since 1997).
You use jQuery.get like this:
$.get('http://www.skysports.com/football/league/0,19540,11660,00.html', function(data) {
//do the parsing here
});
There are several programing languages capable of getting at this information, PHP would be the classic method using curl or file_get_contents and regex parsing to extract the bits you want. You could do it with Yahoo Pipes as well if your web host does not allow remote URL retrieval.
If none of the Java brigade come back with something better contact me and I'll do some rough code for you in PHP.
I want to open a new browser and load a page from the file system (which will be created on clicking that button). My app is a java servlet. Basically I am allowing users to change some HTML on their website. On clicking the button the user get's to see a preview page, that shows what the page looks like with the changes made.
What would be the easiest way to do this and what issues can you see, and possibly how we would we get around them.
I see an issue, for example, if the browser window tries to open the file immediately, it won't even exist. So there needs to perhaps be some delay before trying to read the file.
This can be done using java script
window.open('url','name',....)
you can pass parameters in this function.
google for "javascript window.open function"
Cheers
Maybe you can use jQuery/AJAX or a similar technique to post the data to the server first and then open it in a new window.
see the jQuery documentation for an example: jQuery Post Example
if there is not much data to pass for preview page, you may pass it as url params instead of using jQuery. If data is large it is better to use jQuery. :)
as http://www.xyz.com?name="name".....
I'm not familliar with java servlets, but can't you keep the uploaded html in a server-side session variable, display the contents in your preview page, then save the contents of the variable out to the file system once the user accepts the changes ?
You could use JavaScript. On the button you add:
onClick="doMyOpenWindowFuncion();"
and within the script part of the page you write a function like
function doMyOpenWindowFunction() {
// if you want the client to wait, heres the place to do so.
// I assume you can make your servlet wait so the next line opens a window to the servlet
openWindow('/MyFancyPreviewServlet', 'Preview Window');
}
and in your Servlet you just wait for the file to appear before delivering it.