I am trying to render and export FusionCharts completely on the server. I am aware of solutions such as FCimg and FusionCharts .NET Solution. I have also implemented a Java solution that uses the Process class to run wkhtmltoimage.
However, I am trying to find a pure Java solution of doing this. I have an html file that includes FusionCharts JS Libraries and code to generate the fusion chart. I found JxBrowser that properly renders the chart but it requires X-Server for it to work on Linux. I also have tried Cobra/Lobo Browser but it does not fully support JavaScript. Are there any other ways to render and export fusion charts on the server or atleast render an html file that includes JavaScript completely in Java (and that does not require xserver)?
Thanks in advance for all the help!
Update: Solution that does not require xserver: WebRenderer. The Swing Edition is the only edition that supports HTML5 as of July 9th, 2012. You can use the swing edition to capture the image without a GUI.
I found a way that uses Eclipse's SWT Browser. However this cannot be run in an headless mode. You will have to use xserver to implement this. See this question.
Since this requires xserver and cannot be run in an headless mode, I would suggest using JxBrowser. It is a lot simpler and all you need is to generate an html file with all the fusion charts scripts. See #1, #2, #3
You have to create a template.html file that contains the header
(<html><head>), jquery.min.js, FusionCharts.js,
FusionCharts.HC.js, FusionCharts.HC.Charts.js. Make sure each of
these scripts are in their own script tags (<script type="text/javascript"> [js code] </script>)
Now add another JavaScript function with its own script tags containing the steps to render the chart. For example:
function load() { FusionCharts.setCurrentRenderer('javascript'); var chart = new FusionCharts("swf", 'chart0', "width", "height", "0", "1"); chart.setXMLData("XML DATA HERE"); chart.render("divNAMEHere"); }
Now you need to call the load() function onload, create a div to render the chart in, and end the html file. For example:
`
test
`
Create a new class that imports the eclipse swt browser libraries. Instantiate Display, Shell, and Browser (use this as a guideline to help understand what is happening: http://www.roseindia.net/tutorials/swt/swt-browser.shtml).
Set the text of the browser (browser.setText("htmlcode")) to the html code from template.html. The best way to do this would be to read the file using BufferedReader.
Lastly, the image takes some time to render. Now there is probably a better way to do this but if you want to just get it working, I set up a count and it captures the image after a certain number. This is what you need to add to the end:
int i = 0;
while (!shell.isDisposed())
{
if (!display.readAndDispatch())
{
display.sleep();
i++;
// System.out.println(i);
if(i==100)
{
GC source = new GC (shell);
Image image = new Image(display, browser.getClientArea());
source.copyArea(image, 0, 0);
ImageLoader io = new ImageLoader ();
io.data = new ImageData[] { image.getImageData() };
File f = new File (currentDir+"/workpng.png");
io.save (f.getAbsolutePath(), SWT.IMAGE_PNG);
}
}
}
Related
I am working on an app in Android Studio and am having some trouble web-scraping with JSoup. I have successfully connected to the webpage and returned some basic elements to test the library, but now I cannot actually get the elements I need for my app.
I am trying to get a number of elements with the "data-at" attribute. The weird thing is, a few elements with the "data-at" attribute are returned, but not the ones I am looking for. For whatever reason my code is not extracting all of the elements that share the "data-at" attribute on the web page.
This is the URL of the webpage I am scraping:
https://express.liatoyotaofcolonie.com/inventory?f=dealer.name%3ALia%20Toyota%20of%20Colonie&f=submodel%3ACamry&f=trim%3ALE&f=year%3A2020
The method containing the web-scraping code:
#Override
protected String doInBackground(Void... params) {
String title = "";
Document doc;
Log.d(TAG, queryString.toString());
try {
doc = Jsoup.connect(queryString.toString()).get();
Elements content = doc.select("[data-at]");
for (Element e: content) {
Log.d(TAG, e.text());
}
} catch (IOException e) {
Log.e(TAG, e.toString());
}
return title;
}
The results in Logcat
The element I want to retrieve
One of the elements that is actually being retrieved
This is because some of the content - including the one you are looking for - is created asyncronously and is not present in initial DOM (Javascript ;))
When you view the source of the page you will notice that there is only 17 data-at occurences, while running document.querySelector("[data-at]") 29 nodes are returned.
What you are able to get in the JSoup is static content of the page (initial DOM). You wont be able to fetch dynamically created content as you do not run required JS scripts.
In order to overcome this, you will have to either fetch and parse required resources manually (eg trace what AJAX calls are made by the browser) or use headless browser setup. Selenium + headless Chrome should be enough.
Letter option will allow you to scrape ANY posible web application, including SPA apps, which is not possible using plaing Jsoup.
I don't quite know what to do about this, but I'm going to try one more time... The "Problematic Lines" in your code are these:
doc = Jsoup.connect(queryString.toString()).get();
Elements content = doc.select("[data-at]");
It is the queryString that you have requested - the URL points to a page that contains quite a bit of script code. When you load up a browser and click the button (or menu-option) that reads: "View Source", the HTML you see is not the same exact HTML that is broadcast to and received by JSoup.
If the HTML that is broadcast contains any <SCRIPT TYPE="text/javascript"> ... </SCRIPT> in it (and the named URL in your question does), AND those <SCRIPT> tags are involved in the initial loading of the page, then JSoup will not know anything about it... It only parses what it receives, it cannot process any dynamic content.
There are four ways that I know of to get the "Post Script Loaded" version of the HTML from a dynamic web-page, and I will type them here, now. The first is likely the most popular method (in Java) that I have heard about on Stack Overflow:
Selenium This Answer will show how the tool can run Java-Script. These are some Selenium Docs. And then there is this page right here has a great "first class" for using the tool to retrieve post-script processed HTML. Again, there is no way JSoup can retrieve HTML that is sent to the browser by script (JS/AJAX/Angular/React) since it just a parser.
Puppeteer This requires running a language called Node.js Perhaps calling a simple Node.js program from Java could work, but it would be a "Two Language" solution. I've never used it. Here is an answer that shows getting, sort of, what you are trying to get... The HTML after the script.
WebView Android Java Programmers have a popular class called "WebView" (documented here), that I have recently been told about (yesterday ... but it has been out for years) that will execute script in a browser, and return the HTML. Here is an answer that shows "JavaScript Injection" to retrieve DOM Tree elements from a "WebView" instance (which is how I was told it was done)
Splash My favorite tool, which I don't think anyone has heard of, but has been the simplest for me... So there is an A.P.I. called the "Splash API". Here is their explanation for a "Java-Script Rendering Service." Since this one I have been using... I'll post a code snippet that shows how "Splash Tool" can retrieve post-script processed HTML below.
To run the Splash API (only if you have access to the docker loading program) ... You start a Splash Server as below. These two lines are typed into a GCP (Google Cloud Platform) Shell instance, and the server starts right up without any configurations:
Pull the image:
$ sudo docker pull scrapinghub/splash
Start the container:
$ sudo docker run -it -p 8050:8050 --rm scrapinghub/splash
In your code, just prepend the String to your URL's:
"http://localhost:8050/render.html?url="
So in your code, you would use the following command (instead), and the script would (more likely) load all the HTML Elements that you are not finding:
String SPLASH_URL = "http://localhost:8050/render.html?url=";
doc = Jsoup.connect(SPLASH_URL + queryString.toString()).get();
I am building a project which downloads several PDF files from different URLs, merges them into a single one and downloads it.
I'm trying to use Vaadin's FileDownloader to achieve this:
final FileDownloader fileDownloader = new FileDownloader(new FileResource(resultResource.getFile()));
fileDownloader.extend(download);
The resultResource is the generated PDF which I want to download.
Now the problem is that it takes a short time to generate the PDF, so that sometimes the download happens before the new file is generated, resulting in it downloading the old file, or an empty one.
So what I've been trying to do is something like this:
download.addClickListener(e -> {
try {
// This creates the new PDF
pdfConverter.manipulatePdf(storeNumber.getValue());
fileDownloader.download();
} catch (...) {
...
}
});
But so far without any success. Is there any way to something like this? To disable the "automatic" download and trigger it manually?
There are two approaches you can try
First approach is to refactor your UI so, that PDF file is started to be generated when you enter the view, and once complete you enable the download button. You can have other indicators like progress bar if that is feasible.
In Vaadin 8.4+ you can also setup FileDownloader by extending EventTrigger (see pull request https://github.com/vaadin/framework/pull/10478 ) and API spec https://vaadin.com/download/release/8.4/8.4.2/docs/api/com/vaadin/server/EventTrigger.html That could be something to be exploited if the first way is not applicable for you.
I tried using Swing code in a JSP page. To my surprise it does work well and fine.
But I cannot judge if it is OK to use Swing with JSP?
Basically I want to display some pop up reports from Database. I was thinking to display a JFrame pop up/ applet to do the trick.
But do a web browser require any additional plugin for this?
Or is it fine to do such a thingy? Any guidance will be helpful.
Always remember that every java fragment you insert into your JSP is executed server-side, so it can be deceitful (it may seem to work in your development local machine, but it is only because the server and the client side are running on the same box).
The proper way to do this would be to write an Applet and include it into your page - this way, the browser will download it to client side and run it there. You should subclass JApplet (http://docs.oracle.com/javase/8/docs/api/javax/swing/JApplet.html) and then you will be able to use Swing components at will
The library works but your controls will never be shown at the client side (browser) but at the server (if it is that you have a working window service: Ms Windows, X11, Xorg,...).
I don't think that is a good practice and I would only use Swing library classes not to show GUI components but to use some classes to store special objects such as ImageIcon to store icons. But never to try to paint them.
I have a project where I use JLaTeXMath to generate a PNG within a JSP representing some math equations, in this context, I use javax.swing.JLabel to generate the image:
TeXFormula formula = new TeXFormula(texCode);
TeXIcon texImg = formula.createTeXIcon(TeXConstants.STYLE_DISPLAY, 25);
BufferedImage img = new BufferedImage(texImg.getIconWidth(), texImg.getIconHeight(),
BufferedImage.TYPE_4BYTE_ABGR);
texImg.paintIcon(new JLabel(),img.getGraphics(), 0, 0);
try {
OutputStream os = res.getOutputStream();
res.setContentType("image/png");
ImageIO.write(img, "png", os);
os.close();
res.flushBuffer();
} catch (Exception ex) {
log.warn("LaTeX renderer: " + ex.toString() + "\t" + "Msg: " + ex.getMessage());
return;
}
the JSP would run on the server,and probably display the GUI there,
but why would you want that? In the meantime, the person at the client
who submitted the request would be sitting there waiting for somebody
at the server end to close the Swing window so the JSP could get on
with its work.
So i would say that it is not feasible.
I have an application developed using struts2. One of my web page has a div in which it displays a world map created using Google map API. On click of a button I want to save this map as an image on the server location. I tried this using the ROBOT class but this is not working. My application supports IE8. Below is the code I wrote:
Dimension screenDim = Toolkit.getDefaultToolkit().getScreenSize();
Rectangle rect = new Rectangle(screenDim);
Robot rob = new Robot();
BufferedImage img = rob.createScreenCapture(rect);
String FileName="D:\\SP_Maps\\Map.png";
ImageIO.write(img, "png", new File(FileName));
Basically I tried to take a screen shot of the page on click of a button and save it as an image. This works fine on my local host but, when I deploy this on my server and try to get the screen shot I just get a black page saved as png image.
While you are developing a web application you may use javascript if you are interested..
To get a screen shot and save it with any format you may use PhantomJS
PhantomJS is a headless WebKit scriptable with a JavaScript API. It has fast
and native support for various web standards: DOM handling, CSS selector,
JSON, Canvas, and SVG.
Check those examples written with PhantomJS:
https://github.com/ariya/phantomjs/wiki/Examples
Also check this tutorial Taking website screenshots using PhantomJS
The tutorial is about taking a web shot and saving it as JPEG, PNG, PDF ... etc
Hope this helps you...
Can you explain your use case clearly ? If I understand correctly you have a web application which has a functionality to take a screen capture upon user action? Ideally, Java Robot utility should use to do automated testing of java applications. As per the doc
The primary purpose of Robot is to facilitate automated testing of
Java platform implementations.
Maybe you can try using phantomjs.
example
var page = require('webpage').create();
page.open('http://google.com', function () {
page.render('google.png');
phantom.exit();
});
https://github.com/ariya/phantomjs/wiki/Quick-Start
To perform user actions like button clicks, you can use casper.js
http://casperjs.org/quickstart.html
Can I play video using Vaadin framewotk ?
The main idea is loading video files from local drive in flv or avi formats and play it in web using vaadin framework.
Thanks.
There is a sample in the Sampler: http://demo.vaadin.com/sampler/#FlashEmbed
You can see the source by clicking 'view source', and it will show you something like this:
Embedded e = new Embedded(null, new ExternalResource(
"http://www.youtube.com/v/meXvxkn1Y_8&hl=en_US&fs=1&"));
e.setMimeType("application/x-shockwave-flash");
e.setParameter("allowFullScreen", "true");
e.setWidth("320px");
e.setHeight("265px");
addComponent(e);
Obviously, you'll want to change the ExternalResource to something else (e.g FileResource, ClassResource, StreamResource, ...) in order to play local files.
Vaadin version 6.7 brought a new class Video that uses the new HTML5 "video" element to embed video on a page.
My posting on the Vaadin Forum provides the source code for an example app.
The main part of that code, when populating a window or layout:
Video v = new Video( "video" ); // Instantiate video player widget.
// Specify a list of your video in one or more formats.
// Different browsers support various different video formats.
v.setSources(
new ExternalResource( "http://www.example.com/media/example_video.mp4" ),
new ExternalResource( "http://www.example.com/media/example_video.ogv" )
);
v.setWidth( "640px" ); // Set size of the video player's display area on-screen.
v.setHeight( "360px" );
this.addComponent( v ); // Add the component to the window or layout.
Oops, I just re-read you posting -- you want to play local video files. Do you mean local to the user's computer or the Vaadin app server-side computer? Either way, you may be able to manipulate the "ExternalResource" seen above, or another subclass of Vaadin Resource to access a local file.
You can use the Embedded Class to embed videos.
NOTE: This is for local files:
FileResource fileResource = new FileResource(new File("/Users/user/Downloads/DBTI_1991_teaser_HD.mp4"));
Video vd = new Video();
vd.setAutoplay(true);
vd.setSource(fileResource);
this.addComponent(vd);
Another alternative is the Vaadin add-on "YouTubePlayer" if you want to access a video specifically from YouTube.com.