Background: I'm working on a GWT based application which is deployed on Google App Engine. My application uses a custom library, which requires 6-7 big core javascript files (~3 MB total size) to be loaded up front. So to even load my application's home page, it takes anywhere from 20-40 secs depending upon the bandwidth. Recently I set upon the task to address the slow page load issue.
I made a plain HTML home page, without any GWT or custom library's components. This home page contains a login button and a simple jQuery based image slider. I'm using Google Accounts based authentication. On the home page, I use "asynchronous ordered" Javascript loading technique presented here: http://stevesouders.com/efws/loadscript.php?t=1303979716 to load all the core .js files. So while the user is looking at the home page or interacting with the image slider, the .js files quietly download in the background. After the user logs in, the control is redirected to the Main HTML file, which retrieves the .js files from the cache, and loads the app. This time the loading of the app is significantly faster. So, the home page loads up in 2-5 secs and the main app also loads up pretty fast.
It's exactly what I need. But there's an issue.
ISSUE: If the user clicks on the Login button too fast (i.e. before all the asynchronous .js file downloads are complete), the asynchronous downloads break. In fact, as soon as the user navigates away from the home page (to the Google Accounts sign-in page or any other page), all the asynchronous downloads break. Now when the main app is displayed, it retrieves the partially downloaded .js files from the cache and reports "Unexpected end of line" errors.
I could do one thing to address the problem - disable the login button until the asynchronous downloads complete. This is slightly better than the current situation because it allows the user to at least see the home page and interact with the image slider. But it's not ideal.
Ideally I'm looking for a way to check the hash (or file size) of the .js files in the Main HTML, to see whether they are correct. If there's a mismatch, download the files again. But I'm not a Javascript expert by any means and not even sure whether that's possible. So, I'm looking for help from experts here. This might be a common problem that people have addressed in other ways. So, I'm also open to other alternatives.
Thanks.
UPDATES:
I have multiple (6-7) .js files to be loaded.
The .js files are independent and need to be loaded in a particular order.
3MB of Javascript files is pretty crazy. Do you really need that much? If so you should look to concatinating / minifying them using jsmin or uglifyjs. This will reduce the size by at least 50%. Next you ensure gzipping is enabled on the server hosting these files. That should reduce the size by another 50%.
Lastly you should use an expires header to ensure this file caches clientside forever. To uncache update a query param on the src:
<script src="compressed.js?v=1"></script>
As for checkinging hash of filesizes, not exactly possible, though you could check for the existence of certain variables your libs are introducing into the global namespace. For example:
jQueryLoaded = !!window.jQuery;
In this case, considering we have around 3 MB of javascript to be loaded, I would construct a loading page, with a progress bar, that downloads all the javascript, and shows the progress before showing the actual site. This does not limit user experience to any significant degree, it rather gives you an opportunity to show updates, news (even feeds), help messages while loading. Also have a timer on the loading page, to detect if it takes too long and give them an option to do something else if it does take too long, for example showing the login page with login disabled till scripts are downloaded. I have found scripts that do this here and here
You can actually compress, include versions, minify and bundle the js files this reduces footprint by more than 80%. Have a look at Yahoo's Coding horror. You can do all these using jawr
Related
I am working on a java app, works fine and all, but at the frontend, I am using thymeleaf (first time I use it, moving on from primefaces).
What Ive noticed is that the page loads (working locally) pretty fast, as expected, since the information is not reelevant and no DB is being used by now.
What surprises me is that the images load probably 2 seconds later, I have no idea why, they are stored locally on the app assets folder.
I am incluiding them this way:
<img th:src="#{/assets/images/myimage.png}" />
Is there anyway to make it faster? (Of course, I will later set more memory to my JVM, but this sort of stuff shouldnt take that long...)
Any caching or faster ways?
I am using spring 4 mvc
Thanks.
There are so many variable to this i am not going to list all of them but a few:
Distance between Client and Server
Efficiency of your application
Size of images
Browser and all extensions/plugins attached to it
Power of server
Power of client machine
Delivery method
Potential javascripts that manage loading of images
Malware
As mentioned there is a lot more that I haven't listed and it really is a process of elimination.
Some things we have on our application that avoids the delay in images include
server memory increase from 512 to 1024
Changing location of server to a more local source
changing the location from where the application is sourcing the images (faster raid disks)
delaying full page display until everything is preloaded on client machine (look into Flash of Unstyled Content Fixes)
Performance improvements on web application itself
What you need to do is explore each option and start improving on it and eventually you will get the results you need.
Fyi if you want to avoid loading images from server have them hosted on a CDN for speed of transfer.
We're developing a web application which must tolerate noticeable amount of load. I'm running my tests on a HP (Proliant DL 380) server (two 3.6GHz Xeon cpus, 16 GBs of RAM, ...). I'm using Apache JMeter and pylot to run load tests (they kinda show similar results).
In one scenario, I configured the load testing program to hit my index page ,using just one thread, as much as it can. The index page is about 60KB and consists of about 10 ajax calls, lots of JavaScript and jQuery Codes, required CSS and etc.
The results I got was, well, disappointing.
Full index.jsp page:
Throughput (req/sec): 3.567
Response Time (secs): 0.278
So I removed every ajax call, got rid of charts and also CSS (but not JS)
Throughput (req/sec): 6.082
Response Time (secs): 0.161
Still very low! So I built an static index page in HTML format which contains all the data with the same size (without any server side and client side computation)
Throughput (req/sec): 20.787
Response Time (secs): 0.046
Wow, that was a breakthrough! Now I added some of the JavaScript codes to the index.html page
Throughput (req/sec): 9.617
Response Time (secs): 0.103
Well, I think the bottleneck has been found, Java Script codes. I need to find out how many req/sec "the server" can handle and since java Script is run client side, I don't think I should include it in this test. So should load testing tools process JS codes? (they seem to be doing this)
Another key question is, according to hardware, content size and aforementioned configs, is this amount of throughput plausible? shouldn't I expect more? my expectation is 500 req/sec! is the only solution adding hardware?!
BTW, the webapp has been built using Java+Struts2+JSP+Hibernate+MySQL. it is also distributed over multiple servers using haproxy. but the aforementioned tests was run on a single server.
If what you are looking for is a raw number of how fast the server can serve up the content, then I would say yes, ignore the java-script and focus on how fast it takes to transfer all the various bits of the page (HTML, images, script files, CSS, etc).
However, if you are trying to test user experience, well JS is part of that, so you need to take that into account. From your description, you are not worried about user experience but rather load to the server.
You might want to consider setting up JMeter to make direct calls to the pieces of your pages, as described in the first sentence above.
Is your JavaScript (CSS,images) directly embedded on the page or loaded from a script tag? The latter case will force the browser to download the file from the server ,instantly halving your paged per second.that's one of the reasons you should load jquery from a different server ( eg Google) - this will have a minor hit on the user perceived page load time (one additional DNS query to be made) but really takes the load off your server
Another key question is, according to hardware, content size and aforementioned configs, is this amount of throughput plausible? shouldn't I expect more? my expectation is 500 req/sec! is the only solution adding hardware?!
Writing your application so that pages are cacheable and then putting a http-proxy up in front of your application is often a good strategy.
I want to extract HTML data from a website using JAVA. The problem is the webpage keeps scrolling down once the user reaches the bottom of the page. Number of times it scrolls down is fixed. My JAVA code can extract only for the 1st part. How do I extract for the remaining scrolls? Is there a way to load the whole page at once with JAVA? ANy help would be appreciated :)
This might be the type of thing that PhantomJS (http://phantomjs.org/) was designed for. It will crawl entire web pages and even execute JavaScript, using a "real" browser in headless mode. I suggest stopping what you're doing with Java and take a look at PhantomJS instead. It could save you a LOT of time. :)
This type of behavior is implemented in the browser, interpreting the user's scrolling actions to load more content via AJAX and dynamically modifying the in-memory DOM in the browser. Consider that your Java runs in a web container on the server, and that web container (i.e. Tomcat, JBoss, etc) provides a huge amount of underlying code so your app doesn't have to worry about the plumbing.
Conceptually, a similar thing occurs at the client, with the DHTML web page running in its own "container" (the browser), which provides a wealth of functionality, from UI to networking, to DOM, etc. If you remove the browser from the equation and replace it with a Java program, you will need to provide the equivalent of the browser in which the DHTML/Javascript can execute.
I believe that HTMLUnit may fill the bill, but have not worked with it personally.
I am developing a web based java app, running on jboss and sql server.
I seem to find myself spending an inordinate amount of time recompiling/deploying just to tweak the interface in jquery/javascript/css/html.
Any tips for reducing the turnaround ?
Its deployed to an ear file, so I can not alter the jsps/javascript after deployment(?). Yes, I have created the a static version of the webpage frontends but they do not give me the full functionality - none of the data from db/jstl processing.
To clarify its not so much the actual compile time itself (30seconds) as the ant builds are set-uo well and are very modular; its the subsequent deployment to jboss and accessing the application that cause the real headache.
If you do not work directly in an exploded war inside the hotdeploy folder of JBoss, then strongly consider it.
when developing with application server i've used this product in the past: JRebel from zeroturnaround.
It will prevent having to restart and redeploy an application running within an application server. It works for most scenario's however i found that there were a few occasions when a server restart were required(in my case making changes to the application initialisation). But if you're only working on the interface this product will save you a great number of deployments and restarts.
I have not used Jrebel in combination with JBoss but they mention it as a supported container so thta shouldn't be a problem.
I am an average web designer (at best!) and writing complicated HTML and CSS is a pain for me. A lot of what I do with styles and layout is trial and error and involves a lot of tweaking. I also change my mind frequently about exactly what shade of color I want things. Basically, I'm in the same boat as you.
Long ago I abandoned the idea of the tweak-deploy-test iteration cycle (mvn clean tomcat:deploy takes 2 minutes on my current project) as by the 10th iteration trying to sort a simple layout problem and waiting for the deployment would drive me round the bend. I now use two strategies;
Get a static copy of the HTML I want to work with. This usually means deploying the app, navigating to the page and saving it to a work directory somewhere. This saves the static HTML as well as any images. Next I copy the CSS files from my workspace into the work directory and hand edit the saved HTML file to point to these CSS files.
Open the static HTML page in Firefox. Now I can tweak the CSS or HTML and simply refresh Firefox to show the changes. Iteration time is now down to about 1 second. I can further improve my tweaking using the Firebug addon. This allows you to manipulate the CSS and HTML from within Firefox. This is especially useful for getting margin and padding size right. Once I've tweaked it in Firebug I hand edit the saved HTML and CSS then refresh Firefox to make sure I'm happy with the result.
At certain key stages I then make the changes to my workspace to reflect my tweaking on the static files. I then redeploy and test to make sure I got it right. As I use Firefox for all my development I have to pay special attention to browser compatibility, especially with IE, but this usually comes at a later stage.
Edit:
I didn't mention Javascript, but this process works great for JS too!
We are using GWT Eclipse tool for developing a web application in JAVA and Lightsreamer (third party software) for login purpose and for data streaming.
It takes around 6 mins to load page and about 2 mins to login in production environment.
Previously the Browser folder(WAR) size was 70.5MB. So it was reduced by removing some folders like Web-INF, js, pieces and some files of SC(Deleted subfolders in SC are modules, modules-debug, system) and some files of SmartG and also even optimized the images in Image folder.
And now Image folder got reduced from 40KB to 17KB.
Now the total size of WAR is 15.6MB. It is also works. But now too it takes long time to load the page.
Kindly advice us how we need to optimize WAR folder and suggest the way of optimizing.
The first thing to do is analyse your client application HTTP requests to your webserver using HTTP monitoring tools like Firebug
http://getfirebug.com
Deleting Stuff in your WAR has not impact on the perfs. It's not like the browser loads all that stuff. If after deleting all the garbage, your application still works like before, then that means those was useless.
Now, about your actual issue, as other told you, you should watch where the time is spent. Firebug can be of a good help in that because you will see (Net tab) how long each request takes. Also, speedTracer is a great tool when using GWT because you see exactly where the time is spent. And it can even display you the time spent on the server side!
also some tips :
gwt should compile to obfuscated JS, not pretty or whatever
make sure your images are not too big and loaded only when necessary => image bundle anyone?
compress your css + external JS (yuicompressor)
Have a good caching policy (far futur for gwt stuff since the names are generated)
and lot of others....