We Keep Coding

Automatically navigating a website with java

A few years ago I made a program in .NET that uses the webbrowser control. With that I was able to automatically log in to a website, navigate, and download pictures. It was GUI based since it was using the webbrowser control. It had the advantage that I could follow along and see if something went wrong.
What is the best way forward to replicate that idea in Java? Is there a similar free control that acts as a webbrowser and gives access to the DOM?
I suspect the optimal way would be to use the Google Chrome Developer tools to replicate the login via GET/POST methods, but at first would prefer the webbrowser approach.

You can use Selenium for that. It is a free (open source) automated testing suite for web applications across different browsers and platforms. It mainly focuses on automating web-based applications.

In Java, You can use Selenium which will give you full control on Web-Browsers as well as DOM.
In Selenium Web Driver is a class which provides full automated control of a browser that we want to use.
This may help You!
Thanks!

You could use the JavaFX webView, class javafx.scene.web.WebView.
It uses a Webkit engine that is HTML 5 compliant and seem to be up to date (it was in java 8 & 9).
The engine has interraction with the JS engine that may help to introspect and navigate.
Example to get the "window" JS object:
JSObject window = (JSObject) webView.getEngine().executeScript("window");
Webview ewample:
JavaFx Webview HTML5 DragAndDrop

Java webdriver for angular application not protractor?

I will be responsible for test automation for angular application. I know that we have a protractor tool but i prefer webdriver with java(feel better with this than javascript and protractor). May i use java with webdriver or i must do it using protractor because selenium will not handle it?

Of course, you can still use the regular Java selenium bindings to test AngularJS applications. It's just that Protractor is simply more suitable/convenient to use for specifically AngularJS applications because of the several unique things it provides:
it works in sync with Angular - it always knows when Angular is "ready" to be interacted with
it provides Angular specific locators like by.model, by.binding, by.repeater etc
it allows you to easily mock AngularJS modules on the fly
it is developed and supported by Google developers (and of course the github community) - meaning it is in sort of a sync with Angular development cycle
it has a very nice and documented API
and many more
It's also important to understand that Protractor is actually a wrapper around the WebDriverJS - JavaScript selenium bindings. And, as a side note, Protractor can also be used to test non-angular apps (just turn the sync off).
There is also ngWebDriver package that might actually be your solution:
We have taken JavaScript from Angular's Protractor project. While
ngWebDriver perfectly compliments the Java version of WebDriver, it
has to pass JavaScript up to the browser to inteoprate with Angular,
and the Protractor project has done the hard work (including testing)
to make that solid, and ngWebDriver benefits from that work.
Also see:
Use protractor with Java
how to implement protractor JavaScript API in Java to use in existing Selenium Java Frameworks

Chrome Browser and Java Message Passing

I have developed a Chrome extension and it captures some data in a webpage.
My ultimate goal is to pass this final result to my Java Application.
I have following few options in my mind, but I was not able to find any resources for them yet.
Access the localStorage externally.
Run Chrome browser through the Java app, So I guess we have the control of its data.
If no API found, write the result to a file and access it from the
Java App.
Is there any API to achieve any of the first 2 options? Or any other interface other than the file system?
I checked with berkelium and The Chromium Embedded Framework. But they are just chrome wrappers, and we cannot run a chrome instance from it.
Edit
For the 2nd option I tried with Selenium Webdriver, but I think it hasn't any method to access the localStorage.

It sounds like you are looking for Native Messaging, which allows communication between a Chrome Extension and a native application (e.g. a Java Desktop Application).
There are plenty of question here on SO regarding the implementation of Native Messaging and there is, also, the "official" example.
I suggest the above solution, but if your application will heavily interact with the extension (and you feel like reverse engineering) there is the open-source **[NetBeans Connector Chrome Extension][3]**, which uses a different approach (Sockets or WebSockets - I am not sure).
Take a look at **[this answer][4]** for info on how to get at the sources.

Is there a limitation preventing you from exposing a REST API with your java application?

How to automate PHP-Flash web application using selenium java

I need to automate a web application which is in PHP with flash. Is it possible to automate the application using selenium java with FlexUISelenium or FlexMonkium.
Thanks in advance.

If it's pure Flash without Flex then you should put some external like these methods to your Flash application and call them from your test code in order to verify functionality. If it's Flex then FlexUISelenium and FlexMonkium will help you - but again you should compile these libraries into your Flex app - they contains required methods to be called from Selenium.
In the case you are not allowed to modify your Flash application I would recommend to take a look to image-based tools like Sikuli

canoo or jwebUnit for automated headless web client?

I'm writing an application in groovy with grails that needs to do some automated usage of a couple of websites. So I need something that will perform a similar task to functional web application testing, without a browser being needed. I would also like it to be as lightweight as possible and of course completely headless.
There are two options that I am aware of so far. Canoo which has a nice grails plugin and conveniently also a firefox recorder plugin and EasyB/JWebUnit which can be recorded with Selenium.
Canoo seems very heavy and I don't know what I'd have to do to get it to run in a grails service neatly.
Perhaps I'm thinking about this wrongly though. Has anyone here done this kind of thing in Java or Groovy? Am I better off just dropping out to curl on the command line perhaps?

Have you though about just sending your web requests? Maybe you don't need a full blown headless browser and a line like
def html = new Url("http://example.com").text
is enough for your app?
A testing tool or framework is great when you have to click virtual buttons. But if you only have to remote control a website, sending a request could be enough.
Update: if you need to send POST requests and handle cookies, then the Url-Class isn't enough. But you can use the http builder instead. Here are two URLs which will get you started:
http://blog.swwomm.com/2011/01/groovy-httpbuilder-cookies.html
http://groovy.codehaus.org/modules/http-builder/doc/post.html