Java Robot Class - Add focus to a specific running Application? - java

I am just trying to figure out if/how to get the Java Robot class to change focus from the running java app, to a specific process, such as ms word or firefox.
Thanks!

Robot can't do that automatically. You can activate another application via alt-tab as has been suggested above, but you'll need to know the z-order of the application that you want to activate. I think that to really do this best you'll want to get the window's handle (hWnd) of the top-level window that you want to activate (if this is a Windows app), and then using the Windows user32 library functions activate the desired window. For this I recommend using JNA as one of the easiest ways (when compared to JNI). You would have to first download the JNA jna.jar and platform.jar jar files, and place them on your classpath, and then you can call most OS methods with ease. For instance, I have just this sort of thing up and running for a Windows application where I can get the hWnd for a running top-level Windows application based on the window name (full or partial), and then using that hWnd, call user32's setForegroundWindow function. If you're wanting to activate a Windows application and want to pursue this further, comment to this answer, and I can show you what code I have for this. If so, you'll want to go into greater details on exactly what it is you're trying to do.
Best of luck!

For any that comes across this in Google as I just did:
public class activate {
public interface User32 extends W32APIOptions {
User32 instance = (User32) Native.loadLibrary("user32", User32.class,
DEFAULT_OPTIONS);
boolean ShowWindow(HWND hWnd, int nCmdShow);
boolean SetForegroundWindow(HWND hWnd);
HWND FindWindow(String winClass, String title);
int SW_SHOW = 1;
}
public static void main(String[] args) {
User32 user32 = User32.instance;
HWND hWnd = user32.FindWindow(null, "Downloads"); // Sets focus to my opened 'Downloads' folder
user32.ShowWindow(hWnd, User32.SW_SHOW);
user32.SetForegroundWindow(hWnd);
}
}
Credit: http://www.coderanch.com/t/562454/java/java/FindWindow-ShowWindow-SetForegroundWindow-effect-win

You did not specified system, on Mac, there is possible to do it with AppleScript. AppleScript is integrated to system, so it will be always functional.
https://developer.apple.com/library/content/documentation/AppleScript/Conceptual/AppleScriptLangGuide/reference/ASLR_cmds.html
You need only detect you are on mac and has name of the application.
Runtime runtime = Runtime.getRuntime();
String[] args = { "osascript", "-e", "tell app \"Chrome\" to activate" };
Process process = runtime.exec(args);

Related

Java Selenium Web Scraping Upload Image [duplicate]

I have seen lots of questions and solutions on File upload using Selenium WebDriver on Stack Overflow. But none of them are working for following scenario.
Someone has given a solution as following
// assuming driver is a healthy WebDriver instance
WebElement fileInput = driver.findElement(By.name("uploadfile"));
fileInput.sendKeys("C:/path/to/file.jpg");
But still I can't find window handle. How can I work on that?
I am looking for a solution for the scenario above.
Please check this on any of the following websites.
http://www.uploadify.com/demos/
http://www.zamzar.com/
// assuming driver is a healthy WebDriver instance
WebElement fileInput = driver.findElement(By.name("uploadfile"));
fileInput.sendKeys("C:/path/to/file.jpg");
Hey, that's mine from somewhere :).
In case of the Zamzar web, it should work perfectly. You don't click the element. You just type the path into it. To be concrete, this should be absolutely ok:
driver.findElement(By.id("inputFile")).sendKeys("C:/path/to/file.jpg");
In the case of the Uploadify web, you're in a pickle, since the upload thing is no input, but a Flash object. There's no API for WebDriver that would allow you to work with browser dialogs (or Flash objects).
So after you click the Flash element, there'll be a window popping up that you'll have no control over. In the browsers and operating systems I know, you can pretty much assume that after the window has been opened, the cursor is in the File name input. Please, make sure this assumption is true in your case, too.
If not, you could try to jump to it by pressing Alt + N, at least on Windows...
If yes, you can "blindly" type the path into it using the Robot class. In your case, that would be something in the way of:
driver.findElement(By.id("SWFUpload_0")).click();
Robot r = new Robot();
r.keyPress(KeyEvent.VK_C); // C
r.keyRelease(KeyEvent.VK_C);
r.keyPress(KeyEvent.VK_COLON); // : (colon)
r.keyRelease(KeyEvent.VK_COLON);
r.keyPress(KeyEvent.VK_SLASH); // / (slash)
r.keyRelease(KeyEvent.VK_SLASH);
// etc. for the whole file path
r.keyPress(KeyEvent.VK_ENTER); // confirm by pressing Enter in the end
r.keyRelease(KeyEvent.VK_ENTER);
It sucks, but it should work. Note that you might need these: How can I make Robot type a `:`? and Convert String to KeyEvents (plus there is the new and shiny KeyEvent#getExtendedKeyCodeForChar() which does similar work, but is available only from JDK7).
For Flash, the only alternative I know (from this discussion) is to use the dark technique:
First, you modify the source code of you the flash application, exposing
internal methods using the ActionScript's ExternalInterface API.
Once exposed, these methods will be callable by JavaScript in the browser.
Second, now that JavaScript can call internal methods in your flash app,
you use WebDriver to make a JavaScript call in the web page, which will
then call into your flash app.
This technique is explained further in the docs of the flash-selenium project.
(http://code.google.com/p/flash-selenium/), but the idea behind the technique
applies just as well to WebDriver.
Below code works for me :
public void test() {
WebDriver driver = new FirefoxDriver();
driver.get("http://www.freepdfconvert.com/pdf-word");
driver.findElement(By.id("clientUpload")).click();
driver.switchTo()
.activeElement()
.sendKeys(
"/home/likewise-open/GLOBAL/123/Documents/filename.txt");
driver.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
driver.findElement(By.id("convertButton"));
Using C# and Selenium this code here works for me, NOTE you will want to use a parameter to swap out "localhost" in the FindWindow call for your particular server if it is not localhost and tracking which is the newest dialog open if there is more than one dialog hanging around, but this should get you started:
using System.Threading;
using System.Runtime.InteropServices;
using System.Windows.Forms;
using OpenQA.Selenium;
[DllImport("user32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
private static extern bool SetForegroundWindow(IntPtr hWnd);
[DllImport("user32.dll", EntryPoint = "FindWindow")]
public static extern IntPtr FindWindow(string lpClassName, string lpWindowName);
public static void UploadFile(this IWebDriver webDriver, string fileName)
{
webDriver.FindElement(By.Id("SWFUpload_0")).Click();
var dialogHWnd = FindWindow(null, "Select file(s) to upload by localhost");
var setFocus = SetForegroundWindow(dialogHWnd);
if (setFocus)
{
Thread.Sleep(500);
SendKeys.SendWait(fileName);
SendKeys.SendWait("{ENTER}");
}
}
I made use of sendkeys in shell scripting using a vbsscript file. Below is the code in vbs file,
Set WshShell = WScript.CreateObject("WScript.Shell")
WshShell.SendKeys "C:\Demo.txt"
WshShell.SendKeys "{ENTER}"
Below is the selenium code line to run this vbs file,
driver.findElement(By.id("uploadname1")).click();
Thread.sleep(1000);
Runtime.getRuntime().exec( "wscript C:/script.vbs" );
Find the element (must be an input element with type="file" attribute) and send the path to the file.
WebElement fileInput = driver.findElement(By.id("uploadFile"));
fileInput.sendKeys("/path/to/file.jpg");
NOTE: If you're using a RemoteWebDriver, you will also have to set a file detector. The default is UselessFileDetector
WebElement fileInput = driver.findElement(By.id("uploadFile"));
driver.setFileDetector(new LocalFileDetector());
fileInput.sendKeys("/path/to/file.jpg");
There is a simpler way to solve this then what Slanec described. Hes solution works when you are using an English keyboard, if not you will have a hard time trying to "map" the key for special characters.
Instead of robot.keyPress and robot.keyRelease every single key you can use Toolkit to copy the String to the clipboard and then paste it.
StringSelection s = new StringSelection("Path to the file");
Toolkit.getDefaultToolkit().getSystemClipboard().setContents(s, null);
Robot robot = new Robot();
robot.keyPress(java.awt.event.KeyEvent.VK_ENTER);
robot.keyRelease(java.awt.event.KeyEvent.VK_ENTER);
robot.keyPress(java.awt.event.KeyEvent.VK_CONTROL);
robot.keyPress(java.awt.event.KeyEvent.VK_V);
robot.keyRelease(java.awt.event.KeyEvent.VK_CONTROL);
Thread.sleep(3000);
robot.keyPress(java.awt.event.KeyEvent.VK_ENTER);
First add the file to your project resource directory
then
public YourPage uploadFileBtnSendKeys() {
final ClassLoader classLoader = getClass().getClassLoader();
final File file = new File(classLoader.getResource("yourFile.whatever").getPath());
uploadFileBtn.sendKeys(file.getPath());
return this;
}
Walla, you will see your choosen selected file, and have skipped the file explorer window
Import System.Windows.Forms binary to the test solution and call the following two LOC on clicking the Upload button on the UI.
// Send the file path and enter file path and wait.
System.Windows.Forms.SendKeys.SendWait("complete path of the file");
System.Windows.Forms.SendKeys.SendWait("{ENTER}");
An alternative solution would be to write a script to automate the Open File dialog. See AutoIt.
Also, if you can't "click" the element, my workaround is generally to do this:
element.SendKeys(Keys.Enter);
Hope this helps (Even though it's an old post)
Below code works for me:
// wait for the window to appear
WebDriverWait wait = new WebDriverWait(driver, 10);
wait.until(ExpectedConditions.alertIsPresent());
// switch to the file upload window
Alert alert = driver.switchTo().alert();
// enter the filename
alert.sendKeys(fileName);
// hit enter
Robot r = new Robot();
r.keyPress(KeyEvent.VK_ENTER);
r.keyRelease(KeyEvent.VK_ENTER);
// switch back
driver.switchTo().activeElement();
You have put double slash \\ for the entire absolute path to achieve this
Example:- D:\\images\\Lighthouse.jpg
Steps
- use sendkeys for the button having browse option(The button which will open
your window box to select files)
- Now click on the button which is going to upload you file
driver.findElement(By.xpath("//input[#id='files']")).sendKeys("D:\\images\\Lighthouse.jpg");
Thread.sleep(5000);
driver.findElement(By.xpath("//button[#id='Upload']")).click();
Use AutoIt Script To Handle File Upload In Selenium Webdriver. It's working fine for the above scenario.
Runtime.getRuntime().exec("E:\\AutoIT\\FileUpload.exe");
Please use below link for further assistance:
http://www.guru99.com/use-autoit-selenium.html
webDriver.FindElement(By.CssSelector("--cssSelector--")).Click();
webDriver.SwitchTo().ActiveElement().SendKeys(fileName);
worked well for me. Taking another approach provided in answer above by Matt in C# .net could also work with Class name #32770 for upload box.
The below one had worked for me
webDriver.findElement(By.xpath("//input[#type='file' and #name='importFile']")).sendKeys("C:/path/to/file.jpg");
Double the backslashes in the path, like this:
driver.findElement(browsebutton).sendKeys("C:\\Users\\Desktop\\Training\\Training.jpg");

How to get an instance of Internet Explorer from java?

I have a java 6 application, and i need to get a determined instance of internet explorer from java?. This instance of IE it is running in the same pc that JVM. When i get the instance, i need to change the url.
Anybody has any idea about this?
I searched in google, and i found some related with this. I think, I need to install something from microsoft for to use this package. It is valid for windows XP and Java 6?
I found the page https://support.microsoft.com/en-us/kb/299356.
I found JNA, and i have a pice of code, where i can get a handler of a User32 instance. But i donĀ“t know how i can navigate to an URL.
public static void main(String[] args) {
HWND hwnd = User32.INSTANCE.FindWindow
(null, "PageTitle"); // window title
if (hwnd == null) {
System.out.println("Excel is not running");
}
else{
User32.INSTANCE.ShowWindow(hwnd, 9 ); // SW_RESTORE
User32.INSTANCE.SetForegroundWindow(hwnd); // bring to front
}
}

UISpec4J and external application

I am trying to launch an external application for testing using UISpec4J.
Here are the questions and their answers I referred so far:
How to automate a swing java web start application which runs clicking a link into a web application, which is automated with Selenium WebDriver?
Getting all windows using UISpec4J
UISpec4J Capturing modal dialog, before the trigger finish
my.exe referred below is a Java application wrapped in exe using some tool. Internally it uses the jars and is Java GUI application.
This executable launches a splash screen first, then a dialog to choose where you want to connect to and after that main window is shown. Unless I can automate where I can connect to I won't get main window.
Based on these questions I have come up with following code fragments:
this.setAdapter(new UISpecAdapter() {
#Override
public Window getMainWindow() {
return WindowInterceptor.run(new Trigger() {
#Override
public void run() throws Exception {
// running jnlp by netx launcher
Runtime.getRuntime().exec("C:\\my.exe");
Thread.sleep(10000);
}
});
}
});
In the approach above I simple get "No window was shown" error.
this.setAdapter(new UISpecAdapter() {
#Override
public Window getMainWindow() {
final Window[] result = new Window[1];
WindowInterceptor
.init(new Trigger() {
#Override
public void run() throws Exception {
Runtime.getRuntime().exec("C:\\my.exe");
//Thread.sleep(10000);
}
})
//.processTransientWindow()
.process(new WindowHandler() {
public Trigger process(Window window) throws Exception {
result[0] = window;
return Trigger.DO_NOTHING;
}
})
.run();
return result[0];
}
});
In the second approach above, I still get "No window shown" error AND control never reaches to overriden "process" method.
I referred to http://www.uispec4j.org/reports/apidocs/org/uispec4j/interception/WindowInterceptor.html and recommended approach is to use init to capture modal dialog is init\process sequence.
To capture non-modal it is recommended that we should use following:
Window window = WindowInterceptor.run(panel.getButton("open").triggerClick());
But I have NO idea where and how I am supposed to call it..
From the first question I referred, mentioned above, we should be able to do that because the answer to it mentions launching jnlp application which is external application.
I tried with jre 6 update 0 and I can at least run test. In java update 37, from the third question I referred above, I get abstract method not implemented error.
What am I doing wrong? Any idea?
I am using latest UISpec4J package - version 2.4.
Thanks in advance,
-Neel.
I'm very new to UISpec4J but I'm guessing it needs to run in the same JVM in order to intercept and interact with the GUI components. When you start the exe file with exec, it will create a new process and a new, separate JVM. That'll not work, if I understand UISpec4J correctly.
Regarding the non-modal example, the documentation says "You would retrieve the window from within the test...", so in a setup method or in a test should work.

Java search for on-screen text field

I am trying to create a program that automatically searches for a text field on the screen and types a word repetitively into that text field. Is there any class that can find a text field? Or is there any way in which a text field can be found? Because I know that the Robot class can type text, I just need to either get the cursor onto the text field and use the mousePress() and mouseRelease() methods.
Thanks
I can't directly give you a solution, but I messed around with some code and may be able to point you in the right direction.
Java, as you probably know, runs in the JVM. This allows it to execute in any operating environment. Each operating environment (windows, mac, etc) has its own system for handling edit boxes and setting focus to the right window and whatnot. The following example code is designed for use on windows only, which does not follow the spirit of the Java language. As Adriaan pointed out, there are other languages for this sort of thing, but it IS possible (to an extent) to accomplish with Java alone.
In windows, you must understand how all of the active windows are managed and that everything you see (including edit boxes) are considering a "window" by the Windows OS. I don't truly understand how things work under the hood, so I can't provide much more information than that. In a native language such as C++, there are a few functions provided by the Windows OS API that would be used to accomplish your goal. Namely, EnumWindows(), EnumChildWindows(), GetClassName(), and SetForegroundWindow(). You can find documentation on how to use these functions within a native language by searching the MSDN documentation library.
So with that said, you NEED to be able to call these functions from Java. Under normal circumstances, calling these native methods is not possible. However, there is a library available to help you out: the JNA library. JNA stands for Java Native Access and lets you work with the shiny new functions I mentioned earlier.
So, to accomplish your goal in a native language, normally one would begin with a call toEnumWindows() to return a list of all Parent windows that the OS is aware of. This list will contain window handles of parent windows - windows titled "MSN", "Eclipse", "Microsoft Office", etc. Each of these windows, as a Parent, has children. It is in this list of children that you will find the "control" that you are looking for: an Edit control. Now, many applications use different libraries and non-standard things for text boxes - i.e Pidgin, a messaging application I tested some relevant code with, has every control named "gdkWindowChild" which doesn't exactly tell us which control is actually an EditBox or otherwise a place that allows us to enter text. That's the main problem with your idea; you can't always tell exactly what control you wish to have focus of so that you may enter text. Regardless of that, we'll continue:
After finding the relevant Parent window with EnumWindows(), a call to EnumChildWindows() will give us all of the sub-windows and other "controls" (including potential edit-boxes) that belong to the Parent. EnumChildWindows() calls a callback function for each sub-window it finds, so it's pretty easy to "search" through the list of child windows - using GetClassName() to find the name of a control - to potentially find the HWND (window handle) of the control you want.
Once you have found the correct HWND of the edit box (that, of course, being the difficult part given the general scope of your question) a simple call to SetForegroundWindow(targetHWND) ought to bring the control to the front and set your cursor in a ready-to-type edit box.
Here is some working example code I've written to get you started. This code will iterate through all of the active windows using EnumWindows() and then call EnumChildWindows() on each parent, printing out all of the controls that it finds. Note that this code requires the JNA library to run.
import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.examples.win32.W32API.HWND;
import com.sun.jna.examples.win32.W32API.LPARAM;
import com.sun.jna.win32.StdCallLibrary;
import com.sun.jna.win32.W32APIOptions;
public class IterateChildWindows {
public interface User32 extends StdCallLibrary {
User32 INSTANCE = (User32) Native.loadLibrary("user32", User32.class, W32APIOptions.DEFAULT_OPTIONS);
HWND FindWindow(String lpClassName, String lpWindowName);
int GetWindowRect(HWND handle, int[] rect);
int SendMessage(HWND hWnd, int msg, int wParam, byte[] lParam);
HWND FindWindowEx(HWND parent, HWND child, String className, String window);
boolean EnumWindows(WNDENUMPROC lpEnumFunc, Pointer arg);
boolean EnumChildWindows(HWND parent, WNDENUMPROC callback, LPARAM info);
interface WNDENUMPROC extends StdCallCallback {
boolean callback(HWND hWnd, Pointer arg);
}
int GetWindowTextA(HWND hWnd, byte[] lpString, int nMaxCount);
long GetWindowLong(HWND hWnd, int index);
boolean SetForegroundWindow(HWND in);
int GetClassNameA(HWND in, byte[] lpString, int size);
}
public static void main(String[] args) {
User32.INSTANCE.EnumWindows(new User32.WNDENUMPROC() {
public boolean callback(HWND hWnd, Pointer userData) { // this will be called for each parent window found by EnumWindows(). the hWnd parameter is the HWND of the window that was found.
byte[] textBuffer = new byte[512];
User32.INSTANCE.GetWindowTextA(hWnd, textBuffer, 512);
String wText = Native.toString(textBuffer);
System.out.println("Window found: " + wText);
// now call EnumChildWindows() giving the previously found parent window as the first parameter
User32.INSTANCE.EnumChildWindows(hWnd, new User32.WNDENUMPROC() {
public boolean callback(HWND hWnd, Pointer userData) { // this is called for each child window that EnumChildWindows() finds - just like before with EnumWindows().
byte[] textBuffer = new byte[512];
User32.INSTANCE.GetClassNameA(hWnd, textBuffer, 512);
System.out.println(" - Found sub window / control class: " + new String(textBuffer).trim());
return true;
}
}, null);
return true;
}
}, null);
}
}
Here is an excerpt of output provided by this code:
Window found: Pidgin
- Found sub window / control class: gdkWindowChild
- Found sub window / control class: gdkWindowChild
- Found sub window / control class: gdkWindowChild
- Found sub window / control class: gdkWindowChild
Window found: Malwarebytes Anti-Malware
- Found sub window / control class: Static
- Found sub window / control class: Static
- Found sub window / control class: Button
- Found sub window / control class: Button
- Found sub window / control class: Button
Sending messages directly to the HWND of controls via PostMessage() and SendMessage(), for example to the MalwareBytes Button class, will trigger a button press in the program itself, very similarl to how SetForegroundWindow() should bring an edit-box style control to the front giving you the ability to type. Fun stuff to play with :)
If you wish to visualize what I mean when I am saying "Parent" and "children" and "control", you may find this program helpful: Control Viewer. It can show you each control and highlight it within an applications window and much more - very useful tool.
Sorry if this post left the comfort-zone that java provides, but there's really no other way to accomplish your goal in such a general scope.
I hope I have at least shown you what is necessary to accomplish your goal and pointed you in the right direction. I am no god when it comes to native windows API's, so I may be wrong in some place, however the code does work. Good luck :)
my friend, the Robot class can simulate writing text.
private static void typeOut(String s,Robot bot)
{
try
{
char [] chars = s.toCharArray();
for (char c : chars)
{
bot.keyPress((int)c);
bot.keyRelease((int)c);
}
}
catch (Exception e)
{
System.out.println(e.getMessage());
}
}
and you can use this method by
Robot bot=new Robot();
typeOut("WWW.GOOGLE.COM", bot);
and if in any way you want to read text or write text to a textfield on a browser i would advice you to use selenium.
For this type of problems AutoIt is easier and more versatile than Java.

keyReleases are simulating keyPresses in Linux (java Swing GUI)

I have a kiosk GUI application I'm working on and it requires me to block users from being able to Alt-Tab out of the fullscreen window. I posted a question about this a while back and a member helped me with some code, which worked perfectly under a Windows environment.
Here it is:
public class TabStopper implements Runnable {
private boolean isWorking = false;
private MenuFrame parent;
public TabStopper(MenuFrame parent) {
this.parent = parent;
new Thread(this, "TabStopper").start();
}
public void run() {
this.isWorking = true;
Robot robot;
try {
robot = new Robot();
while (isWorking) {
robot.keyRelease(KeyEvent.VK_ALT);
robot.keyRelease(KeyEvent.VK_TAB);
parent.requestFocus();
Thread.sleep(10);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void stop() {
this.isWorking = false;
}
public boolean isWorking() {
return this.isWorking;
}
}
However, I need this to be able to work in Linux as well. I made an executable jar from the source and brought it into Linux. Everything worked except the Alt and Tab keys were being constantly pressed. The buttons on my GUI were constantly being cycled and I was able to open a terminal (I set a backdoor in the application during testing in case something like this happens) which wouldn't let me type anything because Tab lists all the files in the current directory.
Could anyone tell me if there would be a fix that would work in both Linux and Windows environments. However, if I had to choose, I would go for Linux.
EDIT: I can also confirm that the Alt key is being "pressed". What's with this weird behaviour?
Forget grabbing Alt+Tab with hacks like this. It is a bad hack and it is error-prone. There are also so many other hotkey combinations.
For linux you have two options:
Use a minimal window manager or no window manager at all. For example, with fluxbox you can remove all key bindings alltogether and you can also make your application maximise by default, etc. You can empty the desktop menus such that the user gains no control even when your application crashes. This is a clean solution that really solves your problem instead of some parts of it. There are many ways to fiddle with the system other than Alt+Tab.
Grab input controls completely. This is what games do. For example libSDL does it for you and there are java wrappers for the functionality as well. This should also work as expected, except you use a window manager that does not allow input control grabbing per default (I don't know of any).

Categories