I am trying to create a program that automatically searches for a text field on the screen and types a word repetitively into that text field. Is there any class that can find a text field? Or is there any way in which a text field can be found? Because I know that the Robot class can type text, I just need to either get the cursor onto the text field and use the mousePress() and mouseRelease() methods.
Thanks
I can't directly give you a solution, but I messed around with some code and may be able to point you in the right direction.
Java, as you probably know, runs in the JVM. This allows it to execute in any operating environment. Each operating environment (windows, mac, etc) has its own system for handling edit boxes and setting focus to the right window and whatnot. The following example code is designed for use on windows only, which does not follow the spirit of the Java language. As Adriaan pointed out, there are other languages for this sort of thing, but it IS possible (to an extent) to accomplish with Java alone.
In windows, you must understand how all of the active windows are managed and that everything you see (including edit boxes) are considering a "window" by the Windows OS. I don't truly understand how things work under the hood, so I can't provide much more information than that. In a native language such as C++, there are a few functions provided by the Windows OS API that would be used to accomplish your goal. Namely, EnumWindows(), EnumChildWindows(), GetClassName(), and SetForegroundWindow(). You can find documentation on how to use these functions within a native language by searching the MSDN documentation library.
So with that said, you NEED to be able to call these functions from Java. Under normal circumstances, calling these native methods is not possible. However, there is a library available to help you out: the JNA library. JNA stands for Java Native Access and lets you work with the shiny new functions I mentioned earlier.
So, to accomplish your goal in a native language, normally one would begin with a call toEnumWindows() to return a list of all Parent windows that the OS is aware of. This list will contain window handles of parent windows - windows titled "MSN", "Eclipse", "Microsoft Office", etc. Each of these windows, as a Parent, has children. It is in this list of children that you will find the "control" that you are looking for: an Edit control. Now, many applications use different libraries and non-standard things for text boxes - i.e Pidgin, a messaging application I tested some relevant code with, has every control named "gdkWindowChild" which doesn't exactly tell us which control is actually an EditBox or otherwise a place that allows us to enter text. That's the main problem with your idea; you can't always tell exactly what control you wish to have focus of so that you may enter text. Regardless of that, we'll continue:
After finding the relevant Parent window with EnumWindows(), a call to EnumChildWindows() will give us all of the sub-windows and other "controls" (including potential edit-boxes) that belong to the Parent. EnumChildWindows() calls a callback function for each sub-window it finds, so it's pretty easy to "search" through the list of child windows - using GetClassName() to find the name of a control - to potentially find the HWND (window handle) of the control you want.
Once you have found the correct HWND of the edit box (that, of course, being the difficult part given the general scope of your question) a simple call to SetForegroundWindow(targetHWND) ought to bring the control to the front and set your cursor in a ready-to-type edit box.
Here is some working example code I've written to get you started. This code will iterate through all of the active windows using EnumWindows() and then call EnumChildWindows() on each parent, printing out all of the controls that it finds. Note that this code requires the JNA library to run.
import com.sun.jna.Native;
import com.sun.jna.Pointer;
import com.sun.jna.examples.win32.W32API.HWND;
import com.sun.jna.examples.win32.W32API.LPARAM;
import com.sun.jna.win32.StdCallLibrary;
import com.sun.jna.win32.W32APIOptions;
public class IterateChildWindows {
public interface User32 extends StdCallLibrary {
User32 INSTANCE = (User32) Native.loadLibrary("user32", User32.class, W32APIOptions.DEFAULT_OPTIONS);
HWND FindWindow(String lpClassName, String lpWindowName);
int GetWindowRect(HWND handle, int[] rect);
int SendMessage(HWND hWnd, int msg, int wParam, byte[] lParam);
HWND FindWindowEx(HWND parent, HWND child, String className, String window);
boolean EnumWindows(WNDENUMPROC lpEnumFunc, Pointer arg);
boolean EnumChildWindows(HWND parent, WNDENUMPROC callback, LPARAM info);
interface WNDENUMPROC extends StdCallCallback {
boolean callback(HWND hWnd, Pointer arg);
}
int GetWindowTextA(HWND hWnd, byte[] lpString, int nMaxCount);
long GetWindowLong(HWND hWnd, int index);
boolean SetForegroundWindow(HWND in);
int GetClassNameA(HWND in, byte[] lpString, int size);
}
public static void main(String[] args) {
User32.INSTANCE.EnumWindows(new User32.WNDENUMPROC() {
public boolean callback(HWND hWnd, Pointer userData) { // this will be called for each parent window found by EnumWindows(). the hWnd parameter is the HWND of the window that was found.
byte[] textBuffer = new byte[512];
User32.INSTANCE.GetWindowTextA(hWnd, textBuffer, 512);
String wText = Native.toString(textBuffer);
System.out.println("Window found: " + wText);
// now call EnumChildWindows() giving the previously found parent window as the first parameter
User32.INSTANCE.EnumChildWindows(hWnd, new User32.WNDENUMPROC() {
public boolean callback(HWND hWnd, Pointer userData) { // this is called for each child window that EnumChildWindows() finds - just like before with EnumWindows().
byte[] textBuffer = new byte[512];
User32.INSTANCE.GetClassNameA(hWnd, textBuffer, 512);
System.out.println(" - Found sub window / control class: " + new String(textBuffer).trim());
return true;
}
}, null);
return true;
}
}, null);
}
}
Here is an excerpt of output provided by this code:
Window found: Pidgin
- Found sub window / control class: gdkWindowChild
- Found sub window / control class: gdkWindowChild
- Found sub window / control class: gdkWindowChild
- Found sub window / control class: gdkWindowChild
Window found: Malwarebytes Anti-Malware
- Found sub window / control class: Static
- Found sub window / control class: Static
- Found sub window / control class: Button
- Found sub window / control class: Button
- Found sub window / control class: Button
Sending messages directly to the HWND of controls via PostMessage() and SendMessage(), for example to the MalwareBytes Button class, will trigger a button press in the program itself, very similarl to how SetForegroundWindow() should bring an edit-box style control to the front giving you the ability to type. Fun stuff to play with :)
If you wish to visualize what I mean when I am saying "Parent" and "children" and "control", you may find this program helpful: Control Viewer. It can show you each control and highlight it within an applications window and much more - very useful tool.
Sorry if this post left the comfort-zone that java provides, but there's really no other way to accomplish your goal in such a general scope.
I hope I have at least shown you what is necessary to accomplish your goal and pointed you in the right direction. I am no god when it comes to native windows API's, so I may be wrong in some place, however the code does work. Good luck :)
my friend, the Robot class can simulate writing text.
private static void typeOut(String s,Robot bot)
{
try
{
char [] chars = s.toCharArray();
for (char c : chars)
{
bot.keyPress((int)c);
bot.keyRelease((int)c);
}
}
catch (Exception e)
{
System.out.println(e.getMessage());
}
}
and you can use this method by
Robot bot=new Robot();
typeOut("WWW.GOOGLE.COM", bot);
and if in any way you want to read text or write text to a textfield on a browser i would advice you to use selenium.
For this type of problems AutoIt is easier and more versatile than Java.
Related
I have to create an application that will automatically open a powerpoint file, let it play through, and then close it. Not only do I need to figure out HOW to close it, but I also must detect when it closes or stops.
First option:
I know how long each powerpoint will play for, so I can hardcode when to close the file. I just need to know how to do that. There are no methods in the desktop class (that I could find) for closing.
Second option:
If someone knows a microsoft powerpoint api that lets me open powerpoints and use java to progress through the slideshow and get the state or something, that'd be great. I wouldn't have to go into each presentation and count the number of slides and the transition timer on each slide.
The opening, letting it play, and closing it is a small part of the app I need to create. But here is what I have so far with regards to THIS problem:
File myfile = new File("PowerPoint.ppsx");
try {
Desktop.getDesktop().open(myfile);
} catch (IOException ex) {
Logger.getLogger(Sc.class.getName()).log(Level.SEVERE, null, ex);
}
Probably this is the solution how to close external program:
http://www.java-forums.org/new-java/59691-close-another-program.html#post285956
If you want to detect when program has stopped running then you can start new thread with loop which from time to time will check if the program process is still running, using the same method as mentioned in link.
This is solution only for one (Windows) platform, Java is not the best choice for such tasks.
Here a solution using JNA. First we get the handle, we search using the "class name" of the window. You can determine the class name for a specific program (in this case Powerpoint) with a special utility like Spy++ (included with Visual Studio). It's possible to make the search more precise using the class name and the window caption (but here I use only the class name) so if you have more than one presentation running ... you may not close the good one!.
import com.sun.jna.platform.win32.User32;
import com.sun.jna.platform.win32.WinUser;
import com.sun.jna.platform.win32.WinDef.HWND;
// https://github.com/twall/jna#readme
// you need 2 jars : jna-3.5.1.jar and platform-3.5.1.jar
public class KillMyPP {
public static void main(String[] args) {
HWND hwnd = User32.INSTANCE.FindWindow("screenClass", null);
if (hwnd == null) {
System.out.println("PPSX is not running");
}
else {
User32.INSTANCE.PostMessage(hwnd, WinUser.WM_QUIT, null, null);
}
}
}
I am just trying to figure out if/how to get the Java Robot class to change focus from the running java app, to a specific process, such as ms word or firefox.
Thanks!
Robot can't do that automatically. You can activate another application via alt-tab as has been suggested above, but you'll need to know the z-order of the application that you want to activate. I think that to really do this best you'll want to get the window's handle (hWnd) of the top-level window that you want to activate (if this is a Windows app), and then using the Windows user32 library functions activate the desired window. For this I recommend using JNA as one of the easiest ways (when compared to JNI). You would have to first download the JNA jna.jar and platform.jar jar files, and place them on your classpath, and then you can call most OS methods with ease. For instance, I have just this sort of thing up and running for a Windows application where I can get the hWnd for a running top-level Windows application based on the window name (full or partial), and then using that hWnd, call user32's setForegroundWindow function. If you're wanting to activate a Windows application and want to pursue this further, comment to this answer, and I can show you what code I have for this. If so, you'll want to go into greater details on exactly what it is you're trying to do.
Best of luck!
For any that comes across this in Google as I just did:
public class activate {
public interface User32 extends W32APIOptions {
User32 instance = (User32) Native.loadLibrary("user32", User32.class,
DEFAULT_OPTIONS);
boolean ShowWindow(HWND hWnd, int nCmdShow);
boolean SetForegroundWindow(HWND hWnd);
HWND FindWindow(String winClass, String title);
int SW_SHOW = 1;
}
public static void main(String[] args) {
User32 user32 = User32.instance;
HWND hWnd = user32.FindWindow(null, "Downloads"); // Sets focus to my opened 'Downloads' folder
user32.ShowWindow(hWnd, User32.SW_SHOW);
user32.SetForegroundWindow(hWnd);
}
}
Credit: http://www.coderanch.com/t/562454/java/java/FindWindow-ShowWindow-SetForegroundWindow-effect-win
You did not specified system, on Mac, there is possible to do it with AppleScript. AppleScript is integrated to system, so it will be always functional.
https://developer.apple.com/library/content/documentation/AppleScript/Conceptual/AppleScriptLangGuide/reference/ASLR_cmds.html
You need only detect you are on mac and has name of the application.
Runtime runtime = Runtime.getRuntime();
String[] args = { "osascript", "-e", "tell app \"Chrome\" to activate" };
Process process = runtime.exec(args);
I want to create a special Password Dialog for my eclipse product, which is used with an on screen keyboard.
It would be very nice, if i could use a component like the IPhone Password field. In this field, the added character is shown for a second and after the second it is converted into the '*' character for hiding the complete password.
Did a jar/library exists, this is implemented in AWT or SWT?
Edit:
I could trying to implement it from scratch (SWT), but for these i would have to create a very special and complicated KeyListener for the password Text component. I would have to catch the keyReleased event and set the characters manually into the field.
So far i was not able to find any libraries in the web. Suggestion how this can be implemented are welcome too.
This is not really a full answer, rather than a discussion starter and I don't know of any out-of-the-box widgets which can do that.
My first idea was to inheriting the swt Text widget and overriding setEchoChar et al., but after looking at the code this doesn't really seem feasible, because this method is merely a wrapper around:
OS.SendMessage (handle, OS.EM_SETPASSWORDCHAR, echo, 0);
If anyone would know the OS specific low-level implementation, that might be helpful.
Anyway, on to a different approach. I would avoid the KeyListener and use a ModifyListener on the Text-Widget.
void addModifyListener(ModifyListener listener)
You could then build a wrapper which catches the entered text using this listener, appends it to a locally held string/stringbuffer (or e.g. the Eclipse Preferencestore) and send a modified full text to the Text widget using setText(String s), replacing all characters except the last by an echo character (e.g. *).
myText.setText((s.substring(0, s.length()-1)).replaceAll("[\\s\\S]","*")+s.charAt(s.length()-1));
This is a bit of a kludge, but it should work.
The not so straightforward bit is the 1 second timing, without stalling the whole view...
Depending on what Jules said the following code is some kind of working.
The code is quick and fast and i would like to have a more thread safe solution.
originalString = new StringBuffer();
passwordField.addModifyListener(new ModifyListener() {
public void modifyText(ModifyEvent e) {
synchronized (passwordField) {
String s = passwordField.getText();
String newS = s.replaceAll("[\\s\\S]", "*");
if (newS.equals(s)) {
while (originalString.length() > s.length()) {
originalString = originalString.deleteCharAt(originalString.length() - 1);
}
usernameField.setText(originalString.toString());
return;
}
if (originalString.length() < s.length()) {
originalString.append(s.charAt(s.length() - 1));
}
try {
Thread.sleep(500);
} catch (InterruptedException e1) {
}
passwordField.setText(newS);
}
passwordField.redraw();
passwordField.setSelection(passwordField.getText().length());
}
});
Key Events are cached, so you can add more characters, also when the Thread is waiting.
Another Problem is the Cursor handling. the Cursor always moves to the first position, when you set the Text.
I think when this is working it is very near to the iphone solution.
I am calling UIManager.getSystemLookAndFeelClassName().
And get as result WindowsLookAndFeel, even when setting current theme to WindowsClassicLookAndFeel. But I expect WindowsClassicLookAndFeel.
Question is how can I detect that windows classic theme is now active
It looks like you signed up specifically to ask this question, and now I'm signing up specifically to answer it! I was Googling for something completely different but I was intrigued and did some experimenting, and here's what I found:
You're right about the WindowsClassicLookAndFeel. This class extends WindowsLookAndFeel but doesn't override anything, and doesn't appear to be used at all, even when Windows Classic is active. So, I looked into the code of WindowsLookAndFeel and found some interesting internal code that references the package-private class XPStyle. This class appears to be a singleton and the getter method, getXP(), only returns its instance if the 'XP' theme is active:
/**
* Get the singleton instance of this class
*
* #return the singleton instance of this class or null if XP styles
* are not active or if this is not Windows XP
*/
static synchronized XPStyle getXP() {
if (themeActive == null) {
Toolkit toolkit = Toolkit.getDefaultToolkit();
themeActive =
(Boolean)toolkit.getDesktopProperty("win.xpstyle.themeActive");
if (themeActive == null) {
themeActive = Boolean.FALSE;
}
if (themeActive.booleanValue()) {
GetPropertyAction propertyAction =
new GetPropertyAction("swing.noxp");
if (AccessController.doPrivileged(propertyAction) == null &&
ThemeReader.isThemed() &&
!(UIManager.getLookAndFeel()
instanceof WindowsClassicLookAndFeel)) {
xp = new XPStyle();
}
}
}
return xp;
}
Interestingly, the code checks for the WindowsClassicLookAndFeel again but we know that this is no longer used (perhaps it's changed)... But the best part is the check for the desktop property win.xpstyle.themeActive Extracting this from the code, I made the following test:
package test;
import java.awt.Toolkit;
import java.beans.PropertyChangeEvent;
import java.beans.PropertyChangeListener;
import javax.swing.UIManager;
public class WindowsClassicThemeDetector {
public static boolean isWindowsLAF() {
return UIManager.getLookAndFeel().getID().equals("Windows");
}
public static boolean isWindowsClassicLAF() {
return isWindowsLAF()
&& !(Boolean) Toolkit.getDefaultToolkit().getDesktopProperty(
"win.xpstyle.themeActive");
}
public static void main(String... args) throws Exception {
// Apply the system look and feel (which will be Windows)
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
// Check the initial theme state on startup
System.out.println(UIManager.getLookAndFeel().getClass());
System.out.println("Windows classic is initially: " + isWindowsClassicLAF());
// Register a listener in case the theme changes during runtime.
Toolkit.getDefaultToolkit().addPropertyChangeListener("win.xpstyle.themeActive",
new PropertyChangeListener() {
#Override
public void propertyChange(PropertyChangeEvent evt) {
System.out.println("Windows classic is now: " + isWindowsClassicLAF());
}
});
// Wait until user presses ENTER in the console and then exit.
System.in.read();
}
}
You can import this class into your program and call the isWindowsClassicLAF() method after setting your Look&Feel at any point. I've also given an example of how you can listen out for changes to the theme at run-time.
This is tried and tested on XP. If the user changes from one XP theme to another, the listener doesn't fire, but if the user changes from a theme to Classic, or vice-versa, it will.
I hope that helps!
I'm not quite sure what you are asking.
If you are asking which theme/look and feel SWING is currently using, try
UIManager.getSystemLookAndFeelClassName().
If you are trying to find out which theme Windows is using - I don't know. Probably not quite easy to find that out.
Edit: A quick hack might be (apart from using JNA/JNA to query some Windows API directly which theme is being used) would be creating a new JFrame, place it in some area, capture a small part of that border and compare it with samples of borders (which you need to create beforehand and ship with your code, as to be able to programmatically compare the just-taken-screen-capture-bit and all border-images you ship)
UIManager.getLookAndFeel() returns currently installed LaF. getSystemLookAndFeel() returns look and feel that would look as a current system's theme, e.g. WindowsLookAndFeel on windows GTKLookAndFeel on Linux MOtifLookAndFeel on Solaris etc.
I would like to find the window ID of my SWT program.
I start up my SWT shell in the standard way. How do I then find the ID of the window that's been created? The program is executing on Fedora 10 using the Compiz-Fusion window manager (if that makes a difference).
Code to help explain what I mean:
public static void main(String[] args) {
Display display = new Display();
Shell shell = new Shell();
// find window ID here??
shell.open();
while (!shell.isDisposed()) {
if(!display.readAndDispatch()) {
display.sleep();
}
}
}
Update 6 Mar 2009
After looking at and trying out a whole range of things (thanks to VonC for the suggestions below), I came across something that's worked so far. It's supremely dodgy but at least allows me to get on with some other work for the moment.
The ID returned from Control.embeddedHandle is close to the window ID. It's different by a predictable offset. So my code is:
public static void main(String[] args) {
...
Shell shell = new shell(display, SWT.NONE);
Composite e = new Composite(shell, SWT.EMBEDDED);
long windowID = e.embeddedHandle - WINDOW_ID_MAGIC_NUMBER;
e.dispose();
....
WINDOW_ID_MAGIC_NUMBER is 5+(number of widgets added to shell before the embedded composite).
It seems reliable on my system so far. I have yet to see if it falls over in a different environment.
If you create a Composite with the style SWT.EMBEDDED style, then under SWT/GTK+ Composite.embeddedHandle will be an X window ID you can use for parenting an XEMBED child.
Composite embed = new Composite(shell, SWT.EMBEDDED);
System.out.println ("X window ID: " + embed.embeddedHandle);
int hwndChild = OS.GetWindow ( c.handle, OS.GW_CHILD);
This supports embedding using the XEMBED protocol.
This is similar to the JNI code used to get the window ID from its handle
GtkWidget *widget = (GtkWidget *) handle;
GdkWindow *window = widget->window;
xwinid = GDK_WINDOW_XWINDOW(window);
Example of code here.
Example of class using OS:
org.eclipse.swt.widgets.Tree, org.eclipse.swt.widgets.CoolItem,
OS I can find is indeed org.eclipse.swt.internal.win32.OS, not gtk, and it is not a perfect solution because you would access an internal package, but that can give you an idea where to look.
In your case, org.eclipse.swt.internal.gtk.OS is the right class, and you should look in Tree or CoolItem sources how they are using the GetWindow function.
Some other function need to be called in GTK, like may be gtk_container_get_children(int container);
It is said in an answer to the message I was referring at the beginning
If you need the X window, there's no way to do this from the public SWT API (even going through internals), and furthermore even if there was a way I don't think you could guarantee it for all controls. I'm not sure if this works but the closest you could get might be to:
make Control.fixedHandle public
Use OS.GTK_WIDGET_WINDOW (control.fixedHandle) to get a GdkWindow
Use OS.gdk_x11_drawable_get_xid (gdkWindow) to translate that to an X window
I guess the other way might be to take Control.handle, and then call GTK_WIDGET_WINDOW() on it, or if that's null keep calling it on its parents until you find one with a GdkWindow, and then translate that to an X window.
The difficulty here is that SWT talks to GTK+, which talks to GDK, which then talks to X. So, there's three layers between you and the X window.
Not sure if this still matters to you, 7 years later :-), but this works for me:
private static long getWindowIdFromShell(Shell shell) {
long handle = shell.handle;
long topWidget = OS._gtk_widget_get_toplevel(handle);
long topWindow = OS._gtk_widget_get_window(topWidget);
long topXid = OS._gdk_x11_window_get_xid(topWindow);
return topXid;
}
In particular, the "get_toplevel" step is what jumps to the root widget/window, and so means you don't need the "minus magic window offset" hack (which I was initially stuck doing as well).