I am trying to automate some UI testing for my company. I have written an automation that goes through an entire process. It does everything I need. I am using the Sikuli API from my Java code to do this.
Now, my problem is that my mouse is hijacked during the test process. It would be convenient if I could run these tests without my mouse being affected. My first thought was to run the tests inside a VM and control the "VM Mouse" using the automation instead of running the tests from my desktop and controlling my "Desktop Mouse".
I have seen a few questions and links on this topic, but none with a clear answer. Is it possible to do this? If so, how? Keep in mind any solution works - it doesn't have to be Java or Sikuli, I just want to know if it is possible to automate mouse movement inside a VM without affecting my "real" mouse.
You can't ask a question about something you've done using Sikuli and then say that it doesn't have to be Sikuli. If you are generally interested in a way you can run headless or remote desktop automation, just ask it as it is.
Saying that, you can't use Sikuli without sacrificing your screen and mouse. The reason for this is Sikuli implementation which is done by utilizing the Java Robot class that takes control of user input to interact with underlying software.
There is an attempt to work around this limitation using a VNC. It is described here.
Related
There is a legacy app that uses swing, and it spawns various windows for various purposes at various times. What I'm wondering is if it possible to get a "screenshot" of each window, without having to actually go through the screenshot utility on my OS. I know this is possible somehow, but I'm wondering if it is something nicely supported by swing/java? I'd love to just load a file via this app, iterate over the windows spawned, and write an image out somewhere.
Sikuli is pretty neat, it allows you to automate anything, based on image recognition. Their site: www.sikuli.org. And it can take screenshot while running the automated task you scripted: How to take screenshot with sikuli
I'm not affiliate to sikuli whatsoever, I simply used it in the past (to automate tests on a swing GUI). The description on their site:
Sikuli automates anything you see on the screen. It uses image
recognition to identify and control GUI components. It is useful when
there is no easy access to a GUI's internal or source code.
Ever since i started learning java i wanted to create a way to automate a few actions on a couple websites,
For example, topline is a website that replaces all your ads with its own ads and pays you a bit of money for it, i want to emulate the act of just surfing the web, then start emulating specific tasks like clicking certain buttons or playing flash games (Actually playing the game by using image recognition) and this has to be written in java as i want to run this on a raspberry pi.
any help is appreciated, is there a class that i can use?
any help is appreciated!
Selenium is a good browser automation tool. Refer http://seleniumhq.org/
You can get more info on Google. Let me know if you need help finding resources.
Check the class Robot, it will help you to emulate mouse interactions with the screen, but you have to implement the image recognition though
Although I can't point you to a JAVA solution, I would like to advocate two very interesting tools: PhantomJS and CasperJS. The latter depends on the first, and with them, browser navigation scripting and testing are a breeze.
They both work on Linux, MAC OS and Windows and are as multiplatform as Javascript can be. Naturally, it will work just fine in you Raspberry Pi.
I need to create a Java application that can detect any GUI component of an application on Windows or Linux. For example, you have a Firefox browser running and I want to have a list of interactive components (such as buttons, menus) and possibly drive Firefox with my program (sort of like a remote controller). Do the OSes provide some capabilities that enable this?
I remember from a long time ago when I did some automated software testing, the testing software could tell each GUI component of any application on Windows. I've looked around and found ResourceHacker (http://www.angusj.com/resourcehacker/rh_shot.html) and it's somewhat similar.
Is this possible through Java? If not, what language might be suitable? Any opensource solutions out there?
Any pointer/advice would be appreciated.
Thank you!
How can I disable OS-level keyboard shortcuts (e.g. Alt-Tab, Ctrl-Alt-Left/Right, etc.) on a [Ubuntu] Linux machine? I'm developing a full-screen Java Swing app and don't want the user to be able to task switch away from the program arbitrarily. It's not enough to toggle the "always on top" flag; users mustn't be allowed to switch workspaces, migrate focus or any other such things. The machine must function normally before and after the application is executed. Google says that this will require JNI or JNA but I'm looking for a bit more hand-holding.
There's no point in trying to do this in your application because any of these changes are going to need to be handled by X11 and/or the window manager since those are what respond to the commands. Assuming that you have control of the platform, choose a window manager which supports a kiosk mode. Then use the window manager's settings to start your application and enter kiosk mode.
Options for window managers which can do this include KDE or twm-kiosk.
(And if you don't have control of the platform, you're not likely to be able to have your application intercept things like ctrl-alt-backspace anyway.)
Edit:
In response to a scaled-down version of the question in which he's willing to let things like ctl-alt-backspace go and just wants most of the keys including alt-tab or other similar application switching key combinations, the following should work:
You should be able to do this using XLib's XGrabKeyboard method through JNI. This Java/XLib JNI keypress capture tutorial should be a good starting point. However, it uses XGrabKey which just passively listens for keys and does not prevent other applications from receiving them. You'll instead want to use XGrabKeyboard which actively snags all of the normal keyboard events (which, if the premise of this StackOverflow question is correct, includes the task switching keys).
Note that as a side-effect, key capture in Swing will also probably stop working because your Swing windows are going to be separate from the window you create in C. As such, you will probably have to use your JNI interface to get key presses to your program when needed. (Although I would definitely advise testing it first before writing the code.) You might be able to avoid this if you can get the window using Java AWT Native Interface to get the window ID. (Note that Swing is built on top of AWT, so this will work for Swing.) However, I'm not sure how to do this. It looks like you might be able to navigate the window tree by getting the root window from the Display and going from there to find your Window, but it's all kind of weird. It would be nice if the AWT NI just told you the window ID, but it doesn't look like it does that.
As this warning Reminder: XGrabKeyboard is not a security interface notes, this doesn't make it impossible for other programs to see the keys, but it seems likely that window managers will not be using XQueryKeyMap so it is likely to prevent task switching.
I want to partially automate some integration level tests via the use of the Robot class to send mouse/keyboard events to the the Java application I'm testing. I want the user to be able to click a button and have a number of keyboard events be automatically sent at once.
I spawn the application I want to test from inside my Robot test (using Runtime.exec) and then generate the appropriate events. Unfortunately when the user clicks a button the button has focus and receives the events instead of the events going to the child process as I would like. I would like a way to ensure that the Robot keyboard events are sent to the application I'm trying to test instead.
I've thought of fetching the child process PID and then using the FG command to bring the application I want to the foreground; but this is a Linux specific approach. I would prefer a method that works for Linux or Windows equally as well.
I don't know exactly what you are testing or how you are testing it but you should look into Sikuli - its a visual technology and there are methods that will let you either automatically click on a part of the screen you want to get focus, or use App.focus("application name"). I really recommend it. However using it will not let you use the computer to do other things while testing the program.
Hope that helps!