Java Robot class won't interact with an application? - java

I've been reading up on this and haven't found an answer that has made sense to me.
I'm trying to write a program in Java to interact with an application to see if I can write a program to play a video game for me. The game is on my computer.
Here is an excerpt of code:
public static void main(String[] args) throws java.io.IOException {
Runtime run = Runtime.getRuntime();
run.exec("open /Applications/OpenEmu.app");
try {
Robot robot = new Robot();
System.out.println("Waiting 5 Seconds");
//robot.delay(5000);
System.out.println("Pressed X");
robot.keyPress(KeyEvent.VK_X);
robot.keyPress(KeyEvent.VK_X);
robot.keyPress(KeyEvent.VK_X);
robot.keyPress(KeyEvent.VK_X);
//Starts an easy mode game
It opens the application fine, and in something like notepad, it will type XXXX, but it won't do so for the game?
I've assigned the 'x' key on my keyboard as a command button for the game. My guess is that the 'x' press is internal. All help is appreciated!

If you are trying to simulate input, try add robot.keyRelease as well. Javadoc for robot says for keyPress "Presses a given key. The key should be released using the keyRelease method."
System.out.println("Pressed X");
robot.keyPress(KeyEvent.VK_X);
robot.keyRelease(KeyEvent.VK_X);
...
Also remember this:
https://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html "Note that some platforms require special privileges or extensions to access low-level input control. "

Related

How to intercept music control keyboard shortcuts in Java?

If your keyboard has buttons for play/pause/etc (music control shortcuts), and you press them, iTunes will open (at least on Mac).
If you recently opened another music player, like Spotify, it will actually intercept the shortcut keys, and iTunes won't do anything.
Well, I want to make a music player with Java, and I want to have the same behavior. I want my application to intercept such shortcuts, and other programs shouldn't be able to interfere.
I am using JavaFX, although I don't think that really matters.
How can I achieve this?
I am already able to detect they keys the user presses using JNativeHook, but I do not know how to intercept the keys so that other applications won't do things with them.
Once you detect the keys, you could send the pause key so that the song that is being played by itunes is paused, you could use a boolean variable to detect between the shortcuts being typed on the keyboard or being send by the program(in case if you need)
or
You could use some c code(start the c program along with your java program) take a look at #Dave Delongs answer over here Modify NSEvent to send a different key than the one that was pressed
You could have a different keyboard shortcut and modify the c program to send your shortcut keys while the Itunes Shortcut keys are pressed, if you need the key codes Where can I find a list of Mac virtual key codes?
for example if your music program uses p to play songs and r to listen to the next song, and itunes uses spacebar to play songs and right arrow key to go to the next one, you could do modify #Dave Delongs answer here are the changes :-
#import <Cocoa/Cocoa.h>
CGEventRef myCGEventCallback(CGEventTapProxy proxy, CGEventType type, CGEventRef event, void *refcon) {
//0x31 is the virtual keycode for "Spacebar"
//0x23 is the virtual keycode for "p"
if (CGEventGetIntegerValueField(event, kCGKeyboardEventKeycode) == 0x31) {
CGEventSetIntegerValueField(event, kCGKeyboardEventKeycode, 0x23);
}
//0x7C is the virtual keycode for "Right arrow"
//0x0F is the virtual keycode for "R"
if (CGEventGetIntegerValueField(event, kCGKeyboardEventKeycode) == 0x7C) {
CGEventSetIntegerValueField(event, kCGKeyboardEventKeycode, 0x0F);
}
return event;
}
int main(int argc, char *argv[]) {
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];
CFRunLoopSourceRef runLoopSource;
CFMachPortRef eventTap = CGEventTapCreate(kCGHIDEventTap, kCGHeadInsertEventTap, kCGEventTapOptionDefault, kCGEventMaskForAllEvents, myCGEventCallback, NULL);
if (!eventTap) {
NSLog(#"Couldn't create event tap!");
exit(1);
}
runLoopSource = CFMachPortCreateRunLoopSource(kCFAllocatorDefault, eventTap, 0);
CFRunLoopAddSource(CFRunLoopGetCurrent(), runLoopSource, kCFRunLoopCommonModes);
CGEventTapEnable(eventTap, true);
CFRunLoopRun();
CFRelease(eventTap);
CFRelease(runLoopSource);
[pool release];
exit(0);
}
You may be able to use some of the code from iTunesPatch to accomplish what you're looking for, but it appears that a system daemon may need to be modified upon installation, and you will likely have to use Objective-C/Swift.
There are further details about iTunesPatch in a blog post here.

Control robot using keyboard in JAVA

I started to program a NAO Robot in IntelliJ IDEA and I am new in both, I created some functions that makes NAO to move or to speak, every time to see what is happening I have to run project. Now I want to execute some functions on KeyPress. Could you give me an example.
How to change this code, to allow NAO to Stand, Crouch or Sit on some KeyPress (ex: q-> Stand, w-> Crouch, e-> Sit)
package test;
import com.aldebaran.qi.Application;
import com.aldebaran.qi.helper.proxies.ALMotion;
import com.aldebaran.qi.helper.proxies.ALRobotPosture;
import com.aldebaran.qi.helper.proxies.ALTextToSpeech;
public class StandNao {
private static ALMotion motion;
public static void main(String[] args) throws Exception {
Application application = new NaoSettings().NaoConnect(args);
application.start();
motion = new ALMotion(application.session());
// Create an ALTextToSpeech object and link it to your current session
ALTextToSpeech tts = new ALTextToSpeech(application.session());
// Make your robot say something
tts.say("MAC Start Stand");
motion.killAll();
ALRobotPosture posture = new ALRobotPosture(application.session());
posture.getPostureList();
posture.goToPosture("Stand", 1.0f);
Thread.sleep(10000);
posture.goToPosture("Crouch", 1.0f);
Thread.sleep(10000);
posture.goToPosture("Sit", 1.0f);
}
}
I'll not describe all the possibilities over, but to give you a way to go, you can check many Java libraries who will let you manage poll or event driven keyboard input.
You can then make an infinite loop where you would listen to the keyboard activity, and react depending on the key hit.
It would barely looks like something like that:
Keyboard.poll();
while(Keyboard.next()) {
if(Keyboard.getEventKey() == Keyboard.KEY_LEFT && !Keyboard.getEventKeyState()) {
// do something if the letter left arrow key is released
}
}
Also consider trying to make funny things, if you want to learn both Java and Web aspects, like a Web Application with REST controller, and when you hit buttons a web page, your robot move accordingly.
It can be quickly implemented by using Java Spring Boot.

Java Robot class working in laptop but not pc

This is my first question to stackoverflow. Helped me a lot in the past whenever i was stuck. Anyway here is the problem:
I was using Java Robot in my PC. Everything was fine like i could type in notepad move mouse around in other applications like games etc. But there was only this one game which the java Robot did not work on. Nothing was detected on this game not even mouse movement. I tried to do some research on this and came to a conclusion that maybe that game has some sort of anti-bot system. Keep in mind this was in my PC which is windows 7 64bit. Then i thought to use the same code in my laptop(which is also windows 7 64bit) on the same game and it WORKED!
So my question is why did this happen? Why did it work in my laptop and not my PC?
here is the code:
package test_bot1;
import java.awt.Robot;
import java.awt.event.KeyEvent;
public class test_BOT1 {
public static void main(String[] args) {
try{
Robot bot = new Robot();
bot.delay(3000);
bot.mouseMove(500, 0);
for(int i = 0; i < 10; i++){
bot.keyPress(KeyEvent.VK_A);
}
bot.delay(100);
bot.keyRelease(KeyEvent.VK_A);
bot.delay(100);
bot.keyPress(KeyEvent.VK_TAB);
bot.delay(200);
bot.keyRelease(KeyEvent.VK_TAB);
bot.delay(159);
bot.keyPress(KeyEvent.VK_1);
bot.delay(179);
bot.keyRelease(KeyEvent.VK_1);
}catch(Exception e){
}
}
}
K Out!
Surely the simple answer to this is not to cheat at games?
Try adding e.printStackTrace() to your catch block and look to see if there are errors on the pc version and not on the laptop.

How to close external programs (Powerpoint) running on the native desktop using Java?

I have to create an application that will automatically open a powerpoint file, let it play through, and then close it. Not only do I need to figure out HOW to close it, but I also must detect when it closes or stops.
First option:
I know how long each powerpoint will play for, so I can hardcode when to close the file. I just need to know how to do that. There are no methods in the desktop class (that I could find) for closing.
Second option:
If someone knows a microsoft powerpoint api that lets me open powerpoints and use java to progress through the slideshow and get the state or something, that'd be great. I wouldn't have to go into each presentation and count the number of slides and the transition timer on each slide.
The opening, letting it play, and closing it is a small part of the app I need to create. But here is what I have so far with regards to THIS problem:
File myfile = new File("PowerPoint.ppsx");
try {
Desktop.getDesktop().open(myfile);
} catch (IOException ex) {
Logger.getLogger(Sc.class.getName()).log(Level.SEVERE, null, ex);
}
Probably this is the solution how to close external program:
http://www.java-forums.org/new-java/59691-close-another-program.html#post285956
If you want to detect when program has stopped running then you can start new thread with loop which from time to time will check if the program process is still running, using the same method as mentioned in link.
This is solution only for one (Windows) platform, Java is not the best choice for such tasks.
Here a solution using JNA. First we get the handle, we search using the "class name" of the window. You can determine the class name for a specific program (in this case Powerpoint) with a special utility like Spy++ (included with Visual Studio). It's possible to make the search more precise using the class name and the window caption (but here I use only the class name) so if you have more than one presentation running ... you may not close the good one!.
import com.sun.jna.platform.win32.User32;
import com.sun.jna.platform.win32.WinUser;
import com.sun.jna.platform.win32.WinDef.HWND;
// https://github.com/twall/jna#readme
// you need 2 jars : jna-3.5.1.jar and platform-3.5.1.jar
public class KillMyPP {
public static void main(String[] args) {
HWND hwnd = User32.INSTANCE.FindWindow("screenClass", null);
if (hwnd == null) {
System.out.println("PPSX is not running");
}
else {
User32.INSTANCE.PostMessage(hwnd, WinUser.WM_QUIT, null, null);
}
}
}

Java bot using the Robot class

I'm trying to create a game bot using the Robot class.
I have tried the following code to perform a right click of the mouse:
robot.mousePress(InputEvent.BUTTON3_MASK);
robot.mouseRelease(InputEvent.BUTTON3_MASK);
And it worked.
I'm testing it on a client side 3d online game.
Pressing the key "1" should perform some kind of a movement ingame, and when I tried the following code it didn't worked:
robot.keyPress(KeyEvent.VK_1);
robot.keyRelease(KeyEvent.VK_1);
But it did worked when I used that code while speaking in the chat in game.
It's been tested over and over and I keep getting the same result.
Is it something that I have done wrong?or somehow the game detect that I'm not the one that pressing that key.
You are probably releasing the key too quickly. Try sleeping for 30~60ms before releasing the key:
robot.keyPress(KeyEvent.VK_1);
try {
Thread.sleep(50);
} catch(Exception e) {
e.printStackTrace();
}
robot.keyRelease(KeyEvent.VK_1);

Categories