We Keep Coding

Problem Making A T-Rex Game Bot Using Selenium and Robot Class

I have a problem making T-Rex game bot. The code works fine for a few seconds but then the game gets over.
I have used Selenium and The Robot class for this project.
MyCode>>
public static void main(String[] args) throws InterruptedException, AWTException {
System.setProperty("webdriver.chrome.driver", "C:\\Users\\BHEL\\Downloads\\chromedriver_win32\\chromedriver.exe");
WebDriver driver= new ChromeDriver();
driver.manage().window().maximize();
driver.get("https://trex-runner.com/");
Robot robo = new Robot();
robo.mouseMove(460, 520);
while(true) {
//now i have used for loops to sense the cactus in a square area
for(int i = 0; i<=10;i++) {
for(int j=0;j<=10;j++) {
if(robo.getPixelColor(460-i, 520-j).equals(new Color(83,83,83))) {
robo.keyPress(KeyEvent.VK_UP);
robo.keyRelease(KeyEvent.VK_UP);
Thread.sleep(200-j-i);//Stay in the air for a while
robo.keyPress(KeyEvent.VK_DOWN);
robo.keyRelease(KeyEvent.VK_DOWN);
}
}
}
}
}
The WebSite url for T-Rex game that i used https://trex-runner.com .
I have seen some other tutorials that did the same thing that i did.
Thanks for your help :)

Could be that the coordinates aren't too precise. You could use System.out.println(MouseInfo.getPointerInfo().getLocation()); to get the coordinates of your mouse on your monitor and then change your x and y value to those. That method would get me to 400 points. To get further you have to add more conditions so the robot responds better to the obstacles.
Simple code:
while (true) {
//Got the coordinates from the mouse location ^^^
if (robo.getPixelColor(580 ,530).equals(new Color(83,83,83))) {
robo.keyPress(KeyEvent.VK_SPACE);
robo.keyRelease(KeyEvent.VK_SPACE);
Thread.sleep(150);//Stay in the air for a while
robo.keyPress(KeyEvent.VK_DOWN);
robo.keyRelease(KeyEvent.VK_DOWN);
}
}

How to do smooth-scrolling by Java robot?

I got used to my touch pad, that allows to scroll smoothly and very exactly, but I can not to simulate it by Java robot - mousewheel is getting only integer parameters and a scrolling carried by steps. Can I simulate smoothly scrolling in Java?
robot.mouseWheel(a); // int a

The unit of scrolls will always be by "notches of the wheel" (before you ask: that's how the measurement is named in the docs). This is simply how it's implemented by the hardware (to be more specific: the mouse). How many pixels are scrolled per "notch" is nothing but OS-configuration. You can't mess with that with pure java and I wouldn't recommend it, even if it was possible.
What you can do nevertheless is to slow down the speed at which the robot scrolls:
import java.awt.Robot;
public class Test{
public static void main(String[] args)
throws Exception
{
//time to switch to a specific window where the robot ought to be tested
try{ Thread.sleep(2000); }catch(InterruptedException e){}
Robot r = new Robot();
for(int i = 0; i < 20; i++){
//scroll and wait a bit to give the impression of smooth scrolling
r.mouseWheel(1);
try{ Thread.sleep(50); }catch(InterruptedException e){}
}
}
}

Mouse emulator using Processing, Arduino, and Robot class: inconsistent function

I have built a mouse emulator for a severely disabled person, which has a series of buttons arranged in a + pattern, with the center button being a 'click'. The inputs are read into an Arduino Micro, which then connects through USB to the computer. A processing script, using the Arduino library, responds to these changes by moving the mouse, using the Robot class. I exported an application, for both 32-bit and 64-bit windows, but found that the .exe files were problematic, and instead opted for .bat files using the embedded Java.
This all works perfectly on my computer (Microsoft Surface, Windows 8, 64-bit). But on the client's computer (Windows 7, 32-bit), the results are erratic. Each time I load the program, I get a different result. For example, sometimes up, down, and click work. Sometimes only up and click work. And so forth. The click usually seems to work, but the directions are less reliable, with vertical working more often than horizontal.
I have verified that nothing is wrong with the board. I tried putting in a debug statement which ran println() each time a movement's "if" statement was tripped - I only got printing from directions that worked on that particular run.
I see a number of potential sources for this problem: the differences between 32-bit and 64-bit Windows, possible issues with the Robot class, USB performance issues (though I've tried multiple ports on the client's laptop), etc. The client's laptop has generally poor performance (for example, if I click "Control Panel", it may be a full minute before Control Panel opens), so that may also be related.
Does anyone have other insights on this?
EDIT: As requested, here's a minimal version of the Processing code. The Arduino is just running Firmata:
import processing.serial.*;
import cc.arduino.*;
import java.awt.*;
import java.awt.event.*;
Arduino arduino;
Robot r;
final int upPin=8;
final int downPin=12;
final int leftPin=10;
final int rightPin=11;
final int buttonPin=9;
final int SPEED=8;
boolean arduinoConnected;
int prevButton=Arduino.LOW;
void setup() {
arduinoConnected = (Arduino.list().length>0);
if (arduinoConnected) {
arduino = new Arduino(this, Arduino.list()[Arduino.list().length-1], 57600);
arduino.pinMode(buttonPin, Arduino.INPUT);
arduino.pinMode(upPin, Arduino.INPUT);
arduino.pinMode(downPin, Arduino.INPUT);
arduino.pinMode(leftPin, Arduino.INPUT);
arduino.pinMode(rightPin, Arduino.INPUT);
}
try {
r = new Robot();
}
catch (Exception e) {
e.printStackTrace();
}
}
void draw() {
arduinoConnected = (Arduino.list().length>0);
if (arduinoConnected && arduino==null) setup();
if (arduinoConnected) {
if (arduino.digitalRead(upPin)==Arduino.HIGH) moveMouse(0,0-SPEED);
if (arduino.digitalRead(downPin)==Arduino.HIGH) moveMouse(0,SPEED);
if (arduino.digitalRead(leftPin)==Arduino.HIGH) moveMouse(0-SPEED,0);
if (arduino.digitalRead(rightPin)==Arduino.HIGH) moveMouse(SPEED,0);
int button=arduino.digitalRead(buttonPin);
if (button==Arduino.LOW && prevButton==Arduino.HIGH) clickMouse(InputEvent.BUTTON1_DOWN_MASK);
prevButton=button;
}
}
void moveMouse(int x, int y) {
PointerInfo pinfo = MouseInfo.getPointerInfo();
Point p = pinfo.getLocation();
r.mouseMove((int)p.getX()+x, (int)p.getY()+y);
}
void clickMouse(int whichType) {
r.mousePress(whichType);
r.mouseRelease(whichType);
r.waitForIdle();
}

Java Swing Frame crashes upon opening?

I am programming a chess game in java, and at the moment I am building a basic interface. It is simply an 8x8 array of buttons that will display in a window. I have coded for these buttons, and have gotten the board to display properly. However, when I connect this with the rest of the game, the game window crashes upon running and I have to force quit the java application. This is my code:
package Chess_Game;
import javax.swing.SwingUtilities;
import Chess_Interface.Iboard;
public class Game_Tester
{
public static void main(String[] args)
{
SwingUtilities.invokeLater(new Runnable()
{
public void run()
{
Game G = new Game();
Iboard I = new Iboard(G.getBoard().getArray(), G.getSides());
I.setVisible(true);
while(!(G.isGameOver()))
{
boolean redo = true;
while(redo)
{
int row = 0;
int col = 0;
int nRow = 0;
int nCol = 0;
System.out.println("Click the piece you want to move.");
while(!(I.getBool())){}
if(I.getBool())
{
row = I.getRow();
col = I.getCol();
I.setBool(false);
}
System.out.println("Click the place you want to move to.");
while(!(I.getBool())){}
if(I.getBool())
{
nRow = I.getRow();
nCol = I.getCol();
I.setBool(false);
}
if(G.canMove(row, col, nRow, nCol))
{
G.move(row, col, nRow, nCol, I);
redo = false;
}
else
{
System.out.println("You cant move there! Try again!");
}
}
I.updateBoard(G.getBoard().getArray(), G.getSides());
}
}
});
}
}
The board displays properly when I comment out the main while loop (and everything inside of it), and I assume the problem lies somewhere inside there, but I have been unable to find it. I have also looked online for similar game loop problems, but all of those have been for games involving frame rates and movement across a java swing frame, something that is not present in my code.
Any help would be greatly appreciated.

You have several loops such as
while(!(I.getBool())){}
which could potentially run forever if I does not respond as expected. You could start by printing something out within these loops, and within the following blocks if(I.getBool()){...} to see at what point your application gets stuck.
Checking the user interface in a loop like this is not good practice. It is better to use Listeners to respond to the user interface.
Nor is running the main application on the Swing thread using SwingUtilities.invokeLater(new Runnable(), even though it avoids potential problems of updating the GUI from another thread.
In fact, this may be your root problem, as running the main application loop on the Swing thread (the thread used to run the GUI) like this probably prevents the GUI from ever responding properly. You are putting a task (the entire game) onto the GUI's queue, but that task never completes while(!(G.isGameOver())).

Game "Need for Speed Most Wanted" can't controlled by Java class Robot

I wrote a bot for controlling racing game using Java Robot. The bot works well for Need For Speed Underground except for key "Down" (Up, Left, Right keys work very well). However my bot can't control Need For Speed Most Wanted.
The bot works fine, but the Game doesn't accept the simulated key events. I did some searching, and found the game is DirectX based. In DirectX the keyboard/mouse events are special. It seems that the game "asks" the keyboard directly, not through Windows. And I try my program in CS, and found it works pretty well.
I program in Windows 7, using Eclipse and Java 1.6. So I want to ask why doesn't Need for Speed Most Wanted accept the simulated key events and how to solve this program? Thank you.

I wrote a bot for controlling racing
game using Java Robot.
What KeyEvents to you generate?
For Java Robot's simulated key events
like "VK_Up,
VK_Down,Vk_Left,VK_Right", the "Need
for Speed Most Wanted" ignored. But,
for keys "A-Z", the game accepted!
Maybe you are trying to generate keyTyped events when you should be using keyPressed and keyReleased?
Here is a simple example that works with the right/left/up down keys. Try entering (1, 2, 3) then backspace to the beginning and enter (0). Then press the playback button.
import java.awt.*;
import java.awt.event.*;
import java.util.ArrayList;
import java.util.List;
import javax.swing.*;
public class RobotPlayback extends JFrame implements KeyListener, ActionListener
{
JTextField textField1;
JTextField textField2;
List playback = new ArrayList();
public RobotPlayback()
{
textField1 = new JTextField(30);
textField1.addKeyListener( this );
getContentPane().add(textField1, BorderLayout.NORTH);
//
JButton button = new JButton("Playback");
button.addActionListener( this );
button.setFocusable(false);
getContentPane().add(button);
//
textField2 = new JTextField(30);
getContentPane().add(textField2, BorderLayout.SOUTH);
}
public void keyPressed(KeyEvent e)
{
playback.add(e);
}
public void keyReleased(KeyEvent e)
{
playback.add(e);
}
public void keyTyped(KeyEvent e) {}
public void actionPerformed(ActionEvent e)
{
Thread playback = new Thread()
{
public void run()
{
playback();
}
};
playback.start();
}
private void playback()
{
textField2.requestFocus();
try
{
Robot robot = new Robot();
robot.setAutoDelay( 200 );
for (int i = 0; i < playback.size();i++)
{
KeyEvent event = (KeyEvent)playback.get(i);
if (event.getID() == KeyEvent.KEY_PRESSED)
robot.keyPress( event.getKeyCode() );
else
robot.keyRelease( event.getKeyCode() );
}
}
catch(Exception exc)
{
System.out.println(exc);
}
setVisible(true);
playback = new ArrayList();
textField1.requestFocus();
}
public static void main(String[] args)
throws Exception
{
JFrame frame = new RobotPlayback();
frame.setDefaultCloseOperation( JFrame.EXIT_ON_CLOSE );
frame.pack();
frame.setLocationRelativeTo( null );
frame.setVisible( true );
}
}
I registered the account just in this
morning and 11 reputations only. Not
enough for upvoting
Even with 11 points you should be able to "accept" an answer if it answers your question.

As far as I understand, DirectX doesn't rely on the events generated by Windows to receive input from the keyboard. Thus, firing simulated events won't work for you. I don't think you can do what you want.

The Java Robot has a bug when it comes to arrow keys. It will be pressing the num pad arrows. Currently, it is not possible to press the non num pad arrows.
Ref: http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4838497

Though this is an old question, I'll post my answer in case it helps someone out later.
I recently faced the same issue. I was calling keyPress and keyRelease right after the other for each of the left, right, up and down keys, based on some external input. I tried a bunch of games, but the Robot key presses didn't work in any of them. When I tried the same code with the cursor in a text file, the cursor moved correctly.
The problem was that games typically require continuous key presses, and so the keyPress+keyRelease combination didn't cause any effect. So I fixed the issue by calling keyRelease in the opposite direction for each arrow key press.
For instance, if the left arrow key needed to be pressed, I call
keyRelease(KeyEvent.VK_RIGHT);
keyPress(KeyEVent.VK_LEFT);
This keeps the left key pressed till the right arrow key eventually needs to be pressed. There are more optimal ways to do this by tracking the last key press, and so on, but this method seems to work just fine.

I know this is a very old post but my answer might be helpful for someone who is looking for answer
Answer:-
Its because some games uses DirectInput API for reading keyboard scan codes rather than virtual key codes. As java.awt.Robot performs keystrokes via virtual key codes not scan codes therefore keyboard events simulated using java.awt.Robot may have not effect on the game. You have to make Windows native call from your java application using JNI/JNA.
code is available here https://github.com/umer0586/winKeyboad
Keyboard keyboard = new Keyboard();
keyboard.winKeyPress(ScanCode.DIK_UP);
//Thread.sleep(1000);
keyboard.winKeyRelease(ScanCode.DIK_UP);