We Keep Coding

Is there a way in Java to be able to click through a JFrame?

I'm trying to make an overlay for a HTML-based game running in a browser window and created an JFrame which is opaque. I'd like to be able to still play the game whilst having the overlay above the window. I tried some solutions that I've found but those didn't work for me.
I've thought of catching the click-event on my JFrame and "simulating" the click on the game window. But sadly I don't have an idea how thats possible.
My current code is using the JNA libarys to access the position and scale of the window (in my test code Task-Manager).
I'm fine with using another libary or something like that, if it's even possible.
Thats my code so far:
import com.sun.jna.platform.DesktopWindow;
import com.sun.jna.platform.WindowUtils;
import javax.swing.*;
import java.awt.*;
public class Test {
public static void main(String[] args) throws InterruptedException {
JFrame frame = new JFrame("title");
frame.setUndecorated(true);
frame.setBackground(new Color(255, 69, 0, 100));
frame.setAlwaysOnTop(true);
frame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
Rectangle rect = null;
while (true) {
for (DesktopWindow desktopWindow : WindowUtils.getAllWindows(true)) {
if (desktopWindow.getTitle().contains("Task-Manager")) {
rect = desktopWindow.getLocAndSize();
frame.setSize(rect.width - 16, rect.height - 8);
frame.setLocation(rect.x + 8, rect.y);
frame.setVisible(true);
Thread.sleep(10);
}
}
}
}
}

A JFrame is a heavyweight component. There is a window in the host OS GUI system to go with it. The host OS GUI directs mouse events to the window. Perhaps using a lightweight component for your overlay and then disabling mouse events on it would be a better solution.
Your idea of catching the click event and "simulating it" on you game window should be fairly easy. If your JFrame event processing code has a reference to your game engine, it can determine the relative position of the windows and tell your game engine the corresponding point at which it should act as if it received a click. I.e. just call the same method of your game engine for click events that it received normally and also for the simulated ones.
An ugly hack (there is noticeable flicker) would be to hide the window and send the click through with the Robot class... like this:
import java.awt.AWTException;
import java.awt.Color;
import java.awt.Frame;
import java.awt.Graphics;
import java.awt.Point;
import java.awt.Robot;
import java.awt.event.InputEvent;
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import java.awt.event.MouseMotionListener;
import javax.swing.SwingUtilities;
public class ClickThrough extends Frame implements MouseListener, MouseMotionListener {
private final Robot robot;
private Color bgColor = new Color(0x80808080, true);
private Point dragPoint;
public ClickThrough() throws AWTException {
setAlwaysOnTop(true);
robot = new Robot();
}
public static void main(String[] args) throws AWTException {
ClickThrough w = new ClickThrough();
w.setUndecorated(true);
w.setSize(200, 100);
w.setOpacity(0.7f);
w.addMouseListener(w);
w.addMouseMotionListener(w);
w.setVisible(true);
}
#Override
public void paint(Graphics g) {
int w = getWidth();
int h = getHeight();
g.setColor(Color.GRAY);
g.fillRect(0, 0, w, 16);
g.setColor(bgColor);
g.fillRect(0, 16, w, h-16);
g.setColor(Color.BLACK);
g.drawString("Go ahead, click on me...", 20, 50);
}
private void makeHole(MouseEvent e) {
// Tried making a shape with a hole where the mouse was clicked,... didn't work (macOS).
//setShape(windowWithHoleShape);
setVisible(false);
}
private void repairHole(MouseEvent e) {
//setShape(windowShape);
setVisible(true);
}
#Override
public void mousePressed(MouseEvent e) {
Point p = e.getPoint();
// give it a draggable area at the top
if (p.y < 16) {
dragPoint = p;
return;
}
dragPoint = null;
SwingUtilities.convertPointToScreen(p,this);
makeHole(e);
robot.mouseMove(p.x, p.y);
robot.mousePress(InputEvent.getMaskForButton(e.getButton()));
repairHole(e);
}
#Override public void mouseReleased(MouseEvent e) { }
#Override public void mouseClicked(MouseEvent e) { }
#Override public void mouseEntered(MouseEvent e) { }
#Override public void mouseExited(MouseEvent e) { }
#Override
public void mouseDragged(MouseEvent e) {
if (dragPoint != null) {
Point p = e.getPoint();
SwingUtilities.convertPointToScreen(p, this);
p.translate(-dragPoint.x, -dragPoint.y);
setLocation(p);
}
}
#Override
public void mouseMoved(MouseEvent e) { }
}
I tried to see if I could cut a hole in the window by setting the window Shape, but at least on macOS the hole does not allow the mouse events through.
I should also point out that if you switch your GUI framework to JAvaFX, then you have the option of running your HTML-based game UI in a JavaFX WebView, so you can integrate your game and overlay into a single coherent application. You could specifically make your overlay "mouse transparent". IMO that would be a much better approach than hacking around with the mouse events.

I am not sure what is incorrect

I am creating a small Java Jpanel game in which I am supposed to have a rocket that moves up and down via arrows and fires via space.
The firing method should work like this: Space bar pressed, thing fires and moves across screen , and then when it hits a certain x, it disappears. Also, you can only fire once until the other bullet disappears.
I do not know what I am doing wrong. For one, as soon as my code starts you can see a bullet flying across the screen.
2nd, the bullet is not disappearing.
3rd, even though the other bullet is still visible, it allows me to fire again.
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.event.KeyEvent;
import java.awt.event.KeyListener;
import java.io.File;
import java.io.IOException;
import java.util.Random;
import javax.imageio.ImageIO;
import javax.swing.*;
#SuppressWarnings("serial")
public class SpaceGame extends JPanel implements ActionListener{
Timer t = new Timer(2, this);
private ImageIcon rocket,asteroid,bullet;
private JLabel rocketlabel,ast1,ast2,ast3,bulletLabel;
public static int y=90,dy=0,bulletX=110,bulletY,i=0,canFire;
//public sound sound;
static boolean bulletFired=false;;
static JFrame f = new JFrame();
SpaceGame(){
this.setBackground(Color.black);
rocket = new ImageIcon(getClass().getResource("rocketFinal.png"));
rocketlabel= new JLabel(rocket);
this.add(rocketlabel);
asteroid = new ImageIcon(getClass().getResource("asteroid.png"));
ast1=new JLabel(asteroid);
ast2=new JLabel(asteroid);
ast3=new JLabel(asteroid);
bullet = new ImageIcon(getClass().getResource("bulletReal.png"));
bulletLabel = new JLabel(bullet);
canFire=1;
bulletLabel.setVisible(false);
this.add(ast1);this.add(ast2);this.add(ast3);this.add(bulletLabel);
f.addKeyListener(new controller());
this.setLayout(null);
this.setVisible(true);
}
public class controller implements KeyListener{
#Override
public void keyPressed(KeyEvent e) {
int keyCode = e.getKeyCode();
if(keyCode== KeyEvent.VK_UP) {
dy=-1;
}
if(keyCode== KeyEvent.VK_DOWN) {
dy=1;
}
if(keyCode== KeyEvent.VK_SPACE) {
if(canFire==0) {
System.out.println(String.valueOf(canFire));
bulletFired = true;
bulletY = y;
bulletX=110;
}canFire=1;
}
}
#Override
public void keyReleased(KeyEvent e) {
int key = e.getKeyCode();
switch(key) {
case KeyEvent.VK_UP: dy=0; break;
case KeyEvent.VK_DOWN: dy=0; break;
}
}
#Override
public void keyTyped(KeyEvent e) {
// TODO Auto-generated method stub
}
}
public void paintComponent(Graphics g) {
super.paintComponent(g);
rocketlabel.setBounds(45,y,rocket.getIconWidth(),80);
fireBullet();
paintStars(g);
t.start();
}
public void paintStars(Graphics g) {
g.setColor(Color.yellow);
for(int i=0; i<5;i++) {
Random rand = new Random();
int o = rand.nextInt(500);
int p = rand.nextInt(300);
g.fillOval(o, p, 3, 3);
}
}
public void actionPerformed(ActionEvent e) {
if(y==-20) y=249;
if(y==250)y=-20;
y+=dy;
if(bulletFired=true) {
bulletX++;
if(bulletX==455)bulletFired=false;bulletLabel.setVisible(false);System.out.println(String.valueOf(bulletX)); canFire=0;
}
repaint();
}
public void fireBullet(){
if(bulletFired=true) {
bulletLabel.setVisible(true);
bulletLabel.setBounds(bulletX,bulletY+25,bullet.getIconHeight(),bullet.getIconWidth());
}
}
public static void main(String[] args) {
String filepath = "SpaceGameMusic.wav";
musicStuff musicPlayer = new musicStuff();
musicPlayer.playMusic(filepath);
SpaceGame t = new SpaceGame();
f.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
f.getContentPane().add(t);
f.setSize(500,335);
f.setVisible(true);
f.setResizable(false);
}
}

For one, as soon as my code starts you can see a bullet flying across the screen.
The paintComponent() method is for painting only. You can't control when Swing will determine a component needs to be repainted.
So, for example:
t.start();
should NOT be in the painting method. As soon as the frame is made visible the panel will be painted and the Timer will be started.
You application code should control when the Timer is started.
Other issues:
you should not be using static variables. The variable should simply be instances of your class.
the paintStars() method should not generate random locations. Again. a painting method should only paint the current state of the class. So if you want to change the location of the stars you should have a method like randomizeStars(). In this method you would update an ArrayList of Point objects. Each Point instance would represent the location of a star. Then the paintStars() method would simply iterate through the ArrayList and paint each star.
you should not be using a KeyListener. A KeyListener only works if a component has focus. You can't guarantee that your component will lose focus. Instead you should be using Key Bindings. Key bindings allow you to handle a KeyEvent even if the component doesn't have focus. See Motion Using the Keyboard for more information and a working example.
you can only fire once until the other bullet disappears
Your canFire variable should be a boolean variable so it only has true/false values. Again you have a method that sets the state. Your game logic will then check the state before firing the bullet again.
if(y==-20) y=249;
if(y==250)y=-20;
Don't hardcode values. The number should be based on the size of your panel. So you use methods like getWidth() and getHeight() to determine the current size of the panel.

The problem was quite simply that I had forgotten to use == in my if(boolean) statements.

Java: Repaint in Swing Not Working

I am learning java swing and am having trouble with the following program. It creates a small frame with a quit button at top. The objective is to display coordinates wherever the mouse is clicked. When I click the mouse 2 unwanted things are happening:
the quit button is overridden by the mouse clicks and it no longer responds (instead of responding to event and quitting, it displays coordinates on top of the quit button).
when I click at a new location, the coordinates from the old location persist.
I used removeAll() and revalidate() before doing repaint() based on this discussion but that has not helped. This code is taken from here and the code to says to research online documentation for why this is happening.
Any pointers?
import java.awt.*;
import java.awt.event.*;
import javax.swing.*;
import java.awt.Color;
import java.awt.event.ActionListener;
import java.awt.event.ActionEvent;
import java.awt.BorderLayout;
import java.awt.Graphics;
import javax.swing.JFrame;
import javax.swing.JButton;
import javax.swing.JLabel;
public class QuitCoordinateTest {
public static void main(String[] args){
GUI gui = new GUI();
}
}
class MyFrame extends JFrame implements ActionListener{
int clickX;
int clickY;
public void paint(Graphics g){
g.drawString("" + clickX + ", " + clickY, clickX, clickY);
}
public void actionPerformed(ActionEvent e){
System.exit(0);
}
}
//=======================================================//
class GUI extends MyFrame {
JButton quitButton = new JButton("Quit");
public GUI(){
MyFrame displayWindow = new MyFrame();
displayWindow.setTitle("Title");
/*
JPanel buttonPanel = new JPanel();
buttonPanel.add(quitButton);
displayWindow.getContentPane().add(buttonPanel,BorderLayout.NORTH);
JPanel textPanel = new JPanel();
*/
displayWindow.getContentPane().add(quitButton,BorderLayout.NORTH);
quitButton.addActionListener(displayWindow);
displayWindow.setSize(201,201);
displayWindow.setVisible(true);
// displayWindow.pack();
displayWindow.setDefaultCloseOperation(JFrame.DO_NOTHING_ON_CLOSE);
displayWindow.addMouseListener(new MouseProc(displayWindow));
}//end constructor
}//end class GUI definition
//=======================================================//
//This listener class monitors for mouse presses and
// displays the coordinates of the mouse pointer when the
// mouse is pressed on the source object.
class MouseProc extends MouseAdapter{
MyFrame refToWin;
MouseProc(MyFrame inWin){
refToWin = inWin;
}
//Override the mousePressed method to determine and
// display the coordinates when the mouse is pressed.
public void mousePressed(MouseEvent e){
refToWin.removeAll();
refToWin.clickX = e.getX();
refToWin.clickY = e.getY();
//Force the JFrame object to be repainted in order to
// display the coordinate information.
refToWin.removeAll();
refToWin.validate();
refToWin.repaint();
}
}

repaint() is working fine.
Avoid drawing directly on the JFrame.
Instead draw in the protected void paintComponent(Graphics g) method override of a JPanel that is then displayed in your JFrame.
Be sure to call the super's paintComponent(g) method inside of your paintComponent override -- this will erase the old images and is the reason for one of your problems.
Use reasonable comments in your code. Too many comments and too much text distracts and makes understanding your code harder, not easier.
Calling removeAll() on your JFrame will do just that -- remove all components including your button. Why are you calling this? Are you sure that you want to call this method?
A minor nitpick -- you'll want to avoid directly setting the fields of another class, such as your clickX and clickY fields. Instead, make them private, and only allow outside classes to modify them through public methods. While it may not matter much for this small program, it will matter greatly when you start scaling up your programming and create large programs with complex interactions. The key to success here will be to limit and control all communication between classes to avoid hard to detect side effects.
For example, something like...
#Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
String str = String.format("[%d, %d]", clickX, clickY);
g.drawString(str, clickX, clickY);
}
public int getClickX() {
return clickX;
}
public void setClickX(int clickX) {
this.clickX = clickX;
}
public int getClickY() {
return clickY;
}
public void setClickY(int clickY) {
this.clickY = clickY;
}
For example
import java.awt.*;
import java.awt.event.MouseAdapter;
import java.awt.event.MouseEvent;
import javax.swing.*;
#SuppressWarnings("serial")
public class DetectClicks extends JPanel {
private static final int PREF_W = 800;
private static final int PREF_H = 650;
private int clickX;
private int clickY;
public DetectClicks() {
MyMouseListener mouseAdapter = new MyMouseListener(this);
addMouseListener(mouseAdapter);
addMouseMotionListener(mouseAdapter); // to allow dragging!
}
#Override
public Dimension getPreferredSize() {
return new Dimension(PREF_W, PREF_H);
}
#Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
String str = String.format("[%d, %d]", clickX, clickY);
g.drawString(str, clickX, clickY);
}
public int getClickX() {
return clickX;
}
public void setClickX(int clickX) {
this.clickX = clickX;
}
public int getClickY() {
return clickY;
}
public void setClickY(int clickY) {
this.clickY = clickY;
}
private static void createAndShowGui() {
DetectClicks mainPanel = new DetectClicks();
JFrame frame = new JFrame("DetectClicks");
frame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
frame.getContentPane().add(mainPanel);
frame.pack();
frame.setLocationByPlatform(true);
frame.setVisible(true);
}
public static void main(String[] args) {
SwingUtilities.invokeLater(new Runnable() {
public void run() {
createAndShowGui();
}
});
}
}
class MyMouseListener extends MouseAdapter {
private DetectClicks detectClicks;
public MyMouseListener(DetectClicks detectClicks) {
this.detectClicks = detectClicks;
}
#Override
public void mousePressed(MouseEvent evt) {
showPoint(evt);
}
#Override
public void mouseDragged(MouseEvent evt) {
showPoint(evt);
}
private void showPoint(MouseEvent evt) {
detectClicks.setClickX(evt.getX());
detectClicks.setClickY(evt.getY());
detectClicks.repaint();
}
}

Your event is getting consumed by the handler that prints the coordinates, you need to redispatch the event so that the button can see it. You can do it like this, inside the coordinate display event handler:
Component c = e.getComponent();
c.getParent().dispatchEvent( e );
Also, I'd be tempted to use the glass pane of the frame, and put a JLabel on it with the co-ordinates rather than messing with the paint method.

you don't have to use any of repaint(),invalidate() etc.
i highly recommend to use
SwingUtilities.invokeLater(new Runnable() {
public void run() {
//TODO udpdate UI compontents, layouts etc.
}
});
this guarantees that UI components update on real time. Because we don't know when the system update UI hierarchy so we can't force it. This allow system to determine by it's self.

Manual Positioning of Image in JFrame

I am trying to create a JFrame that displays an image from a file path onto a particular position on the JFrame. At a later time (when a button is clicked), I want the image to move positions, say, 50 pixles to the left. If a layout manager is necessary, I want to use the null layout, as this is a project for myself and I am not quite ready to learn how to write my own layout manager.
So far, I have managed to display a BufferedImage in a frame, but I do not know how to specify its position.
Is using a BufferedImage even the correct approach? What is the best way to go about doing this?
Update: I tried to follow your suggestion of using mouselistener and it resulted in this:
class ImgComponent extends JComponent implements ChangeListener, MouseListener {
MovableImage mi;
public ImgComponent(MovableImage mi) {
this.mi = mi;
mi.addListener(this);
mi.addListener1(this);
}
#Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
g.drawImage(mi.i, mi.getX(), mi.getY(), null);
}
#Override
public void stateChanged(ChangeEvent e) {
repaint();
}
#Override
public void mouseClicked(MouseEvent e) {
mi.setPos(100, 100);
System.out.println("yay");
}
}
But unfortinely, the mouseClicked event never triggers. I just want that damn image to move, lol.

Here's a complete example that uses the model/view/controller pattern. (Just dump all snippets after each other in a single .java file.)
import java.awt.*;
import java.awt.event.ActionEvent;
import java.util.*;
import java.util.List;
import javax.swing.*;
import javax.swing.event.*;
// A class encapsulating an image and a x-coordinate (a "model")
class MovableImage {
Image i = new ImageIcon("duke.png").getImage();
private int x = 0;
// Observers that are interested in movements.
List<ChangeListener> listeners = new ArrayList<ChangeListener>();
public void addListener(ChangeListener cl) {
listeners.add(cl);
}
public int getX() {
return x;
}
public void incrementX() {
x += 10;
// Notify those interested.
for (ChangeListener cl : listeners)
cl.stateChanged(null);
}
}
// A graphical component displaying the model.
// Object of this class are interested in movement because when the image moves,
// this component needs to be repainted.
class ImgComponent extends JComponent implements ChangeListener {
// The movable image to present.
MovableImage mi;
public ImgComponent(MovableImage mi) {
this.mi = mi;
mi.addListener(this);
}
#Override
protected void paintComponent(Graphics g) {
super.paintComponent(g);
g.drawImage(mi.i, mi.getX(), 10, null);
}
// This method is called from MovableImage when the position changes.
#Override
public void stateChanged(ChangeEvent e) {
repaint();
}
}
// Main class.
public class FrameTestBase extends JFrame {
public static void main(String args[]) {
// Create the "model".
final MovableImage mi = new MovableImage();
FrameTestBase t = new FrameTestBase();
t.setLayout(new BorderLayout());
// Add a component presenting the model.
t.add(new ImgComponent(mi), BorderLayout.CENTER);
// Create a button which increments x when clicked on.
t.add(new JButton(new AbstractAction("Move right") {
#Override
public void actionPerformed(ActionEvent e) {
mi.incrementX();
}
}), BorderLayout.SOUTH);
// Show it.
t.setDefaultCloseOperation(EXIT_ON_CLOSE);
t.setSize(400, 400);
t.setVisible(true);
}
}
Regarding your edit:
You need to add the mouse listener as well. In the constructor:
public ImgComponent(MovableImage mi) {
this.mi = mi;
mi.addListener(this);
mi.addListener1(this);
}
add the following line at the bottom:
addMouseListener(this);

Mouse Pointer Problem in Java Swing

I have created the following simple Java Swing program which outputs a 3*3 square in the window every time the user clicks their mouse. The squares remain in the window even if the user clicks more than once. The program compiles and runs just fine, however, when one clicks in the window the square is drawn far below where the mouse pointer is. I've been racking my brain over this one for a while -- what can I change here to get the square to appear exactly with the pointer on each click? Many thanks for any help!
import java.awt.Graphics;
import java.awt.Graphics2D;
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import java.awt.geom.Point2D;
import java.awt.geom.Rectangle2D;
import java.util.ArrayList;
import javax.swing.JComponent;
import javax.swing.JFrame;
public class ClickCloud extends JComponent {
final ArrayList<Point2D> points = new ArrayList<Point2D>();
public void addPoint(Point2D a) {
points.add(a);
}
public void paintComponent(Graphics g) {
Graphics2D g2 = (Graphics2D) g;
for (int i = 0; i < points.size(); i++) {
Point2D aPoint = points.get(i);
g2.draw(new Rectangle2D.Double(aPoint.getX(), aPoint.getY(), 3, 3));
}
}
public static void main(String[] args) {
final ClickCloud cloud = new ClickCloud();
JFrame aFrame = new JFrame();
class ClickListen implements MouseListener {
#Override
public void mouseClicked(MouseEvent arg0) {
}
#Override
public void mouseEntered(MouseEvent arg0) {
}
#Override
public void mouseExited(MouseEvent arg0) {
}
public void mousePressed(MouseEvent arg0) {
cloud.addPoint(arg0.getPoint());
cloud.repaint();
}
#Override
public void mouseReleased(MouseEvent arg0) {
}
}
aFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
aFrame.setSize(500, 500);
aFrame.add(cloud);
aFrame.addMouseListener(new ClickListen());
aFrame.setVisible(true);
}
}

You're adding the MouseListener to the JFrame, but displaying the results in the JComponent and relative to the JComponent. So the location of the Point clicked will be relative to the JFrame's coordinates, but then displayed relative to the JComponent's coordinates which will shift things down by the distance of the title bar. Instead simply add the MouseListener to the same component that is responsible for displaying the results so that the display and clicking coordinates match:
aFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
aFrame.setSize(500, 500);
aFrame.add(cloud);
//!! aFrame.addMouseListener(new ClickListen()); // !! Removed
cloud.addMouseListener(new ClickListen()); // !! added
aFrame.setVisible(true);
By the way: Thanks for creating and posting a decent SSCCE as this makes it so much easier to analyse and solve your problem.