Control robot using keyboard in JAVA - java

I started to program a NAO Robot in IntelliJ IDEA and I am new in both, I created some functions that makes NAO to move or to speak, every time to see what is happening I have to run project. Now I want to execute some functions on KeyPress. Could you give me an example.
How to change this code, to allow NAO to Stand, Crouch or Sit on some KeyPress (ex: q-> Stand, w-> Crouch, e-> Sit)
package test;
import com.aldebaran.qi.Application;
import com.aldebaran.qi.helper.proxies.ALMotion;
import com.aldebaran.qi.helper.proxies.ALRobotPosture;
import com.aldebaran.qi.helper.proxies.ALTextToSpeech;
public class StandNao {
private static ALMotion motion;
public static void main(String[] args) throws Exception {
Application application = new NaoSettings().NaoConnect(args);
application.start();
motion = new ALMotion(application.session());
// Create an ALTextToSpeech object and link it to your current session
ALTextToSpeech tts = new ALTextToSpeech(application.session());
// Make your robot say something
tts.say("MAC Start Stand");
motion.killAll();
ALRobotPosture posture = new ALRobotPosture(application.session());
posture.getPostureList();
posture.goToPosture("Stand", 1.0f);
Thread.sleep(10000);
posture.goToPosture("Crouch", 1.0f);
Thread.sleep(10000);
posture.goToPosture("Sit", 1.0f);
}
}

I'll not describe all the possibilities over, but to give you a way to go, you can check many Java libraries who will let you manage poll or event driven keyboard input.
You can then make an infinite loop where you would listen to the keyboard activity, and react depending on the key hit.
It would barely looks like something like that:
Keyboard.poll();
while(Keyboard.next()) {
if(Keyboard.getEventKey() == Keyboard.KEY_LEFT && !Keyboard.getEventKeyState()) {
// do something if the letter left arrow key is released
}
}
Also consider trying to make funny things, if you want to learn both Java and Web aspects, like a Web Application with REST controller, and when you hit buttons a web page, your robot move accordingly.
It can be quickly implemented by using Java Spring Boot.

Related

Java Robot class won't interact with an application?

I've been reading up on this and haven't found an answer that has made sense to me.
I'm trying to write a program in Java to interact with an application to see if I can write a program to play a video game for me. The game is on my computer.
Here is an excerpt of code:
public static void main(String[] args) throws java.io.IOException {
Runtime run = Runtime.getRuntime();
run.exec("open /Applications/OpenEmu.app");
try {
Robot robot = new Robot();
System.out.println("Waiting 5 Seconds");
//robot.delay(5000);
System.out.println("Pressed X");
robot.keyPress(KeyEvent.VK_X);
robot.keyPress(KeyEvent.VK_X);
robot.keyPress(KeyEvent.VK_X);
robot.keyPress(KeyEvent.VK_X);
//Starts an easy mode game
It opens the application fine, and in something like notepad, it will type XXXX, but it won't do so for the game?
I've assigned the 'x' key on my keyboard as a command button for the game. My guess is that the 'x' press is internal. All help is appreciated!
If you are trying to simulate input, try add robot.keyRelease as well. Javadoc for robot says for keyPress "Presses a given key. The key should be released using the keyRelease method."
System.out.println("Pressed X");
robot.keyPress(KeyEvent.VK_X);
robot.keyRelease(KeyEvent.VK_X);
...
Also remember this:
https://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html "Note that some platforms require special privileges or extensions to access low-level input control. "

How to close external programs (Powerpoint) running on the native desktop using Java?

I have to create an application that will automatically open a powerpoint file, let it play through, and then close it. Not only do I need to figure out HOW to close it, but I also must detect when it closes or stops.
First option:
I know how long each powerpoint will play for, so I can hardcode when to close the file. I just need to know how to do that. There are no methods in the desktop class (that I could find) for closing.
Second option:
If someone knows a microsoft powerpoint api that lets me open powerpoints and use java to progress through the slideshow and get the state or something, that'd be great. I wouldn't have to go into each presentation and count the number of slides and the transition timer on each slide.
The opening, letting it play, and closing it is a small part of the app I need to create. But here is what I have so far with regards to THIS problem:
File myfile = new File("PowerPoint.ppsx");
try {
Desktop.getDesktop().open(myfile);
} catch (IOException ex) {
Logger.getLogger(Sc.class.getName()).log(Level.SEVERE, null, ex);
}
Probably this is the solution how to close external program:
http://www.java-forums.org/new-java/59691-close-another-program.html#post285956
If you want to detect when program has stopped running then you can start new thread with loop which from time to time will check if the program process is still running, using the same method as mentioned in link.
This is solution only for one (Windows) platform, Java is not the best choice for such tasks.
Here a solution using JNA. First we get the handle, we search using the "class name" of the window. You can determine the class name for a specific program (in this case Powerpoint) with a special utility like Spy++ (included with Visual Studio). It's possible to make the search more precise using the class name and the window caption (but here I use only the class name) so if you have more than one presentation running ... you may not close the good one!.
import com.sun.jna.platform.win32.User32;
import com.sun.jna.platform.win32.WinUser;
import com.sun.jna.platform.win32.WinDef.HWND;
// https://github.com/twall/jna#readme
// you need 2 jars : jna-3.5.1.jar and platform-3.5.1.jar
public class KillMyPP {
public static void main(String[] args) {
HWND hwnd = User32.INSTANCE.FindWindow("screenClass", null);
if (hwnd == null) {
System.out.println("PPSX is not running");
}
else {
User32.INSTANCE.PostMessage(hwnd, WinUser.WM_QUIT, null, null);
}
}
}

Eclipse RCP IPageLayout woes

I have an Eclipse RCP application with a sort of three column layout:
The editor area is at the extreme right. Now, when you get an IPageLayout to work with, the editor area is already added in. That's fine: we add area B to the left of the editor, and area A to the left of B, and the layout is exactly what we need.
The issue is that when you move the sash between A and B, views A and B change without resizing the editor area (good;) but when you move the other sash between B and the editor area, all three views are resized; the layout manager acts to maintain the ratio of the widths of A and B, and that's not what we want. We want the user to be able to move each sash independently, and have it influence only the two views it touches.
It seems like the root cause of this is that the editor is in place when you get your IPageView, and therefore you have to position the IFolderLayouts relative to it. If you could position the editor relative to B, instead, then resize would do the right thing.
So my questions:
Is there any way to tell the IPageView to position the editor relative to a view, instead of the other way around?
Barring that, is there any other way to influence the layout algorithm, like writing some kind of layout manager?
I know of no way to alter the layout tree of IPageLayout in Eclipse 3.x. In Eclipse 4.2, however, the Application Model can be changed dynamically at runtime.
So, if you would consider migrating your application to Eclipse 4, this solution could be an option. To keep the original application and UI code as untouched as possible, this solution will
take full advantage of the compatibility layer of Eclipse 4 to create an Application Model from the Eclipse 3 based RCP application. There is no need to create an Application Model or alter the UI code of the application.
rearrange the editor area's layout after the application is active. This is done by creating an addon class in a separate plugin.
allow easy migration to more Eclipse 4 functionality in the future: Should you decide to build an own Application Model, you can just unhook the addon plugin.
I started with the regular RCP Mail template of Eclipse 3 and altered the perspective to recreate the problem. This is the Perspective class I used in my test application:
import org.eclipse.ui.IPageLayout;
import org.eclipse.ui.IPerspectiveFactory;
public class Perspective implements IPerspectiveFactory {
public static final String ID = "wag.perspective";
public void createInitialLayout(IPageLayout layout) {
String editorArea = layout.getEditorArea();
layout.setEditorAreaVisible(true);
layout.addStandaloneView(AView.ID, false, IPageLayout.LEFT,
0.25f, editorArea);
layout.addStandaloneView(BView.ID, false, IPageLayout.LEFT,
0.25f, editorArea);
layout.getViewLayout(AView.ID).setCloseable(false);
layout.getViewLayout(BView.ID).setCloseable(false);
}
}
It basically creates the scenario you described: a three column layout where one sash effects all three parts and the other one only two.
I then proceeded to migrate the application and alter the Application Model.
Migrate the Eclipse 3 based RCP application to Eclipse 4
There are online tutorials available for this process. I found Eclipse 4.1: Run your 3.x RCP in 4.1 and Eclipse 4 and the Compatibility Layer - Tutorial to be very helpful.
I recommend including the org.eclipse.e4.tools.emf.liveeditor and its required plug-ins in your product dependencies. With the live editor, you can take a look at the Application Model that is created by the compatibility layer.
Once the application starts, thet sashes will still behave the same way. Open the live editor on your application window and take a look at your model.
You can see that the PartSashContainer including the placeholder for the AView contains another PartSashContainer. Moving the sash between AView and that container will update the rest of the layout tree, while moving the sash between BView and the editor does not effect other parts of the layout.
You could now drag the placeholder for the AView to the container where the BView and the editor are located. This would instantly create the effect you desire: The sashes will only affect their direct neighbours. But these changes will only be saved in one's own runtime workspace. Something else is needed to alter the layout structure automatically.
Altering the Application Model at runtime
Since I didn't want to touch the original code if possible, I created another plugin to make a contribution to the Application Model.
Create a Plug-In Project without an Activator without using a template.
Add an Addon class: select New->Other->Eclipse 4->Classes->New Addon Class
Add a Model Fragment: select New->Other-Eclipse 4->Model->New Model Fragment. Open the created fragment.e4xmi file and add a Model Fragment. For the Element Id, put org.eclipse.e4.legacy.ide.application (this is the standard id of legacy applications) and for the Featurename addons. Add an Addon to the Model Fragment. Enter an ID and set the Class URI to your addon class.
Now add your fragment.e4xmi to your org.eclipse.e4.workbench.model extension point:
<extension
id="id1"
point="org.eclipse.e4.workbench.model">
<fragment
uri="fragment.e4xmi">
</fragment>
</extension>
Add your contribution plugin to the dependencies of your application product. When you start your application and look at the model with the live editor, you should see your Addon listed in the model.
Now we can implement the Addon. This is the code of my Addon class:
package wag.contribution.addons;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;
import javax.annotation.PostConstruct;
import javax.annotation.PreDestroy;
import javax.inject.Inject;
import org.eclipse.e4.core.services.events.IEventBroker;
import org.eclipse.e4.ui.model.application.MApplication;
import org.eclipse.e4.ui.model.application.ui.MElementContainer;
import org.eclipse.e4.ui.model.application.ui.MUIElement;
import org.eclipse.e4.ui.model.application.ui.advanced.MPlaceholder;
import org.eclipse.e4.ui.workbench.modeling.EModelService;
import org.osgi.service.event.Event;
import org.osgi.service.event.EventHandler;
public class LayoutSorter {
#Inject private IEventBroker broker;
private EventHandler handler;
// The part IDs we are interested in, sorted in the sequence they should be
// shown
private static List<String> PART_IDS = Arrays.asList(new String[] {
"wag.aView", "wag.bView", "org.eclipse.ui.editorss" });
// Listen to the e4 core service's event broker to find the magical time
// when the application is created and try to sort the layout.
#PostConstruct
void hookListeners(final MApplication application,
final EModelService service) {
if (handler == null) {
handler = new EventHandler() {
// Try to sort the layout. Unsubscribe from event broker if
// successful.
#Override
public void handleEvent(Event event) {
try {
sort(application, service);
// sort did finish: stop listening to the broker.
broker.unsubscribe(handler);
} catch (Exception e) {
// Something went wrong, the application model was not ready yet.
// Keep on listening.
}
}
};
// Subscribe "ServiceEvent.MODIFIED" to grab the application.STARTED
// event. Does anybody know how to do this in a better way?
broker.subscribe("org/osgi/framework/ServiceEvent/MODIFIED",
handler);
}
}
private void sort(MApplication application, EModelService service) {
// find all placeholders
List<MPlaceholder> placeholders = service.findElements(application,
null, MPlaceholder.class, null);
// only keep the ones we are interested in
for (int i = placeholders.size() - 1; i > -1; i--) {
if (!PART_IDS.contains(placeholders.get(i).getElementId())) {
placeholders.remove(i);
}
}
// find the parents of the placeholders
List<MElementContainer<MUIElement>> parents = new ArrayList<>(
placeholders.size());
for (MPlaceholder placeholder : placeholders) {
parents.add(placeholder.getParent());
}
// find the parent that is "deepest down" in the tree
MElementContainer<MUIElement> targetParent = null;
for (MElementContainer<MUIElement> parent : parents) {
for (MUIElement child : parent.getChildren()) {
if (parents.contains(child)) {
continue;
}
targetParent = parent;
}
}
// move all parts to the target parent
if (targetParent != null) {
for (int i = 0; i < placeholders.size(); i++) {
if (targetParent != placeholders.get(i).getParent()) {
service.move(placeholders.get(i), targetParent, i);
}
}
}
}
#PreDestroy
void unhookListeners() {
if (handler != null) {
// in case it wasn't unhooked earlier
broker.unsubscribe(handler);
}
}
}
(Please note that the code above is a bit of a hack because it is only really suited for this specific problem.)
After a restart, the application should now behave in the desired way. Take a look at the Application Model to see your changes.
One thing to be aware of is that local changes are saved in the runtime workspace in the file .metadata\.plugins\org.eclipse.e4.workbench\workbench.xmi if saving is switched on, so for recreating the unaltered model for testing this file has to be deleted.
I don't think, it's possible to achieve exactly what you want (so the answers to your questions would be 1. no, 2. no). But there it a 3rd alternative, which IMO behaves quite nicely.
When trying in Eclipse: Start with viewA on left and Editor on right. Then when you drag viewB to the right side of viewA, you get the (wrong) setup you describe. But then you drag it to the left part of the Editor, then you get different configuration, where dragging right sash behaves as you want. Dragging of left sash resizes viewA and Editor and MOVES viewB.
I would say that the code to achieve this would be:
IFolderLayout areaA = layout.createFolder("A", IPageLayout.LEFT, 0.33f, editorArea);
IFolderLayout areaB = layout.createFolder("B", IPageLayout.LEFT, 0.5f, editorArea);

keyReleases are simulating keyPresses in Linux (java Swing GUI)

I have a kiosk GUI application I'm working on and it requires me to block users from being able to Alt-Tab out of the fullscreen window. I posted a question about this a while back and a member helped me with some code, which worked perfectly under a Windows environment.
Here it is:
public class TabStopper implements Runnable {
private boolean isWorking = false;
private MenuFrame parent;
public TabStopper(MenuFrame parent) {
this.parent = parent;
new Thread(this, "TabStopper").start();
}
public void run() {
this.isWorking = true;
Robot robot;
try {
robot = new Robot();
while (isWorking) {
robot.keyRelease(KeyEvent.VK_ALT);
robot.keyRelease(KeyEvent.VK_TAB);
parent.requestFocus();
Thread.sleep(10);
}
} catch (Exception e) {
e.printStackTrace();
}
}
public void stop() {
this.isWorking = false;
}
public boolean isWorking() {
return this.isWorking;
}
}
However, I need this to be able to work in Linux as well. I made an executable jar from the source and brought it into Linux. Everything worked except the Alt and Tab keys were being constantly pressed. The buttons on my GUI were constantly being cycled and I was able to open a terminal (I set a backdoor in the application during testing in case something like this happens) which wouldn't let me type anything because Tab lists all the files in the current directory.
Could anyone tell me if there would be a fix that would work in both Linux and Windows environments. However, if I had to choose, I would go for Linux.
EDIT: I can also confirm that the Alt key is being "pressed". What's with this weird behaviour?
Forget grabbing Alt+Tab with hacks like this. It is a bad hack and it is error-prone. There are also so many other hotkey combinations.
For linux you have two options:
Use a minimal window manager or no window manager at all. For example, with fluxbox you can remove all key bindings alltogether and you can also make your application maximise by default, etc. You can empty the desktop menus such that the user gains no control even when your application crashes. This is a clean solution that really solves your problem instead of some parts of it. There are many ways to fiddle with the system other than Alt+Tab.
Grab input controls completely. This is what games do. For example libSDL does it for you and there are java wrappers for the functionality as well. This should also work as expected, except you use a window manager that does not allow input control grabbing per default (I don't know of any).

can't detect that windows classic theme active

I am calling UIManager.getSystemLookAndFeelClassName().
And get as result WindowsLookAndFeel, even when setting current theme to WindowsClassicLookAndFeel. But I expect WindowsClassicLookAndFeel.
Question is how can I detect that windows classic theme is now active
It looks like you signed up specifically to ask this question, and now I'm signing up specifically to answer it! I was Googling for something completely different but I was intrigued and did some experimenting, and here's what I found:
You're right about the WindowsClassicLookAndFeel. This class extends WindowsLookAndFeel but doesn't override anything, and doesn't appear to be used at all, even when Windows Classic is active. So, I looked into the code of WindowsLookAndFeel and found some interesting internal code that references the package-private class XPStyle. This class appears to be a singleton and the getter method, getXP(), only returns its instance if the 'XP' theme is active:
/**
* Get the singleton instance of this class
*
* #return the singleton instance of this class or null if XP styles
* are not active or if this is not Windows XP
*/
static synchronized XPStyle getXP() {
if (themeActive == null) {
Toolkit toolkit = Toolkit.getDefaultToolkit();
themeActive =
(Boolean)toolkit.getDesktopProperty("win.xpstyle.themeActive");
if (themeActive == null) {
themeActive = Boolean.FALSE;
}
if (themeActive.booleanValue()) {
GetPropertyAction propertyAction =
new GetPropertyAction("swing.noxp");
if (AccessController.doPrivileged(propertyAction) == null &&
ThemeReader.isThemed() &&
!(UIManager.getLookAndFeel()
instanceof WindowsClassicLookAndFeel)) {
xp = new XPStyle();
}
}
}
return xp;
}
Interestingly, the code checks for the WindowsClassicLookAndFeel again but we know that this is no longer used (perhaps it's changed)... But the best part is the check for the desktop property win.xpstyle.themeActive Extracting this from the code, I made the following test:
package test;
import java.awt.Toolkit;
import java.beans.PropertyChangeEvent;
import java.beans.PropertyChangeListener;
import javax.swing.UIManager;
public class WindowsClassicThemeDetector {
public static boolean isWindowsLAF() {
return UIManager.getLookAndFeel().getID().equals("Windows");
}
public static boolean isWindowsClassicLAF() {
return isWindowsLAF()
&& !(Boolean) Toolkit.getDefaultToolkit().getDesktopProperty(
"win.xpstyle.themeActive");
}
public static void main(String... args) throws Exception {
// Apply the system look and feel (which will be Windows)
UIManager.setLookAndFeel(UIManager.getSystemLookAndFeelClassName());
// Check the initial theme state on startup
System.out.println(UIManager.getLookAndFeel().getClass());
System.out.println("Windows classic is initially: " + isWindowsClassicLAF());
// Register a listener in case the theme changes during runtime.
Toolkit.getDefaultToolkit().addPropertyChangeListener("win.xpstyle.themeActive",
new PropertyChangeListener() {
#Override
public void propertyChange(PropertyChangeEvent evt) {
System.out.println("Windows classic is now: " + isWindowsClassicLAF());
}
});
// Wait until user presses ENTER in the console and then exit.
System.in.read();
}
}
You can import this class into your program and call the isWindowsClassicLAF() method after setting your Look&Feel at any point. I've also given an example of how you can listen out for changes to the theme at run-time.
This is tried and tested on XP. If the user changes from one XP theme to another, the listener doesn't fire, but if the user changes from a theme to Classic, or vice-versa, it will.
I hope that helps!
I'm not quite sure what you are asking.
If you are asking which theme/look and feel SWING is currently using, try
UIManager.getSystemLookAndFeelClassName().
If you are trying to find out which theme Windows is using - I don't know. Probably not quite easy to find that out.
Edit: A quick hack might be (apart from using JNA/JNA to query some Windows API directly which theme is being used) would be creating a new JFrame, place it in some area, capture a small part of that border and compare it with samples of borders (which you need to create beforehand and ship with your code, as to be able to programmatically compare the just-taken-screen-capture-bit and all border-images you ship)
UIManager.getLookAndFeel() returns currently installed LaF. getSystemLookAndFeel() returns look and feel that would look as a current system's theme, e.g. WindowsLookAndFeel on windows GTKLookAndFeel on Linux MOtifLookAndFeel on Solaris etc.

Categories