How does DRMAA work? Can a local Java program using DRMAA start jobs on a remote cluster over SSH (so that nothing will need to be installed on the server-side)?
Background:
I'm developing a general (or as general as possible) HPC client in Java/Eclipse RCP, and
wanted to use DRMAA in order to support any resource manager as backend.
I have the SSH connection functionality through the Remote System Explorer (RSE) Eclipse plugin already.
Most of the DRMAA implementations use the native API calls (the same which are used inside the qsub/bsub/sbatch... commands). One can see DRMAA as the "ODBC for the batch systems".
Most of the DRMAA implementations require you to run it from submit host of your local cluster, there is no SSH inside. What you can try to do is to build a portable, DRMAA based "drmaa-run" command (example: http://apps.man.poznan.pl/trac/drmaa-misc/browser/drmaa_utils/trunk/drmaa_utils/drmaa_run.c) and run it via SSH.
Related
I have a machine that runs batch scripts over ssh on windows machine using open ssh and cygwin (copssh)
I'm looking to change this mechanism since this tool requires configuration where many mistakes can be made.
I will also need to collect the results of the script
Any ideas on how it can be done?
Thanks
You can use RMI and on the remote side just use Runtime.getRuntime().exec
I am trying to make a terminal emulator in Java. The java program will accept the commands from user, and show its output to them. I can emulate simple commands like 'ls', but I don't know how to handle commands like 'cd'. This is because, I am using exec() method for executing terminal commands. So, all the commands are executed at current directory. The commands like 'cd ..' are executed, but then they have no persistent effect, because each command is separately executed by exec().
Any Ideas How I can emulate a whole session??
If you are executing commands with exec(), you are not writing a terminal emulator; you are writing a shell. In that case, you will need to keep track of things the shell keeps track of, like environment variables and working directory.
If you really want to write a terminal emulator, you would be talking to a shell process through a pseudo-terminal. Then your program would just be keeping track of the things a terminal keeps track of, like the line state and what appears on the screen.
Working with a pseudo-terminal from Java will be a little tricky, because most of the documentation assumes you are using a C api. man pty should get you started. Your Java process will have to open the master side of the pseudo-terminal with FileStream objects. I'm not sure there is a way within Java to get a child process to open the slave side of the pseudo-terminal; you might have to invoke a shell command with exec() that starts another shell command with standard input/output/error redirected to the slave side of the pseudo terminal.
JSch is a pure Java implementation of SSH2.
JSch allows you to connect to an sshd server and use port forwarding, X11 forwarding, file transfer, etc., and you can integrate its functionality into your own Java programs.
http://www.jcraft.com/jsch/
You should really give a try to Ganymed.
Ganymed SSH-2 for Java is a library which implements the SSH-2
protocol in pure Java (tested on J2SE 1.4.2 and 5.0). It allows one to
connect to SSH servers from within Java programs. It supports SSH
sessions (remote command execution and shell access), local and remote
port forwarding, local stream forwarding, X11 forwarding, SCP and
SFTP.
http://www.ganymed.ethz.ch/ssh2/
Ganymed along with apache FTP client you can also download and upload files.
Also there is a inbuilt example code for terminal emulation in Ganymed.
The following is a link to a project which is did using Ganymed along with apache FTP client.
GITHUB
Happy Coding!!
I would like to debug my Java application usually locally, but sometimes on a remote server. I was thinking about rsyncing the class files and the jar dependencies to the remote server (perharps in an ant script) that is run occasionally, and then running the application remotely by ssh and using an ssh tunnel for connecting to the remote JVM.
This is easily achieved by running the rsync part as a Builder and the ssh tunnel as an external tool which is ran before debugging sessions. However, I would like to simply launch the remote debug configuration and have it up & running. Ideally, I would like to execute some code snippet before Eclipse tries to connect to the remote JVM, and possibly have its output appear in a Console view.
Is there any way of achieving this with some plugins (it is not supported out of the box)? I suppose I could write a quick hack as an Eclipse plugin, but I would prefer not to reinvent the wheel.
If installing CDT is an option, then you could use its launch groups for running your external tool together with the normal launch configuration:
Alternatively you may want to have a look at the EclipseRunner plugin. While it can organize launch configurations in groups, I'm not exactly sure if the groups can be launched as such.
I've got this new project at work. We are using Eclipse for the project. There are two run configurations, server and client. I have to launch the server and the client independently, and connect to the server using the client. Now, it so happens that this has to be done on both Windows and Linux. (four possible combinations: WS-LC, WC-LS, WS-WC, LS-LC, where W-windows, L-linux, C-client and S-server)
I have Linux on my machine (in which Eclipse is running) and run Windows on a VM. Is there a way I can make Eclipse launch the application in the VM?
I understand I have to build the application locally to a shared folder and send a launch command to windows (using openssh?, not sure). What are the best practices used in this scenario.
EDIT: I need to use this during development, to test my changes to the code. The same application provides both client & server. (yes, horrible, i know :X )
You can publish the server functions as JMX Beans using the MBean interface standard. Then use JMX Console to remotely connect to the server JVM and launch the application.
Eclipse has integration points for remote servers, look to "tomcat configuration" for an example of how it integrates with one remote server.
Whether your application can use an existing server integration solution or not depends heavily on details which aren't present. If you want to actually launch a stand-alone Java process from your remote machine, you generally need a program to capture the request and launch the process.
How to execute a batch file located on Windows remote system? Batch file should run on remote system.
Abhinav,
For your problem I see RMI is the quickest possible solution
Check out the basics from these links (1,2)
Start the server from where you want to run the batch
In the Remote object on the server side Use Runtime.getRuntime().exec() to run your batch.
From the client machine give call to this remote object and method.
Another approach is to use SSH like sshj. This only requires the remote system to have SSH installed and is more secure than RMI.
You can use Jsch and Expect4j for executing commands on remote machine(window/Linux). Further more, if your system allows, transfer the batch file on remote machine using some FTP utility like Apache Commons Net and then execute commands that executes the script.