I need some advice on the best way to implement a software that I need to distribute and install on both windows and linux machines.
I have written a small Java messaging application that runs locally on my machine and listens to a particular port for streaming messages. This application pushes the messages onto a JMS queue. My web application then polls the JMS queue (also local) and runs as expected.
I now wish to scale my application significantly. This means I want to be able to distribute a messaging application to other users (potentially anywhere in the world) who can install it then select a port number using a simple user interface and then run the application. This application would perform the same as the messaging application I have written in JAVA. It would process messages on the selected port and then publish them to a queue on my queue server - from which my Java Application would poll the messages.
I am wondering a couple of things...
I am a Java programmer but wonder what would be the easiest way to produce the remote client that can be easily installed on various platforms? I would rather people could download and install rather than dealing with complex platform dependencies. This might mean a Windows installer...
I was thinking of using the AMQP messaging as the backbone - such as RabbitMQ. This is especially important if I decide to shy away from distributing a Java based message application.
Just some thoughts on the approach would be appreciated.
what would be the easiest way to produce the remote client that can be easily installed on various platforms?
Pure Java, with a GUI, deployed using Java Web Start. The user clicks a link, follows the prompts, and the app. is on screen (with automatic update).
Designed by Sun (maintained by Oracle) to be used on Windows, *nix & OS X.
I have a web application running with support for some specific pieces of hardware. This is achieved in the following steps:
User runs a small installer that places java files (and a couple
others) on the client machine. The main piece is a jar called "hardwareManager"
User visits web app. The web app runs a java applet which, due to
a .java.policy file placed during the install, has permission to
interact with the client machine outside the browser sandbox.
The applet checks to make sure the hardwareManager is running,
and if not runs a command to start it.
User interacts with the web app which sends commands to the applet via
javascript. The applet then writes commands to a text file
on the client machine. The text file is constantly monitored by the
hardwareManager which runs any commands it reads in.
This works, but seems clunky. I have a couple ideas on how to improve it, but I don't know which, if any, are even worth trying.
Would it be better to set up the hardwareManager as a socketServer and have the applet connect directly to it, rather than going through text files? Is that even possible?
Is there a way to eliminate the applet altogether and have the javascript talk directly to the hardwareManager? Maybe by writing the hardwareManager to be a local http server? What port should it run on? Do javascript xss limitations fit in here somewhere?
It would be less clunky to start the Java application using Java Web Start. This would remove the need to daemonize or install the Java hardware manager.
Another alternative is to use a built-in browser inside Java. I supose this is not an option, since you depend heavily on Javascript (I suppose to provide a rich client experience).
If you already have to install something on the client machine, why did you make the choice to go with a web application?
Talking from experience: We had a Java EE application which needed to print to PoS printers at the client site. We installed a small "synchronizer" application that connects through SSH and synchronizes all clients files. Afterwards, it loads the JAR and executes the program. This program connects through RMI with the server and subscribes to a JMS queue to receive the print assignments.
Applied to your case: Why not let your Java application connect to the server directly? You can use HTTP, SOAP or even JMS over RMI. You can then launch the hardware command from the server (instead of from the limited JavaScript webbrowser environment). This way, you get tons of features: authentication, buffering of commands, and you can even share hardware between multiple clients.
Schematic:
<----AJAX------> Web browser
ApplicationServer
<---HTTP/SOAP--> Java hardware manager application
You can launch the Java application using Java Web Start, which allows you to update the application automatically (instead of needing to pass every client a new installer).
I want to place a server program written in Java on the cloud. It would accept TCP socket connections from clients (clients are android phones using 3G), do some computations, save stuff to a MySQL database (also on EC2), and send stuff back to the clients over the TCP connections. It may even be necessary to create several instances of the server (i.e. a process group).
Is this easy to do? I think I can make a AMI, but I'm not sure how to upload Java files, compile and run them, and create a MySQL database etc
Any help would be much appreciated.
Take a look at using Amazon Elastic Beanstalk. Beanstalk is Amazon's PaaS offering and it will alleviate a lot of the system administration burden. Here's a quick description from their docs:
AWS Elastic Beanstalk is an even
easier way for you to quickly deploy
and manage applications in the AWS
cloud. You simply upload your
application, and Elastic Beanstalk
automatically handles the deployment
details of capacity provisioning, load
balancing, auto-scaling, and
application health monitoring.
Also, if you're interested in using MySQL then you should look at Amazon RDS. Again, this will alleviate the system administration burden for your database tier. Here's a quick description from their docs:
Amazon Relational Database Service
(Amazon RDS) is a web service that
makes it easy to set up, operate, and
scale a relational database in the
cloud. It provides cost-efficient and
resizable capacity while managing
time-consuming database administration
tasks, freeing you up to focus on your
applications and business.
Is this easy to do? I think I can make
a AMI, ...
I think the answer depends on how comfortable you are with system administration in general. Creating a AMI to run in EC2 is really pretty much the same as creating a physical server or a VM image. You'll need to install an operating system, and then install tools, libraries and programs you need (like mysql, the jdk, ssh, etc).
You can save yourself a little work by using one of Amazon's pre-built AMI's http://aws.amazon.com/amis/. But, ultimately, you'll be responsible for all system administration of the server. If you've never built a server from the ground up, you have a pretty big learning curve ahead of you. It's not insurmountable, but just be warned that the devil is in the details; there's a ton of stuff you'll need to learn ;-)
... but I'm not sure how to upload Java
files, compile and run them, ...
Once the server is setup and running in EC2, compiling them and running java files is just the same as compiling and running on your local. Normally, you probably want to compile and package your java app into a jar or war and then transfer that up to your EC2 server. If you install linux os on your EC2 server, you can use scp or a FTP client to transfer your files over sftp to move the files from your local up to the server. Once the latest files are up on your server, you can ssh to the server and start your app.
... and create a MySQL database etc ...
Installing mysql is going to be specific to the OS you choose to install on your server. For example, you can install mysql easily on Ubuntu with a command like:
sudo aptitude install mysql
Again, there will be more system-admin-type stuff to learn here specific to mysql databases.
So, it's definitely doable. An experienced sys admin could build a AMI instance pretty easily/quickly. If this is your first experience with system administration, I'd suggest finding an old Desktop you have lying around and try installing Ubuntu and all the required libraries and tools you need (mysql, jdk, ssh, etc..). Get your java program working on the old desktop and then it should be pretty easy to create an AMI from that. Then you can run your custom AMI on EC2 and will be set up.
If you don't have a spare desktop lying around, you can use one of the Virtual Machine products like VMWare Player or Sun's VirtualBox and build a server instance on one of those.
If you want to avoid the hassle of managing the entire install of the Operating system, you might want to look at services like slicehost and/or linode instead of EC2. They give you ssh access to a pre-installed server. And it's as easy as clicking a button to install programs like mysql, etc.
Hope this answer is relevant and helpful, good luck.
- Dave
If you can use Tomcat as your server - you might want to try Amazon Web Services Elastic Beanstalk It will greatly simplify your task by providing an easy instance of Tomcat to deploy.
EDIT: AWS has a full section on how to develop using Java here: http://aws.amazon.com/java/
How comfortable are you with remote administering a server and solving problems most people have never heard of? That's what you're talking about.
You'll create your EC2 instance, log into it and configure it like you would any other server you're working with. You can download JDKs to it, dbs etc. You might consider using a tool like Chef to help you. You'll use ftp and scp to copy files to the server.
You're probably going to want your Java server on one box and have it talk to a separate db server since you say you may want multiple servers.
Once the server is working the way you like it you can create an image of it to use to launch multiple instances, then configure a load balancer to point at your servers.
If you can create a MySql db on your local box you can create it in the cloud. If not?....
Running Applications needs app. s/w to run them.
Apps such as Cyber Duck helps to upload files via SFTP.
I am successful in doing the same problem addressed.. Trust me, YOU CAN DO IT. All u need is interfaces for server services such as MySQL (use WorkBench to connect using key pair), terminal access (using PUTTY/SSH via MAC) and well you are good to go
I think to get started is to use a a whole unit packed into one archive.
Think of it as you have your Java files compiled, a embedded http server say tomcat. Now all of this packed in a jar ready to be deployed on Amazon's EC2. Use this link for more on embedded servers and relevant code.
As far as database is concerned, you can use Amazon's RDS. You can configure mysql on RDS and obtain a link to it for connection. Amazon RDS would help you to get started with database.
Now that you have everything ready to start and work with application. Now is the time to deploy on Amazon AMI
Perform Following steps to deploy you app on Amazon EC2:
Create a server instance on Amazon EC2 refer EC2 getting started.
Receive Private key file i.e. .pem(receive from Step 1), this file would help you to login to your server instance and perform SFTP.
Use Putty or similar SSH client to login to your EC2 server instance created on Step-1 using .pem file and server's public DNS, refer Accessing Instance for SSH login.
Transfer the archived files packed into jar to server using SFTP client, WinnSCP being one such client, refer SFTP for more on it.
Run application. Note in case of jar file you can simply fire java -jar TomcatApp.jar.
If everything is configured properly you would be to access the app using EC2 instance's public DNS or public IP, which would be of the form:
http://<public_dns_address>:<port number>/servlet
Hope it helps you to get started and provide you an overall view.
I am developing a java based application; its pertinent requirements are listed below
Large datasets exist on several machines on network. my program needs to (remotely) execute a java program to process these data sets and fetch the results
A user on a windows desktop will need to process datasets (several gigs) on machine A. My program can reside on the user's machine. He will execute my program from his machine and initiate the dataset processing on remote machine(s)
Instead of getting the dataset over the network from the remote machine to his machine, he will execute the program on the remote machine and fetch results
The user may have open access to the other machines but ftp is the requirement
Data should not be brought through network to the user's machine.
Users have windows OS
My question(s)
How can I perform this kind of remote process execution ? Any ideas?
I am looking at hadoop; I am working on Windows XP. I was unable to get hadoop working for a single node cluster; I am unable to find good documentation. I therefore haven't quite tested hadoop. Any comments on if I am on the right track?
Any links any of you has found useful for installation of hadoop and trouble shooting?
Thanks in advance for any responses. Do please let me know if I should provide any more/specific details.
-jv
Java has a RMI API that you could use, assuming that you can have a JAVA VM running on your remote machines. That's the lightest weight solution. The next lightest weight would be straight socket communication. After that you're getting into EJB servers or Web Servers, which is probably overkill.
Have a look at how to write web services with Java 6. That allows you to publish a method as a web service with an annotation. A web service client is small and does not require additional software. I found the Idea IntelliJ IDE easy to use, and generated a pure Java 6 client.
Then it essentially boils down to making a "normal" method call, and processing the result.
Keep it simple. Grid software is most likely not what you want.
I've got the following problem.
A client is looking for better latency to access a forex trading java web app (.jnlp) that is stored on a third party server. I can provide him access to one of our servers (running linux) that is geographicaly closer to the trading portal (rather than connecting directly to the trading server, less latency, which is critical to the client).
I'm trying to find a way to 'export' the web app interface (basically a simple window) back to the client without having to export the full desktop (using vnc, x export, etc). The client is new to linux and a 'one-click' solution would be the best.
Client is running Windows based OS, Server that will export him the java web app is running linux (debian).
What would be the best mix of technologies to get started ?
I think what you actually need is not better latancy to the jnlp app itself (it is propably insignificant really how long the app needs to start up).
You want lower latancy for the communication between the app and the trading server. Which means the machine on which the app runs, needs a broad/fast connection to the trading server.
This means any remote desktop application might be a solution to your problem.
(TeamViewer for example allows you to only share a single application (and not the whole desktop). Runs on linux using wine, but I'm not aware of all the compatibility issues.)