I have been tasked to set up a Java based development environment across multiple Windows machines. The problem is that I want to the process to be done automatically and easily on each machine so the developers don't have to waste time downloading and installing all the different applications. Ideally, I would like to have the following:
Automated and unattended initial installs
Some sort of a monitor on those installations that would make sure the settings remain constant between all machines
A possibility to push new settings/programs/upgrades when required.
I've looked into several tools for the job. Currently the most promising one seems to be Puppet . However, Puppet doesn't work as well in Windows...
Using a VM image would solve the first requirement, but it is out of the question since the hardware is different across the machines and upgrades won't come easy.
Has anyone had any experience with this sort of task? How would you solve it?
I've been playing with Vagrant for a couple weeks and finding it a fantastic tool for this. It uses Puppet, Chef, or a customer "provisioner" on top of a VirtualBox, and is controlled by a simple command. They have a great tutorial/tour that will show you what it's capable of.
As an example, the direction I've been heading this week is writing Puppet scripts/modules to setup my production server, and all the dynamic parts are handled by parameterized classes. So my development environment will have the same OS, the same firewall settings, the same daemons, etc--all without affecting my host OS or doing any manual configuration steps.
That being said, I've not used it on a day-to-day basis so I don't know if there are any blocking issues, but I have used manually managed VirtualBox for the same purpose without trouble, so I don't foresee any problems.
The more interesting functionality is pulling information from the developers machines. The development environment changes, and different developers try out new things/programs/settings at a rate that is difficult to keep up with if it is not automated. Only having one configuration (the centralized model) kills your ability to respond to change. It is important to understand the differences between configurations, though.
One interesting option might be to standardize on the Eclipse IDE plus a set of plugins (SCM, testing, J2EE development etc.) and use the Eclipse update mechanism to deploy an identical configuration to every machine. Dependencies, synchronization and suchlike would be handled automatically by the Eclipse platform.
This might not work for you if you need some specific tools that are not available inthe Eclipse ecosystem, but my personal development environment is 100% Eclipse based so it is certainly possible to make this work.
Java can be installed globally easily. For Windows have your system administrator push out the MSI file embedded in the Java JRE installation executable. For Ubuntu ssh in and have the sun-java6-jdk module installed.
Then Eclipse is just a matter of pushing out an unzipped distribution to the users.
Most developers like to customise their setups, so I'm not sure this is going to be popular. You could go bleeding edge and look to provide them with instances on the cloud (once you've got one set-up correctly you can clone away!).
1) Use a Disk Image.
or
2) Put everything (Eclipse executable etc.) in SVN (or some other source repository). Then they just have to install SVN and checkout.
Related
DISCLAIMER: I'm relatively new to Linux. I debated putting this on SuperUser or ServerFault because the answer does require an explanation of Linux as a system (and not a specific programming problem), however I'm interested in this from a Java developer's perspective, and I argue that this is a specific problem because where I install GlassFish greatly impacts my Java configuration, and ultimately, app configuration. Not to mention posting this question on those other sites probably won't garnish any attention from a Java developer, more over system admins, who may not know enough about Java to weigh-in fully on this decision.
I'm trying to decide where to install GlassFish and it seems to me I have (realistically) 4 viable options:
/opt/glassfish/
/usr/local/glassfish/
/home/myUsers/glassfish/ (which is what all the OGS docs show in their examples)
/home/ogs/glassfish (as its own user, similar to how Apache web server is sometimes setup)
I'm wondering what the pros (taking into consideration the nature of how Linux treats these directories differently, the FHS, etc.) and cons of each approach are.
I've read that there are mounting/paritioning benefits to installing it under opt/. However, I normally install 3rd party software to usr/local/, so I'm a little unsure about that as a strategy.
The Oracle GlassFish Server (OGS) docs all demonstrate (but never outright recommend) GlassFish being installed under your home directory (home/myUser/).
Then again, I've read that it is fairly common to install daemon-type services (which is what I would imagine I would be using GlassFish as - where I start it once and it only comes down for routine maintenance or crashes) as their own user (home/ogs/glassfish/).
I'm sure this decision is also impacted by how I'm going to use GlassFish, so let me qualify this question with a few constraints:
I intend to deploy 4 OGS instances across 4 VMs on the same physical machine and cluster them into the same domain (1 of the 4 server instances will be the admin server for the domain)
Several applications will be deployed to this cluster (all 3 non-admin nodes) at the same time, and should be running 24/7/365 except for when they crash (hopefully not often!) or when I need to maintenance or tune them
Each application will be farily large and I would like to configure them with real admin, not "bare bones" de minimis, default settings
If these don't provide specific-enough information to help make this choice, please ask and I can be more specific.
I guess, when the dust settles at the end of the day, I'm looking (more or less) for a matrix with each of the four directory options (plus any other obvious ones I've omitted) compared against their respective pros & cons.
There's also the option of using your Linux distro's package management system to install Glassfish. On Ubuntu for example, you could install it using
sudo apt-get install glassfish-appserv
and let it get installed to wherever the package owner thought it should be installed.
I myself tend to steer away from the above option, as I like to exert my own control over which exact version of Glassfish (or any other Java server/software) is installed and where, but I just wanted to throw that out there, as it's one of the things you could do.
Now for the individual options you've provided:
/opt/glassfish/
This is the preferred option as far as I'm concerned. It keeps the software on a separate directory outside of the regular Linux installation, and allows the mounting and partitioning benefits you mention.
/usr/local/glassfish/
I don't like this much, because /usr/local is generally used by third-party software that is installed using the distro's package management software (apt/yum/etc), and on most distros has directories like bin, etc and lib under it. Putting a directory for glassfish under it, would make it out of place.
Also I prefer to keep system directories separate from custom software that doesn't use the distro's package management tools.
/home/myUsers/glassfish/ , /home/ogs/glassfish
These 2, I would not recommend.
They are only described in most places, because the author doesn't want to assume that the user has root access on those boxes, in which case the home directory would be the only one you'd be guaranteed to own. If you own the system and are managing it, those restrictions don't apply.
Remember, home directories are for specific users. I always recommend server software being managed with individual users' accounts that have the required privileges. Putting software in someone's home directory would mean that you either
Give everyone who needs to manage Glassfish, the password to that user account
Give multiple users read/write access to the home directory of a specific user.
Either way, that's not good system administration policy.
There's not much of a Java perspective here, but if you ask me, there doesn't need to be.
Regarding the prior recommendation, some tradeoffs remain:
If placed in /opt or /usr/local, you will have to have read/write access to those directories, meaning that you will have to have root access to the box. In some environments (where IT controls the platform), IT will not let you have root access. You would have to delegate to IT the responsibility of installing, patching, and upgrading the GlassFish binaries.
If placed in /opt or /usr/local, then you will also have to place the domain directories (--domaindir) in a separate location unless you want them owned by root (unlikely). This was the default in GlassFish 2.x RPM installs on Linux. GlassFish 3.x does not have RPM installs (from Oracle, anyway), but you can still split the two. This isn't a bad tradeoff, but something you should understand.
If placed in a "home directory", then you have rights to upgrade the core binaries, install patches, etc, separate from IT. There is good/bad/ugly in this approach depending on organizational responsibilities.
Hope this helps.
I'm trying to migrate from Windows to Linux as a Java development platform, and while the transition has generally been pretty painless, there are a few points of uncertainty that I'd like some feedback on. I'm running openSUSE 11.4, but I'm open to hear what works on other distros.
Where do you install your JDK from? This one is surprisingly not as cut and dry as most people make it out to be. OpenJDk 6 is available in the openSUSE repositories, and was very easy to install. However it's currently update 21, and right now the Oracle release is at update 24. I'm used to a little alert in Windows notifying me that my Java needs updating but that doesn't appear to be the norm in Linux. Do Java developers forgo the JDK in their package manager and install the binary directly? Or is there another way?
Where do you install Eclipse? There seems to be a general agreement online that Eclipse is best installed by simply downloading the binary and extracting it somewhere, but where's the usual place I would extract a program like Eclipse or Ant? I've seen votes for /usr/local and /opt online, but no definitive answer.
Where do you put your Jetty/Tomcat? Similar to the eclipse question, where do most Linux Java developers put their Jetty/Tomcat/other container.
What are some of the differences between the way you setup development versus production At the very least it seems I don't want to run my servlet container as root, that makes sense to me. But what other practices should I watch out for? Is there anything else that could make my development environment easier, but perhaps less secure?
I found this question was similar but ultimately too high level and didn't get into details of how actual developers are setting up their environment. If there's other resources you feel answer these questions, please share them here.
Thanks for your time.
Q> Where do you install your JDK from?
A> I never bother with other JDKs coming from outside Sun/Oracle mainly because our product is only certified to work with Sun/Oracle JRE. On my desktop, I run Kubuntu, but I never use apt-get for this but always download them manually. Reasons:
distro maintainers rarely rush to upgrade packages, as their primary concern is to make dependant apps (such as OpenOffice) work. If JDK changes from 1.6.0_20 to 1.6.0_21, they simply don't care. I might do because a newer patch might have an important bugfix or I simply want to try if my app still passes all the unit tests.
it might be a nightmare to retain old JDK versions. We still support older versions of our product and if I upgrade to a newer Kubuntu, I don't have guarantees that some ancient JDK will still be available as a package.
I am not sure some distros even support multiple existence of JDKs on the same machine.
My preference is to keep all JDKs/JREs in /opt and make a symlink to the newest one or the one I need most. I simply don't see why installing JDK manually is a problem.
I also set the PATH to the newest JDK/JRE.
Same thing (and similar arguments) apply to Ant and Maven.
Q> Where do you install Eclipse?
A> I use IntelliJ but the same applies. I store IDE in my home folder. This allows me to have different versions of it, update them without needing sudo, etc. I could as well install it in /opt but I guess I got this habit when I was downloading and testing newest IntelliJ IDEA EAP every week so I can quickly delete the older versions and do not pollute /opt. Finally, other programs might require Ant/Maven/JDK but it's only me who uses IntelliJ hence the different approach.
Q> Where do you put your Jetty/Tomcat?
A> I have a separate folder tomcats under /home where I have ~10 different Tomcat instances. Each of Tomcats is used for a different version of my app (we bundle Tomcat with our app). This is necessary because one deployment of our app can have different Tomcat settings (or even version) than another.
Q> What are some of the differences between the way you setup development versus production
A> It very much depends on your app. For example, we need some partitions to have lower access latencies but having less space (e.g. gigabytes for Lucene indexes) VS others which can have higher latencies but require more space (e.g. terabytes for content repositories). We, however, design our app so that all these different aspects can reside on different partitions which are configurable. Some partitions need to have special limitations (e.g. file upload) so this doesn't overflow other partitions. There is no simple one-for-all answer to this question, but obviously most of these concerns don't matter that much for a development environment.
Where do you install your JDK from?
I use Arch Linux myself, and we have the oracle jdk/jre in the repository itself. Hence, use your distro-repository if it has the oracle jdk/jre else get it from oracle itself.
Where do you install Eclipse?
Again, the same answer as above applies to this as well. If however, there is any issue with the distro provided version, I always put my custom installs in /opt/ - /opt/java , /opt/eclipse, /opt/netbeans - etc. I dont install stuff in my home folder (except in circumstances where I don't have permission anywhere else - rare), since that would mean that other users would need access to my home folder to run the stuff. I don't want production (or development for that matter) stuff having direct access to my home.
Where do you put your Jetty/Tomcat?
The same answer as above applies here as well. Only in circumstances, where I have installed more than one version, I create an /opt/experimental/ and install there so that I know which one my production is running and which one I can remove when no longer required.
What are some of the differences
between the way you setup development
versus production?
If possible, I always setup different machines for production and development work. Different computers, but exactly identical setup. The only systems that can push code to the production system are those in the development group. Where this segregation is not possible, I prefer to have different install for the servers, so that while I am tweaking the development configuration, my main servers don't crash or something. Also development setup will generally include a clean_up script that makes it ready for production (dropping unnecessary priviledges for db accounts, cleaning up, etc.
Have whatever, setup you will, just make sure you have different database setup for development and production purposes.
The Sun version of Java for openSUSE is on the nonOSS disk (go here and scroll down for an ISO), which is an additional disk image, not part of the main install disk ISO (or you can pull the RPMs from here).
As far as eclipse is concerned, if I'm installing at the system level, I tend to drop it in /opt. You might want to read this article on how best to handle plugins.
(Don't let them steer you off openSUSE, it is the best distro for KDE IMO.)
Working with linux is a lot less hand holding then the windows environment that you're use too. If I were you I would switch distros to either RedHat or Ubuntu, I use to use SUSE and never looked back since I switched.
You can put your JDK/Eclipse/Tomcat binaries in a couple of different places. If you are the only one going to be using them I suggest you put them in your home directory somewhere. For your Eclipse/Tomcat stuff you can but them in a local bin and then add that local bin to your PATH in your .bashrc. You can also set the location of your JDK to a JAVA_HOME env variable in your bash. If you need any more specific help with setup let me know.
1) We have had several problems with OpenJDK (bugs, etc) so we always use the sun jdk.
2/3) A good rule to live by when living in Linux, is always install your custom software under your home folder. Linux needs to be reinstalled from time to time, but everything in your home folder is in a separate disk-partition, so it lives on. I always installs/unzips custom software such as eclipse to /home/myuser/opt, so my eclipse lives at ~/opt/eclipse. I also symlinks the eclipse-binary to my /home/myuser/bin folder for easy access. When I upgrade or change any software, I just rewire the symlink.
4) Production and development environments should be as close as possible to identical. This elimenate loads of bugs caused by different configurations etc.
I create a /usr/local/java directory and unpack the JDK(s), Eclipse, Maven, Ant, Groovy and Grails in there, then create symlinks to /usr/local/bin.
I would like to know if there is any way that I could build a very simple GUI app (it doesn't even have to look good) that will run on a fresh install of Windows Vista and OS X with no other installations needed by the user. I would perfer not to use Java (just out of personal programming preference). I will use it though, if it is the only way. Specically, I am wondering if I can write a swing app with Scala or Groovy and run in on windows without them having to install anything. Sorry if this is a silly question, I am a Obj-C developer by trade.
You can pack the Scala jars into your own, which should work as long as Java is installed (which it usually is on a 3rd party vendor install of Vista or OS X). If you use Java web start, no installations are needed beyond Java itself. Plus, if you're going to install your own code, why not just copy along the Scala jars also?
If you really mean a fresh install--nothing but what the OS provides--then no, I don't think so.
Edit: You do always have javascript on the browser(s). I assume this won't cut it for what you want?
If you really, really don't want to install anything (or carry anything in your app), then write the application as a web app (possibly a javascript app). Then any user can run that UI from any machine with a decent browser. But then, this will require that you host the app somewhere.
If that is not an option, you can develop your app to as a single html/xhtml file containing a self-contained, self-modifiable javascript application (like TiddlyWiki which I use a lot). Then the user user can download on it on his machine, point his browser to it and voila.
If you combine javascript with HTML5 (and assuming the user has a HTML5 compliant browser like safari), your application can use localStorage to keep its state in the user's machine (thus no longer needing to be self-modifiable to save state as TiddlyWiki does.)
But this would break your rule of not downloading anything on the user's host machine. It is a chicken-and-egg problem that has no solution since each OS implements its own set of application libraries. For multi-platform support, you must use a layer that abstracts out differences between operating systems, be it a vm (like JVM, Ruby or Mono) or a set of libraries (Qt, Gnome).
As far as i know you won't be able to accomplish that with no other installations needed by the user. If you violate this restriction, mono (with gtk#) is a good choice.
Scala and Groovy will have the same deployment issues as Java; all of these require a JVM to be installed. You generally have to first install the JVM (which is not included with Windows) and then install your program. Java is included in OS X, however.
It is possible to use Ruby or Python and one of the cross platform libraries (like wxWidgets) and compile these to an executable file that includes the entire set of runtime libraries (e.g. all of ruby and python).
REAL Studio (formally REALbasic) certainly meets this requirement. It creates native applications that have no external dependencies for OS X and Windows (plus Linux).
In theory you could write a .net application using Mono that it should run without issues on any other one with the .net runtime environment installed.
But I'm not sure if it will work on practice.
I've had some success with XulRunner
There's also a couple of recommendations from these questions I asked
Building Cross Platform app - recommendation
Building XUL app a-la SongBird
XULRunner is pretty cool once you get into it, but it's a tad confusing at first (I thought).. the folks on the mozilla google groups are really nice and helpful though!
I am working on a java based application and we are looking to ease our deployment of updates. Up until now, we've always simply sent out new install packs & had the sysadmin's on our customer sites roll out the upgrades - painful for a large number of users.
what I'd like to do is something similar to java webstart (or eclipse p2) - when the application starts, it checks for updates in a specified location and then downloads the updates prior to starting.
But here's my problem - I want more control over what's done outside of the scope of plugins & jar files. For example:
I'd like to be able to upate my JVM (we ship a modified version with additional security features).
I need to install DLL's - possibly local to the jar files, sometimes to windows
Occasiontally run MSI's to install windows components (e.g. printer drivers).
I need to modify config files & the registry.
I have found a few applications that support this (such as AppLifeUpdate at http://www.kineticjump.com/) but they tend to be .NET focused and it seems a bit perverse to introduce a .NET dependancy on a java application ;)
I know I could write my own here, but if there is already a 3rd party library out there that supports this kind of facility, then it would make my life a lot easier.
So, has anyone else had a similar problem & knows of some products I could look at?
Thanks, Brian.
I have done similar things but on Linux/Unix, so can't suggest any specific products for Windows. But I always found helpful to either package everything in OS format and distribute everything that way (rpms, debian pkg, Solaris pkg, ...) or have the platform deployed through OS mechanisms and have the application deployed through Java mechanism (Web start).
The reason is that sysadmins typically don't like things that magically update themselves without them knowing or in a way that does not follow standard practices. That is: you can't expect to make your life easier if you are not sure that you are not making their life more difficult...
Also: whatever work I do at the OS level, I do it with OS tools. If on Windows it means using .NET, I would very much consider using that...
Hope this helps!
Most of the project artifacts (build time, run time libraries) needed by our development environment is handled dynamically via maven dependencies. But there are other programs which have to be used for all developer accounts before they can reliably use the system (e.g. JDK, Ant, svn client or tortoise svn, eclipse, maven, jboss appserver ...)
Is there a way to automate this, so that you can get the developer quickly up to speed on the development environment. How an where do you store such artifacts (currently we point to a http server to download such artifacts, but its quite burdensome over a period of time), what other options exist over here.
EDIT 1:
Some of the tools don't make changes to the registry, predominantly you unzip and run (e.g. ANT, JBoss, ...).
Some of the tools say Tortoise SVN is painful as you need to install them via their installer. If the vendor starts pushing out their delta changes, there is no clear way to push these changes to all developer machines)
We've been using Virtual Machines with Ubuntu Linux for a year now, to do exactly what you're describing. That way, the only thing developer need to do to get started is run a virtual machine with all the necessary stuff on it.
It has a little bit of performance penalty, but you can work perfectly with it.
The simplest solution would be to prepare a disk image for the whole machine which would contain all these tools. But that is not a 'modular' option as it also dictates the operating system.
The best solution would be 1 big archive that contains all the tools needed. The problem with this option are the files that would be missing due to not installing the tools properly (such as the svn shell extensions that are defined in the Windows Registry).
Unfortunately, there is no common method to do all this. It's an extremely painful process to set everything up, took me around 4 hours to set up my current workstation (eclipse, svn, maven, ..)
But it could be worse: Imagine you used MS Visual Studio with MS Sql Server. Their setup scripts take a HORRIBLE time installing. Service Pack 1 for VS took 90 minutes to install.
One options is to create a system image but it also have its downsides.
Our system support department tried this option but quickly found out that some of the tools installed on the machine that was used to create the system image were storing user/license information during installation. When the image was restored on a brand new system, license collisions were reported.