I'm just figuring out Mesos and testing it on a vagrant node.
While in the process of checking out mesos, my work ran into the JVM memory leak in Docker issue posted here -> https://github.com/docker/docker/issues/15020
I would like to know if there are some tips or examples where we can test the containerization on Mesos to see if the memory problems are specific to Docker Containerizers. So far most of the examples are for marathon running docker, but not much else on non-docker Mesos.
Do I need to write my own Mesos JVM Containerizer Isolator?
Do containerizer just parse job details, and executes a script?
Can I pass the Mesos containerizer a .war and job package and have it run my Java App?
Any simple samples or specific code on large github projects that illustrates this?
My work is deciding if we should deploy our Java apps without containerization or if changing a containerization solution may help resolve the issue.
Any tips or links to documentation I can check to look at how to setup a Tomcat JVM app on Mesos Containerizers? I am still learning how mesos works, so anything helps!
So let me step back a bit: you don't run a Java app on Mesos, you run it on one of its frameworks, such as Marathon (for long-running tasks such as a webserver) or Chronos (for scheduled batch jobs). Think of Mesos as the Linux kernel of a distributed operating system, that might help.
Coming back to your question, and unless you plan to write your own framework (for example tobilg/mesos-js-framework), the following assumes you use Marathon to launch Tomcat (since it is a long-running task).
Marathon, as most frameworks, doesn't do the heavy lifting, it leverages Mesos core features such as the Fetcher to pull resources necessary to execute an app into the sandbox on the Mesos Agent. Via uris you can specify a list of URLs that Marathon (via the Mesos Fetcher) downloads and, depending on the file extension, also extracts before it launches the app specified via the content of cmd.
Related
I have followed the quickstart guide on the Spring website and successfully run the Spring app hello world server via ./mvnw spring-boot:run.
And it works.
I did som benchmarking (on my local machine) but its on the RoR7 requests-per-second response level and I was hoping to get something at least Express like numbers - so at least something that is 15 - 20x or more times faster than RoR.
Is there a command line flag parameter like production or --release to get a release-finetuned-app` that is faster?
UPDATE:
even if I do mvn build and run java -jar and the jar target name it is still the same ruby on rails level. Fr example, an express app of that same complexity is 40x faster and in Rust(Actix) 400x faster, while it should be something like the same or faster than Express and perhaps 3 - 5x slower than Rust. I need to turn off debugging and other stuff and produce a release build of a spring app somehow. Any idea how to realize that?
You have a Web-Application. Web-Applications on Spring are using the Servlet-Specification.
The filetype of your application is a bundled .war-file, who is ready for deployment on a Servlet-Container.
In production state-of-the-art is a servlet-container like
Tomcat
JBoss / Wildfly
Jetty
They are running performance-optimized on a hardened operation-systems like Linux or Solaris. Starting a JRE who can not debug by default.
They use a VM-flag called --server what brings big performace-improvements from about 5-10%. But the 20x times faster execution does not seems a compiler-flag problem.
Do you have a Antivirus-Program running? That might give such a impact.
Try to install tomcat, package the application for production and copy the .war-file to tomcat's webapps-folder. This might give the performace-improvement you are looking for.
I am about to deploy a set of JAVA based microservices.
I am confused as to whether:
Run them as simple jars via "java -jar [JAR_NAME]"
Run them in a JAVA based docker container.
Run them as a war.
Please offer me pros and cons of each implementation as this will save me a lot of headache if I use the suggested best approach :)
Thanks in advance.
Definitely Docker. Using containerization gives you max flexibility.
In your first approach, you jar is dependent on Java. Whenever you create new VM, you need to install fix set of software to support you application.
Benefits in second approach,
First, everything is going to be in single container.
You can install all required software in container and that container can be user in any VM. You have flexibility to use java of your choice for each microservices. Only install docker and everything is going to be worked.
Second, Dev Prod Parity
If you thing very much of microservice architecture and 12-factor apps. Then docker helps to support lots of factors.
Your java and other software are going to be unique in all your environment. That means you are never going to get surprise whether it is working in QA and not in Prod due to some version mismatch of runtime environment.
Third, Flexibility
If you go into microservice architecture, then why only java. You can also go with GO, Python or other languages. At this time, rather installing runtime environment for each platform on each VM it is very useful to have microservice in containers.
Last, Deployment Easiness
You can use docker-compose or docker swarm to run 100s of mivroservice in single command.
I have a Java Spring Boot Application, and I build it with Maven. With
spring-boot-maven-plugin,
I can create fat, executable jar file.
Then I copy it to the remote server and run. But sometimes,
I change only one line or event one word in my code and I had to do whole build/copy step again. I'm sure that I'm doing it wrong, but I couldn't find another way that more efficient (Like capistrano in Rails).
At this point, I'm planning to clone source code to server, push from local, pull from remote, build and run approach. What is the correct (or elegant) way of doing this deployment?
For automatic build and deployment process (continuous integration), you can use Jenkins. Refer this documentation for more details: https://jenkins.io/doc/
I would say it depends where are you trying to do it.
The best and the most agile way to do it for a controlled environment is surely a CI-CD (Continuous Integration and Continuous Deployment) pipelines, which complies-builds-tests-deploys your code against every commit made to the source code BUT it may be too slow to use CI-CD for a development environment where you had like to have a shorter feedback cycle and faster feedback to see how the code is progressing.
However, if you are talking about development environment, I will hit another chord and ask you why to deploy to the external server AT ALL while developing. When you use Spring Boot, which helps you develop a self-contained application, you get the Tomcat Server embedded with it for free. That gives you the choice to run the code anywhere you develop and test to move forward.
A simple maven goal - mvn spring-boot:run can make the code run anywhere you had like.
There is another magical library available in Spring-Boot, known as Devtools, which is meant to support agile developers. The library once in the app classpath, performs hot-swapping of byte-code to auto reload of code into the running application (running locally with embedded Tomcat) as soon there is a saved change. This is one of the coolest gadget that a developer can have.
Use of Spring-Loaded (or JRebel for non spring-boot apps) libraries can also help a developer do hot-swapping of byte code to load changes in running application as soon saved.
I hope it helps.
I am working on a enterprise product and primarily there are 3 pieces to it swing based client, DB, Server(for now we can ignore DB part). Being enterprise product Client and Server comes with their own installer(it is not like configuring apache or JBOSS and deploy war's on it).
We have CI configured to generate the nightly OS specific builds for Client and server which can be installed.
So we have to test these build regularly on specific OS, which requires a lot of manual process of installing and creating system with X version client on Y OS OR X version server on Y OS. This is becoming very tedious since we are all on windows and doing next-> next -> really sucks(I have created a script which installed our product via shell but then it is still steps which I believe can be automated, but don't how). And also we need an isolation.
Now I am thinking how can we automate this process of creating these test machine. I have just started exploring Vagrant/Docker if they can be helpful to me (and under the their concept, still doesn't understand Puppet/Chef though) and I am confused in which strategy should I adopt
Create VM via vagrant and run my installation script on that box (This will require one VM per Client or per server)
Create VM via vagrant and run my client docker containers on it (this I guess, will require one VM for multiple client or server, since they would be under container)
Note: I have to create VM, since we are on window.either via vagrant or via boot2docker
So my question are
If these 2 strategy are valid and not wrong then out of these 2 which strategy should I adopt out of two ?
Are there any different strategy that I am missing or am I approaching this in right way ?
If strategy #2 is to be adopted then how can I create container/docker images in which my client is installed
how can I create container/docker images in which my client is installed
You must put in a Dockerfile all what you do in order to have your client started and configured.
In order to do so, you can either create a container, do all the stuff, and then docker commit or the better way is to put all the required commands in a Dockerfile, so that when you do a slight modification, you build a new version easily with a basic docker build -t myclient_version_n .
Check the docs
https://docs.docker.com/examples/mongodb/#creating-a-dockerfile-for-mongodb
and how to automate builds
http://docs.docker.com/docker-hub/builds/#automated-builds
how to create a Dockerfile
https://docs.docker.com/examples/nodejs_web_app/#creating-a-dockerfile
and have a look at existing Dockerfiles of containerized application in the docker Hub
https://registry.hub.docker.com/
An alternative to Vagrant would be to use Docker Machine. You could leverage the cloud providers as #m1keil mentioned too. Machine can provision Docker hosts on a number of providers and they are ready to go.
Disclosure: I work at Docker and am the maintainer of Machine :)
Your strategies seem valid to me. The addition of containers (docker) to your process might help you speed up and parallelize the testing process (if it's fully automatic testing) since the initialization time and the general resource consumption of a container are lower. However one cannot give you definitive answer without inspecting your testing process first. And since you haven't provided any details about it, it would be hard to tell you if you should use the first or the second strategy.
You can take advantage of the cloud and use services such as AWS, Azure, GCE, etc to initialize machines and run your tests. You can use Vagrant to do this, or skip Vagrant and create your own simple scripts by using the appropriate APIs of your chosen Cloud provider.
Also you can take a look at services such as Travis.ci, Circle.ci, and others, which might help you created automated testing pipe without the need to spend too much time on the plumbing.
I really like docker's ease of use via the Dockerfile. The Dockerfile let's you very easily update and control the software in the docker image, and then you can provision it in you CI/testing environment. Docker now has native Windows support, so this shouldn't prevent you from being able to use it: https://docs.docker.com/docker-for-windows/ Furthermore, I like that you can setup very lightweight, minimal machines, with only the build and runtime dependencies needed for your project, and store them for free on hub.docker.com. Depending on how long it takes to build & install certain dependencies, this can speed up your testing because you can just download a docker image with everything already installed and built, and then just build and test your actual project.
I use this for https://github.com/sourceryinstitute/opencoarrays, which is GCC's official implementation of Coarray Fortran. I have a little project https://github.com/zbeekman/nightly-docker-rebuild that lets you setup nightly docker image builds on hub.docker.com in under two minutes. I use this to trigger builds of https://github.com/zbeekman/nightly-gcc-trunk-docker-image because I can't rebuild GCC from source on Travis-CI.org without the build timing out. This way, I delegate the GCC nightly build to hub.docker.com and then just docker pull zbeekman/nightly-gcc-trunk-docker-image into a travis-ci instance to test OpenCoarrays against the latest GCC trunk.
I have written a little netty server application, packed in a jar file that I want to deploy on a linux server.
Since I have no professional experience with deploying java applications, I was wondering if it is enough to start the netty server by doing:
java -jar NettyServer.jar NettyServer &
Obviously a script could be created to ensure the correct user starts the process etc., but is this the way (stand-alone) java services is being deployed?
It seems almost too easy, considering every other question/answer seems to mention some big hunky container-bean-glassfish-tomcat-whatnot (which I might consider later on if/when issues arise)
yes thats the way - no container needed!! I built a middleware (http://sourceforge.net/projects/serviceconnecto/) using netty as underlaying framework. It's the way i start my server as well! Just verify the classpath is set correctly - meaning libraries are in correct place and the jar archive is correctly built.
I personally prefer Upstart to start services on linux. http://upstart.ubuntu.com/
It is very easy to use, and can also restart your application on crash.
I hope it helps.