How to make shell visible to java in docker container - java

I am trying to run jython.sh from my Docker container and below are the Jython and Java versions.
We really need to suppress this error, this is very misleading and looks messy. This error is displayed before the execution of every call to Jython and looks like it cannot find the shell.
Is there a way I can fix the error/exception thrown below?
How to make shell visible to java in docker?
Commands used:
docker run -d <image_name>
docker exec -it <container_name> bash
Dockerfile
FROM alpine:latest
ARG IMAGE_NAME
ENV PATH=$PATH:/bin:/opt/user/jre/bin
ENV LIBRARY_PATH=$LIBRARY_PATH:/opt/user/jre/lib/amd64/server:/opt/user/jre/lib/amd64/jli:/opt/user/jre/lib/amd64
USER user
VOLUME /opt/user/etc/
VOLUME /var/log/user
EXPOSE 6800
ENTRYPOINT ["/bin/bash", "/opt/user/bin/startup.sh", "user"]
#startup.sh script contains some container specific initialization
System requirements
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43)
[OpenJDK 64-Bit Server VM (AdoptOpenJDK)] on java1.8.0_292
Type "help", "copyright", "credits" or "license" for more information.
Actual result
$ ./jython.sh
[ERROR] Failed to construct terminal; falling back to unsupported
java.io.IOException: Cannot run program "sh": error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at org.python.jline.internal.TerminalLineSettings.exec(TerminalLineSettings.java:308)
at org.python.jline.internal.TerminalLineSettings.stty(TerminalLineSettings.java:282)
at org.python.jline.internal.TerminalLineSettings.get(TerminalLineSettings.java:143)
at org.python.jline.internal.TerminalLineSettings.<init>(TerminalLineSettings.java:108)
at org.python.jline.internal.TerminalLineSettings.getSettings(TerminalLineSettings.java:123)
at org.python.jline.UnixTerminal.<init>(UnixTerminal.java:60)
at org.python.jline.UnixTerminal.<init>(UnixTerminal.java:50)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at java.lang.Class.newInstance(Class.java:442)
at org.python.jline.TerminalFactory.getFlavor(TerminalFactory.java:211)
at org.python.jline.TerminalFactory.create(TerminalFactory.java:102)
at org.python.jline.TerminalFactory.get(TerminalFactory.java:186)
at org.python.jline.TerminalFactory.get(TerminalFactory.java:192)
at org.python.jline.console.ConsoleReader.<init>(ConsoleReader.java:243)
at org.python.util.JLineConsole.install(JLineConsole.java:107)
at org.python.core.Py.installConsole(Py.java:1744)
at org.python.core.PySystemState.initConsole(PySystemState.java:1269)
at org.python.core.PySystemState.doInitialize(PySystemState.java:1119)
at org.python.core.PySystemState.initialize(PySystemState.java:1033)
at org.python.core.PySystemState.initialize(PySystemState.java:989)
at org.python.core.PySystemState.initialize(PySystemState.java:984)
at org.python.util.jython.run(jython.java:263)
at org.python.util.jython.main(jython.java:142)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
at java.lang.ProcessImpl.start(ProcessImpl.java:134)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 26 more
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43)
[OpenJDK 64-Bit Server VM (AdoptOpenJDK)] on java1.8.0_292
Type "help", "copyright", "credits" or "license" for more information.
>>>
Expected result
$ ./jython.sh
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43)
[OpenJDK 64-Bit Server VM (AdoptOpenJDK)] on java1.8.0_292
Type "help", "copyright", "credits" or "license" for more information.
>>>
Working environment
However, I'm able to run Jython without any error/exception with Alpine image & OpenJDK8
Dockerfile
FROM alpine:latest
RUN apk update && apk add --no-cache openjdk8 \
&& apk add --no-cache bash \
bash-5.1$ ./jython.sh
Jython 2.7.1 (default:0df7adb1b397, Jun 30 2017, 19:02:43)
[OpenJDK 64-Bit Server VM (IcedTea)] on java1.8.0_302
Type "help", "copyright", "credits" or "license" for more information.
>>>

The Alpine distribution does not come with bash, but a lightweight shell called ash.
From there on, you have two possibilities, either your script is simple enough to be executed by ash, and you can adapt your shebang and usage accordingly, or, you install bash in your Alpine container.
So, either, in your script:
#!/usr/bin/env ash
echo 'I work'
Or, in your Dockerfile
FROM alpine:latest
RUN apk add --no-cache bash
# the rest of your container comes here
If your script does confuse sh for bash, what you could also do is to change the shell where /bin/sh points, which is, per default /bin/busybox by changing the symlink of /bin/sh, this way:
FROM alpine:latest
RUN apk add --no-cache bash \
&& ln -sf /bin/bash /bin/sh
# the rest of your container comes here
And that is actually what you are doing already in the other container that you said is working
FROM alpine:latest
RUN apk update && apk add --no-cache openjdk8 \
&& apk add --no-cache bash
### ^--- Here you are installing bash
Extra note: apk update is not needed when installing packages with the --no-cache flag.
Worth reading: Alpine Dockerfile Advantages of --no-cache Vs. rm /var/cache/apk/*

Related

Install rust on openjdk docker image

I'm trying to write a dockerfile to run a rust test suite that requires a jar to be running on a separate port. I've downloaded the jar into the project and would like to start with the openjdk docker image and download rust as well. That way I can use the java command to run the jar then cargo to run the tests.
# Start with java
FROM openjdk:latest
# Add rust
RUN <one line command to download rust>
# Install production dependencies and build a release artifact.
RUN cargo build --release
# Start chromedriver, geckodriver, and selenium standalone binary
RUN ./thirtyfour/tests/binaries_and_jars/chromedriver
RUN ./thirtyfour/tests/binaries_and_jars/geckodriver
RUN java -jar ./thirtyfour/tests/binaries_and_jars/selenium.jar standalone --port 1234
# Run the tests
RUN cargo test
I was hoping to download rust using the curl command provided on the website, passing all the options as cli arguments, but I still get the prompt. Running
curl https://sh.rustup.rs -sSf | sh -s -- --default-toolchain stable --default-host x86_64-unknown-linux-gnu --profile default
still produces the confirmation prompt.
I'm open to any solution here, including
A one-line command to install rust
Starting from the rust image + a one-line command to install java.
You can skip the confirmation prompt by adding the -y argument. This can be found in the help page of installer script.
The installer for rustup
USAGE:
rustup-init [FLAGS] [OPTIONS]
FLAGS:
-v, --verbose Enable verbose output
-q, --quiet Disable progress output
-y Disable confirmation prompt.
--no-modify-path Don't configure the PATH environment variable
-h, --help Prints help information
-V, --version Prints version information
OPTIONS:
--default-host <default-host> Choose a default host triple
--default-toolchain <default-toolchain> Choose a default toolchain to install
--default-toolchain none Do not install any toolchains
--profile [minimal|default|complete] Choose a profile
-c, --component <components>... Component name to also install
-t, --target <targets>... Target name to also install

JLink does not produce redistributable image

For a while now I've been working with modular projects, but due to being constrained with filename and automatic modules, I had never got a chance to work with jlink tool to produce a redistributable application image. Today I've opted to start an independent project which does not import any external dependencies to prevent usage of the compatibility mode. The project consists of 3 modules and is in maven, so I will only be posting the jlink command snippet I'm using.
Project for reference: https://gitlab.com/Dragas/edu-day-demo, checkout the modules-full tag. Project is built with package goal, to prevent polluting your local .m2 repository. Project is already configured to pull dependencies so packaging and deployment would be easier.
The command I used to generate the jlinked image was as follows:
jlink \
--module-path edu-day-runtime/target/dependency/:edu-day-runtime/target/ \
--add-modules ALL-MODULE-PATH \
--output edu-day-jlinked \
--launcher edurun=edu.day.runtime
Invoking the command does indeed generate a jlinked image, which contains minimum required modules, java libraries and JVM binaries to run the project. Invoking the machine that built the image
edu-day-jlinked/bin/edurun 1 1
does run the project and outputs the following
Result of sum is 2
Meanwhile, attempting to run the same in containerized environment (here i'm using bash:5, a non-java image to simulate an environment where java is not installed) does not yield similar results. Instead, the shell does not seem to find a binary named java
docker run -it -v "$(pwd)/edu-day-jlinked:/app" bash:5
...(in container)
bash-5.0# /app/bin/edurun 1 1
/app/bin/edurun: line 4: /app/bin/java: not found
Upon inspection, the folder does indeed contain the binary called java
bash-5.0# ls -la
total 52
drwxr-xr-x 2 1000 1000 4096 Aug 23 07:53 .
drwxr-xr-x 7 1000 1000 4096 Aug 23 07:53 ..
-rwxr-xr-x 1 1000 1000 116 Aug 23 07:53 edurun
-rwxr-xr-x 1 1000 1000 16688 Aug 23 07:53 java
-rwxr-xr-x 1 1000 1000 16712 Aug 23 07:53 keytool
But even invoking it directly (to show the help message) does not yield any results, besides the same message that the binary cannot be found
(in /app/bin/ folder)
bash-5.0# ./java
bash: ./java: No such file or directory
What is more interesting is that even the keytool binary returns the same error
(in /app/bin/ folder)
bash-5.0# ./keytool
bash: ./keytool: No such file or directory
This raises a question: what went wrong? I haven't yet delved deeper into how jlink works, but my speculation is that it copies the binaries from my own java installation (openjdk 11.0.8+10 from arch repositories), and considers that to be redistributable. Or did I just miss some command line options?
Your issue is that the test container (bash:5) doesn't use the same version of the run-time linker as the java environment.
The binary produced by the jlink will only run if there is a compatible linux run-time linker on the system.
The purpose of the run-time linker is to configure the binary for execution on the system - at the time you are building an executable the default run-time linker is hard-coded into the binary. You can inspect the run-time linker using a tool such as readelf -l, or ldd (ldd only works if it can find the run-time linker)
The default run-time linker for amd64 linux (e.g. ubuntu) is: /lib64/ld-linux-x86-64.so.2
The default run-time linker for i386 linux is: /lib/ld-linux.so.2
On a bash:5 container, the default run-time linker is: /lib/ld-musl-x86_64.so.1
This is not compatible with the run-time linker for the jdk
The error: /app/bin/java: not found is caused because the run-time linker cannot be found for the binary. A dirty test of a jlinked VM in a bash:5 container gives the same error.
When I get the run-time linker for the java I've used:
$ docker run --rm -it -v (pwd)/edu-day-jlinked64:/app -w /here bash:5 bash
bash-5.0# /app/bin/java
bash: /app/bin/java: No such file or directory
bash-5.0# strings -a /app/bin/java | grep '^/lib'
/lib64/ld-linux-x86-64.so.2
bash-5.0# ls -l /lib64/ld-linux-x86-64.so.2
ls: /lib64/ld-linux-x86-64.so.2: No such file or directory
Testing with the run-time linker that's on-board:
bash-5.0# /lib/ld-musl-x86_64.so.1 --list /app/bin/java
/lib64/ld-linux-x86-64.so.2 (0x7fe2852a3000)
libjli.so => /app/bin/../lib/libjli.so (0x7fe28528c000)
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7fe2852a3000)
libz.so.1 => /lib/libz.so.1 (0x7fe285272000)
libdl.so.2 => /lib64/ld-linux-x86-64.so.2 (0x7fe2852a3000)
libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x7fe2852a3000)
Error relocating /app/bin/../lib/libjli.so: __snprintf_chk: symbol not found
Error relocating /app/bin/../lib/libjli.so: __vfprintf_chk: symbol not found
Error relocating /app/bin/../lib/libjli.so: __read_chk: symbol not found
Error relocating /app/bin/../lib/libjli.so: __memmove_chk: symbol not found
Error relocating /app/bin/../lib/libjli.so: __printf_chk: symbol not found
Error relocating /app/bin/../lib/libjli.so: __fprintf_chk: symbol not found
Error relocating /app/bin/../lib/libjli.so: __sprintf_chk: symbol not found
so it definitely won't work here.
Let's use something 'standard'. As I had built the jlinked app in an ubuntu:focal container, with an installed version of java let's use one that doesn't have java built-in:
$ docker run --rm -it -v $(pwd)/edu-day-jlinked64:/app -w /here ubuntu:focal bash
root#865c9c12c029:/here# /app/bin/java
Usage: java [options] <mainclass> [args...]
(to execute a class)
or java [options] -jar <jarfile> [args...]
(to execute a jar file)
or java [options] -m <module>[/<mainclass>] [args...]
java [options] --module <module>[/<mainclass>] [args...]
(to execute the main class in a module)
or java [options] <sourcefile> [args]
(to execute a single source-file program)
so it will work in this case.
Reproducibility:
Built using:
$ docker run --rm -it -v $(pwd):/here -w /here ubuntu:focal bash
# apt-get update
# DEBIAN_FRONTEND=noninteractive apt-get install -y git openjdk-14-jdk maven
# git clone https://gitlab.com/Dragas/edu-day-demo .
# git checkout modules-full
# ./mvnw package
# rm -rf edu-day-runtime/target/classes
# jlink --module-path edu-day-runtime/target/dependency/:edu-day-runtime/target/ --add-modules ALL-MODULE-PATH --output edu-day-jlinked --launcher edurun=edu.day.runtime
# ./edu-day-jlinked/bin/edurun 1 1
Result of sum is 2
In an adjacent directory:
$ docker run --rm -it -v $(pwd)/edu-day-jlinked:/app -w /here bash:5 bash
bash-5.0# /app/bin/edurun 1 1
/app/bin/edurun: line 4: /app/bin/java: not found
In another directory:
$ docker run --rm -it -v $(pwd)/edu-day-jlinked:/app -w /here ubuntu:focal bash
root#200b4a98f9ee:/here# /app/bin/edurun 1 1
Result of sum is 2
TL;DR — The bash:5 image uses a C library that is binary incompatible with the C library that was linked with your edu-day-jlinked/bin/java executable.
The long version
„…This raises a question: what went wrong?…“
What's going wrong is your app/bin/java binary is failing to find the C library that it was originally linked to when you built your edu-day-jlinked executable on whatever machine you built it on locally.
The problem occurs because the java binary that jlink produced is linked to the GNU glibc library that your locally-installed JDK uses…
$ ldd edu-day-demo-modules-full/edu-day-jlinked/bin/java
…
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fa61a95b000)
…
Whereas, the bash:5 image runs in the Busybox Linux distribution. And Busybox does not use glibc…
bash-5.0# ldd app/bin/java
…
libjli.so => app/bin/../lib/jli/libjli.so (0x7f572a16d000)
…
libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x7f572a19f000)
Error relocating app/bin/../lib/jli/libjli.so: __snprintf_chk: symbol not found
Error relocating app/bin/../lib/jli/libjli.so: __vfprintf_chk: symbol not found
Error relocating app/bin/../lib/jli/libjli.so: __read_chk: symbol not found
Error relocating app/bin/../lib/jli/libjli.so: __memmove_chk: symbol not found
Error relocating app/bin/../lib/jli/libjli.so: __printf_chk: symbol not found
Error relocating app/bin/../lib/jli/libjli.so: __fprintf_chk: symbol not found
Error relocating app/bin/../lib/jli/libjli.so: __sprintf_chk: symbol not found
It uses a different C library: musl…
bash-5.0# find / -name '*musl*'
/lib/libc.musl-x86_64.so.1
/lib/ld-musl-x86_64.so.1
It helps to understand the Linking process. And it also helps to bear in mind that JLink builds a custom executable for a specific environment.
Your trial run on your local machine worked, because jlink built the executable specifically for your local environment.
The proposed solution
„…docker here is intended to simulate an environment which does not have java installed…“
Here is a Dockerfile that successfully builds your application and the resulting image „does not have java installed“…
FROM maven:3.6.1-jdk-13-alpine as build
WORKDIR /app
COPY pom.xml .
COPY edu-day-sum edu-day-sum
COPY edu-day-runtime edu-day-runtime
COPY edu-day-api edu-day-api
RUN mvn package && \
--module-path ${JAVA_HOME}/jmods:edu-day-runtime/target/dependency/:edu-day-runtime/target/edu-day-runtime-1.0-SNAPSHOT.jar \
--add-modules ALL-MODULE-PATH \
--output edu-day-jlinked \
--launcher edurun=edu.day.runtime
FROM alpine:latest
COPY --from=build /app/edu-day-jlinked /app
ENTRYPOINT ["/app/bin/edurun"]
CMD ["1", "1"]
Docker best practice advises: „Use multi-stage builds“ (like in the above Dockerfile) when your aim is to build „an environment which does not have java installed“.
The FROM maven:3.6.1-jdk-13-alpine stage of the multi-stage build, uses an alpine Linux image that has both Maven and a JDK specifically built to be compatible with the alpine distribution.
The FROM alpine:latest is a very small linux distro that does not have Java on it. The maven:3.6.1-jdk-13-alpine layer is discarded as the Docker best practice docs says. The only java in the resulting image is the one in app/bin.

icns file not found error with javapackager on macOS High Sierra

I just installed macOS High Sierra (10.13) and am attempting to update a tutorial I wrote that shows how to packaging a Java Swing application (executable JAR) into a macOS installer.
The tutorial's steps worked fine on macOS Serra (10.12).
Now the javapackager command crashes out with:
ERROR: File Not Found. (-43) on file: /var/folders/c1/hkb.../images/ShowTime/.VolumeIcon.icns
System info:
Darwin Comet.local 17.0.0 Darwin Kernel Version 17.0.0: Thu Aug 24 21:48:19 PDT 2017; root:xnu-4570.1.46~2/RELEASE_X86_64 x86_64
java version "9"
Java(TM) SE Runtime Environment (build 9+181)
Java HotSpot(TM) 64-Bit Server VM (build 9+181, mixed mode)
Simplified steps to reproduce:
curl --remote-name https://centerkey.com/mac/java/ShowTime.java
javac ShowTime.java
echo "Main-Class: ShowTime" > MainClass.txt
jar cmf MainClass.txt ShowTime.jar *.class
jdk=$(/usr/libexec/java_home)
$jdk/bin/javapackager -deploy -native dmg -name ShowTime \
-srcdir . -srcfiles ShowTime.jar -appclass ShowTime \
-outdir deploy -outfile ShowTime -v
Full error:
Running [/usr/bin/SetFile, -c, icnC, /var/folders/c1/hkb5ph9s7l352nhr17n310vc0000gn/T/fxbundler10109585143778151656/images/ShowTime/.VolumeIcon.icns]
ERROR: File Not Found. (-43) on file: /var/folders/c1/hkb5ph9s7l352nhr17n310vc0000gn/T/fxbundler10109585143778151656/images/ShowTime/.VolumeIcon.icns
java.io.IOException: Exec failed with code 2 command [[/usr/bin/SetFile, -c, icnC, /var/folders/c1/hkb5ph9s7l352nhr17n310vc0000gn/T/fxbundler10109585143778151656/images/ShowTime/.VolumeIcon.icns] in unspecified directory
at jdk.packager/com.oracle.tools.packager.IOUtils.exec(IOUtils.java:169)
at jdk.packager/com.oracle.tools.packager.IOUtils.exec(IOUtils.java:142)
at jdk.packager/com.oracle.tools.packager.IOUtils.exec(IOUtils.java:136)
at jdk.packager/com.oracle.tools.packager.mac.MacDmgBundler.buildDMG(MacDmgBundler.java:391)
at jdk.packager/com.oracle.tools.packager.mac.MacDmgBundler.bundle(MacDmgBundler.java:92)
at jdk.packager/com.oracle.tools.packager.mac.MacDmgBundler.execute(MacDmgBundler.java:549)
at jdk.packager/com.sun.javafx.tools.packager.PackagerLib.generateNativeBundles(PackagerLib.java:371)
at jdk.packager/com.sun.javafx.tools.packager.PackagerLib.generateDeploymentPackages(PackagerLib.java:348)
at jdk.packager/com.sun.javafx.tools.packager.Main.main(Main.java:496)
Config files are saved to /var/folders/c1/hkb5ph9s7l352nhr17n310vc0000gn/T/fxbundler10109585143778151656/macosx. Use them to customize package.
Exception in thread "main" com.sun.javafx.tools.packager.PackagerException: Error: Bundler "DMG Installer" (dmg) failed to produce a bundle.
at jdk.packager/com.sun.javafx.tools.packager.PackagerLib.generateNativeBundles(PackagerLib.java:374)
at jdk.packager/com.sun.javafx.tools.packager.PackagerLib.generateDeploymentPackages(PackagerLib.java:348)
at jdk.packager/com.sun.javafx.tools.packager.Main.main(Main.java:496)
Full tutorial and steps:
Mac Java! | commands.txt
The error occurs whether I supply an icon file or not. Any ideas on how to fix it?
As GabeV commented, the -native dmg option is broken but -native image works.
Bug report:
https://bugs.java.com/bugdatabase/view_bug.do?bug_id=JDK-8188763
Fortunately, the -native pkg option also works. It produces a .pkg installer file instead of a .dmg disk image, but an installer file is a desired result for many situations.
New steps with pkg workaround:
curl --remote-name https://centerkey.com/mac/java/ShowTime.java
javac ShowTime.java
echo "Main-Class: ShowTime" > MainClass.txt
jar cmf MainClass.txt ShowTime.jar *.class
jdk=$(/usr/libexec/java_home)
$jdk/bin/javapackager -deploy -native pkg -name ShowTime \
-srcdir . -srcfiles ShowTime.jar -appclass ShowTime \
-BappVersion=1.0.0 -outdir deploy -v
ls -l deploy
open deploy/ShowTime-1.0.0.pkg
Documentation:
https://docs.oracle.com/javase/9/tools/javapackager.htm
Update (December 4, 2017):
Apparently the problem is related to a bug of not detecting the new Apple File System (APFS) for macOS 10.3 (High Sierra). See: https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8190758

Permissions error running Java Timezone Updater JAR in Dockerfile

I have a fairly standard Java image with some dependencies added in that I'm writing this Dockerfile on top of. Here's where I'm having a problem in my Dockerfile:
RUN which java
RUN ["/usr/bin/java", "-jar", "tzupdater.jar", "-v", "-l"]
I thought that every Dockerfile command was run as root? But when I try to build from this Dockerfile, I'm running into this permissions issue which also happened in my local environment when I forgot to run this JAR as sudo:
$ docker build -t container-w-tz-update .
Uploading context 1.122 GB
Uploading context
Step 0 : FROM company/java-img-with-dependencies:1.0
---> 0101010101
...
Step 3 : RUN which java
---> Running in 0101010101
/usr/bin/java
---> 0101010101
Step 4 : RUN ["/usr/bin/java", "-jar", "tzupdater.jar", "-v", "-l"]
---> Running in 0101010101
java.vendor: Sun Microsystems Inc.
java.version: 1.6.0_45
tzupdater version 2.1.1-b01
Downloaded file to /tmp/tz.tmp/tzdata.tar.gz
Downloaded file to /tmp/tz.tmp/sha512hash
failed.
Cant rename {0} to {1}.
com.sun.tools.tzupdater.TzRuntimeException: com.sun.tools.tzupdater.TzRuntimeException: Cant rename {0} to {1}.
Caused by: com.sun.tools.tzupdater.TzRuntimeException: Cant rename {0} to {1}.
at com.sun.tools.tzupdater.TimezoneUpdater.update(TimezoneUpdater.java:301)
at com.sun.tools.tzupdater.TimezoneUpdater.run(TimezoneUpdater.java:249)
at com.sun.tools.tzupdater.TimezoneUpdater.main(TimezoneUpdater.java:643)
2017/04/05 22:20:23 The command [/usr/bin/java -jar tzupdater.jar -v -l] returned a non-zero code: 1
I was able to run this utility as sudo on my local with the exact same version of Java with no issues. When I tried to run it locally without sudo, I received the same error. Thoughts?
(This is more like a comment but the message length is beyond the comment limit, so I leave it here as an answer so that other people can maybe give more useful info based on what I've done.)
Seems it's really an unresolved issue, I reproduced your problem by image enoniccloud/java6 running with root user, I also tried docker run -it --privileged ... to run it manually but it does not help. I also tried jdk8 with base image alpine:3.3 but also failed with:
Downloaded file to /tmp/tz.tmp/sha512hash
Renaming /opt/jdk1.8.0_91/jre/lib/tzdb.dat to /opt/jdk1.8.0_91/jre/lib/tzdb.dat.tzdata2016a failed.
Cant rename {0} to {1}.
Validating for : tzdata2017b
Validation complete
JRE updated to version : tzdata2017b
I searched and the only info I found is as follows:
An open issue: https://github.com/docker/hub-feedback/issues/896
Related but not the same: https://forums.docker.com/t/update-docker-container-jre-to-set-timezone-correctly/24426
Before you find the root cause for this issue, I think an alternative way to build your image could be:
Download a JDK tarball for your platform
Untar it, setup JAVA_HOME and PATH, run java -jar tzupdater.jar -v -l on your host
Build your base image based on this updated JDK, ADD to image and setup environment variable like JAVA_HOME and PATH
Hope this could be helpful to you :-)
docker build will complete RUN steps as the last USER set in the Dockerfile.
To reset:
RUN whoami
USER root
RUN ["/usr/bin/java", "-jar", "tzupdater.jar", "-v", "-l"]
USER "whatever whoami reported"
I got a workaround for that. Adding some mv commands (that have no real effect) made it work:
FROM openjdk:7u211-jdk-alpine3.9
ADD tzupdater.jar tzupdater.jar
RUN mv /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi.tzdata2018g && \
mv /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi.tzdata2018g /usr/lib/jvm/java-1.7-openjdk/jre/lib/zi && \
java -jar tzupdater.jar -v -l https://.../tzdata-latest.tar.gz
I have no idea why it works, I got it by trial and error.
Notice the two mv commands do like mv a b && mv b a, so nothing really changes.
The names of the files (dirs, actually) used in the mv are the ones it outputs when it errors (before the workaround), so change it accordingly.

Puppetserver service fails to start

I have a Vagrant CentOS VM running with ps.memory = 2048 RAM allocated.
When I try to start the puppetserver service:
$ puppet --version
4.4.0
$ sudo puppet resource service puppetserver ensure=running
Error: Could not start Service[puppetserver]: Execution of '/bin/systemctl start puppetserver' returned 1: Job for puppetserver.service failed. See 'systemctl status puppetserver.service' and 'journalctl -xn' for details.
Error: /Service[puppetserver]/ensure: change from stopped to running failed: Could not start Service[puppetserver]: Execution of '/bin/systemctl start puppetserver' returned 1: Job for puppetserver.service failed. See 'systemctl status puppetserver.service' and 'journalctl -xn' for details.
service { 'puppetserver':
ensure => 'stopped',
}
$ journalctl -xn
No journal files were found.
$ systemctl status puppetserver.service
puppetserver.service - puppetserver Service
Loaded: loaded (/usr/lib/systemd/system/puppetserver.service; disabled)
Process: 4708 ExecStartPre=/usr/bin/install --directory --owner=puppet --group=puppet --mode=775 /var/run/puppetlabs/puppetserver (code=exited, status=0/SUCCESS)
Main PID: 4709 (java); : 4710 (bash)
CGroup: /system.slice/puppetserver.service
├─4709 /usr/bin/java -Xms1g -Xmx1g -XX:MaxPermSize=1g -XX:OnOutOfMemoryError=kill -9 %p -Djava.security.egd=/...
└─control
├─4710 /bin/bash /opt/puppetlabs/server/apps/puppetserver/ezbake-functions.sh wait_for_app
└─4755 sleep 1
My JAVA_ARGS from /etc/sysconfig/puppetserver:
JAVA_ARGS="-Xms1g -Xmx1g -XX:MaxPermSize=1g"
As requested, the puppetserver.service file:
$ cat /usr/lib/systemd/system/puppetserver.service
[Unit]
Description=puppetserver Service
After=syslog.target network.target
[Service]
Type=simple
EnvironmentFile=/etc/sysconfig/puppetserver
User=puppet
TimeoutStartSec=120
TimeoutStopSec=60
Restart=on-failure
StartLimitBurst=5
PermissionsStartOnly=true
ExecStartPre=/usr/bin/install --directory --owner=puppet --group=puppet --mode=775 /var/run/puppetlabs/puppetserver
ExecStart=/usr/bin/java $JAVA_ARGS \
'-XX:OnOutOfMemoryError=kill -9 %%p' \
-Djava.security.egd=/dev/urandom \
-cp "${INSTALL_DIR}/puppet-server-release.jar" clojure.main \
-m puppetlabs.trapperkeeper.main \
--config "${CONFIG}" \
-b "${BOOTSTRAP_CONFIG}" $#
KillMode=process
ExecStartPost=/bin/bash "${INSTALL_DIR}/ezbake-functions.sh" wait_for_app
SuccessExitStatus=143
StandardOutput=syslog
[Install]
WantedBy=multi-user.target
An attempt at running the ExecStartPost command by hand:
$ /usr/bin/java -Xms1g -Xmx1g -XX:MaxPermSize=1g -XX:OnOutOfMemoryError='kill -9 %%p' -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg
RuntimeError: Got 2 failure(s) while initializing: File[/var/log/puppetlabs/puppetserver]: change from 0700 to 0750 failed: failed to set mode 0700 on /var/log/puppetlabs/puppetserver: Operation not permitted - No message available; File[/var/run/puppetlabs/puppetserver]: change from 0775 to 0755 failed: failed to set mode 0775 on /var/run/puppetlabs/puppetserver: Operation not permitted - No message available
So I tried again, but this time I changed some directory permissions, but still similar error (which doesn't make sense given I just changed the mode?):
$ sudo chown -R vagrant:vagrant /var/run/puppetlabs/
$ sudo chown -R vagrant:vagrant /var/log/puppetlabs/
$ sudo chmod -R 0755 /var/run/puppetlabs/
$ /usr/bin/java -Xms1g -Xmx1g -XX:MaxPermSize=1g -XX:OnOutOfMemoryError='kill -9 %%p' -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1g; support was removed in 8.0
RuntimeError: Got 1 failure(s) while initializing: File[/var/run/puppetlabs/puppetserver]: change from 0775 to 0755 failed: failed to set mode 0775 on /var/run/puppetlabs/puppetserver: Operation not permitted - No message available
use at /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/settings.rb:1007
What could be the issue?
Are you certain that it’s an OnOutOfMemory error? I ask because I found that the latest PuppetServer includes a newer version of logback, as shown by this message in /var/log/messages:
Mar 18 01:56:21 puppetserver java: Exception in thread "main" java.lang.AbstractMethodError: ch.qos.logback.core.net.SyslogAppenderBase.createOutputStream()Lch/qos/logback/core/net/SyslogOutputStream;
Mar 18 01:56:21 puppetserver java: at ch.qos.logback.core.net.SyslogAppenderBase.start(SyslogAppenderBase.java:62)
Mar 18 01:56:21 puppetserver java: at ch.qos.logback.classic.net.SyslogAppender.start(SyslogAppender.java:48)
If you see, this replace “classic.net.Syslog” in logback.xml with "core.net.Syslog"
sed -i_old -e 's/classic.net.Syslog/core.net.Syslog/' /etc/puppetlabs/puppetserver/logback.xml
If that's not the problem, please post your logfiles.
You have provided log as
OpenJDK 64-Bit Server VM warning: ignoring option MaxPermSize=1g;
support was removed in 8.0
So, it is clear that you are using jdk 8 which removed permgen space.
The Permanent Generation (PermGen) space has completely been removed and is kind of replaced by a new space called Metaspace. The consequences of the PermGen removal is that obviously the PermSize and MaxPermSize JVM arguments are ignored and you will never get a java.lang.OutOfMemoryError: PermGen error.
So please remove the -XX:MaxPermSize=1g portion from JAVA_ARGS of location /etc/sysconfig/puppetserver
JAVA_ARGS="-Xms1g -Xmx1g"
So
And then,
your command will be look like below:
$ /usr/bin/java -Xms1g -Xmx1g -XX:OnOutOfMemoryError='kill -9 %%p' -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg
or
$ /usr/bin/java -Xms1g -Xmx1g -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg
Please try this 2 commands. I hope both will run successfully. For caution purpose, I have added both.
N.B: There is no need to change the file permission. keep them as before you using.
Otherwise, if you don't want to change anything, you can downgrade java 8 to java 7
Related Link:
PermGen elimination in JDK 8
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize
JAVA 8 XX:PERMSIZE AND XX:MAXPERMSIZE DISAPPEARING
JDK 8 Milestones
JEP 122: Remove the Permanent Generation
#lollercoster, you're starting your service with:
sudo puppet resource service puppetserver ensure=running
But your next command is not telling which user it is
/usr/bin/java -Xms1g -Xmx1g -XX:MaxPermSize=1g -XX:OnOutOfMemoryError='kill -9 %%p' -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg
Please run whoami directly before your run the above command:
whoami
/usr/bin/java -Xms1g -Xmx1g -XX:MaxPermSize=1g -XX:OnOutOfMemoryError='kill -9 %%p' -Djava.security.egd=/dev/urandom -cp /opt/puppetlabs/server/apps/puppetserver/puppet-server-release.jar clojure.main -m puppetlabs.trapperkeeper.main --config /etc/puppetlabs/puppetserver/conf.d -b /etc/puppetlabs/puppetserver/bootstrap.cfg
Why this commands?
I am pretty confident that your issue is a permission/usercontext issue and following commands make it more worse:
$ sudo chown -R vagrant:vagrant /var/run/puppetlabs/
$ sudo chown -R vagrant:vagrant /var/log/puppetlabs/
$ sudo chmod -R 0755 /var/run/puppetlabs/
with the above cases you have to do a sudo on the right user context to have the files writable, but it depends if the other data is writable. So you can give it a try, but I would use the puppet user created for the service
User=puppet
So your puppet service runs as user puppet, but your logfiles are under vagrant user control, and not writable for puppet user.
So I would also try to switch back to:
$ sudo chown -R puppet /var/run/puppetlabs/
$ sudo chown -R puppet /var/log/puppetlabs/
$ sudo chmod -R 0755 /var/run/puppetlabs/
Java
Also I suggest you NOT to mess around with the JVM heap parameters, except you know what you do. Since you do not need to enforce a lower and a max bound on Java VM heap since JDK5. I would only limit to the upper bound
-Xmx1g
The heap and the real allocated mem will be managed well by the javaVM. As JVM needs more it will step by step increase the heap and keep the needed size (no downsizing to a lower value will occur). So you will have a better used RAM on your machine.
Also please switch to the Oracle JVM. I am facing often issues with security and Compatibility with OpenJDK. So my first test is to run it on the latest needed JDK of Oracle. In your case Oracle JDK8.
But please try first the permission things I mentioned before.
Good luck, and please keep me updated.
I have seen this behaviour before. Do not try to run the commands manually as root, because you will mess up the permissions. Instead read the logs carefully with journalctl -xe and try to understand why the service is failing. In my case I could not start the puppetserver service because there was already a private key, but not yet a public certificate.
ls /etc/puppetlabs/puppet/ssl/certs/`hostname -f`.pem
ls /etc/puppetlabs/puppet/ssl/private_keys/`hostname -f`.pem
If one of them is present without the other one, the service will fail to start up. You will see something like this in the log file:
Exception in thread "main" java.lang.IllegalStateException: Cannot initialize master with partial state; need all files or none.
If you are trying to install puppetserver for the first time, so there is no way to break an existing deployment, then you can probably just remove the exisiting private key and puppetserver will automatically generate a new pair.
rm /etc/puppetlabs/puppet/ssl/certs/`hostname -f`.pem
rm /etc/puppetlabs/puppet/ssl/private_keys/`hostname -f`.pem
And finally try again to start the puppetserver service.
service puppetserver start
/usr/bin/java -Xms1g -Xmx1g -XX:MaxPermSize=1g
Xms and Xms are less than MaxPermSize. this worked for me.

Categories