Getting error resulting in crash loop elasticsearch (GCE/kubernetes) - java

im getting an error while running elasticsearch on kubernetes. I dont believe this is a memory allocation issue but i dont know. Trying to set it up with discovery - not a single node.
here is my kubernetes config for elasticsearch - https://hastebin.com/ohiyivinit.bash
Here is my error on startup from kubectl logs
Exception in thread "main" java.lang.RuntimeException: starting java failed with [137]
output:
error:
OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
at org.elasticsearch.tools.launchers.JvmErgonomics.flagsFinal(JvmErgonomics.java:111)
at org.elasticsearch.tools.launchers.JvmErgonomics.finalJvmOptions(JvmErgonomics.java:79)
at org.elasticsearch.tools.launchers.JvmErgonomics.choose(JvmErgonomics.java:57)
at org.elasticsearch.tools.launchers.JvmOptionsParser.main(JvmOptionsParser.java:89)
EDIT: my error was that the requested memory was the same as the max memory in limits

The error code strongly suggests this is a memory issue.
Since you're using GKE, if Stackdriver is enabled, you can use the following advanced filter to confirm if this is an OOM kill:
resource.type="container"
resource.labels.cluster_name="YOUR_CLUSTER"
resource.labels.namespace_id="NAMESPACE"
resource.labels.project_id="PROJECT_ID"
resource.labels.zone:"ZONE"
resource.labels.container_name="CONTAINER_NAME" # Container name, not pod name
resource.labels.pod_id:"POD_NAME-" # Notice that is not the full pod ID
"OOM"
If you find out this is an OOM issue, you can set requests and limits to your deployment to ensure the resources are enough to run your application.

enter image description hereTry to run command prompt run as administartor

Related

FAILURE Build failed with an exception

how to solve this?
command ionic cordova build android
FAILURE: Build failed with an exception.
What went wrong:
Unable to start the daemon process.
This problem might be caused by incorrect configuration of the daemon.
For example, an unrecognized jvm option is used.
Please refer to the user guide chapter on the daemon at https://docs.gradle.org/ 4.1/userguide/gradle_daemon.html
Please read the following process output to find out more:
Error occurred during initialization of VM
Could not reserve enough space for 2097152KB object heap
Set the JAVA_OPTS system variable (-Xmszzzm -minimum , -Xmxzzzm - maximum)
JAVA_OPTS="-Xms128m -Xmx512m"
(set as per requirement)
...hope this will help you

error in import project from github

When I'm importing following a project from GitHub, I am getting 2 error's
Project
https://github.com/davidanthonymalone/Beer-Finder
Error
Error: Unable to start the daemon process.
This problem might be caused by incorrect configuration of the daemon.
For example, an unrecognized jvm option is used.
Please refer to the user guide chapter on the daemon at https://docs.gradle.org/2.14.1/userguide/gradle_daemon.html
Please read the following process output to find out more:
Error occurred during initialization of VM
Could not reserve enough space for 2097152KB object heap
Java HotSpot(TM) Client VM warning: ignoring option MaxPermSize=512m; support was removed in 8.0

Java/Spring application cannot start up on cloudfoundry: OutOfMemoryError Compressed class space

After deploying a new version of our Java/Spring Boot software to the Swisscom Developer Cloud, running on CloudFoundry, the startup suddenly failed with the following error: OutOfMemoryError: Compressed class space. So, we decided to deploy a previous version of the software, the version which was running actually just before: The same error occurred. We did not switch from Java7 to Java8, nor did we change any configuration. This leads to the question: Is this really an error on our side or rather on the server's side?
We then tried to increase the MaxMetaspaceSize by setting the variable JBP_CONFIG_OPEN_JDK_JRE to one of the following lines:
[jre: {version: 1.8.0_+}, memory_calculator: {memory_sizes: {metaspace: 128m}}]
{memory_calculator: {memory_sizes: {metaspace: 128m}}}
{memory_sizes: {metaspace: 128m}}
The application always warned, that the value of memory_sizes was invalid. What is the correct format of this YAML variable?
[ConfigurationUtils] WARN User config value for 'memory_sizes' is not valid, existing property not present
We then deleted the Java app and the database service on the Swisscom Developer Console and recreated it. It had no effect, the same error occured.
And finally, do you know why this error does suddenly occur, even with a version which was just running fine a few minutes ago?
EDIT:
This is the manifest ([database-service-name] and [application-name] were replaced):
---
path: .
instances: 1
buildpack: https://github.com/cloudfoundry/java-buildpack
services:
- [database-service-name]
applications:
- name: [application-name]
domain: scapp.io
host: [application-name]
memory: 1024M
disk_quota: 1024M
env:
SPRING_PROFILES_ACTIVE: stage, cloudfoundry
The Java buildpack version is (according to the logs):
2017-03-03 11:47:02 [STG/0] OUT -----> Java Buildpack Version: b08a692 | https://github.com/cloudfoundry/java-buildpack#b08a692
This command seems to be executed (in the logs after the crash):
2017-03-03 11:46:25 [APP/PROC/WEB/0] OUT vcap 8 0 99 10:46 ? 00:01:09 /home/vcap/app/.java-buildpack/open_jdk_jre/bin/java -Djava.io.tmpdir=/home/vcap/tmp -XX:OnOutOfMemoryError=/home/vcap/app/.java-buildpack/open_jdk_jre/bin/killjava.sh -XX:MaxDirectMemorySize=10M -XX:MaxMetaspaceSize=68540K -XX:ReservedCodeCacheSize=240M -XX:CompressedClassSpaceSize=8731K -Xmx408104K -Djavax.net.ssl.trustStore=/home/vcap/app/.java-buildpack/container_certificate_trust_store/truststore.jks -Djavax.net.ssl.trustStorePassword=java-buildpack-trust-store-password -cp /home/vcap/app/. org.springframework.boot.loader.WarLauncher
The OutOfMemory error occurred because the Java buildpack was changed to use version 3.x of the memory calculator. Similar problems arising from this change are under discussion in GitHub issue 390. Please refer to this issue for details.
In general, v3.x of the memory calculator chooses values for various JVM memory settings based on the number of class files in the application and some default values which depend on the version of Java. It then sets the maximum heap size to the remaining amount of memory.
The previous version of the memory calculator was configured by setting JBP_CONFIG_OPEN_JDK_JRE. However, v3.x can be configured simply by setting the corresponding Java memory settings in JAVA_OPTS. For example, you could set the maximum metaspace size to 100 Mb as follows:
cf set-env app-name JAVA_OPTS '-XX:MaxMetaspaceSize=100m'
If you simply want a workaround, you can use the version of the Java buildpack released before the memory calculator change:
cf push -b https://github.com/cloudfoundry/java-buildpack.git\#v3.14 ...
comment from Swisscom Java developer:
For sure they will be reverting the memory heuristics to what they
were in 3.13 or at least refine the calculations. The current
recommendation to all customers is either
to use 3.13 or
to use some explicit options with the environment variable JAVA_OPTS.

Error while start Glassfish (errno = 22)

I have a problem when starting my server glassfish. When I start it loads in about 1 minute and then immediately sends the error message quoted below.
Glassfish domain config:
<java-config debug-options="-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=9009" system-classpath="" classpath-suffix="">
<jvm-options>-Djava.awt.headless=true</jvm-options>
<jvm-options>-XX:+AggressiveOpts</jvm-options>
<jvm-options>-Djava.security.policy=${com.sun.aas.instanceRoot}/config/server.policy</jvm-options>
<jvm-options>-Xmn1500m</jvm-options>
<jvm-options>-XX:+UseLargePages</jvm-options>
<jvm-options>-Dfelix.fileinstall.disableConfigSave=false</jvm-options>
<jvm-options>-Dosgi.shell.telnet.maxconn=1</jvm-options>
<jvm-options>-Dfelix.fileinstall.poll=5000</jvm-options>
<jvm-options>-XX:NewRatio=2</jvm-options>
<jvm-options>-Djava.endorsed.dirs=${com.sun.aas.installRoot}/modules/endorsed${path.separator}${com.sun.aas.installRoot}/lib/endorsed</jvm-options>
<jvm-options>-Dosgi.shell.telnet.port=6666</jvm-options>
<jvm-options>-Dcom.sun.enterprise.config.config_environment_factory_class=com.sun.enterprise.config.serverbeans.AppserverConfigEnvironmentFactory</jvm-options>
<jvm-options>-Djava.ext.dirs=${com.sun.aas.javaRoot}/lib/ext${path.separator}${com.sun.aas.javaRoot}/jre/lib/ext${path.separator}${com.sun.aas.instanceRoot}/lib/ext</jvm-options>
<jvm-options>-XX:+UseParallelOldGC</jvm-options>
<jvm-options>-XX:PermSize=2048m</jvm-options>
<jvm-options>-Dgosh.args=--nointeractive</jvm-options>
<jvm-options>-Dwebconsole.type=properties</jvm-options>
<jvm-options>-Djava.rmi.server.hostname=localhost</jvm-options>
<jvm-options>-Dwebconsole.jms.url=tcp://localhost:61616</jvm-options>
<jvm-options>-Djavax.management.builder.initial=com.sun.enterprise.v3.admin.AppServerMBeanServerBuilder</jvm-options>
<jvm-options>-Dcom.sun.enterprise.security.httpsOutboundKeyAlias=s1as</jvm-options>
<jvm-options>-XX:MaxPermSize=4096m</jvm-options>
<jvm-options>-XX:LargePageSizeInBytes=2048k</jvm-options>
<jvm-options>-XX:+UnlockDiagnosticVMOptions</jvm-options>
<jvm-options>-XX:ParallelGCThreads=16</jvm-options>
<jvm-options>-Dwebconsole.jmx.url=service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi</jvm-options>
<jvm-options>-Dfelix.fileinstall.bundles.startTransient=true</jvm-options>
<jvm-options>-Dfelix.fileinstall.bundles.new.start=true</jvm-options>
<jvm-options>-Dfelix.fileinstall.dir=${com.sun.aas.installRoot}/modules/autostart/</jvm-options>
<jvm-options>-Djava.security.auth.login.config=${com.sun.aas.instanceRoot}/config/login.conf</jvm-options>
<jvm-options>-Dosgi.shell.telnet.ip=127.0.0.1</jvm-options>
<jvm-options>-Dfelix.fileinstall.log.level=2</jvm-options>
<jvm-options>-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/config/keystore.jks</jvm-options>
<jvm-options>-server</jvm-options>
<jvm-options>-Djdbc.drivers=org.apache.derby.jdbc.ClientDriver</jvm-options>
<jvm-options>-DAllowMediatedWriteInDefaultFetchGroup=true</jvm-options>
<jvm-options>-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/config/cacerts.jks</jvm-options>
<jvm-options>-Dcom.sun.enterprise.server.ss.ASQuickStartup=false</jvm-options>
<jvm-options>-Xms2048m</jvm-options>
<jvm-options>-DANTLR_USE_DIRECT_CLASS_LOADING=true</jvm-options>
<jvm-options>-Xmx5020m</jvm-options>
<jvm-options>-XX:+UseTLAB</jvm-options>
<jvm-options>-XX:+UseParallelGC</jvm-options>
</java-config>
Error description:
Waiting for domain1 to start ..................................Error starting domain domain1.
The server exited prematurely with exit code 0.
Before it died, it produced the following output:
Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 22).
Java HotSpot(TM) 64-Bit Server VM warning: Failed to reserve shared memory (errno = 22).
Launching GlassFish on Felix platform
Exception: java.lang.NullPointerException thrown from the UncaughtExceptionHandler in thread "Grizzly-kernel-thread(1)"
Exception: java.lang.NullPointerException thrown from the UncaughtExceptionHandler in thread "Grizzly-kernel-thread(1)"
Command start-domain failed.
If someone know what i can do to fix this problem will help me a lot.
Thanks!
This problem occurs when you are using -XX:+UseLargePages and no more HugePages are available or they are not accessible for the current user.
You can see if HugePages are available if you run: cat /proc/meminfo (on *nix)
Try removing the JVM option and see if it helps. In the debian wiki you can get some detailed information about HugePages and how they are used.

Unable to start Neo4j on Windows, no error messages

I'm using the following on Windows 7:
neo4j-community-1.9.2
Java 7 Update 25
I have Windows Firewall disabled.
When I start Neo4j.bat, both as Administrator and normally, I get the following message:
c:\Tools\neo4j\bin>Neo4j.bat
28/07/13 9:34:27 PM org.neo4j.server.AbstractNeoServer INFO: Setting startup tim
eout to: 120000ms based on -1
A blank Java console window pops up, no messages, then the window disappears after a minute or so.
When I go to http://localhost:7474/, I just get a server not found error.
If I try to install it as a service, the service wouldn't start.
Any ideas what I need to do to start Neo4j? I see here that Neo4j 1.9 does support Java 7:
http://blog.neo4j.org/2013/03/neo4j-19m05-released-wrapping-up.html
base.bat correctly points to windows-service-wrapper-5.jar, so not the windows-service-wrapper-*.jar problem that some people have had in the past. It would be helpful if it outputted an error message.
I just tried it with neo4j 2.0 milestone, and it didn't make any difference. The Java window pops up with no messages, http://localhost:7474/ gets a server not found error, then the Java window closes after a minute or two.
I added JAVA_HOME and JRE_HOME system variables, made no difference. I think I would get an error if Neo4j was not able to locate Java anyway.
I've fixed it.
With help from Neo4j's Google Group, I looked at the logs in data/logs/*.log and data/graph.db/messages.log, and found the following exception:
Caused by: java.rmi.server.ExportException: Port already in use: 1337; nested exception is:
java.net.BindException: Address already in use: JVM_Bind
Once I disabled my other service that uses that port, Neo4j's web admin at http://localhost:7474 worked. I then found out port 1337 is used by Neo4j's remote shell, I disabled it by adding enable_remote_shell = false to neo4j.properties, then I was able to have both Neo4j and my other service running.
I'm using the version 2.2.1 community edition of Neo4j on Windows 7, Java 1.8 update 45
In my case the log file messages.log was under default.graphdb directory. In there, you'll see the root cause of the problem.

Categories