Logstash unable to start pipeline - java

I am working with Logstash v7.15.1 and when I am trying to run it, I am getting below error which I am unable to figure out. Will someone help me to fix this?
Note - I have removed several plugins comes bundled with Logstash as all were not part of my requirement.
[2021-11-10T11:38:45.045Z][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:logstash, :exception=>"Java::JavaLang::NoSuchMethodError", :message=>"java.nio.MappedByteBuffer.position(I)Ljava/nio/MappedByteBuffer;", :backtrace=>["org.logstash.ackedqueue.io.MmapPageIOV2.create(MmapPageIOV2.java:194)", "org.logstash.ackedqueue.Queue.newCheckpointedHeadpage(Queue.java:352)", "org.logstash.ackedqueue.Queue.openPages(Queue.java:209)", "org.logstash.ackedqueue.Queue.open(Queue.java:176)", "org.logstash.ackedqueue.ext.JRubyAckedQueueExt.open(JRubyAckedQueueExt.java:123)", "org.logstash.ackedqueue.ext.JRubyWrappedAckedQueueExt.initialize(JRubyWrappedAckedQueueExt.java:65)", "org.logstash.ackedqueue.QueueFactoryExt.create(QueueFactoryExt.java:62)", "org.logstash.execution.AbstractPipelineExt.openQueue(AbstractPipelineExt.java:205)", "org.logstash.execution.AbstractPipelineExt$INVOKER$i$0$0$openQueue.call(AbstractPipelineExt$INVOKER$i$0$0$openQueue.gen)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:355)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:144)", "org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)", "org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:86)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:73)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:85)", "org.jruby.RubyClass.newInstance(RubyClass.java:939)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "opt.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/opt/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:52)", "opt.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/opt/logstash/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:80)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:70)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:207)", "opt.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/opt/logstash/logstash-core/lib/logstash/agent.rb:391)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)", "org.jruby.runtime.Block.call(Block.java:139)", "org.jruby.RubyProc.call(RubyProc.java:318)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"]}
warning: thread "Converge PipelineAction::Create<logstash>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::NoSuchMethodError` for `PipelineAction::Create<logstash>`
create at org/logstash/execution/ConvergeResultExt.java:135

Apologies for posting incomplete information. After hours of troubleshooting, I was able to fix the issue. It was related to using different version of ruby and java. FYI, I am running Logstash in container and then deploying it on K8s cluster. I have removed plugins from Logstash source code itself and then build it.

Related

start Logback SimpleSocketServer to centralize logging

I'm trying to use Logback to centralize the logging of 2 microservices.
Therefore reading the documention I understood I should setup a simpleLogServer to which send the logging events of my others 2 java microservices.
https://logback.qos.ch/manual/appenders.html#serverSocketAppender
Assuming you are in the logback-examples/ directory, start SimpleSocketServer with the following command:
java ch.qos.logback.classic.net.SimpleSocketServer 6000 src/main/java/chapters/appenders/socket/server1.xml
So I've clone the git hub repo https://github.com/qos-ch/logback and from inside the logback-examples/ directory I've run the command only to get:
Error: Could not find or load main class ch.qos.logback.classic.net.SimpleSocketServer
I can't find an example of what I'm trying to achieve, and also I don't understand how can I start a SimpleSocketServer on my prod server only launching a java command and without installing anything.
Can someone clarify that for me? Thanks

Unable to Connect to PowerDesigner through java

I have a requirement to automate the process of connecting to PowerDesigner through java to extract the models from repository. I have used the OLE Automation JavaProxy document to configure the process.
when trying to run, I'm getting this error:
Exception in thread "main" java.lang.NoSuchMethodError: org.eclipse.swt.internal.ole.win32.COM.CoCreateInstance(Lorg/eclipse/swt/internal/ole/win32/GUID;IILorg/eclipse/swt/internal/ole/win32/GUID;[I)I
at
com.sybase.stf.powerdesigner.PdCommon.Application.start(Application.java:57)
at
com.sybase.stf.powerdesigner.PdCommon.Application.(Application.java:31)
at
com.sybase.stf.powerdesigner.PdCommon.Application.getInstance(Application.java:42)
at OOM.src.createOOM.CreateOOM.CreateAndDisplayOOM(CreateOOM.java:37)
at OOM.src.createOOM.CreateOOM.main(CreateOOM.java:28)
Versions:
PowerDesigner - 16.5.5.0
Eclipse 4.18
Supporting Jars:
pdj2com_16.5.5.jar
org.eclipse.swt.win32.win32.x86_64_3.115.100.v20201202-1103.jar
The above issue is resolved Thanks!
Now i'm facing trouble connecting again with the below issue.
org.eclipse.swt.SWTException: Action can not be performed. result = -2137456383 (com.sybase.stf.powerdesigner.com.COMException: COM Failure [HRESULT: 0x80990101])
at org.eclipse.swt.ole.win32.OLE.error(OLE.java:345)
at com.sybase.stf.powerdesigner.com.COMException.raiseOnFail(COMException.java:83)
at com.sybase.stf.powerdesigner.com.IDispatchEx.raisingInvoke(IDispatchEx.java:231)
at com.sybase.stf.powerdesigner.PdCommon.IApplication.CreateModel(IApplication.java:84)
at OOM.src.createOOM.CreateOOM.createOOM(CreateOOM.java:68)
at OOM.src.createOOM.CreateOOM.CreateAndDisplayOOM(CreateOOM.java:42)
at OOM.src.createOOM.CreateOOM.main(CreateOOM.java:28)
Caused by: com.sybase.stf.powerdesigner.com.COMException: COM Failure [HRESULT: 0x80990101]
at com.sybase.stf.powerdesigner.com.COMException.raiseOnFail(COMException.java:88)
... 5 more
Is there anything else to add to the code to connect?
I am not able to find any solution online. Kindly help me figure this out. Thanks in advance.
The swt.win.win32.x86_64 jar is for 64-bit, so I assume you're using a 64-bit Java.
In my case, the pdj2com jar is named something like pdj2com_16.N.Nx64.jar for the 64-bit version. pdj2com_16.N.N.jar is the name of the 32-bit version.

java.lang.NoClassDefFoundError due to Native library issue?

I am trying to use timezone finding service (https://github.com/dustin-johnson/timezonemap) in my Flink java job.
It worked fine in local machine and it worked fine initially in Flink server, but after 2-3 restart of the job it started giving NoClassDefFoundError error-
java.lang.NoClassDefFoundError: Could not initialize class com.github.luben.zstd.ZstdInputStream
at org.apache.commons.compress.compressors.zstandard.ZstdCompressorInputStream.<init>(ZstdCompressorInputStream.java:43)
at us.dustinj.timezonemap.data.DataLocator.getDataInputStream(DataLocator.java:22)
at us.dustinj.timezonemap.TimeZoneMap.forRegion(TimeZoneMap.java:92)
at us.dustinj.timezonemap.TimeZoneMap.forEverywhere(TimeZoneMap.java:60)
at dtap.service.TimezoneService.<init>(TimezoneService.java:17)
at functions.processors.KernelIntgrationProcessor.processElement(KernelIntgrationProcessor.java:47)
at functions.processors.KernelIntgrationProcessor.processElement(KernelIntgrationProcessor.java:23)
at org.apache.flink.streaming.api.operators.ProcessOperator.processElement(ProcessOperator.java:66)
at org.apache.flink.streaming.runtime.io.StreamInputProcessor.processInput(StreamInputProcessor.java:202)
at org.apache.flink.streaming.runtime.tasks.OneInputStreamTask.run(OneInputStreamTask.java:105)
at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:300)
at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704)
at java.lang.Thread.run(Thread.java:748)
When I looked at the code of ZstdCompressorInputStream class, it is loading native libraries.
And I found from below page that native library unloading issue could be the reason for this.
https://www.ibm.com/support/knowledgecenter/SS7K4U_8.5.5/com.ibm.websphere.zseries.doc/ae/rtrb_classload_viewer.html
Does anyone has faced similar issue or has idea to solve this problem?
Thanks
Mahendra

JBOSS came UP but GUI doesn't

Recently, we were facing some issues related to timesten configuration and as per Oracle's suggestion we gave a system reboot to verify the changes.
Post system reboot we found that one of our internal GUI ( related to the application ) didn't came up automatically. Although we restarted the JBOSS process manually post-reboot , process came UP well but GUI doesn't.
I went through the JBOSS startup.log as well but found to see so many errors there like:
error installing to real name=vfsfile jboss
org.jboss.deployers.spi.DeploymentException: Error deploying: jboss:service=Hypersonic,database=localDB
ERROR [AbstractKernelController] Error installing to Start: name=jboss:database=localDB,service=Hypersonic state=Create mode=Manual requiredState=Installed
log4j error: setFile(null,true) call failed.
java.sql.SQLException: The database is already in use by another process: org.hsqldb.persist.NIOLockFile
........and many more
Literally, i have gone through every possible workaround available on internet with no success. Please pardon my question's framing methodology if it's sounds too be least technical.
I am new to JBOSS so need your help in resolving this issue.
Environment is Solaris 10 and JBOSS version is "jboss-5.1.0.GA"

Broken pipe error when running Gradle test

I've got a problem regarding running tests in gradle. I know that in other machines my gradle config works but in mine unfortunately not. We have got junit test and testNG and both of them, when try to execute, produces stacktrace like this:
Could not write standard input into: Gradle Worker 1.
java.io.IOException: The pipe is being closed
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:318)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:53)
at org.gradle.internal.concurrent.DefaultExecutorFactory$StoppableExecutorImpl$1.run(DefaultExecutorFactory.java:66)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Which continuously will produce this error with different number (Gradle Worker 2, Gradle Worker 3, etc). Has anybody ever faced similar problem?
I'm using gradle 1.6.
EDIT: I forgot to tell that I'm using gradle wrapper
EDIT: After changing to version 1.12 I'm receiving error:
Caused by: org.gradle.api.InvalidUserDataException: Could not create task '(custome taks name)': Unknown argument(s) in task definition: [mustRunAfter]
11:20:17.990 [ERROR] [org.gradle.BuildExceptionReporter] at org.gradle.api.internal.project.taskfactory.TaskFactory.validateArgs(TaskFactory.java:147)
11:20:17.991 [ERROR] [org.gradle.BuildExceptionReporter] at org.gradle.api.internal.project.taskfactory.TaskFactory.checkTaskArgsAndCreateDefaultValues(TaskFactory.java:134)
11:20:17.991 [ERROR] [org.gradle.BuildExceptionReporter] at org.gradle.api.internal.project.taskfactory.TaskFactory.createTask(TaskFactory.java:72)
EDIT: Ok I think I have found what cause this problem - it was Antivirus. When I disable it script moves a little bit forward but ended with:
org.gradle.messaging.remote.internal.ConnectException: Could not connect to server [e42b57ea-ced6-4bb6-9369-3186ab4983d6 port:63631, addresses:[/127.0.0.1, /0:0:0:0:0:0:0:1]]. Tried addresses: [/127.0.0.1, /0:0:0:0:0:0:0:1].
at org.gradle.messaging.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:62)
at org.gradle.messaging.remote.internal.hub.MessageHubBackedClient.getConnection(MessageHubBackedClient.java:35)
at org.gradle.process.internal.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:54)
at org.gradle.process.internal.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:35)
at org.gradle.process.internal.child.ImplementationClassLoaderWorker.execute(ImplementationClassLoaderWorker.java:85)
at org.gradle.process.internal.child.ImplementationClassLoaderWorker.execute(ImplementationClassLoaderWorker.java:41)
at org.gradle.process.internal.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:43)
at org.gradle.process.internal.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:32)
at org.gradle.process.internal.launcher.BootstrapClassLoaderWorker.call(BootstrapClassLoaderWorker.java:46)
at org.gradle.process.internal.launcher.BootstrapClassLoaderWorker.call(BootstrapClassLoaderWorker.java:32)
at jarjar.org.gradle.process.internal.launcher.GradleWorkerMain.run(GradleWorkerMain.java:32)
at jarjar.org.gradle.process.internal.launcher.GradleWorkerMain.main(GradleWorkerMain.java:37)
Caused by: java.net.ConnectException: Connection refused: connect
Searching with google on this error gives me nothing (only some old gradle 1.1 errors info). Anybody faced this kind of problem?
Whew, after whole day struggling with gradle I think I've finally found what cause problems. Guilty of that was antivirus - COMODO. After uninstalling it and installing another one, everything started to working fine. So all Gradle Users - please be careful with COMODO because even disabled, it could cause problems when using Gradle . Be aware :)
I faced the similar issue while running the test classes. I can able to build the project, but I couldn't able to run, so after some research I found one of the solution, which is to remove the some of the workerThread-jar from the .gradle/cache folder. I tried and it worked for me. You can use below commands to solve this.
In terminal I invoked gradle test --info command,
In response, I receive the logs, where I searched the for the
Gradle worker Daemon
key, to fetch the location of my gradle-worker.jar. Typically in windows you can find this under the following location C:\Users\.gradle\caches\\workerMain\gradle-worker.jar.
To fix this, one approach is to remove the the workerMain folder itself, and run the test classes.
You really don't need to disable or remove COMODO Antivirus. Just do the following steps:
Go to COMODO Client Panel
Open the Advanced Tasks option under the Task menu.
See all active tasks and select the Gradle process that is being blocked.
Right click on the task and 'Add to Trusted Files'
I got this same error, except it said Gradle Worker 2. I restarted my command line (powershell) and then tried to build again with gradle and it worked. I had changed my environment variables to point to the Java version the project was expecting and had not thought to restart- not sure if that had anything to do with it.
None of the other answers here were helpful in my case.
What I had to do was disable the coroutine agent.
Open Settings (File - Settings)
Select Kotlin under Build... - Debugger - Data Views
Check the box "Disable coroutine agent"
Alternatively, go to your build.gradle file and upgrade the version used for the coroutines dependencies.

Categories