We Keep Coding

Elastic APM Agent Configuration Options are not working

Elasticsearch,kibana and apm-server are installed in a ec2 server
I have installed automatic java agent attach to another server to track jenkins app
Agent is getting attached to the process but dynamic configuration options are not working
Apmagent directory: (command ls)
apm-agent-attach-standalone.jar elasticapm.properties
elasticapm.properties file
service_name="jenkins-dev"
server_url="http://x.x.x.x:8200"
recording=true
enabled=true
log_level="DEBUG"
log_file=_AGENT_HOME_/logs/elastic-apm.log
Attach Command:
sudo java -jar apm-agent-attach-standalone.jar --include '.jenkins.'
->This doesn't pick configuration file but attached the agent
so i used below command to update
sudo java -jar apm-agent-attach-standalone.jar --include '.*jenkins.*' --config recording=false,enabled=false
sudo java -jar apm-agent-attach-standalone.jar --include '.*jenkins.*' --config
config_file=elasticapm.properties log_file=/etc/apmagents/apm.log
Log:
2021-04-12 10:47:20,338 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error trying to connect to APM Server. Some details about SSL configurations corresponding the current connection are logged at INFO level.
2021-04-12 10:47:20,339 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type JSON_WRITER with this error: Connection refused (Connection refused)
2021-04-12 10:47:20,339 [elastic-apm-server-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 36 seconds (+/-10%)
2021-04-12 10:47:24,345 [elastic-apm-remote-config-poller] ERROR co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Connection refused (Connection refused)
Query:
1.Which is the right way to use the configuration options in command line?
2.Do we need to create a log file or it will create if log_file is used..now its polluting the application log

Try to specify the config_file using the following notation:
-Delastic.apm.config_file=elasticapm.properties
The attacher can create the log file depending on the settings configured during startup. See the [1] current code for a better understanding.
[1] https://github.com/elastic/apm-agent-java/blob/0465d479430172c3e745afd2ef5b62a3da6b60aa/apm-agent-attach-cli/src/main/java/co/elastic/apm/attach/AgentAttacher.java#L79

Timed out waiting for container port to open (localhost ports: [32773] should be listening)

I am trying to use https://github.com/testcontainers/testcontainers-scala that is inherent from https://www.testcontainers.org/ as the following:
final class MessageSpec extends BddSpec
with ForAllTestContainer
with BeforeAndAfterAll {
override val container = GenericContainer("sweetsoft/sapmock").configure{ c =>
c.addExposedPort(8080)
c.withNetwork(Network.newNetwork())
}
override def beforeAll() {
}
feature("Process incoming messages") {
When I run the test with the command sbt test, I've got the following exception:
15:22:23.171 [pool-7-thread-2] ERROR 🐳 [sweetsoft/sapmock:latest] - Could not start container
org.testcontainers.containers.ContainerLaunchException: Timed out waiting for container port to open (localhost ports: [32775] should be listening)
at org.testcontainers.containers.wait.strategy.HostPortWaitStrategy.waitUntilReady(HostPortWaitStrategy.java:47)
at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:35)
at org.testcontainers.containers.wait.HostPortWaitStrategy.waitUntilReady(HostPortWaitStrategy.java:23)
at org.testcontainers.containers.wait.strategy.AbstractWaitStrategy.waitUntilReady(AbstractWaitStrategy.java:35)
at org.testcontainers.containers.GenericContainer.waitUntilContainerStarted(GenericContainer.java:582)
The image is a local image:
docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
sweetsoft/sapmock latest f02be90356e7 3 hours ago 664MB
openjdk 8 bec43387959a 11 days ago 625MB
quay.io/testcontainers/ryuk 0.2.3 64849fd2d464 3 months ago 10.7MB
The question is, why is it waiting for 32775 port? And for what is the port good for?
Update
Maybe this log will help:
15:47:47.274 [pool-7-thread-4] INFO org.testcontainers.dockerclient.DockerClientProviderStrategy - Found Docker environment with Environment variables, system properties and defaults. Resolved:
dockerHost=unix:///var/run/docker.sock
apiVersion='{UNKNOWN_VERSION}'
registryUrl='https://index.docker.io/v1/'
registryUsername='developer'
registryPassword='null'
registryEmail='null'
dockerConfig='DefaultDockerClientConfig[dockerHost=unix:///var/run/docker.sock,registryUsername=developer,registryPassword=<null>,registryEmail=<null>,registryUrl=https://index.docker.io/v1/,dockerConfigPath=/home/developer/.docker,sslConfig=<null>,apiVersion={UNKNOWN_VERSION},dockerConfig=<null>]'
15:47:47.275 [pool-7-thread-4] INFO org.testcontainers.DockerClientFactory - Docker host IP address is localhost
15:47:47.277 [pool-7-thread-4] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: com.github.dockerjava.core.exec.InfoCmdExec#51a07bb5
15:47:47.389 [pool-7-thread-4] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: com.github.dockerjava.core.exec.VersionCmdExec#70fc9b37
15:47:47.392 [pool-7-thread-4] INFO org.testcontainers.DockerClientFactory - Connected to docker:
Server Version: 18.09.6
API Version: 1.39
Operating System: Ubuntu 18.04.2 LTS
Total Memory: 7976 MB
15:47:47.395 [pool-7-thread-4] DEBUG com.github.dockerjava.core.command.AbstrDockerCmd - Cmd: ListImagesCmdImpl[imageNameFilter=quay.io/testcontainers/ryuk:0.2.3,showAll=false,filters=com.github.dockerjava.core.util.FiltersBuilder#0,execution=com.github.dockerjava.core.exec.ListImagesCmdExec#562a343]
15:47:47.417 [pool-7-thread-4] DEBUG org.testcontainers.utility.RegistryAuthLocator - Looking up auth config for image: quay.io/testcontainers/ryuk:0.2.3
15:47:47.417 [pool-7-thread-4] DEBUG org.testcontainers.utility.RegistryAuthLocator - RegistryAuthLocator has configFile: /home/developer/.docker/config.json (does not exist) and commandPathPrefix:
15:47:47.418 [pool-7-thread-4] WARN org.testcontainers.utility.RegistryAuthLocator - Failure when attempting to lookup auth config (dockerImageName: quay.io/testcontainers/ryuk:0.2.3, configFile: /home/developer/.docker/config.json. Falling back to docker-java default behaviour. Exception message: /home/developer/.docker/config.json (No such file or directory)
15:47:47.418 [pool-7-thread-4] DEBUG org.testcontainers.dockerclient.auth.AuthDelegatingDockerClientConfig - Effective auth config [null]

Original java library has answer to your port question.
https://www.testcontainers.org/features/networking/
Note that this exposed port number is from the perspective of the
container.
From the host's perspective Testcontainers actually exposes this on a
random free port. This is by design, to avoid port collisions that may
arise with locally running software or in between parallel test runs.
Because there is this layer of indirection, it is necessary to ask
Testcontainers for the actual mapped port at runtime. This can be done
using the getMappedPort method, which takes the original (container)
port as an argument
In Scala library you can get this mapped port by calling
container.mappedPort(yourExposedPort)
Error is most likely related to this concept, you need to expose that port in advance, inside your docker image. Make sure that you either have expose 8080 command somewhere in your dockerfile or any image that is used to build yours have it

HBase 1.2.1 standalone in Docker unable to connect

I want to connect to HBase running in standalone in a docker, using Java and the HBase API
I use this code to connect :
Configuration config = HBaseConfiguration.create();
config.set("hbase.zookeeper.quorum", "163.172.142.199");
config.set("hbase.zookeeper.property.clientPort","2181");
HBaseAdmin.checkHBaseAvailable(config);
Here is my /etc/hosts file
127.0.0.1 localhost
XXX.XXX.XXX.XXX hbase-srv
Here is the /etc/hosts file from my docker (named hbase-srv)
XXX.XXX.XXX.XXX hbase-srv
With this configuration, I get a connection refused error :
INFO | Initiating client connection, connectString=163.172.142.199:2181 sessionTimeout=90000 watcher=hconnection-0x6aba2b860x0, quorum=163.172.142.199:2181, baseZNode=/hbase
INFO | Opening socket connection to server 163.172.142.199/163.172.142.199:2181. Will not attempt to authenticate using SASL (unknown error)
INFO | Socket connection established to 163.172.142.199/163.172.142.199:2181, initiating session
INFO | Session establishment complete on server 163.172.142.199/163.172.142.199:2181, sessionid = 0x15602f8d8dc0002, negotiated timeout = 40000
INFO | Closing zookeeper sessionid=0x15602f8d8dc0002
INFO | Session: 0x15602f8d8dc0002 closed
INFO | EventThread shut down
org.apache.hadoop.hbase.MasterNotRunningException: com.google.protobuf.ServiceException: java.net.ConnectException: Connection refused
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$StubMaker.makeStub(ConnectionManager.java:1560)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation$MasterServiceStubMaker.makeStub(ConnectionManager.java:1580)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveMasterService(ConnectionManager.java:1737)
at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.isMasterRunning(ConnectionManager.java:948)
at org.apache.hadoop.hbase.client.HBaseAdmin.checkHBaseAvailable(HBaseAdmin.java:3159)
at hbase.Benchmark.main(Benchmark.java:26)
However, if I remove the lines XXX.XXX.XXX.XXX hbase-srv from both /etc/hosts files I get the error unknown host : hbase-srv
I have also checked, I can successfully telnet to my hbase docker on the client port.
On the docker, all the ports used by HBase are opened and binded to the same number (60000 on 60000, 2181 on 2181, etc).
I also wanted to add that all was fine when I used this configuration on localhost.
If you can't give me an answer to my problem, could you at least give me a procedure to deploy a standalone hbase on a docker.
UPDATE : Here is my Docker file
FROM java:openjdk-8
ADD hbase-1.2.1 /hbase-1.2.1
WORKDIR /hbase-1.2.1
# ZooKeeper
EXPOSE 2181
# HMaster
EXPOSE 60000
# HMaster Web
EXPOSE 60010
# RegionServer
EXPOSE 60020
# RegionServer Web
EXPOSE 60030
EXPOSE 16010
RUN chmod 755 /hbase-1.2.1/bin/start-hbase.sh
CMD ["/hbase-1.2.1/bin/start-hbase.sh"]
My HBase shell is working, I also tried to open the port using iptables for tcp and udp but still the same problem

There are two problems with your Dockerfile:
use hbase master start instead of start-hbase.sh
regionserver is actually not running on 60020
The 2nd problem is not so easy to solve. If run hbase standalone with version >= 1.2.0 (not sure, I'm running 1.2.0), hbase will use ephemeral port instead of the default port or the port you provide in hbase-site.xml which makes it very hard to provide hbase service in docker using the original version.
I add a property named hbase.localcluster.port.ephemeral and managed to build a standalone hbase in docker, which you can reference here.

mongoDB, connection refused

Show my code
conf.set( "mongo.input.uri" , "mongodb://127.0.0.1/stackoverflow.mrtest" );
conf.set( "mongo.output.uri" , "mongodb://127.0.0.1/stackoverflow.mrtest_out2" );
the code runs without error when the host is localhost or 127.0.0.1. But when the host changed to my ip wlan0 192.168.1.102, it returned the following error
Cluster created with settings {hosts=[192.168.1.102:27017], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms', maxWaitQueueSize=500}
Exception in monitor thread while connecting to server 192.168.1.102:27017
com.mongodb.MongoSocketOpenException: Exception opening socket
at com.mongodb.connection.SocketStream.open(SocketStream.java:63)
at com.mongodb.connection.InternalStreamConnection.open(InternalStreamConnection.java:114)
at com.mongodb.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:127)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.net.ConnectException: connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at com.mongodb.connection.SocketStreamHelper.initialize(SocketStreamHelper.java:50)
at com.mongodb.connection.SocketStream.open(SocketStream.java:58)
... 3 more
I have open the port 27017.
sudo iptables -A INPUT -ptcp --dport 27017 -j ACCEPT
My OS is Ubuntu 14.04.
How should I fix it? Thank you!

By default MongoDB only binds to the loopback interface which makes it only accessible from localhost. To change that you need to edit this line in mongod.conf file;
# /etc/mongod.conf
# Listen to local interface only. Comment out to listen on all interfaces.
bind_ip = 127.0.0.1
you can change it to bind_ip = 127.0.0.1,192.168.1.102 to allow LAN and local connections or you can remove or comment out that line to allow all connections.
For more info : MongoDB – Allow remote access

There could be several reasons of it , which in short can be concluded as Your Application is unable to Communicate mongoDB service
1.Check your MongoDB using the same IP configured in your application.yml file,
If not then configure the same used by MongoDB:
spring:
profiles:
active: dev
---
spring:
profiles: dev
data:
mongodb:
host: localhost
port: 27017
Here i assumed my mongo running on localhost, and port 27017, so i configured accordingly.
Check whether your MongoDB service up and running , How to check ?
Execute following command in your terminal
sudo service mongodb status
<pre><code>
● mongodb.service - An object/document-oriented database
Loaded: loaded (/lib/systemd/system/mongodb.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2020-07-03 20:10:15 IST; 1min 54s ago
Docs: man:mongod(1)
Main PID: 14305 (mongod)
Tasks: 23 (limit: 4915)
CGroup: /system.slice/mongodb.service
└─14305 /usr/bin/mongod --unixSocketPrefix=/run/mongodb --config /etc/mongodb.conf</pre></code>
If Status not visible as active and running, you need to start/restart the service
sudo service mongodb restart

I was able to determine that it was an issue with the bind parameter in the /etc/mongod.conf. Instead of commenting it out I set it to 0.0.0.0 to all for remote access.
# network interfaces
net:
port: 27017
bindIp: 0.0.0.0

Exception: com.mongodb.MongoSocketOpenException: Exception opening socket
Solution:
Verify whether you have started "mongo daemon" or not.
Windows Terminal: mongod.exe
Linux Termina: mongod

If you're using Spring Boot and are following the Quick Start, make sure you put this configuration in your application.properties
spring.data.mongodb.uri=[YOUR_URI]

Stop jboss 5.x from shell script

I am writing some small shell script utility which automates deployment process. I ready many links related to command for shutting down the jboss server in linux so that I can shutdown the server from my utility.
Read some links which will tell how to do it in jboss 7 or later but not in old jboss versions like 5.x. One such is
"$JBOSS_HOME/bin/jboss-cli.sh" --connect command=:shutdown
I want to shutdown the jboss 5.2 version from my shell script?
Edit: Jboss 5 version has shutdown.sh in the bin folder. But I am getting the below exception when I say
sh shutdown.sh -S
[john#mymachine bin]$ sh shutdown.sh -S
Exception in thread "main" javax.naming.CommunicationException: Could not obtain connection to any of these urls: localhost:1099 [Root exception is javax.naming.CommunicationException: Failed to connect to server localhost/127.0.0.1:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost/127.0.0.1:1099 [Root exception is java.net.ConnectException: Connection refused]]]
at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1851)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:720)
at org.jnp.interfaces.NamingContext.lookup(NamingContext.java:713)
at javax.naming.InitialContext.lookup(InitialContext.java:411)
at org.jboss.Shutdown.main(Shutdown.java:225)
Caused by: javax.naming.CommunicationException: Failed to connect to server localhost/127.0.0.1:1099 [Root exception is javax.naming.ServiceUnavailableException: Failed to connect to server localhost/127.0.0.1:1099 [Root exception is java.net.ConnectException: Connection refused]]
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:362)
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:287)
at org.jnp.interfaces.NamingContext.checkRef(NamingContext.java:1821)
... 4 more
Caused by: javax.naming.ServiceUnavailableException: Failed to connect to server localhost/127.0.0.1:1099 [Root exception is java.net.ConnectException: Connection refused]
at org.jnp.interfaces.NamingContext.getServer(NamingContext.java:332)
... 6 more
Caused by: java.net.ConnectException: Connection refused
Any idea ??
Regards

JBoss 5 has a shutdown.sh, from the docs:
usage: shutdown [options] <operation>
options:
-h, --help Show this help message (default)
-D<name>[=<value>] Set a system property
-- Stop processing options
-s, --server=<url> Specify the JNDI URL of the remote server
-n, --serverName=<url> Specify the JMX name of the ServerImpl
-a, --adapter=<name> Specify JNDI name of the MBeanServerConnection to use
-u, --user=<name> Specify the username for authentication
-p, --password=<name> Specify the password for authentication
operations:
-S, --shutdown Shutdown the server
-e, --exit=<code> Force the VM to exit with a status code
-H, --halt=<code> Force the VM to halt with a status code

I believe you should check your JBoss configuration and look for the JNP port. Probably you are running with a non default port and that's why you receive the connection refused exception.
Have a look at the file server/default/conf/bindingservice.beans/META-INF/bindings-jboss-beans.xml.
Reference https://community.jboss.org/wiki/ConfigurePorts

If you have on ubuntu with default visual interface, open the "System Monitor", change to tab Process and filter by "jboss", it's will result in a java process, kill then.