Connecting to MongoDB from Maven exec fails but not from the jar - java

The command below resolves all required dependencies to execute my program:
mvn exec:java -Dexec.mainClass='com.cnrg.cdproc.App'
but I can't connect to MongoDB with it, when it's possible to execute the same code from the jar using this command: java -jar target/cdproc-1.0.0-SNAPSHOT-shaded.jar.
Here's the error I'm getting starting the app via maven:
[Thread-1] INFO org.mongodb.driver.cluster - Cluster created with settings {hosts=[localhost:9032], mode=SINGLE, requiredClusterType=UNKNOWN, serverSelectionTimeout='30000 ms'}
[cluster-ClusterId{value='5fc969c67d495439ce9a3796', description='null'}-localhost:9032] INFO org.mongodb.driver.cluster - Exception in monitor thread while connecting to server localhost:9032
com.mongodb.MongoInterruptedException: Interrupted acquiring a permit to retrieve an item from the pool
at com.mongodb.internal.connection.ConcurrentPool.acquirePermit(ConcurrentPool.java:203)
at com.mongodb.internal.connection.ConcurrentPool.get(ConcurrentPool.java:140)
at com.mongodb.internal.connection.ConcurrentPool.get(ConcurrentPool.java:123)
at com.mongodb.internal.connection.PowerOfTwoBufferPool.getByteBuffer(PowerOfTwoBufferPool.java:82)
at com.mongodb.internal.connection.PowerOfTwoBufferPool.getBuffer(PowerOfTwoBufferPool.java:77)
at com.mongodb.internal.connection.SocketStream.getBuffer(SocketStream.java:93)
at com.mongodb.internal.connection.InternalStreamConnection.getBuffer(InternalStreamConnection.java:684)
at com.mongodb.internal.connection.ByteBufferBsonOutput.getByteBufferAtIndex(ByteBufferBsonOutput.java:93)
at com.mongodb.internal.connection.ByteBufferBsonOutput.getCurrentByteBuffer(ByteBufferBsonOutput.java:82)
at com.mongodb.internal.connection.ByteBufferBsonOutput.writeByte(ByteBufferBsonOutput.java:77)
at org.bson.io.OutputBuffer.write(OutputBuffer.java:150)
at org.bson.io.OutputBuffer.writeInt32(OutputBuffer.java:56)
at com.mongodb.internal.connection.RequestMessage.writeMessagePrologue(RequestMessage.java:158)
at com.mongodb.internal.connection.RequestMessage.encode(RequestMessage.java:137)
at com.mongodb.internal.connection.CommandMessage.encode(CommandMessage.java:59)
at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:269)
at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:83)
at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:33)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:107)
at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initialize(InternalStreamConnectionInitializer.java:62)
at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:144)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:188)
at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:144)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.lang.InterruptedException
at java.base/java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1343)
at java.base/java.util.concurrent.Semaphore.acquire(Semaphore.java:318)
at com.mongodb.internal.connection.ConcurrentPool.acquirePermit(ConcurrentPool.java:199)
... 23 more
What's the difference between starting the program through the jar and starting the program through maven? How to troubleshoot that issue if possible and start the program from maven?

Related

java.lang.NoSuchMethodError: 'java.lang.String org.slf4j.helpers.Util.safeGetSystemProperty(java.lang.String)'

Getting an issue when starting Spark on Kubernetes in client mode using JupyterHub.
21/10/05 03:54:33 ERROR SparkUncaughtExceptionHandler: Uncaught exception in thread Thread[Executor task launch worker-0,5,main]
java.lang.NoSuchMethodError: 'java.lang.String org.slf4j.helpers.Util.safeGetSystemProperty(java.lang.String)'
at org.slf4j.impl.VersionUtil.getJavaMajorVersion(VersionUtil.java:11)
at org.slf4j.impl.Log4jMDCAdapter.<clinit>(Log4jMDCAdapter.java:37)
at org.slf4j.impl.StaticMDCBinder.getMDCA(StaticMDCBinder.java:59)
at org.slf4j.MDC.<clinit>(MDC.java:74)
at org.apache.spark.executor.Executor.org$apache$spark$executor$Executor$$setMDCForTask(Executor.scala:740)
at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:432)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
21/10/05 03:54:33 INFO MemoryStore: MemoryStore cleared
21/10/05 03:54:33 INFO BlockManager: BlockManager stopped
21/10/05 03:54:33 INFO ShutdownHookManager: Shutdown hook called
I've also confirmed that the slf4j versions and jars are the same in the driver and executor as some articles suggest that it could be a package version mismatch.
Jars installed are
jcl-over-slf4j-1.7.30.jar
jul-to-slf4j-1.7.30.jar
slf4j-api-1.7.30.jar
slf4j-log4j12-1.7.30.jar
in both executor and driver pods. Would you know if there's something I missed? Thank you.
Was able to make it work. Built the executor pods from Spark's build tool and added new packages on top of that. Also ensured jars exist in the driver pod (Jupyter Notebook) and executor pods.

Elastic APM Agent Configuration Options are not working

Elasticsearch,kibana and apm-server are installed in a ec2 server
I have installed automatic java agent attach to another server to track jenkins app
Agent is getting attached to the process but dynamic configuration options are not working
Apmagent directory: (command ls)
apm-agent-attach-standalone.jar elasticapm.properties
elasticapm.properties file
service_name="jenkins-dev"
server_url="http://x.x.x.x:8200"
recording=true
enabled=true
log_level="DEBUG"
log_file=_AGENT_HOME_/logs/elastic-apm.log
Attach Command:
sudo java -jar apm-agent-attach-standalone.jar --include '.jenkins.'
->This doesn't pick configuration file but attached the agent
so i used below command to update
sudo java -jar apm-agent-attach-standalone.jar --include '.*jenkins.*' --config recording=false,enabled=false
sudo java -jar apm-agent-attach-standalone.jar --include '.*jenkins.*' --config
config_file=elasticapm.properties log_file=/etc/apmagents/apm.log
Log:
2021-04-12 10:47:20,338 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Error trying to connect to APM Server. Some details about SSL configurations corresponding the current connection are logged at INFO level.
2021-04-12 10:47:20,339 [elastic-apm-server-reporter] ERROR co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Failed to handle event of type JSON_WRITER with this error: Connection refused (Connection refused)
2021-04-12 10:47:20,339 [elastic-apm-server-reporter] INFO co.elastic.apm.agent.report.IntakeV2ReportingEventHandler - Backing off for 36 seconds (+/-10%)
2021-04-12 10:47:24,345 [elastic-apm-remote-config-poller] ERROR co.elastic.apm.agent.configuration.ApmServerConfigurationSource - Connection refused (Connection refused)
Query:
1.Which is the right way to use the configuration options in command line?
2.Do we need to create a log file or it will create if log_file is used..now its polluting the application log
Try to specify the config_file using the following notation:
-Delastic.apm.config_file=elasticapm.properties
The attacher can create the log file depending on the settings configured during startup. See the [1] current code for a better understanding.
[1] https://github.com/elastic/apm-agent-java/blob/0465d479430172c3e745afd2ef5b62a3da6b60aa/apm-agent-attach-cli/src/main/java/co/elastic/apm/attach/AgentAttacher.java#L79

Not able to start SonarCube server in MacOSx

Running the latest version of sonarcube 7.1.2 and getting the following error.:
Command executed: sudo ./sonar.sh
Wrapper Started as Console
Launching a JVM...
Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
2018.07.01 18:36:05 INFO app[][o.s.a.AppFileSystem] Cleaning or
creating temp directory /Users/aneeshgoel/Downloads/sonarqube-7.2.1/temp
2018.07.01 18:36:05 INFO app[][o.s.a.es.EsSettings] Elasticsearch
listening on /127.0.0.1:9001
2018.07.01 18:36:05 INFO app[][o.s.a.p.ProcessLauncherImpl] Launch
process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from
[/Users/aneeshgoel/Downloads/sonarqube-7.2.1/elasticsearch]:
/Users/aneeshgoel/Downloads/sonarqube-
7.2.1/elasticsearch/bin/elasticsearch -
Epath.conf=/Users/aneeshgoel/Downloads/sonarqube-7.2.1/temp/conf/es
2018.07.01 18:36:05 INFO app[][o.s.a.SchedulerImpl] Waiting for
Elasticsearch to be up and running
2018.07.01 18:36:10 INFO app[][o.e.p.PluginsService] no modules loaded
2018.07.01 18:36:10 INFO app[][o.e.p.PluginsService] loaded plugin
[org.elasticsearch.transport.Netty4Plugin]
2018.07.01 18:36:16 WARN app[][o.s.a.p.AbstractProcessMonitor] Process
exited with exit value [es]: 1
2018.07.01 18:36:16 INFO app[][o.s.a.SchedulerImpl] Process [es] is
stopped
2018.07.01 18:36:16 INFO app[][o.s.a.SchedulerImpl] SonarQube is
stopped
<-- Wrapper Stopped
Then I tried with non sudo user. Command ./sonar.sh
Error got is :
--> Wrapper Started as Console
Launching a JVM...
Wrapper (Version 3.2.3) http://wrapper.tanukisoftware.org
Copyright 1999-2006 Tanuki Software, Inc. All Rights Reserved.
2018.07.01 18:18:16 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /Users/aneeshgoel/Downloads/sonarqube-7.2.1/temp
WrapperSimpleApp: Encountered an error running main: java.nio.file.AccessDeniedException: /Users/aneeshgoel/Downloads/sonarqube-7.2.1/temp/conf/es/elasticsearch.yml
java.nio.file.AccessDeniedException: /Users/aneeshgoel/Downloads/sonarqube-7.2.1/temp/conf/es/elasticsearch.yml
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.implDelete(UnixFileSystemProvider.java:244)
at sun.nio.fs.AbstractFileSystemProvider.delete(AbstractFileSystemProvider.java:103)
at java.nio.file.Files.delete(Files.java:1126)
at org.sonar.process.FileUtils2$DeleteRecursivelyFileVisitor.visitFile(FileUtils2.java:170)
I have tried giving write access also to the directory , but still no luck. Please can someone help in debugging the issue.
SonarQube installation guide, unfortunately, doesn’t say a thing about configuring user for the analysis server. People that are installing it can later forget about it, leaving SonarQube running with root rights for a while.
It is, however pretty simple and straightforward. Prepare sonar system user and change installation directory rights:
You have to run the sonar in the context of sonar user. To create a user called sonar, follow these steps:
groupadd sonar useradd -c "Sonar System User" -d /opt/sonarqube -g
sonar -s /bin/bash sonar chown -R sonar:sonar /opt/sonarqube
Then edit the file present here:
/opt/sonarqube/bin/sonar.sh
Find the line which reads RUN_AS_USER=sonar which will be commented, and then change as sonar and try to run the app now.

Hadoop Yarn job: Wrong FS

I installed a cloudera cluster with a vagrant box.
I get an error when I launch the following example:
hadoop jar /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar grep input output23 'dfs[a-z.]+'
I went to check the log in /var/log/hadoop-yarn.
There several log file, in yarn-yarn-nodemanager-cdh-master.log, there is the following stackstrace:
2015-06-17 11:42:42,398 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1434535025160_0001_000001 (auth:SIMPLE)
2015-06-17 11:42:42,597 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1434535025160_0001_01_
000001 by user vagrant
2015-06-17 11:42:42,762 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Creating a new application reference for app appli
cation_1434535025160_0001
2015-06-17 11:42:42,776 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1434535025160_0001 tran
sitioned from NEW to INITING
2015-06-17 11:42:42,778 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=vagrant IP=10.10.50.5 OPERATION=Start Container Request
TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1434535025160_0001 CONTAINERID=container_1434535025160_0001_01_000001
2015-06-17 11:42:43,997 FATAL org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
java.lang.IllegalArgumentException: Wrong FS: hdfs://var/log/hadoop-yarn, expected: hdfs://cdh-master:8020
at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:645)
at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:193)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:105)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1128)
at org.apache.hadoop.hdfs.DistributedFileSystem$18.doCall(DistributedFileSystem.java:1124)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1124)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:192)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:319)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:443)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:67)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:173)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:106)
at java.lang.Thread.run(Thread.java:744)
2015-06-17 11:42:44,000 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Adding container_1434535025160_0001_01_000001 t
o application application_1434535025160_0001
2015-06-17 11:42:44,001 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..
2015-06-17 11:42:44,034 INFO org.mortbay.log: Stopped HttpServer2$SelectChannelConnectorWithSafeStartup#0.0.0.0:8042
2015-06-17 11:42:44,035 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Applications still running : [application_14345350
I've seen this error
java.lang.IllegalArgumentException: Wrong FS:
hdfs://var/log/hadoop-yarn, expected: hdfs://cdh-master:8020
in the following post: Failed to start Jobtracker and Tasktracker in CDH pseudo cluster, but this did not helped me much.
Does anyone has an idea?
Thx
Change the property yarn.nodemanager.remote-app-log-dir in yarn-site.xml config file either to:
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>hdfs://cdh-master:8020/var/log/hadoop-yarn/apps</value>
</property>
or
<property>
<name>yarn.nodemanager.remote-app-log-dir</name>
<value>/var/log/hadoop-yarn/apps</value>
</property>
The second option will use the default filesystem that should be set to HDFS anyway.

Tez crashes on Hadoop-2.5.2 cluster

I successfully Build Tez-0.6.0 against Hadoop-2.5.2
Then I configured Tez-0.6.0 as like in http://tez.apache.org/install.html
Moved Tez lib package to HDFS location and updated my tez-site.xml
<property>
<name>tez.lib.uris</name>
<value>${fs.default.name}/apps/Tez/,${fs.default.name}/apps/Tez/lib/</value>
</property>
After that I tried the sample test for tez
hadoop jar tez-examples-0.6.0.jar orderedwordcount <input> <output>
But I face following error while running this command
Running OrderedWordCount
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Hadoop/
share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBind
er.class]
SLF4J: Found binding in [jar:file:/C:/Tez/lib
/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
15/04/15 10:47:57 INFO client.TezClient: Tez Client Version: [ component=tez-api
, version=0.6.0, revision=${buildNumber}, SCM-URL=scm:git:https://git-wip-us.apa
che.org/repos/asf/tez.git, buildTime=2015-04-15T01:13:02Z ]
15/04/15 10:48:00 INFO client.TezClient: Submitting DAG application with id: app
lication_1429073725727_0005
15/04/15 10:48:00 INFO Configuration.deprecation: fs.default.name is deprecated.
Instead, use fs.defaultFS
15/04/15 10:48:00 INFO client.TezClientUtils: Using tez.lib.uris value from conf
iguration: hdfs://HA-Cluster/apps/Tez/,hdfs://HA-Cluster/apps/Tez/lib/
15/04/15 10:48:01 INFO client.TezClient: Stage directory /tmp/app/tez/sta
ging doesn't exist and is created
15/04/15 10:48:01 INFO client.TezClient: Tez system stage directory hdfs://HA-cluster
/tmp/app/tez/staging/.tez/application_1429073725727_0005 doesn't ex
ist and is created
15/04/15 10:48:02 INFO client.TezClient: Submitting DAG to YARN, applicationId=a
pplication_1429073725727_0005, dagName=OrderedWordCount
15/04/15 10:48:03 INFO impl.YarnClientImpl: Submitted application application_14
29073725727_0005
15/04/15 10:48:03 INFO client.TezClient: The url to track the Tez AM: http://syn
cserver34:8088/proxy/application_1429073725727_0005/
15/04/15 10:48:03 INFO client.DAGClientImpl: Waiting for DAG to start running
15/04/15 10:48:09 INFO client.DAGClientImpl: DAG completed. FinalState=FAILED
OrderedWordCount failed with diagnostics: [Application application_1429073725727
_0005 failed 2 times due to AM Container for appattempt_1429073725727_0005_00000
2 exited with exitCode: -1073741515 due to: Exception from container-launch: Ex
itCodeException exitCode=-1073741515:
ExitCodeException exitCode=-1073741515:
at org.apache.hadoop.util.Shell.runCommand(Shell.java:538)
at org.apache.hadoop.util.Shell.run(Shell.java:455)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:
702)
at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.la
unchContainer(DefaultContainerExecutor.java:195)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.C
ontainerLaunch.call(ContainerLaunch.java:300)
at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.C
ontainerLaunch.call(ContainerLaunch.java:81)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
.java:615)
at java.lang.Thread.run(Thread.java:744)
1 file(s) moved.
Container exited with a non-zero exit code -1073741515
.Failing this attempt.. Failing the application.]
While Seeing at Resourcemanager log:
15/04/15 12:56:15 ERROR scheduler.SchedulerApplicationAttempt: Error trying to a
ssign container token and NM token to an allocated container container_142908227
1173_0001_01_000001
java.lang.IllegalArgumentException: java.net.UnknownHostException: MasterNode
at org.apache.hadoop.security.SecurityUtil.buildTokenService(SecurityUti
l.java:373)
at org.apache.hadoop.yarn.server.utils.BuilderUtils.newContainerToken(Bu
ilderUtils.java:247)
at org.apache.hadoop.yarn.server.resourcemanager.security.RMContainerTok
enSecretManager.createContainerToken(RMContainerTokenSecretManager.java:199)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerAppl
icationAttempt.pullNewlyAllocatedContainersAndNMTokens(SchedulerApplicationAttem
pt.java:425)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.common.fica.F
iCaSchedulerApp.getAllocation(FiCaSchedulerApp.java:248)
at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.Capa
cityScheduler.allocate(CapacityScheduler.java:736)
at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAtte
mptImpl$AMContainerAllocatedTransition.transition(RMAppAttemptImpl.java:816)
at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAtte
mptImpl$AMContainerAllocatedTransition.transition(RMAppAttemptImpl.java:809)
at org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.
doTransition(StateMachineFactory.java:385)
at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMa
chineFactory.java:302)
at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMach
ineFactory.java:46)
at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine
.doTransition(StateMachineFactory.java:448)
at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAtte
mptImpl.handle(RMAppAttemptImpl.java:649)
at org.apache.hadoop.yarn.server.resourcemanager.rmapp.attempt.RMAppAtte
mptImpl.handle(RMAppAttemptImpl.java:104)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$Applica
tionAttemptEventDispatcher.handle(ResourceManager.java:761)
at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$Applica
tionAttemptEventDispatcher.handle(ResourceManager.java:742)
at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher
.java:173)
at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.ja
va:106)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.UnknownHostException: MasterNode
... 19 more
Problem might be while connecting to nodemanager it unable to handshake with ResourceManager.
If I try in single node hadoop cluster mean It working correctly.
Try to add property
yarn.nodemanager.delete.debug-delay-sec
1200
One thing while running the "launchcontainer.cmd" located in hadoop \tmp..\appcache location.It arise an issue in accessing the Dll for running mapreduce on windows platform, ie MSVCR100.dll is missing to handle the Tez job.As bellow
"The program can't start because MSCVR100.dll is missing from your
computer. Try reinstalling the program to fix this issue"
Provide Full privilege to hadoop-tmp directory and try to replaced/Moved msvcr100.dll(C:\Windows\System32) file in windows machine to run the mapreduce program for TEZ job.

Categories