Jboss server startup exception - java

I am seeing the following exception while starting Jboss-4.0.3SP1 on my Windows 7 64 bit machine. I am researching on this to see if there any clash for the ports etc.. Any thoughts from you would be appreciated!
14:52:13,067 WARN [ServiceController] Problem starting service jboss:service=invoker,type=pooled
java.net.SocketException: Unrecognized Windows Sockets error: 0: JVM_Bind
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at org.jboss.invocation.pooled.server.PooledInvoker.startService(PooledInvoker.java:205)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:274)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:230)
at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:592)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:943)
at $Proxy0.start(Unknown Source)
at org.jboss.system.ServiceController.start(ServiceController.java:428)
at org.jboss.system.ServiceController.start(ServiceController.java:446)
at org.jboss.system.ServiceController.start(ServiceController.java:446)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:592)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:72)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
at $Proxy4.start(Unknown Source)
at org.jboss.deployment.SARDeployer.start(SARDeployer.java:285)
at org.jboss.deployment.MainDeployer.start(MainDeployer.java:989)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:790)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:753)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:737)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:592)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:141)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:80)
at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:118)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:74)
at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:127)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:74)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:245)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:644)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:176)
at $Proxy5.deploy(Unknown Source)
at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:453)
at org.jboss.system.server.ServerImpl.start(ServerImpl.java:330)
at org.jboss.Main.boot(Main.java:187)
at org.jboss.Main$1.run(Main.java:438)
at java.lang.Thread.run(Thread.java:595)

Most likely port conflict. Check list of ports open on your computer with 'netstat' command for example.

Check for whether any of these ports are already running on your machine with the netstat command: 1098, 1099 4444, 4445, 8009, 8080, 8083, 8093. Most likely port 8080 is already running on your machine but check for all of them.

Some times,it might be due to firewall service.
Its also better to try changing port number in server.xml, which is found in \server\default\deploy\jbossweb-tomcat55.sar

Yeah! Fixed the issue, one of the port was in use.
<attribute name="ServerBindPort">4445</attribute>
4445 was in use, so changed to 4446
File name : jboss-4.0.3SP1\server\default\conf\jboss-service.xml
netstat was useful to identify the port
netstat -an |find /i "4445"
TCP 0.0.0.0:4445 0.0.0.0:0 LISTENING

Related

Address Bind exception in Tomcat

We are running Java 8 web application in Apache Tomcat 8.0.36 server since 1 year.
From past few days sometimes when we restart the Tomcat server , the application is not working. Logs are showing address bind exception for Https port.
I tried checking the process associated with the port using below command in cmd:
netstat -aon | findstr [port]
but it is not showing any process associated with the port.
I have tried closing tomcat process from task manager and also tried executing tomact shutdown.bat and then started tomcat again but nothing helped.
If the system is restarted where application is deployed the issue gets resolved but we can't restart the system anytime
Please help what could be the issue and what else can be checked?
Below is the error in tomcat error log:
03-Aug-2018 11:42:32.008 INFO [main] org.apache.coyote.AbstractProtocol.init
Initializing ProtocolHandler ["http-nio-443"]
03-Aug-2018 11:42:32.008 SEVERE [main]
org.apache.coyote.AbstractProtocol.init Failed to initialize end point
associated with ProtocolHandler ["http-nio-443"]
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:351)
at
org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:748)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:459)
at
org.apache.coyote.http11.AbstractHttp11JsseProtocol.init(AbstractHttp11Jss
eProtocol.java:120)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:960)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardService.initInternal
(StandardService.java:568)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardServer.initInternal
(StandardServer.java:869)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.startup.Catalina.load(Catalina.java:580)
at org.apache.catalina.startup.Catalina.load(Catalina.java:603)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:310)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:484)
03-Aug-2018 11:42:32.008 SEVERE [main]
org.apache.catalina.core.StandardService.initInternal Failed to initialize
connector [Connector[HTTP/1.1-443]]
org.apache.catalina.LifecycleException: Failed to initialize component
[Connector[HTTP/1.1-443]]
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:106)
at org.apache.catalina.core.StandardService.initInternal
(StandardService.java:568)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardServer.initInternal
(StandardServer.java:869)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.startup.Catalina.load(Catalina.java:580)
at org.apache.catalina.startup.Catalina.load(Catalina.java:603)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:310)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:484)
Caused by: org.apache.catalina.LifecycleException: Protocol handler
initialization failed
at org.apache.catalina.connector.Connector.initInternal(Connector.java:962)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
... 12 more
Caused by: java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:351)
at
org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:748)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:459)
at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init
(AbstractHttp11JsseProtocol.java:120)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:960)
... 13 more03-Aug-2018 11:42:32.008 INFO [main]
org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["http-
nio-443"]
03-Aug-2018 11:42:32.008 SEVERE [main]
org.apache.coyote.AbstractProtocol.init
Failed to initialize end point associated with ProtocolHandler ["http-nio-
443"]
java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:351)
at org.apache.tomcat.util.net.AbstractEndpoint.init
(AbstractEndpoint.java:748)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:459)
at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init
(AbstractHttp11JsseProto col.java:120)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:960)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardService.initInternal
(StandardService.java:568)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardServer.initInternal
(StandardServer.java:869)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.startup.Catalina.load(Catalina.java:580)
at org.apache.catalina.startup.Catalina.load(Catalina.java:603)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:310)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:484)
03-Aug-2018 11:42:32.008 SEVERE [main]
org.apache.catalina.core.StandardService.initInternal Failed to initialize
connector [Connector[HTTP/1.1-443]]
org.apache.catalina.LifecycleException: Failed to initialize component
[Connector[HTTP/1.1-443]]
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:106)
at org.apache.catalina.core.StandardService.initInternal
(StandardService.java:568)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.core.StandardServer.initInternal
(StandardServer.java:869)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
at org.apache.catalina.startup.Catalina.load(Catalina.java:580)
at org.apache.catalina.startup.Catalina.load(Catalina.java:603)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:310)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:484)
Caused by: org.apache.catalina.LifecycleException: Protocol handler
initialization failed
at org.apache.catalina.connector.Connector.initInternal(Connector.java:962)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:102)
... 12 more
Caused by: java.net.BindException: Address already in use: bind
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.Net.bind(Unknown Source)
at sun.nio.ch.ServerSocketChannelImpl.bind(Unknown Source)
at sun.nio.ch.ServerSocketAdaptor.bind(Unknown Source)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:351)
at org.apache.tomcat.util.net.AbstractEndpoint.init
(AbstractEndpoint.java:748)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:459)
at org.apache.coyote.http11.AbstractHttp11JsseProtocol.init
(AbstractHttp11JsseProtocol.java:120)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:960)
... 13 more
See the update below. We think we have found the root cause and a solution
We have the same problem: A tomcat service that was operating without problems for nearly a year was not able to bind to the port which it used after it was stopped and then restarted.
We usually stop and restart our tomcat service regularly every night, and it used to work without problems. The problem first occurred on July 24th, so after a lot of research that I and our IT have done, I suspect that some kind of change in our IT infrastructure that happened shortly before July 24th causes this problem.
Our server is a Windows Server 2012 R2, 64 Bit, but our IT environment is virtualized, running on vSphere Host Version 6. On what server/infrastructure is your tomcat operating? Is it a virtualized system or is it running on a physical server?
In my opinion, the problem came in either with the last installed patches for Windows
(KB4338815 and/or KB4339093) or with some Hypervisor updates. But our IT is still researching.
For the moment as a workaround we do a server restart when we shutdown the service
and restart it regularly every night. But because the problem does not reliable occur on
each and every shutdown/restart, we restart the server only if it is really necessary. To achieve this we use a listener we have implemented and configured in the server.xml that scans the stderr.log just after the tomcat service has started. If it detects, that a Bind Exception occurs it restarts the server (tomcat service is installed as automatic). If you find the cause of this problem, please post it here, as I will, if I or our IT will find it.
Update August 7th:
We think we have found the root cause. Obviously the Windows Monthly Quality Rollup for July introduced this problem. In our case (Windows Server 2012 R2, 64 Bit) the problematic update was KB4338815 (see https://support.microsoft.com/en-us/help/4338815/windows-81-update-kb4338815).
If you do a quick research on the internet, the problem, which also affects Windows IIS Webserver and MS SQL Server is mostly described as related to KB4338818 (see https://support.microsoft.com/en-us/help/4338818/windows-7-update-kb4338818) but this is the same kind of update for Windows Server 2008 R2 and Windows 7.
Microsoft is already aware of the problem and describes it in its documentation for KB4338815 and KB4338818 in the "Known issues in this update" section of the mentioned articles. Microsoft states that the problem is resolved in KB4338831 (for Windows Server 2012 R2, Windows 8.1) and KB4338821 (for Windows Server 2008 R2, Windows 7)
In our case we installed hot fix KB4345424 (see https://support.microsoft.com/en-us/help/4345424/improvements-and-fixes-windows-8-1-and-server-2012-r2), also available from Microsoft, which addresses this bug and as far as we see, the problem does not exist anymore.

Connection timmed out when accessing Tomcat web manager

I installed tomcat8 on a amazon ubuntu server from a windows computer using an online tutorial as a guide.
It started from command line, a message stated it was really running, but when I try to access the webapp, I get connection timmed out.
I went through a lot of possible online solutions, so I edited my tomcat-users.xml, my server.xml, but I dont get to load the app when I enter http://server_ip:8080
When I try localhost:8080 I get access denied.
How can I verify what is wrong, please?
Oh, I also added tomcat user, group and exported catalina path.
I'm really new to ubuntu, could you guys help me? Obrigada
this is what i get from catalina.out
17-Oct-2016 14:41:36.789 SEVERE [main] org.apache.coyote.AbstractProtocol.init Failed to initialize end point associated with ProtocolHandler ["ajp-nio-8009"]
java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:463)
at sun.nio.ch.Net.bind(Net.java:455)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:228)
at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:866)
at org.apache.tomcat.util.net.AbstractJsseEndpoint.init(AbstractJsseEndpoint.java:213)
at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:575)
at org.apache.catalina.connector.Connector.initInternal(Connector.java:944)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
at org.apache.catalina.core.StandardService.initInternal(StandardService.java:549)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:873)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
at org.apache.catalina.startup.Catalina.load(Catalina.java:606)
at org.apache.catalina.startup.Catalina.load(Catalina.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:311)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:494)
17-Oct-2016 14:41:36.791 SEVERE [main] org.apache.catalina.core.StandardService.initInternal Failed to initialize connector [Connector[AJP/1.3-8009]]
org.apache.catalina.LifecycleException: Failed to initialize component [Connector[AJP/1.3-8009]]
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:112)
at org.apache.catalina.core.StandardService.initInternal(StandardService.java:549)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:873)
at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:107)
at org.apache.catalina.startup.Catalina.load(Catalina.java:606)
at org.apache.catalina.startup.Catalina.load(Catalina.java:629)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.load(Bootstrap.java:311)
17-Oct-2016 05:04:02.282 SEVERE [main] org.apache.catalina.core.StandardServer.await StandardServer.await: create[localhost:8005]:
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:440)
at org.apache.catalina.startup.Catalina.await(Catalina.java:743)
at org.apache.catalina.startup.Catalina.start(Catalina.java:689)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:355)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:495)
The thing is, Im not using these ports anymore, I changed them in server.xml file to port 80.
What actually worked for me was opening 8080 port on aws. As simple as that haha :)

Hadoop word count example without root permissions

I am trying to run this hadoop word count example on a Linux machine without root permissions. I keep getting this error all the time, though.
13/08/27 16:00:08 ERROR security.UserGroupInformation: PriviledgedActionException as:priyankara cause:org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.mapred.JobTrackerNotYetInitializedException: JobTracker is not yet RUNNING
at org.apache.hadoop.mapred.JobTracker.checkJobTrackerState(JobTracker.java:5199)
at org.apache.hadoop.mapred.JobTracker.getNewJobId(JobTracker.java:3543)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)
org.apache.hadoop.ipc.RemoteException: org.apache.hadoop.mapred.JobTrackerNotYetInitializedException: JobTracker is not yet RUNNING
at org.apache.hadoop.mapred.JobTracker.checkJobTrackerState(JobTracker.java:5199)
at org.apache.hadoop.mapred.JobTracker.getNewJobId(JobTracker.java:3543)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:587)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1432)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1428)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1426)
at org.apache.hadoop.ipc.Client.call(Client.java:1113)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:229)
at org.apache.hadoop.mapred.$Proxy2.getNewJobId(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:85)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:62)
at org.apache.hadoop.mapred.$Proxy2.getNewJobId(Unknown Source)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:944)
at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:550)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:580)
at org.apache.hadoop.examples.WordCount.main(WordCount.java:82)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160)
What is happening? How can I fix this?
I think the problem is I don't have root permission on that machine. Is there a way to run this example without root permission or not?
The problem was solved by editing the etc/hosts file.
We have to put our Ip address and host name to /etc/hosts file.
example ,
If your Ip is 192.255.50.1 then your user name is 'user' you have to put them both on your hosts file as ,
192.255.50.1 user
Please check your hostname, is it "localhost" or different name?
You can try to change the host name to be "localhost", than re-start the Hadoop. For instance,
$ hostname localhost
$ $HADOOP_HOME/bin/stop-all.sh
$ $HADOOP_HOME/bin/start-all.sh
This works for my case. For your information, good luck!
I was trying to use minicluster on Windows ,
this suggestion of 192.255.50.1 user fixed my issue , I put 127.0.0.1 myloginname(user). it fixed my issue of Datanode denied communication with namenode:
thanks DilanG

Hadoop Program cannot find installed binary

I have hadoop setup and running on my machine , I am trying to run PEGASUS peta scale mining program , but when I do make , there are lot of warning and following is part of it:
13/03/21 12:58:42 WARN hdfs.DFSClient: DataStreamer Exception: org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/hadi_edge/catepillar_star.edge could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1066)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3507)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3370)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2700(DFSClient.java:2586)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2826)
13/03/21 12:58:42 WARN hdfs.DFSClient: Error Recovery for block null bad datanode[0] nodes =
= null
13/03/21 12:58:42 WARN hdfs.DFSClient: Could not get block locations. Source file "/user/hadoop/hadi_edge/catepillar_star.edge" - Aborting...
put: java.io.IOException: File /user/hadoop/hadi_edge/catepillar_star.edge could only be replicated to 0 nodes, instead of 1
13/03/21 12:58:42 ERROR hdfs.DFSClient: Exception closing file /user/hadoop/hadi_edge/catepillar_star.edge : org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/hadi_edge/catepillar_star.edge could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
org.apache.hadoop.ipc.RemoteException: java.io.IOException: File /user/hadoop/hadi_edge/catepillar_star.edge could only be replicated to 0 nodes, instead of 1
at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:1558)
at org.apache.hadoop.hdfs.server.namenode.NameNode.addBlock(NameNode.java:696)
at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:563)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1382)
at org.apache.hadoop.ipc.Client.call(Client.java:1066)
at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
at $Proxy1.addBlock(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:82)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:59)
at $Proxy1.addBlock(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.locateFollowingBlock(DFSClient.java:3507)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3370)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2700(DFSClient.java:2586)
at org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2826)
I thought this one is a permission issue then I tried sudo make and then I get the folloing error:
Hadoop is not installed in the system.
Please install Hadoop and make sure the hadoop binary is accessible.
make: *** [demo_hadi] Error 127
Hadoop is already installed as i print my jps :
hadoop#polaris:~/PEGASUS$ jps
7814 JobTracker
8061 TaskTracker
3799 FsShell
7718 SecondaryNameNode
9155 FsShell
8881 RunJar
7235 NameNode
6339 RunJar
9236 Jps
Thanks for the time , I really want to know what is going wrong !!!
As visible, Datanode is not up on your system (Same is the probable reason for "could only be replicated to 0 nodes, instead of 1")
It happens sometimes.
Solution : Stop all the hadoop daemons and start them again. Look if all the 5 daemons are running (namenode, Secondary namenode,datanode, jobtracker, tasktracker).
Try running some sample code to test that framework is running fine.
Restart the machine, if the above solution isnt working.
If issue persists, reformat the namenode (not a very good advise though; should be kept as last solution).

How to fast fail JBoss (ie System.exit()) during container initialization when exception is propagated to ServerImpl.doStart()

Specifically, consider the case where JBoss attempts to open a socket on 1099 during ServiceMBeanSupport (rmi) initialization, but port 1099 is already taken.
Current behavior is that the exception is system.err'd and JBoss continues to initialize. Later, during application runtime, a jndi lookup causes an execution thread to hang -- not good and more difficult to diagnose than a failed container initialization.
So the question is, how to fast fail JBoss on initialization exception?
Here's the trace:
java.net.BindException: Address already in use: JVM_Bind
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.PlainSocketImpl.bind(PlainSocketImpl.java:359)
at java.net.ServerSocket.bind(ServerSocket.java:319)
at java.net.ServerSocket.<init>(ServerSocket.java:185)
at javax.net.DefaultServerSocketFactory.createServerSocket(ServerSocketFactory.java:170)
at org.jnp.server.Main.initBootstrapListener(Main.java:331)
at org.jnp.server.Main.start(Main.java:282)
at org.jboss.naming.NamingService.startService(NamingService.java:236)
at org.jboss.system.ServiceMBeanSupport.jbossInternalStart(ServiceMBeanSupport.java:289)
at org.jboss.system.ServiceMBeanSupport.jbossInternalLifecycle(ServiceMBeanSupport.java:245)
at sun.reflect.GeneratedMethodAccessor3.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.system.ServiceController$ServiceProxy.invoke(ServiceController.java:978)
at $Proxy0.start(Unknown Source)
at org.jboss.system.ServiceController.start(ServiceController.java:417)
at org.jboss.system.ServiceController.start(ServiceController.java:435)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:86)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
at $Proxy4.start(Unknown Source)
at org.jboss.deployment.SARDeployer.start(SARDeployer.java:302)
at org.jboss.deployment.MainDeployer.start(MainDeployer.java:1007)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:808)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:771)
at org.jboss.deployment.MainDeployer.deploy(MainDeployer.java:755)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.jboss.mx.interceptor.ReflectedDispatcher.invoke(ReflectedDispatcher.java:155)
at org.jboss.mx.server.Invocation.dispatch(Invocation.java:94)
at org.jboss.mx.interceptor.AbstractInterceptor.invoke(AbstractInterceptor.java:133)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.interceptor.ModelMBeanOperationInterceptor.invoke(ModelMBeanOperationInterceptor.java:142)
at org.jboss.mx.server.Invocation.invoke(Invocation.java:88)
at org.jboss.mx.server.AbstractMBeanInvoker.invoke(AbstractMBeanInvoker.java:264)
at org.jboss.mx.server.MBeanServerImpl.invoke(MBeanServerImpl.java:659)
at org.jboss.mx.util.MBeanProxyExt.invoke(MBeanProxyExt.java:210)
at $Proxy5.deploy(Unknown Source)
at org.jboss.system.server.ServerImpl.doStart(ServerImpl.java:482)
at org.jboss.system.server.ServerImpl.start(ServerImpl.java:362)
at org.jboss.Main.boot(Main.java:200)
at org.jboss.Main$1.run(Main.java:464)
You can't, to my knowledge, but you're right, it is annoying when it does this.
If you're running this on *nix, my suggestion would be to write a bash script which checks for port availability in advance, and only launches JBoss if they're available. Maybe you could do something similar with cygwin on windows.
I also agree that jboss could fail the startup, just as Apache httpd does.
A script to pre-check may not detect that after the check and before JBoss grabs the port another process grabs it. Maybe a rare situation, but possible and also hard to detect.
Since JBoss does not offer to fail when a port cannot be opened, I suggest monitoring the logging. It could be helpful to configure log4j to alert via mail|SNMP|whatever if this situation occurs and propert management could kick in straight away.

Categories