While running the command:
storm jar target/crawlIndexer-1.0-SNAPSHOT.jar org.apache.storm.flux.Flux --local es-injector.flux --sleep 86400000
I get an error saying:
8710 [Thread-26-status-executor[4 4]] ERROR c.d.s.e.p.StatusUpdaterBolt -
Can't connect to ElasticSearch
When running http://localhost:9200/ in browser ES successfully loads up. Kibana also connects to ES. So it must just be the connection from StromCrawler to ElasticSearch.
What could be the issue?
Snippet of full error:
8710 [Thread-26-status-executor[4 4]] ERROR c.d.s.e.p.StatusUpdaterBolt - Can't connect to ElasticSearch
java.lang.RuntimeException: java.net.UnknownHostException: http: nodename nor servname provided, or not known
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:107) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getConnection(ElasticSearchConnection.java:151) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:141) [crawlIndexer-1.0-SNAPSHOT.jar:?]
at org.apache.storm.daemon.executor$fn__5044$fn__5057.invoke(executor.clj:791) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:482) [storm-core-1.1.0.jar:1.1.0]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.net.UnknownHostException: http: nodename nor servname provided, or not known
at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[?:1.8.0_151]
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928) ~[?:1.8.0_151]
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName0(InetAddress.java:1276) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_151]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[?:1.8.0_151]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:104) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
... 6 more
8710 [Thread-18-__metricscom.digitalpebble.stormcrawler.elasticsearch.metrics.MetricsConsumer-executor[2 2]] ERROR c.d.s.e.m.MetricsConsumer - Can't connect to ElasticSearch
java.lang.RuntimeException: java.net.UnknownHostException: http
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:107) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getConnection(ElasticSearchConnection.java:151) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getConnection(ElasticSearchConnection.java:133) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at com.digitalpebble.stormcrawler.elasticsearch.metrics.MetricsConsumer.prepare(MetricsConsumer.java:68) [crawlIndexer-1.0-SNAPSHOT.jar:?]
at org.apache.storm.metric.MetricsConsumerBolt.prepare(MetricsConsumerBolt.java:77) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.daemon.executor$fn__5044$fn__5057.invoke(executor.clj:791) [storm-core-1.1.0.jar:1.1.0]
at org.apache.storm.util$async_loop$fn__557.invoke(util.clj:482) [storm-core-1.1.0.jar:1.1.0]
at clojure.lang.AFn.run(AFn.java:22) [clojure-1.7.0.jar:?]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_151]
Caused by: java.net.UnknownHostException: http
at java.net.InetAddress.getAllByName0(InetAddress.java:1280) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1192) ~[?:1.8.0_151]
at java.net.InetAddress.getAllByName(InetAddress.java:1126) ~[?:1.8.0_151]
at java.net.InetAddress.getByName(InetAddress.java:1076) ~[?:1.8.0_151]
at com.digitalpebble.stormcrawler.elasticsearch.ElasticSearchConnection.getClient(ElasticSearchConnection.java:104) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
... 8 more
8714 [Thread-26-status-executor[4 4]] ERROR o.a.s.util - Async loop died!
java.lang.RuntimeException: java.lang.RuntimeException: java.net.UnknownHostException: http: nodename nor servname provided, or not known
at com.digitalpebble.stormcrawler.elasticsearch.persistence.StatusUpdaterBolt.prepare(StatusUpdaterBolt.java:145) ~[crawlIndexer-1.0-SNAPSHOT.jar:?]
at org.apache.storm.daemon.executor$fn__5044$fn__5057.invoke(executor.clj:791) ~[storm-core-1.1.0.jar:1.1.0]
Perhaps there is something I need to modify in elasticsearch.yml or es-conf.yml? (They both have default settings)
Got the application to work by playing around (for many hours).
Steps:
Reverted back Storm-Crawler-Core and Storm-Crawler-Elastic-Search to version 1.5 in the maven file.
Used the 1.5 flux/yaml config file versions from Storm-Crawler-Elastic-Search
Using Elastic Search 5.4.0
Using Kibana 5.4.0
Your error message above mentions ElasticSearchConnection.java:104 so you must be using a release and not the code from the master branch. Make sure the es conf corresponds to the same version. Did you copy the conf from the master branch? The next release will have a different format for the addresses
Related
I bootstrapped the springboot application with JHipster but I have not been able to get it start up . It says build success but won't start up on a server . There is an embedded tomcat server added to the POM.XML , what could be wrong ?
Caused by: java.lang.IllegalArgumentException: Unable to resolve Configuration with the provided Issuer of "http://localhost:9080/auth/realms/jhipster"
at org.springframework.security.oauth2.client.registration.ClientRegistrations.getBuilder(ClientRegistrations.java:221)
at org.springframework.security.oauth2.client.registration.ClientRegistrations.fromIssuerLocation(ClientRegistrations.java:145)
at org.springframework.boot.autoconfigure.security.oauth2.client.OAuth2ClientPropertiesRegistrationAdapter.getBuilderFromIssuerIfPossible(OAuth2ClientPropertiesRegistrationAdapter.java:83)
at org.springframework.boot.autoconfigure.security.oauth2.client.OAuth2ClientPropertiesRegistrationAdapter.getClientRegistration(OAuth2ClientPropertiesRegistrationAdapter.java:59)
at org.springframework.boot.autoconfigure.security.oauth2.client.OAuth2ClientPropertiesRegistrationAdapter.lambda$getClientRegistrations$0(OAuth2ClientPropertiesRegistrationAdapter.java:53)
at java.base/java.util.HashMap.forEach(HashMap.java:1425)
at org.springframework.boot.autoconfigure.security.oauth2.client.OAuth2ClientPropertiesRegistrationAdapter.getClientRegistrations(OAuth2ClientPropertiesRegistrationAdapter.java:52)
at org.springframework.boot.autoconfigure.security.oauth2.client.servlet.OAuth2ClientRegistrationRepositoryConfiguration.clientRegistrationRepository(OAuth2ClientRegistrationRepositoryConfiguration.java:49)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:64)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:564)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
... 118 common frames omitted
Caused by: org.springframework.web.client.ResourceAccessException: I/O error on GET request for "http://localhost:9080/auth/realms/jhipster/.well-known/openid-configuration": Connection refused: connect; nested exception is java.net.ConnectException: Connection refused: connect
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:785)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:670)
at org.springframework.security.oauth2.client.registration.ClientRegistrations.lambda$oidc$0(ClientRegistrations.java:156)
at org.springframework.security.oauth2.client.registration.ClientRegistrations.getBuilder(ClientRegistrations.java:209)
... 130 common frames omitted
Caused by: java.net.ConnectException: Connection refused: connect
at java.base/sun.nio.ch.Net.connect0(Native Method)
at java.base/sun.nio.ch.Net.connect(Net.java:574)
at java.base/sun.nio.ch.Net.connect(Net.java:563)
at java.base/sun.nio.ch.NioSocketImpl.connect(NioSocketImpl.java:588)
at java.base/java.net.Socket.connect(Socket.java:648)
at java.base/java.net.Socket.connect(Socket.java:597)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:182)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1261)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1194)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1082)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1016)
at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:76)
at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48)
at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:66)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:776)
... 133 common frames omitted
Ensure that keycloak is started before running the springboot app because jhispter provides ready to use keycloak config. The app tries to reach keycloak at localhost:9080.
If you have docker and docker-compose installed you can use the provided docker-compose configuration and start keycloak via docker-compose -f src/main/docker/keycloak.yml up
If you want to start it manually you need to make sure to setup the jhipster realm in your keycloak instance.
You have to check 9080 port
Is it use by another process
You can do it by this command : netstat -aon
Then kill process by processor: taskkill -f /pid process_id
I have a JSF+Richfaces based application that has Axis2 1.6.2 SOAP Webservice implemented on JDK6. The application works fine when I deploy it in Tomcat 6 or 7. But when we deploy it in WebSphere 8.5.5 Network Deployment it gives following error whenever we try to start the application:
[9/28/20 17:02:26:798 PKT] 00000040 WarBasedAxisC E org.apache.axis2.deployment.WarBasedAxisConfigurator <init> org.apache.commons.fileupload.FileUploadException
org.apache.axis2.deployment.DeploymentException: org.apache.commons.fileupload.FileUploadException
at org.apache.axis2.deployment.AxisConfigBuilder.processMessageBuilders(AxisConfigBuilder.java:774)
at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:234)
at org.apache.axis2.deployment.DeploymentEngine.populateAxisConfiguration(DeploymentEngine.java:859)
at org.apache.axis2.deployment.WarBasedAxisConfigurator.<init>(WarBasedAxisConfigurator.java:156)
at org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:569)
at org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:457)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:338)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.init(ServletWrapperImpl.java:168)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.loadOnStartupCheck(ServletWrapper.java:1363)
at com.ibm.ws.webcontainer.webapp.WebApp.doLoadOnStartupActions(WebApp.java:606)
at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinally(WebApp.java:576)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:425)
at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:169)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:749)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:634)
at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:426)
at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:718)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1175)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1370)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:968)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:774)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2182)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:445)
at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:388)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:116)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:994)
at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:502)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)
Caused by: java.lang.NoClassDefFoundError: org.apache.commons.fileupload.FileUploadException
at java.lang.J9VMInternals.verifyImpl(Native Method)
at java.lang.J9VMInternals.verify(J9VMInternals.java:93)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:170)
at java.lang.J9VMInternals.newInstanceImpl(Native Method)
at java.lang.Class.newInstance(Class.java:1564)
at org.apache.axis2.deployment.DescriptionBuilder.processMessageBuilders(DescriptionBuilder.java:230)
at org.apache.axis2.deployment.AxisConfigBuilder.processMessageBuilders(AxisConfigBuilder.java:771)
... 30 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.fileupload.FileUploadException
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:506)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:650)
... 37 more
[9/28/20 17:02:26:811 PKT] 00000040 AxisServlet I org.apache.axis2.util.OnDemandLogger info org.apache.axis2.deployment.DeploymentException: org.apache.commons.fileupload.FileUploadException
[9/28/20 17:02:26:896 PKT] 00000040 FfdcProvider W com.ibm.ws.ffdc.impl.FfdcProvider logIncident FFDC1003I: FFDC Incident emitted on D:\Program Files (x86)\IBM\WebSphere\AppServer\profiles\AppSrv01\logs\ffdc\server1_ad76d55e_20.09.28_17.02.26.838698289724018241690.txt com.ibm.ws.webcontainer.servlet.ServletInstance.init 172
[9/28/20 17:02:26:906 PKT] 00000040 ServletWrappe E com.ibm.ws.webcontainer.servlet.ServletWrapper init Uncaught.init.exception.thrown.by.servlet
[9/28/20 17:02:26:908 PKT] 00000040 webapp E com.ibm.ws.webcontainer.webapp.WebApp commonInitializationFinally SRVE0266E: Error occured while initializing servlets: {0}
javax.servlet.ServletException: javax.servlet.ServletException: org.apache.axis2.deployment.DeploymentException: org.apache.commons.fileupload.FileUploadException
at org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:490)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:338)
at com.ibm.ws.webcontainer.servlet.ServletWrapperImpl.init(ServletWrapperImpl.java:168)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.loadOnStartupCheck(ServletWrapper.java:1363)
at com.ibm.ws.webcontainer.webapp.WebApp.doLoadOnStartupActions(WebApp.java:606)
at com.ibm.ws.webcontainer.webapp.WebApp.commonInitializationFinally(WebApp.java:576)
at com.ibm.ws.webcontainer.webapp.WebAppImpl.initialize(WebAppImpl.java:425)
at com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWebApplication(WebGroupImpl.java:88)
at com.ibm.ws.webcontainer.VirtualHostImpl.addWebApplication(VirtualHostImpl.java:169)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApp(WSWebContainer.java:749)
at com.ibm.ws.webcontainer.WSWebContainer.addWebApplication(WSWebContainer.java:634)
at com.ibm.ws.webcontainer.component.WebContainerImpl.install(WebContainerImpl.java:426)
at com.ibm.ws.webcontainer.component.WebContainerImpl.start(WebContainerImpl.java:718)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:1175)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.fireDeployedObjectStart(DeployedApplicationImpl.java:1370)
at com.ibm.ws.runtime.component.DeployedModuleImpl.start(DeployedModuleImpl.java:639)
at com.ibm.ws.runtime.component.DeployedApplicationImpl.start(DeployedApplicationImpl.java:968)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:774)
at com.ibm.ws.runtime.component.ApplicationMgrImpl.start(ApplicationMgrImpl.java:2182)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:445)
at com.ibm.ws.runtime.component.CompositionUnitImpl.start(CompositionUnitImpl.java:123)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.start(CompositionUnitMgrImpl.java:388)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl.access$500(CompositionUnitMgrImpl.java:116)
at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:994)
at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:502)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1862)
Caused by: javax.servlet.ServletException: org.apache.axis2.deployment.DeploymentException: org.apache.commons.fileupload.FileUploadException
at org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:576)
at org.apache.axis2.transport.http.AxisServlet.init(AxisServlet.java:457)
... 25 more
Caused by: org.apache.axis2.deployment.DeploymentException: org.apache.commons.fileupload.FileUploadException
at org.apache.axis2.deployment.AxisConfigBuilder.processMessageBuilders(AxisConfigBuilder.java:774)
at org.apache.axis2.deployment.AxisConfigBuilder.populateConfig(AxisConfigBuilder.java:234)
at org.apache.axis2.deployment.DeploymentEngine.populateAxisConfiguration(DeploymentEngine.java:859)
at org.apache.axis2.deployment.WarBasedAxisConfigurator.<init>(WarBasedAxisConfigurator.java:156)
at org.apache.axis2.transport.http.AxisServlet.initConfigContext(AxisServlet.java:569)
... 26 more
Caused by: java.lang.NoClassDefFoundError: org.apache.commons.fileupload.FileUploadException
at java.lang.J9VMInternals.verifyImpl(Native Method)
at java.lang.J9VMInternals.verify(J9VMInternals.java:93)
at java.lang.J9VMInternals.initialize(J9VMInternals.java:170)
at java.lang.J9VMInternals.newInstanceImpl(Native Method)
at java.lang.Class.newInstance(Class.java:1564)
at org.apache.axis2.deployment.DescriptionBuilder.processMessageBuilders(DescriptionBuilder.java:230)
at org.apache.axis2.deployment.AxisConfigBuilder.processMessageBuilders(AxisConfigBuilder.java:771)
... 30 more
Caused by: java.lang.ClassNotFoundException: org.apache.commons.fileupload.FileUploadException
at org.eclipse.osgi.internal.loader.BundleLoader.findClassInternal(BundleLoader.java:506)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:422)
at org.eclipse.osgi.internal.loader.BundleLoader.findClass(BundleLoader.java:410)
at org.eclipse.osgi.internal.baseadaptor.DefaultClassLoader.loadClass(DefaultClassLoader.java:107)
at java.lang.ClassLoader.loadClass(ClassLoader.java:650)
... 37 more
I have done a very detailed research on the issue and found that Websphere has its own Axis library and to over ride this, Class loader should be configured to Parent last. I also tried to deploy the sample Axis2 1.6.2 application by following steps given in url just to make sure is Axis2 1.6.2 works or not:
http://manishcusat68.blogspot.com/2015/08/axis2-installation-in-websphere.html
The sample application is deployed and I am able to see the wsdl file.
I am using the same version of axis2 i.e 1.6.2 in my application but I always get the above exception and application never starts.
I have searched every forum, have tried each and every step I found on IBM Websphere website/stackoverflow but I am not able to find any solution of this.
I tried by putting commons-fileupload jar in lib/ext but it didn't work.
I tried to add the patch of commom-fileupload jar in system environment path but it didn't work either.
I have also tried to delete jars from IBM installation/plugin folder as per suggestions in many forums and put commons-fileupload jar in my application/lib but Websphere didn't start.
I desperately need the solution of that and request assistance from any expert please.
There's a JVM property to disable the JAXWS engine that's built into WAS. It's not perfect but it would be worth a try if you haven't already. https://www.ibm.com/support/knowledgecenter/SSAW57_9.0.5/com.ibm.websphere.nd.multiplatform.doc/ae/twbs_thirdparty.html
I Installed Spark on Linux machine. the version is spark-1.6.2-bin-hadoop2.6.tgz.
and then I start the Spark using ./sbin/start-all.sh
I trid to run the examples JavaWordCount.java in Eclipse.
but always failed. someone can helps me?
the version of Spark Master is: Welcome to version 1.6.2, Using Scala version 2.10.5 (Java HotSpot(TM) Server VM, Java 1.8.0_101)
the version of Spark on Eclipse side is:
The exception is as follows:
16/07/25 12:01:20 INFO StandaloneAppClient$ClientEndpoint: Connecting to master spark:// hostname:7077...
16/07/25 12:01:20 WARN StandaloneAppClient$ClientEndpoint: Failed to connect to master hostname:7077
org.apache.spark.SparkException: Exception thrown in awaitResult
at org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:77)
at org.apache.spark.rpc.RpcTimeout$$anonfun$1.applyOrElse(RpcTimeout.scala:75)
at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:33)
at org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at org.apache.spark.rpc.RpcTimeout$$anonfun$addMessageIfTimeout$1.applyOrElse(RpcTimeout.scala:59)
at scala.PartialFunction$OrElse.apply(PartialFunction.scala:162)
at org.apache.spark.rpc.RpcTimeout.awaitResult(RpcTimeout.scala:83)
at org.apache.spark.rpc.RpcEnv.setupEndpointRefByURI(RpcEnv.scala:88)
at org.apache.spark.rpc.RpcEnv.setupEndpointRef(RpcEnv.scala:96)
at org.apache.spark.deploy.client.StandaloneAppClient$ClientEndpoint$$anonfun$tryRegisterAllMasters$1$$anon$1.run(StandaloneAppClient.scala:109)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: java.io.InvalidClassException: org.apache.spark.rpc.RpcEndpointRef; local class incompatible: stream classdesc serialVersionUID = -1223633663228316618, local class serialVersionUID = 18257903091306170
at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:616)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1630)
at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1521)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1781)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2018)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1942)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1808)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1353)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:373)
at org.apache.spark.serializer.JavaDeserializationStream.readObject(JavaSerializer.scala:76)
at org.apache.spark.serializer.JavaSerializerInstance.deserialize(JavaSerializer.scala:109)
at org.apache.spark.rpc.netty.NettyRpcEnv$$anonfun$deserialize$1$$anonfun$apply$1.apply(NettyRpcEnv.scala:258)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
at org.apache.spark.rpc.netty.NettyRpcEnv.deserialize(NettyRpcEnv.scala:310)
at org.apache.spark.rpc.netty.NettyRpcEnv$$anonfun$deserialize$1.apply(NettyRpcEnv.scala:257)
at scala.util.DynamicVariable.withValue(DynamicVariable.scala:57)
at org.apache.spark.rpc.netty.NettyRpcEnv.deserialize(NettyRpcEnv.scala:256)
at org.apache.spark.rpc.netty.NettyRpcHandler.internalReceive(NettyRpcEnv.scala:588)
at org.apache.spark.rpc.netty.NettyRpcHandler.receive(NettyRpcEnv.scala:570)
at org.apache.spark.network.server.TransportRequestHandler.processRpcRequest(TransportRequestHandler.java:149)
at org.apache.spark.network.server.TransportRequestHandler.handle(TransportRequestHandler.java:102)
at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:104)
at org.apache.spark.network.server.TransportChannelHandler.channelRead0(TransportChannelHandler.java:51)
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:266)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:294)
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:308)
This issue is caused by version not matched.
I tried to install hadoop and use spark-assembly-1.6.2-hadoop2.6.0.jar, it works fine now.
I've a web application which sending mail. To perform sending mail I use library which itself based on Apache Commons Email 1.4. When I've used Websphere Liberty 8.5.5.7 everything worked fine, but after I've upgraded to 8.5.5.9 it stopped to work and throwing an exception: javax.mail.NoSuchProviderException the full stack trace I posted at the end of the question.
Again, I've tried to run it on both installations of Websphere Liberty 8.5.5.7 and 8.5.5.9, on first one it works fine on second it's broken. I've tried to look what the difference and it looks that Liberty itself has com.ibm.ws.javax.mail-1.5_1.5.xx.jar package which interfere with Java javax-mail-1.5.2 which used by commons-email 1.4, see its pom.xml here.
After some debugging I've paid attention that:
Liberty 8.5.5.9 uses com.ibm.ws.javax.mail-1.5_1.5.12.jar while Liberty 8.5.5.7 uses com.ibm.ws.javax.mail-1.5_1.5.10.jar possible this is caused to my bug.
So, my question is how can I eliminate the influence of com.ibm.ws.javax.mail which comes together with liberty to make it work or is there another workaround?
Also, where can I submit bug report for Liberty Team?
Update:
It looks that Liberty has JavaMail 1.5 feature which can be enabled or disabled, I didn't enable it explicitly, it looks that on of my features enabled it implicitly, this is my features list:
<featureManager>
<feature>jdbc-4.1</feature>
<feature>adminCenter-1.0</feature>
<feature>jndi-1.0</feature>
<feature>localConnector-1.0</feature>
<feature>servlet-3.1</feature>
<feature>ssl-1.0</feature>
<feature>jaxrs-2.0</feature>
<feature>cdi-1.2</feature>
<feature>ejbLite-3.2</feature>
<feature>jsf-2.2</feature>
</featureManager>
Update 2
I've solved this issue by using this article: Overriding a provided API with an alternative version
Just make in your server.xml the following changes:
<application id="" name="Scholar" type="ear" location="scholar.ear">
<classloader delegation="parentLast" />
</application>
And now your libraries will receive higher precedence then Liberty's libraries.
Full stack trace of the exception:
org.apache.commons.mail.EmailException: Sending the email to the following server failed : relay.******.com:25
at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1421) ~[commons-email-1.4.jar:1.4]
at org.apache.commons.mail.Email.send(Email.java:1448) ~[commons-email-1.4.jar:1.4]
at my.package.mailer.sendHtmlMail(IBMMail.java:96) ~[classes/:?]
at my.package.mailer.EmailGateway.sendMail(EmailGateway.java:24) [classes/:?]
at my.package.mailer.EventMailGateway.eventCreatedNotification(EventMailGateway.java:52) [classes/:?]
at **********.restful.EventsAPI.saveEvent(EventsAPI.java:99) [classes/:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_45]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_45]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_45]
at java.lang.reflect.Method.invoke(Method.java:497) ~[?:1.8.0_45]
at com.ibm.ws.jaxrs20.server.LibertyJaxRsServerFactoryBean.performInvocation(LibertyJaxRsServerFactoryBean.java:636) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.performInvocation(LibertyJaxRsInvoker.java:115) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at org.apache.cxf.service.invoker.AbstractInvoker.invoke(AbstractInvoker.java:97) [cxf-core-3.0.3.jar:3.0.3]
at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:210) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:200) [cxf-rt-frontend-jaxrs-3.0.3.jar:3.0.3]
at com.ibm.ws.jaxrs20.server.LibertyJaxRsInvoker.invoke(LibertyJaxRsInvoker.java:381) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at org.apache.cxf.jaxrs.JAXRSInvoker.invoke(JAXRSInvoker.java:99) [cxf-rt-frontend-jaxrs-3.0.3.jar:3.0.3]
at org.apache.cxf.interceptor.ServiceInvokerInterceptor$1.run(ServiceInvokerInterceptor.java:59) [cxf-core-3.0.3.jar:3.0.3]
at org.apache.cxf.interceptor.ServiceInvokerInterceptor.handleMessage(ServiceInvokerInterceptor.java:96) [cxf-core-3.0.3.jar:3.0.3]
at org.apache.cxf.phase.PhaseInterceptorChain.doIntercept(PhaseInterceptorChain.java:307) [cxf-core-3.0.3.jar:3.0.3]
at org.apache.cxf.transport.ChainInitiationObserver.onMessage(ChainInitiationObserver.java:124) [com.ibm.ws.jaxrs-2.0.common_1.0.12.jar:3.0.3]
at org.apache.cxf.transport.http.AbstractHTTPDestination.invoke(AbstractHTTPDestination.java:256) [cxf-rt-transports-http-3.0.3.jar:3.0.3]
at com.ibm.ws.jaxrs20.endpoint.AbstractJaxRsWebEndpoint.invoke(AbstractJaxRsWebEndpoint.java:134) [com.ibm.ws.jaxrs-2.0.common_1.0.12.jar:?]
at com.ibm.websphere.jaxrs.server.IBMRestServlet.handleRequest(IBMRestServlet.java:149) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at com.ibm.websphere.jaxrs.server.IBMRestServlet.doPost(IBMRestServlet.java:107) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) [com.ibm.ws.javaee.servlet.3.1_1.0.12.jar:?]
at com.ibm.websphere.jaxrs.server.IBMRestServlet.service(IBMRestServlet.java:99) [com.ibm.ws.jaxrs-2.0.server_1.0.12.jar:?]
at com.ibm.ws.webcontainer.servlet.ServletWrapper.service(ServletWrapper.java:1290) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:778) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.webcontainer.servlet.ServletWrapper.handleRequest(ServletWrapper.java:475) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.webcontainer.filter.WebAppFilterManager.invokeFilters(WebAppFilterManager.java:1161) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.webcontainer.servlet.CacheServletWrapper.handleRequest(CacheServletWrapper.java:82) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.webcontainer.WebContainer.handleRequest(WebContainer.java:928) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.webcontainer.osgi.DynamicVirtualHost$2.run(DynamicVirtualHost.java:262) [com.ibm.ws.webcontainer_1.1.12.jar:?]
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink$TaskWrapper.run(HttpDispatcherLink.java:955) [com.ibm.ws.transport.http_1.0.12.jar:?]
at com.ibm.ws.http.dispatcher.internal.channel.HttpDispatcherLink.ready(HttpDispatcherLink.java:341) [com.ibm.ws.transport.http_1.0.12.jar:?]
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleDiscrimination(HttpInboundLink.java:471) [com.ibm.ws.transport.http_1.0.12.jar:?]
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.handleNewRequest(HttpInboundLink.java:405) [com.ibm.ws.transport.http_1.0.12.jar:?]
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.processRequest(HttpInboundLink.java:285) [com.ibm.ws.transport.http_1.0.12.jar:?]
at com.ibm.ws.http.channel.internal.inbound.HttpInboundLink.ready(HttpInboundLink.java:256) [com.ibm.ws.transport.http_1.0.12.jar:?]
at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.sendToDiscriminators(NewConnectionInitialReadCallback.java:174) [com.ibm.ws.channelfw_1.0.12.jar:?]
at com.ibm.ws.tcpchannel.internal.NewConnectionInitialReadCallback.complete(NewConnectionInitialReadCallback.java:83) [com.ibm.ws.channelfw_1.0.12.jar:?]
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.requestComplete(WorkQueueManager.java:504) [com.ibm.ws.channelfw_1.0.12.jar:?]
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.attemptIO(WorkQueueManager.java:574) [com.ibm.ws.channelfw_1.0.12.jar:?]
at com.ibm.ws.tcpchannel.internal.WorkQueueManager.workerRun(WorkQueueManager.java:929) [com.ibm.ws.channelfw_1.0.12.jar:?]
at com.ibm.ws.tcpchannel.internal.WorkQueueManager$Worker.run(WorkQueueManager.java:1018) [com.ibm.ws.channelfw_1.0.12.jar:?]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_45]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_45]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_45]
Caused by: javax.mail.NoSuchProviderException: smtp
at javax.mail.Session.getService(Session.java:820) ~[javax.mail-1.5.2.jar:1.5.2]
at javax.mail.Session.getTransport(Session.java:742) ~[javax.mail-1.5.2.jar:1.5.2]
at javax.mail.Session.getTransport(Session.java:682) ~[javax.mail-1.5.2.jar:1.5.2]
at javax.mail.Session.getTransport(Session.java:662) ~[javax.mail-1.5.2.jar:1.5.2]
at javax.mail.Session.getTransport(Session.java:719) ~[javax.mail-1.5.2.jar:1.5.2]
at javax.mail.Transport.send0(Transport.java:248) ~[javax.mail-1.5.2.jar:1.5.2]
at javax.mail.Transport.send(Transport.java:124) ~[javax.mail-1.5.2.jar:1.5.2]
at org.apache.commons.mail.Email.sendMimeMessage(Email.java:1411) ~[commons-email-1.4.jar:1.4]
... 48 more
I've solved this issue by using this article: Overriding a provided API with an alternative version
Just make in your server.xml the following changes:
<application id="" name="Scholar" type="ear" location="scholar.ear">
<classloader delegation="parentLast" />
</application>
And now your libraries will receive higher precedence then Liberty's libraries.
I hit javax.mail.NoSuchProviderException: smtp after updating from Liberty 8.5.5.8 to 8.5.5.9. Adding <feature>javaMail-1.5</feature> to my server.xml worked for me.
I'm trying to index a realtime steam using Storm and ElasticSearch and I'm getting this exception. I'm using the following version of ElasticSearch
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>0.90.1</version>
</dependency>
Any pointers about where I should start looking ?
org.elasticsearch.transport.RemoteTransportException: Failed to deserialize exception response from stream
Caused by: org.elasticsearch.transport.TransportSerializationException: Failed to deserialize exception response from stream
at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:171)
at org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:125)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline$DefaultChannelHandlerContext.sendUpstream(DefaultChannelPipeline.java:791)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:296)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.unfoldAndFireMessageReceived(FrameDecoder.java:462)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.callDecode(FrameDecoder.java:443)
at org.elasticsearch.common.netty.handler.codec.frame.FrameDecoder.messageReceived(FrameDecoder.java:303)
at org.elasticsearch.common.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:70)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)
at org.elasticsearch.common.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:268)
at org.elasticsearch.common.netty.channel.Channels.fireMessageReceived(Channels.java:255)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.read(NioWorker.java:88)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.process(AbstractNioWorker.java:107)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:312)
at org.elasticsearch.common.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:88)
at org.elasticsearch.common.netty.channel.socket.nio.NioWorker.run(NioWorker.java:178)
at org.elasticsearch.common.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)
at org.elasticsearch.common.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.StreamCorruptedException: unexpected end of block data
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1370)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:499)
at java.lang.Throwable.readObject(Throwable.java:913)
at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
at java.io.ObjectInputStream.defaultReadObject(ObjectInputStream.java:499)
at java.lang.Throwable.readObject(Throwable.java:913)
at sun.reflect.GeneratedMethodAccessor59.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1004)
at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1891)
at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
at org.elasticsearch.transport.netty.MessageChannelHandler.handlerResponseError(MessageChannelHandler.java:169)
... 23 more
I upgraded to 0.90.1 on all my clients and servers and it works now. I"m not sure if this was the root cause or something else. If I see that error again in the new version I'll post an update here.
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>0.90.2</version>
</dependency>
I found that the problem was something I was doing with the data. I was creating an index with document type as _tweet. Elasticsearch doesn't allow _ in the beginning of the type.
I suggest you do elasticsearch -f -D es.config=config/elasticsearch.yml to start the service in foreground. Then you can read the error to see why you get this problem.
It might be useful to check if the elastic search client and server are running the same version. I was using a 0.90.5 server with a 0.90.3 client and was able to get rid of the exception when I upgraded my Java client to 0.90.5