We currently develop a crawler using Nutch2 with an HBase backend. We create a plugin for the web parsers using Selenium web driver. Everything just fine in Nutch2 local mode. But, when we are trying to deploy it on the cluster using Nutch deploy mode, we got errors that stated "unable to parse successfully." Below are the errors.
java.util.concurrent.ExecutionException: java.lang.NoSuchFieldError:
INSTANCE
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at org.apache.nutch.parse.ParseUtil.runParser(ParseUtil.java:164)
at org.apache.nutch.parse.ParseUtil.parse(ParseUtil.java:146)
at org.apache.nutch.parse.ParserChecker.run(ParserChecker.java:142)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.parse.ParserChecker.main(ParserChecker.java:214)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.NoSuchFieldError: INSTANCE
at org.apache.http.conn.ssl.SSLConnectionSocketFactory.(SSLConnectionSocketFactory.java:144)
at org.openqa.selenium.remote.internal.HttpClientFactory.getClientConnectionManager(HttpClientFactory.java:71)
at org.openqa.selenium.remote.internal.HttpClientFactory.(HttpClientFactory.java:57)
at org.openqa.selenium.remote.internal.HttpClientFactory.(HttpClientFactory.java:60)
at org.openqa.selenium.remote.internal.ApacheHttpClient$Factory.getDefaultHttpClientFactory(ApacheHttpClient.java:251)
at org.openqa.selenium.remote.internal.ApacheHttpClient$Factory.(ApacheHttpClient.java:228)
at org.openqa.selenium.remote.HttpCommandExecutor.getDefaultClientFactory(HttpCommandExecutor.java:96)
at org.openqa.selenium.remote.HttpCommandExecutor.(HttpCommandExecutor.java:70)
at org.openqa.selenium.remote.HttpCommandExecutor.(HttpCommandExecutor.java:58)
at org.openqa.selenium.firefox.internal.NewProfileExtensionConnection.start(NewProfileExtensionConnection.java:97)
at org.openqa.selenium.firefox.FirefoxDriver.startClient(FirefoxDriver.java:271)
at org.openqa.selenium.remote.RemoteWebDriver.(RemoteWebDriver.java:117)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:216)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:211)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:207)
at org.openqa.selenium.firefox.FirefoxDriver.(FirefoxDriver.java:124)
at org.apache.nutch.store.readable.seleniumhandlers.HTTPWebClient$1.initialValue(HTTPWebClient.java:148)
at org.apache.nutch.store.readable.seleniumhandlers.HTTPWebClient$1.initialValue(HTTPWebClient.java:49)
at java.lang.ThreadLocal.setInitialValue(ThreadLocal.java:180)
at java.lang.ThreadLocal.get(ThreadLocal.java:170)
at org.apache.nutch.store.readable.seleniumhandlers.HTTPWebClient.getHtmlPage(HTTPWebClient.java:318)
at org.apache.nutch.store.readable.seleniumhandlers.HTTPWebClient.getHtmlPage(HTTPWebClient.java:309)
at org.apache.nutch.store.readable.parserhandlers.JsoupTokopedia.constructJson(JsoupTokopedia.java:108)
at org.apache.nutch.store.readable.StoreReadable.addJsonToPage(StoreReadable.java:349)
at org.apache.nutch.store.readable.StoreReadable.getParse(StoreReadable.java:311)
at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:36)
at org.apache.nutch.parse.ParseCallable.call(ParseCallable.java:23)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
It looks like Selenium won't work on Hadoop. I think this is the related issue click. Is it because Selenium can't run on Hadoop, or any suggestions on these issues?
Related
Client use Windows xp 32-bit so I don't have other choices.
I've developed JavaFX application using JDK 1.8.0u131 and NetBeans 8.2 32-bit
I'm getting the following error:
Graphics Device initialization failed for : d3d, sw
Error initializing QuantumRenderer: no suitable pipeline found
java.lang.RuntimeException: java.lang.RuntimeException: Error initializing QuantumRenderer: no suitable pipeline found
at com.sun.javafx.tk.quantum.QuantumRenderer.getInstance(QuantumRenderer.java:280)
at com.sun.javafx.tk.quantum.QuantumToolkit.init(QuantumToolkit.java:221)
at com.sun.javafx.tk.Toolkit.getToolkit(Toolkit.java:205)
at com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:209)
at com.sun.javafx.application.LauncherImpl.startToolkit(LauncherImpl.java:675)
at com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:337)
at com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:328)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
Caused by: java.lang.RuntimeException: Error initializing QuantumRenderer: no suitable pipeline found
at com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.init(QuantumRenderer.java:94)
at com.sun.javafx.tk.quantum.QuantumRenderer$PipelineRunnable.run(QuantumRenderer.java:124)
at java.lang.Thread.run(Thread.java:748)
Exception in thread "main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
Caused by: java.lang.RuntimeException: No toolkit found
at com.sun.javafx.tk.Toolkit.getToolkit(Toolkit.java:217)
at com.sun.javafx.application.PlatformImpl.startup(PlatformImpl.java:209)
at com.sun.javafx.application.LauncherImpl.startToolkit(LauncherImpl.java:675)
at com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:337)
at com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:328)
... 5 more
Java Result: 1
I've no choice to use upgraded versions of libraries to get rid of this issue.
I've installed JDK7u4 and NetBeans IDE 7.1 and problem is solved but I can't migrate my project cause I have used higher version of libraries to develop JavaFX application.
Use JDK-7u4 and NetBeans IDE 7.1 for developing JavaFX on Windows XP.
Need to switch back to swing.
Hello I am trying to optimise my application, I've had JProfiler installed for the last 2 weeks but the trial period is up and I can't afford the megabucks.
I want to use VisualVM but I can't get it to work.
I installed version 1.4 from their website but when I try and open my application for profiling it does nothing. (Other available processes do open, like the intellij ide)
The error reported is as follows:
java.lang.IllegalArgumentException: Unexpected composite type for ThreadInfo
at sun.management.ThreadInfoCompositeData.validateCompositeData(ThreadInfoCompositeData.java:372)
at sun.management.ThreadInfoCompositeData.getInstance(ThreadInfoCompositeData.java:68)
at java.lang.management.ThreadInfo.<init>(ThreadInfo.java:263)
at java.lang.management.ThreadInfo.from(ThreadInfo.java:794)
Caused: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.Trampoline.invoke(MethodUtil.java:71)
at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at sun.reflect.misc.MethodUtil.invoke(MethodUtil.java:275)
at com.sun.jmx.mbeanserver.DefaultMXBeanMappingFactory$CompositeBuilderViaFrom.fromCompositeData(DefaultMXBeanMappingFactory.java:1018)
Caused: java.io.InvalidObjectException: Failed to invoke from(CompositeData)
at com.sun.jmx.mbeanserver.DefaultMXBeanMappingFactory.invalidObjectException(DefaultMXBeanMappingFactory.java:1457)
at com.sun.jmx.mbeanserver.DefaultMXBeanMappingFactory$CompositeBuilderViaFrom.fromCompositeData(DefaultMXBeanMappingFactory.java:1021)
at com.sun.jmx.mbeanserver.DefaultMXBeanMappingFactory$CompositeMapping.fromNonNullOpenValue(DefaultMXBeanMappingFactory.java:919)
at com.sun.jmx.mbeanserver.DefaultMXBeanMappingFactory$NonNullMXBeanMapping.fromOpenValue(DefaultMXBeanMappingFactory.java:133)
at com.sun.jmx.mbeanserver.ConvertingMethod.fromOpenReturnValue(ConvertingMethod.java:131)
at com.sun.jmx.mbeanserver.MXBeanProxy.invoke(MXBeanProxy.java:168)
at javax.management.MBeanServerInvocationHandler.invoke(MBeanServerInvocationHandler.java:258)
Caused: java.lang.reflect.UndeclaredThrowableException
at com.sun.proxy.$Proxy14.getThreadInfo(Unknown Source)
at com.sun.tools.visualvm.jmx.impl.JmxSupport.isReadOnlyConnection(JmxSupport.java:126)
at com.sun.tools.visualvm.jmx.impl.JmxModelImpl.isTakeThreadDumpSupported(JmxModelImpl.java:369)
at com.sun.tools.visualvm.application.views.threads.ApplicationThreadsViewProvider.resolveThreads(ApplicationThreadsViewProvider.java:85)
at com.sun.tools.visualvm.application.views.threads.ApplicationThreadsViewProvider.supportsViewFor(ApplicationThreadsViewProvider.java:49)
at com.sun.tools.visualvm.application.views.threads.ApplicationThreadsViewProvider.supportsViewFor(ApplicationThreadsViewProvider.java:45)
at com.sun.tools.visualvm.core.ui.DataSourceViewsManager.getViews(DataSourceViewsManager.java:136)
at com.sun.tools.visualvm.core.ui.DataSourceWindowManager.openWindowAndAddView(DataSourceWindowManager.java:206) at com.sun.tools.visualvm.core.ui.DataSourceWindowManager.access$000(DataSourceWindowManager.java:50)
at com.sun.tools.visualvm.core.ui.DataSourceWindowManager$2.run(DataSourceWindowManager.java:117)
at org.openide.util.RequestProcessor$Task.run(RequestProcessor.java:1443)
at org.netbeans.modules.openide.util.GlobalLookup.execute(GlobalLookup.java:68)
at org.openide.util.lookup.Lookups.executeWith(Lookups.java:303)
It looks like something to do with incompatible Java types? My application was compiled and run with JDK9. I've noticed VisualVM isn't even included in the java download any more. Is there a way around this?
Thanks.
I am trying to run the CountVectorizerDemo program provided here:
https://github.com/apache/spark/blob/master/examples/src/main/java/org/apache/spark/examples/ml/JavaCountVectorizerExample.java
I'm getting the following error and don't know what the problem is.
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.ml.util.SchemaUtils$.checkColumnType$default$4()Ljava/lang/String;
at org.apache.spark.ml.feature.CountVectorizerParams$class.validateAndTransformSchema(CountVectorizer.scala:71)
at org.apache.spark.ml.feature.CountVectorizer.validateAndTransformSchema(CountVectorizer.scala:107)
at org.apache.spark.ml.feature.CountVectorizer.transformSchema(CountVectorizer.scala:168)
at org.apache.spark.ml.PipelineStage.transformSchema(Pipeline.scala:59)
at org.apache.spark.ml.feature.CountVectorizer.fit(CountVectorizer.scala:130)
at com.bah.ossem.spark.topic.CountVectorizerDemo.main(CountVectorizerDemo.java:42)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.spark.deploy.SparkSubmit$.org$apache$spark$deploy$SparkSubmit$$runMain(SparkSubmit.scala:664)
at org.apache.spark.deploy.SparkSubmit$.doRunMain$1(SparkSubmit.scala:169)
at org.apache.spark.deploy.SparkSubmit$.submit(SparkSubmit.scala:192)
at org.apache.spark.deploy.SparkSubmit$.main(SparkSubmit.scala:111)
at org.apache.spark.deploy.SparkSubmit.main(SparkSubmit.scala)
The problem was that my cluster was using Spark core 1.4 but my application was using Spark core 1.5.1 and MLlib 1.5.1. I updated my AWS cluster to Spark 1.5.1 and that fixed the problem.
I'm starting with Play Framework and when I'm sending the following commands, it throws a exception.
# play
In Play Console:
[play-java] $ run
It throws:
org.jboss.netty.channel.ChannelException: Failed to bind to: /0.0.0.0:9000
at org.jboss.netty.bootstrap.ServerBootstrap.bind(ServerBootstrap.java:272)
at play.core.server.NettyServer$$anonfun$10.apply(NettyServer.scala:171)
at play.core.server.NettyServer$$anonfun$10.apply(NettyServer.scala:168)
at scala.Option.map(Option.scala:145)
at play.core.server.NettyServer.<init>(NettyServer.scala:168)
at play.core.server.NettyServer$$anonfun$mainDev$1.apply(NettyServer.scala:347)
at play.core.server.NettyServer$$anonfun$mainDev$1.apply(NettyServer.scala:345)
at play.utils.Threads$.withContextClassLoader(Threads.scala:18)
at play.core.server.NettyServer$.mainDev(NettyServer.scala:344)
at play.core.server.NettyServer$.mainDevHttpMode(NettyServer.scala:340)
at play.core.server.NettyServer.mainDevHttpMode(NettyServer.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at play.PlayRun$$anonfun$playRunTask$1$$anonfun$apply$1.apply(PlayRun.scala:233)
at play.PlayRun$$anonfun$playRunTask$1$$anonfun$apply$1.apply(PlayRun.scala:91)
at scala.Function7$$anonfun$tupled$1.apply(Function7.scala:35)
at scala.Function7$$anonfun$tupled$1.apply(Function7.scala:34)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
Caused by: java.net.BindException: Address already in use
at sun.nio.ch.Net.bind0(Native Method)
at sun.nio.ch.Net.bind(Net.java:463)
at sun.nio.ch.Net.bind(Net.java:455)
at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
at org.jboss.netty.channel.socket.nio.NioServerBoss$RegisterTask.run(NioServerBoss.java:193)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.processTaskQueue(AbstractNioSelector.java:366)
at org.jboss.netty.channel.socket.nio.AbstractNioSelector.run(AbstractNioSelector.java:290)
at org.jboss.netty.channel.socket.nio.NioServerBoss.run(NioServerBoss.java:42)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I'm trying using my local user, root user, changing ports (ie: run 8080), start instead of run, and it always throw the same exception.
By the way, port 9000 is not in use by another application.
Anyone has any clue?
I'm using Ubuntu 12.04 LTS, and Play Framework 2.2.6.
Thanks.
The error is that your port is being used by another process.
In a UNIX machine the command lsof -i:9000 will give the processes that are using the port.
As you have noted correctly processes with root permissions can be hidden if you use the terminal not as root
fed up with raspberry pi.....
i have created a javaFX application from windows 7 using netbeans 7.3.1 , jdk 1.7 ..
application has included some external jar including javafx-dialogs-0.0.3,jaybird etc..
its all works fine with windows 7,ubuntu and centOS
Then i tried to run the same application jar from command line in raspberry pi..
but it gives Exception like "noSuchMethoderror " and exiting the application..
what may be the issue ???
am using same jar during comple and run time..
here is full stack
Exception in Application start method
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at com.sun.javafx.application.LauncherImpl.launchApplicationWithArgs(LauncherImpl.java:367)
at com.sun.javafx.application.LauncherImpl.launchApplication(LauncherImpl.java:305)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at sun.launcher.LauncherHelper$FXHelper.main(LauncherHelper.java:767)
Caused by: java.lang.RuntimeException: Exception in Application start method
at com.sun.javafx.application.LauncherImpl.launchApplication1(LauncherImpl.java:894)
at com.sun.javafx.application.LauncherImpl.access$000(LauncherImpl.java:56)
at com.sun.javafx.application.LauncherImpl$1.run(LauncherImpl.java:158)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.NoSuchMethodError: com.sun.javafx.css.StyleManager.getPseudoclassMask(Ljava/lang/String;)J
at javafx.scene.control.Dialogs$FXDialog$RootPane.<clinit>(Dialogs.java:1310)
at javafx.scene.control.Dialogs$FXDialog.<init>(Dialogs.java:1131)
at javafx.scene.control.Dialogs$FXDialog.<init>(Dialogs.java:1088)
at javafx.scene.control.Dialogs$DialogTemplate.<init>(Dialogs.java:616)
at javafx.scene.control.Dialogs.showSimpleContentDialog(Dialogs.java:494)
at javafx.scene.control.Dialogs.showInformationDialog(Dialogs.java:266)
at displayshelftry.Utilities.propertySettings(Utilities.java:325)
at displayshelftry.DisplayShelfTry.run(DisplayShelfTry.java:356)
at displayshelftry.DisplayShelfTry.start(DisplayShelfTry.java:1150)
at com.sun.javafx.application.LauncherImpl$8.run(LauncherImpl.java:837)
at com.sun.javafx.application.PlatformImpl$7.run(PlatformImpl.java:331)
at com.sun.javafx.application.PlatformImpl$6$1.run(PlatformImpl.java:297)
at com.sun.javafx.application.PlatformImpl$6$1.run(PlatformImpl.java:294)
at java.security.AccessController.doPrivileged(Native Method)
at com.sun.javafx.application.PlatformImpl$6.run(PlatformImpl.java:294)
at com.sun.glass.ui.lens.LensApplication$RunnableEvent.dispatch(LensApplication.java:182)
at com.sun.glass.ui.lens.LensApplication._runLoop(LensApplication.java:861)
at com.sun.glass.ui.lens.LensApplication.access$1700(LensApplication.java:58)
at com.sun.glass.ui.lens.LensApplication$4.run(LensApplication.java:914)
... 1 more
thanks...
Try building you app with jdk8. jdk8 usually should run jdk7 built jars as well, but it's not yet released and rasberry pi was supported only since jdk8.
Also official pi vs javafx page claims that not all demos will work on pi (note it's not yet released product): https://wiki.openjdk.java.net/display/OpenJFX/OpenJFX+on+the+Raspberry+Pi
So you may want to try BrickBreaker instead of DisplayShelf :)