Using this guide to get a Standalone. But whenever I perform the "sbin\start-master.sh" command, a new CMD window pops up and then disappears immediately. No URL is being printed.
I've done the initial setup and set all the environment variables properly but it still doesn't seem to work. How can I fix this?
Edit: Also tried this example using Intelli-J. Its showing the following without any other output:
[Thread-0] INFO org.eclipse.jetty.util.log - Logging initialized #568ms
[Thread-0] INFO spark.webserver.JettySparkServer - == Spark has ignited ...
[Thread-0] INFO spark.webserver.JettySparkServer - >> Listening on 0.0.0.0:4567
[Thread-0] INFO org.eclipse.jetty.server.Server - jetty-9.3.2.v20150730
[Thread-0] INFO org.eclipse.jetty.server.ServerConnector - Started ServerConnector#e92ccd{HTTP/1.1,[http/1.1]}{0.0.0.0:4567}
[Thread-0] INFO org.eclipse.jetty.server.Server - Started #942ms
Related
Error says
Testing started at 12:03 ...
12:03:10.624 [main] DEBUG io.github.bonigarcia.wdm.cache.ResolutionCache - Resolution chrome=103 in cache (valid until 12:20:19 30/07/2022 BST)
12:03:10.634 [main] DEBUG io.github.bonigarcia.wdm.cache.ResolutionCache - Resolution chrome103=103.0.5060.134 in cache (valid until 17:22:03 30/07/2022 BST)
12:03:10.635 [main] INFO io.github.bonigarcia.wdm.WebDriverManager - Using chromedriver 103.0.5060.134 (resolved driver for Chrome 103)
12:03:10.678 [main] DEBUG io.github.bonigarcia.wdm.WebDriverManager - Driver chromedriver 103.0.5060.134 found in cache
12:03:10.681 [main] INFO io.github.bonigarcia.wdm.WebDriverManager - Exporting webdriver.chrome.driver as C:\Users\lukem.cache\selenium\chromedriver\win32\103.0.5060.134\chromedriver.exe
0 Scenarios
0 Steps
0m1.300s
Selenium-server-standalone has a 5 seconds delay while starting. If I run selenium with debug option it shows that the delay is caused by New random session seed.
I start Selenium server with java -jar selenium-server-standalone-2.44.0.jar.
If I want to specify chromedriver then I write java -jar selenium-server-standalone-2.44.0.jar -Dwebdriver.chrome.driver=chromedriver_2.11
It happens with:
OSX 10.9.5 and OSX 10.10
Java 1.7.0_51 (update 67 and update 71)
selenium-server-standalone 2.42.0, 2.43.1, 2.44.0
with and without chromedriver (2.9, 2.10, 2.11)
running as sudo and as non-priveleged user
From what I know on other machines with the same configuration there is no such delay.
It may seem that 5 seconds is nothing, but I have another issue that most probably is caused by the same problem. It gives me 1 minute of waiting instead of few seconds.
Here is the log
11:22:50.445 INFO - Launching a standalone server
11:22:50.820 INFO - Java: Oracle Corporation 24.51-b03
11:22:50.820 INFO - OS: Mac OS X 10.10 x86_64
11:22:50.948 INFO - v2.44.0, with Core v2.44.0. Built from revision 76d78cf
11:22:50.948 INFO - Selenium server running in debug mode.
11:22:51.001 DEBUG - add component: SocketListener0#0.0.0.0:4444
11:22:51.022 DEBUG - add component: org.openqa.jetty.http.ResourceCache#50265e47
11:22:51.046 DEBUG - add component: org.openqa.selenium.server.ProxyHandler in HttpContext[/,/]
11:22:51.065 DEBUG - add component: HttpContext[/,/]
11:22:51.070 DEBUG - Added HttpContext[/,/] for host *
11:22:51.071 DEBUG - add component: org.openqa.jetty.http.ResourceCache#36264c17
11:22:51.073 DEBUG - added SC{BASIC,null,user,CONFIDENTIAL} at /org/openqa/selenium/tests/html/basicAuth/*
11:22:51.107 DEBUG - add component: org.openqa.jetty.http.handler.SecurityHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.111 DEBUG - add component: org.openqa.selenium.server.StaticContentHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.123 DEBUG - add component: org.openqa.selenium.server.SessionExtensionJsHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.125 DEBUG - add component: org.openqa.selenium.server.htmlrunner.SingleTestSuiteResourceHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.126 DEBUG - add component: org.openqa.selenium.server.htmlrunner.SeleniumHTMLRunnerResultsHandler#6d9d3901
11:22:51.132 DEBUG - add component: HttpContext[/selenium-server,/selenium-server]
11:22:51.132 DEBUG - Added HttpContext[/selenium-server,/selenium-server] for host *
11:22:51.242 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: MAC
11:22:51.446 DEBUG - add component: org.openqa.jetty.http.ResourceCache#61f3318a
11:22:51.483 DEBUG - add component: org.openqa.selenium.server.SeleniumDriverResourceHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.483 DEBUG - add component: HttpContext[/selenium-server/driver,/selenium-server/driver]
11:22:51.484 DEBUG - Added HttpContext[/selenium-server/driver,/selenium-server/driver] for host *
11:22:51.484 DEBUG - add component: org.openqa.jetty.http.ResourceCache#c3b626c
11:22:51.528 DEBUG - add component: WebDriver remote server
11:22:51.568 DEBUG - add component: org.openqa.jetty.jetty.servlet.HashSessionManager#5298d146
11:22:51.568 DEBUG - add component: org.openqa.jetty.jetty.servlet.ServletHandler#2ed37507
11:22:51.596 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
11:22:51.596 DEBUG - add component: HttpContext[/wd,/wd]
11:22:51.597 DEBUG - Added HttpContext[/wd,/wd] for host *
11:22:51.597 DEBUG - Starting org.openqa.jetty.jetty.Server#a5bdce3
11:22:51.598 INFO - Version Jetty/5.1.x
11:22:51.598 DEBUG - LISTENERS: [SocketListener0#0.0.0.0:4444]
11:22:51.598 DEBUG - HANDLER: {null={/selenium-server/driver/*=[HttpContext[/selenium-server/driver,/selenium-server/driver]], /selenium-server/*=[HttpContext[/selenium-server,/selenium-server]], /=[HttpContext[/,/]], /wd/*=[HttpContext[/wd,/wd]]}}
11:22:51.599 DEBUG - Starting HttpContext[/selenium-server/driver,/selenium-server/driver]
11:22:51.612 DEBUG - Init classloader from null, sun.misc.Launcher$AppClassLoader#5eb1404f for HttpContext[/selenium-server/driver,/selenium-server/driver]
11:22:51.612 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
11:22:51.612 DEBUG - Starting HttpContext[/selenium-server,/selenium-server]
11:22:51.612 DEBUG - Init classloader from null, sun.misc.Launcher$AppClassLoader#5eb1404f for HttpContext[/selenium-server,/selenium-server]
11:22:51.613 DEBUG - Started org.openqa.jetty.http.handler.SecurityHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.613 DEBUG - Started org.openqa.selenium.server.StaticContentHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.613 DEBUG - Started org.openqa.selenium.server.SessionExtensionJsHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.613 DEBUG - Started org.openqa.selenium.server.htmlrunner.SingleTestSuiteResourceHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.613 DEBUG - Started org.openqa.selenium.server.SeleniumDriverResourceHandler in HttpContext[/selenium-server,/selenium-server]
11:22:51.613 INFO - Started HttpContext[/selenium-server,/selenium-server]
11:22:51.614 DEBUG - Starting HttpContext[/,/]
11:22:51.614 DEBUG - Init classloader from null, sun.misc.Launcher$AppClassLoader#5eb1404f for HttpContext[/,/]
11:22:51.614 DEBUG - Started org.openqa.selenium.server.ProxyHandler in HttpContext[/,/]
11:22:51.614 INFO - Started HttpContext[/,/]
11:22:51.614 DEBUG - Starting HttpContext[/wd,/wd]
11:22:51.614 DEBUG - Init classloader from null, sun.misc.Launcher$AppClassLoader#5eb1404f for HttpContext[/wd,/wd]
11:22:51.614 DEBUG - Starting org.openqa.jetty.jetty.servlet.ServletHandler#2ed37507
11:22:51.614 DEBUG - New random session seed
11:22:56.882 DEBUG - Session scavenger period = 30s
11:22:56.890 DEBUG - Started holder of class org.openqa.selenium.remote.server.DriverServlet
11:22:56.891 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler#2ed37507
11:22:56.891 INFO - Started HttpContext[/wd,/wd]
11:22:56.928 INFO - Started SocketListener on 0.0.0.0:4444
11:22:56.928 INFO - Started org.openqa.jetty.jetty.Server#a5bdce3
I'm trying to use JSonLoader form elephant-bird-pig package.
My script is simple:
register elephant-bird-pig-4.5.jar
register elephant-bird-hadoop-compat-4.5.jar
A = load '1_record_2.json' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad');
DUMP A
And I get an error:
2014-09-30 16:15:32,439 [main] INFO org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-09-30 16:15:32,447 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2014-09-30 16:15:32,448 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier]}
2014-09-30 16:15:32,449 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-09-30 16:15:32,450 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-09-30 16:15:32,450 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-09-30 16:15:32,464 [main] INFO org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at hadoop1/10.242.8.131:8050
2014-09-30 16:15:32,466 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-09-30 16:15:32,466 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2014-09-30 16:15:32,467 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. com/twitter/elephantbird/util/HadoopCompat
Details at logfile: pig_1412081068149.log
I don't know what is missing. Can you please suggest something?
File pig_1412081068149.log contains:
Pig Stack Trace
---------------
ERROR 2998: Unhandled internal error. com/twitter/elephantbird/util/HadoopCompat
java.lang.NoClassDefFoundError: com/twitter/elephantbird/util/HadoopCompat
at com.twitter.elephantbird.pig.load.LzoBaseLoadFunc.setLocation(LzoBaseLoadFunc.java:93)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.getJob(JobControlCompiler.java:477)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:298)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:191)
at org.apache.pig.PigServer.launchPlan(PigServer.java:1324)
at org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1309)
at org.apache.pig.PigServer.storeEx(PigServer.java:980)
at org.apache.pig.PigServer.store(PigServer.java:944)
at org.apache.pig.PigServer.openIterator(PigServer.java:857)
at org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:774) etc...
What class is missing (java.lang.NoClassDefFoundError) ? What libraries should I add more?
Thanks
pawel
I've checked if registered libraries exist and classes are inside the libraries.
Everythig looked fine, but I was still getting this error.
So I left pig shell, and opened it once again - the same script works fine.
You missed two important jars to register.
register elephant-bird-core-4.5.jar
register json-simple-1.1.1.jar
Add these two to your pig script and everything should work fine.
I installed pig on centos 6. I am new to pig.
I opened pig in local mode using $pig -x local.
only getting error while doing DUMP.
error message is : ERROR 2998: Unhandled internal error. org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
I have set JAVA_HOME and java version is 1.7. But hadoop is not installed.
=================
grunt> A = load '/etc/passwd' using PigStorage(':');
grunt> B = foreach A generate $0 as id; grunt> dump B;
2014-06-13 16:24:33,039 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig features used in the script: UNKNOWN
2014-06-13 16:24:33,040 [main] INFO org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer, PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], RULES_DISABLED=[FilterLogicExpressionSimplifier]}
2014-06-13 16:24:33,041 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - File concatenation threshold: 100 optimistic? false
2014-06-13 16:24:33,042 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size before optimization: 1
2014-06-13 16:24:33,042 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer - MR plan size after optimization: 1
2014-06-13 16:24:33,043 [main] INFO org.apache.hadoop.metrics.jvm.JvmMetrics - Cannot initialize JVM Metrics with processName=JobTracker, sessionId= - already initialized
2014-06-13 16:24:33,044 [main] INFO org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added to the job
2014-06-13 16:24:33,044 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
2014-06-13 16:24:33,052 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler - Setting up single store job
2014-06-13 16:24:33,052 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Key [pig.schematuple] is false, will not generate code.
2014-06-13 16:24:33,053 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Starting process to move generated code to distributed cache
2014-06-13 16:24:33,053 [main] INFO org.apache.pig.data.SchemaTupleFrontend - Distributed cache not supported or needed in local mode. Setting key [pig.schematuple.local.dir] with code temp directory: /tmp/1402656873052-0
2014-06-13 16:24:33,062 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2998: Unhandled internal error. org.apache.hadoop.mapred.jobcontrol.JobControl.addJob(Lorg/apache/hadoop/mapred/jobcontrol/Job;)Ljava/lang/String;
Details at logfile: /home/ripsme/pig/pig_1402655166073.log
grunt>
Thank you very much.
for Redhat systems, pls do
$sudo yum install pig
and open pig in local mode. this worked for me.
I am trying to use the Selenium standalone server with Python. However, when I type
"C:\Program Files (x86)\Java\jre7\bin\Java.exe" -jar selenium-server-standalone-2.32.0.jar
into the command prompt, the setup gets stuck displaying
May 22, 2013 3:38:18 PM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
15:38:18.388 INFO - Java: Oracle Corporation 23.21-b01
15:38:18.389 INFO - OS: Windows 7 6.1 x86
15:38:18.394 INFO - v2.32.0, with Core v2.32.0. Built from revision 6c40c18
15:38:18.473 INFO - RemoteWebDriver instances should connect to: http://127.0.0.
1:4444/wd/hub
15:38:18.474 INFO - Version Jetty/5.1.x
15:38:18.475 INFO - Started HttpContext[/selenium-server/driver,/selenium-server
/driver]
15:38:18.475 INFO - Started HttpContext[/selenium-server,/selenium-server]
15:38:18.476 INFO - Started HttpContext[/,/]
15:38:18.498 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler#166faa
c
15:38:18.499 INFO - Started HttpContext[/wd,/wd]
15:38:18.501 INFO - Started SocketListener on 0.0.0.0:4444
15:38:18.502 INFO - Started org.openqa.jetty.jetty.Server#144752d
I'm not sure how to get around this problem, any help would be much appreciated.
I don't think you have a problem - that looks to me like the selenium server has started successfully, and is now running. Do you have a problem connecting to it and running a script?