java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled - java

When I run my Ktor application with gradle run then I've got the following exception:
19:21:11.795 [main] DEBUG io.netty.util.internal.logging.InternalLoggerFactory - Using SLF4J as the default logging framework
19:21:11.810 [main] DEBUG io.netty.util.internal.PlatformDependent0 - -Dio.netty.noUnsafe: false
19:21:11.810 [main] DEBUG io.netty.util.internal.PlatformDependent0 - Java version: 11
19:21:11.811 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
19:21:11.812 [main] DEBUG io.netty.util.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
19:21:11.812 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Buffer.address: available
19:21:11.814 [main] DEBUG io.netty.util.internal.PlatformDependent0 - direct buffer constructor: unavailable
java.lang.UnsupportedOperationException: Reflective setAccessible(true) disabled
at io.netty.util.internal.ReflectionUtil.trySetAccessible(ReflectionUtil.java:31)
at io.netty.util.internal.PlatformDependent0$4.run(PlatformDependent0.java:225)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:219)
at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
19:21:11.814 [main] DEBUG io.netty.util.internal.PlatformDependent0 - java.nio.Bits.unaligned: available, true
19:21:11.815 [main] DEBUG io.netty.util.internal.PlatformDependent0 - jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
java.lang.IllegalAccessException: class io.netty.util.internal.PlatformDependent0$6 cannot access class jdk.internal.misc.Unsafe (in module java.base) because module java.base does not export jdk.internal.misc to unnamed module #557caf28
at java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
at java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
at java.base/java.lang.reflect.Method.invoke(Method.java:558)
at io.netty.util.internal.PlatformDependent0$6.run(PlatformDependent0.java:335)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at io.netty.util.internal.PlatformDependent0.<clinit>(PlatformDependent0.java:326)
at io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:273)
at io.netty.util.internal.PlatformDependent.<clinit>(PlatformDependent.java:92)
at io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:225)
at io.netty.channel.epoll.Native.<clinit>(Native.java:57)
at io.netty.channel.epoll.Epoll.<clinit>(Epoll.java:39)
at io.ktor.server.netty.EventLoopGroupProxy$Companion.create(NettyApplicationEngine.kt:189)
at io.ktor.server.netty.NettyApplicationEngine.<init>(NettyApplicationEngine.kt:74)
at io.ktor.server.netty.EngineMain.main(EngineMain.kt:22)
the content of build.gradle.kts file
plugins {
application
kotlin("jvm") version "1.3.61"
}
group = "io.flatmap"
version = "1.0-SNAPSHOT"
val ktor_version = "1.3.0"
repositories {
mavenCentral()
jcenter()
}
dependencies {
implementation(kotlin("stdlib-jdk8"))
compile("io.ktor:ktor-server-netty:$ktor_version")
compile("io.ktor:ktor-server-core:$ktor_version")
compile("ch.qos.logback:logback-classic:1.2.3")
testCompile(group = "junit", name = "junit", version = "4.12")
}
tasks {
compileKotlin {
kotlinOptions.jvmTarget = "11"
}
compileTestKotlin {
kotlinOptions.jvmTarget = "11"
}
}
application {
mainClassName = "io.ktor.server.netty.EngineMain"
}
I am using Zulu OpenJDK 11:
java --version
openjdk 11.0.6 2020-01-14 LTS
OpenJDK Runtime Environment Zulu11.37+17-CA (build 11.0.6+10-LTS)
OpenJDK 64-Bit Server VM Zulu11.37+17-CA (build 11.0.6+10-LTS, mixed mode)
What am I doing wrong?

The access level of each object member cannot be increased anymore. The operation is not allowed anymore since Java 9 and has been hardly disabled in release 11.
You need to run this application with Java 8.
See https://www.oracle.com/technetwork/java/javase/9-relnote-issues-3704069.html (search for "setAccessible"). There is a new Object which must now be used to allow it again: https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/reflect/AccessibleObject.html

Related

Warning messages using java azure client

Using azure-storage-blob-client, I see these:
10:46:38 [main] WARN JacksonVersion - Could not find version of 'jackson-annotations'.
10:46:38 [main] ERROR JacksonVersion - Version 'unknown' of package 'jackson-annotations' is not supported (older than earliest supported version - `2.10.0`), please upgrade.
10:46:38 [main] WARN JacksonVersion - Could not find version of 'jackson-core'.
10:46:38 [main] ERROR JacksonVersion - Version 'unknown' of package 'jackson-core' is not supported (older than earliest supported version - `2.10.0`), please upgrade.
10:46:38 [main] WARN JacksonVersion - Could not find version of 'jackson-databind'.
10:46:38 [main] ERROR JacksonVersion - Version 'unknown' of package 'jackson-databind' is not supported (older than earliest supported version - `2.10.0`), please upgrade.
10:46:38 [main] INFO JacksonVersion - Package versions: jackson-annotations=unknown, jackson-core=unknown, jackson-databind=unknown, jackson-dataformat-xml=2.12.5, jackson-datatype-jsr310=2.12.5, azure-core=1.21.0
10:46:38 [main] DEBUG MemberNameConverterImpl - Failed to retrieve MethodHandles used to get naming strategy. Falling back to BeanUtils.
java.lang.ClassNotFoundException: com.fasterxml.jackson.databind.introspect.AccessorNamingStrategy.Provider
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at com.azure.core.implementation.jackson.MemberNameConverterImpl.<clinit>(MemberNameConverterImpl.java:55)
at com.azure.core.implementation.jackson.ObjectMapperShim.<init>(ObjectMapperShim.java:139)
at com.azure.core.implementation.jackson.ObjectMapperShim.createHeaderMapper(ObjectMapperShim.java:127)
at com.azure.core.util.serializer.JacksonAdapter.<init>(JacksonAdapter.java:75)
at com.azure.core.util.serializer.JacksonAdapter.<init>(JacksonAdapter.java:53)
at com.azure.core.util.serializer.JacksonAdapter.createDefaultSerializerAdapter(JacksonAdapter.java:111)
at com.azure.storage.blob.implementation.util.ModelHelper.<clinit>(ModelHelper.java:53)
at com.azure.storage.blob.BlobUrlParts.parse(BlobUrlParts.java:371)
at com.azure.storage.blob.BlobServiceClientBuilder.endpoint(BlobServiceClientBuilder.java:147)
at com.azure.storage.blob.BlobServiceClientBuilder.connectionString(BlobServiceClientBuilder.java:286)
Why do I get these warning messages?
Turns out one of my dependencies was a "shaded jar" including the jackson components. This shading processes apparently stripped from the jar some kind of version info of which version of jackson was defined. It could safely be ignored.

Sonarqube 7.9.1 refuses to start after an upgrade from 6.7.7

The previous version of SQ was 6.7.7 LTS, so i upgraded to 7.9.1 LTS.
I installed a new JDK
java version "11.0.4" 2019-07-16 LTS
Java(TM) SE Runtime Environment 18.9 (build 11.0.4+10-LTS)
Java HotSpot(TM) 64-Bit Server VM 18.9 (build 11.0.4+10-LTS, mixed mode)
but still SQ stopped.
This is a snippet of the sonar.log
2019.08.01 14:50:19 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /data/sonarqube/sonarqube-7.9.1/temp
2019.08.01 14:50:19 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001
2019.08.01 14:50:19 INFO app[][o.s.a.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/data/sonarqube/sonarqube-7.9.1/elasticsearch]: /data/sonarqube/sonarqube-````7.9.1/elasticsearch/bin/elasticsearch
2019.08.01 14:50:19 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running
Java HotSpot(TM) 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
2019.08.01 14:50:20 INFO app[][o.e.p.PluginsService] no modules loaded
2019.08.01 14:50:20 INFO app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin]
2019.08.01 14:50:22 WARN app[][o.s.a.p.AbstractManagedProcess] Process exited with exit value [es]: 1
2019.08.01 14:50:22 INFO app[][o.s.a.SchedulerImpl] Process[es] is stopped
2019.08.01 14:50:22 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped
2019.08.01 14:50:22 INFO app[][o.e.c.t.TransportClientNodesService] failed to get node info for {#transport#-1}{GK1cYpSNTM-eycINcVQsOA}{127.0.0.1}{127.0.0.1:9001}, disconnecting...
java.lang.IllegalStateException: Future got interrupted
and a snippet of es.log
2019.08.01 14:50:22 ERROR es[][o.e.b.Bootstrap] Exception
java.lang.RuntimeException: can not run elasticsearch as root
at org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:103) ~[elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170) ~[elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:333) [elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:159) [elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:150) [elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) [elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:124) [elasticsearch-cli-6.8.0.jar:6.8.0]
at org.elasticsearch.cli.Command.main(Command.java:90) [elasticsearch-cli-6.8.0.jar:6.8.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:116) [elasticsearch-6.8.0.jar:6.8.0]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:93) [elasticsearch-6.8.0.jar:6.8.0]
of course i start SQ (and ES) with a non-root user. That’s why i don’t understand the error in es.log
I can't find the solution at all. Please can you help me?
With kind regards, William

SpringBoot Project build failure with Open JDK 11.0.2 in Intellij 2019.1

I recently upgraded my project from Oracle JDK8 to OpenJDK 11.0.2
Java Version in my system
openjdk 11.0.2 2019-01-15
OpenJDK Runtime Environment 18.9 (build 11.0.2+9)
OpenJDK 64-Bit Server VM 18.9 (build 11.0.2+9, mixed mode)
When I build my project using maven from the terminal it compiles fine but is failing while building the same using IntelliJ. There is no stack trace to look upon, every time I build the project it shows me this error.
This says it's a NullPointerException and I know we have lots of duplicate questions on SO for the same, I am not asking to resolve NullPointer but instead if someone can help me just figure out a way to check the logs while building the project using play button in IntelliJ, I will help myself afterward.
I checked the idea.log file and there is nothing to be suspicious about.
019-04-30 16:44:54,956 [1342048] INFO - rationStore.ComponentStoreImpl - Saving Project '/Users/neeraj/Projects/office/PartnerPortal' PartnerPortallibraryTable took 19 ms
2019-04-30 16:44:54,995 [1342087] INFO - ij.compiler.impl.CompileDriver - COMPILATION STARTED (BUILD PROCESS)
2019-04-30 16:44:55,154 [1342246] INFO - j.compiler.server.BuildManager - Using preloaded build process to compile /Users/neeraj/Projects/office/PartnerPortal
2019-04-30 16:44:56,433 [1343525] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 51 ms
2019-04-30 16:44:57,683 [1344775] INFO - over.idea.build.CloverCompiler - External build - file generated: outputRoot=</Users/neeraj/Projects/office/PartnerPortal/target/classes> relativePath=<META-INF/PartnerPortal.kotlin_module>
2019-04-30 16:44:58,102 [1345194] INFO - lij.compiler.impl.CompilerUtil - COMPILATION FINISHED (BUILD PROCESS); Errors: 1; warnings: 0 took 3111 ms: 0 min 3sec
2019-04-30 16:44:58,103 [1345195] INFO - over.idea.build.CloverCompiler - CLOVER: COMPILATION IN EXTERNAL BUILD PROCESS HAS FINISHED
2019-04-30 16:44:58,106 [1345198] INFO - s.CompilerReferenceServiceBase - backward reference index reader is opened
2019-04-30 16:44:58,455 [1345547] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stdout]: Build process started. Classpath: /Applications/IntelliJ IDEA.app/Contents/lib/jps-launcher.jar:/Library/Java/JavaVirtualMachines/jdk-11.0.2.jdk/Contents/Home/lib/classes.zip
2019-04-30 16:44:58,486 [1345578] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
2019-04-30 16:44:58,486 [1345578] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: SLF4J: Defaulting to no-operation (NOP) logger implementation
2019-04-30 16:44:58,486 [1345578] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
2019-04-30 16:44:59,360 [1346452] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: WARNING: An illegal reflective access operation has occurred
2019-04-30 16:44:59,360 [1346452] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: WARNING: Illegal reflective access by com.google.protobuf.UnsafeUtil (file:/Applications/IntelliJ%20IDEA.app/Contents/lib/protobuf-java-3.5.1.jar) to field java.nio.Buffer.address
2019-04-30 16:44:59,360 [1346452] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: WARNING: Please consider reporting this to the maintainers of com.google.protobuf.UnsafeUtil
2019-04-30 16:44:59,360 [1346452] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
2019-04-30 16:44:59,361 [1346453] INFO - j.compiler.server.BuildManager - BUILDER_PROCESS [stderr]: WARNING: All illegal access operations will be denied in a future release
2019-04-30 16:45:04,513 [1351605] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 47 ms
2019-04-30 16:45:21,297 [1368389] INFO - mponents.impl.stores.StoreUtil - saveProjectsAndApp took 41 ms
Disabling "Atlassian Clover for IDEA" plugin helped in this case.
In my case it was due to
<arg>-J--permit-illegal-access</arg>
flag in maven-compiler-plugin.
Checking intelij logs also helps
Help -> Collect logs and diagnostics data
in InteliJ menu

Swing Application not launching when trying to test with robotframework

I am trying to run some automated tests on a swing based java application using the robotframework's swing library. I think I have got all the libraries installed. However, when I run the test it is unable to actually launch the application properly (i.e. the fail tests), and I am getting whole bunch of warnings(?) before the test actually starts. Also, I am briefly see the Java icon appear on the dock, but no window actually opens.
I don't have any problem with testing the demo application here though. That works fine.
Below is my robot script:
*** settings ***
Library SwingLibrary
*** Test Cases ***
Test Push Button
Start Test Application
*** Keywords ***
Start Test Application
Start Application com.xyz.app.ui.MainFrame
Select Main Window
Here is the output:
davids$ sudo CLASSPATH=lib/swinglibrary-1.5.1.jar:app.jar jybot app.robot
Password:
10:31:34.578 [MainThread] DEBUG o.p.n.u.i.l.InternalLoggerFactory - Using SLF4J as the default logging framework
10:31:34.613 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent0 - java.nio.Buffer.address: available
10:31:34.613 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent0 - sun.misc.Unsafe.theUnsafe: available
10:31:34.614 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent0 - sun.misc.Unsafe.copyMemory: available
10:31:34.615 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent0 - java.nio.Bits.unaligned: true
10:31:34.616 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - Java version: 8
10:31:34.616 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - -Dio.netty.noUnsafe: false
10:31:34.616 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - sun.misc.Unsafe: available
10:31:34.617 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - -Dio.netty.noJavassist: false
10:31:34.697 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - Javassist: available
10:31:34.698 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - -Dio.netty.tmpdir: /var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/T (java.io.tmpdir)
10:31:34.698 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - -Dio.netty.bitMode: 64 (sun.arch.data.model)
10:31:34.698 [MainThread] DEBUG o.p.n.u.internal.PlatformDependent - -Dio.netty.noPreferDirect: false
10:31:34.698 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numHeapArenas: 4
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.numDirectArenas: 4
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.pageSize: 8192
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxOrder: 11
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.chunkSize: 16777216
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.tinyCacheSize: 512
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.smallCacheSize: 256
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.normalCacheSize: 64
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.maxCachedBufferCapacity: 32768
10:31:34.699 [MainThread] DEBUG o.p.n.buffer.PooledByteBufAllocator - -Dio.netty.allocator.cacheTrimInterval: 8192
10:31:34.720 [MainThread] DEBUG o.p.n.c.MultithreadEventLoopGroup - -Dio.netty.eventLoopThreads: 8
10:31:34.768 [MainThread] DEBUG o.p.netty.channel.nio.NioEventLoop - -Dio.netty.noKeySetOptimization: false
10:31:34.768 [MainThread] DEBUG o.p.netty.channel.nio.NioEventLoop - -Dio.netty.selectorAutoRebuildThreshold: 512
==============================================================================
app
==============================================================================
Test Push Button | FAIL |
com.xyz.app.ui.MainFrame
------------------------------------------------------------------------------
app | FAIL |
1 critical test, 0 passed, 1 failed
1 test total, 0 passed, 1 failed
==============================================================================
Output: /Users/davids/swinglibrary-demo/output.xml
Log: /Users/davids/swinglibrary-demo/log.html
Report: /Users/davids/swinglibrary-demo/report.html
Also, I have verified that the app.jar file does in fact contain the path to the MainFrame class: com/xyz/app/ui/MainFrame.class
UPDATE:
Using "Start Application In Separate Thread" keyword instead of "Start Application" gives a new error. It seems that it can't find the main class, even though I have manually confirmed the path of the main class by unzipping the jar file.:
Exception in thread "Thread-14" java.lang.RuntimeException: java.lang.ClassNotFoundException: com.xyz.app.ui.MainFrame
at org.robotframework.swing.keyword.launch.ApplicationLaunchingKeywords$1.run(ApplicationLaunchingKeywords.java:53)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.ClassNotFoundException: com.xyz.app.ui.MainFrame
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:264)
at org.robotframework.swing.keyword.launch.ApplicationLaunchingKeywords.getMainMethod(ApplicationLaunchingKeywords.java:64)
at org.robotframework.swing.keyword.launch.ApplicationLaunchingKeywords.launchApplication(ApplicationLaunchingKeywords.java:32)
at org.robotframework.swing.keyword.launch.ApplicationLaunchingKeywords$1.run(ApplicationLaunchingKeywords.java:51)
... 1 more
Following solves the issue:
Setting CLASSPATH using export instead of defining it inline.
Using the fullpath to the .jars.

ERROR SparkContext: Error initializing SparkContext.. IntelliJ and Scala

I'm using Window 10, Scala 2.10.2, Spark 1.6.1, and Java 1.8.
Below is the code I'm trying to run.
import org.apache.spark.SparkContext
import org.apache.spark.SparkConf
object WordsCount {
def main(args: Array[String]): Unit = {
val sparkConf = new SparkConf().setAppName("SparkWordCount").setMaster("local[2]")
val sc = new SparkContext(sparkConf)
val text = sc.textFile("shortTwitter.txt")
val counts = text.flatMap(line=>line.split(" ")).map(word=>(word,1)).reduceByKey(_+_)
counts.foreach(println)
}
}
and I got the following error.
"C:\Program Files\Java\jdk1.8.0_101\bin\java" -Didea.launcher.port=7533 "-Didea.launcher.bin.path=C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2.3\bin" -Dfile.encoding=UTF-8 -classpath "C:\Program Files\Java\jdk1.8.0_101\jre\lib\charsets.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\deploy.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\access-bridge-64.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\cldrdata.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\dnsns.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\jaccess.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\jfxrt.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\localedata.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\nashorn.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\sunec.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\sunjce_provider.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\sunmscapi.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\sunpkcs11.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\ext\zipfs.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\javaws.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\jce.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\jfr.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\jfxswt.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\jsse.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\management-agent.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\plugin.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\resources.jar;C:\Program Files\Java\jdk1.8.0_101\jre\lib\rt.jar;C:\Users\Administrator\IdeaProjects\choi_jieun_assignment1\out\production\choi_jieun_assignment1;C:\scala\lib\jline.jar;C:\scala\lib\scalap.jar;C:\scala\lib\diffutils.jar;C:\scala\lib\akka-actors.jar;C:\scala\lib\scala-swing.jar;C:\scala\lib\scala-actors.jar;C:\scala\lib\scala-library.jar;C:\scala\lib\scala-partest.jar;C:\scala\lib\scala-reflect.jar;C:\scala\lib\scala-compiler.jar;C:\scala\lib\typesafe-config.jar;C:\scala\lib\scala-actors-migration.jar;C:\Users\Administrator\.ivy2\cache\org.scala-lang\scala-library\jars\scala-library-2.10.2.jar;C:\Users\Administrator\.ivy2\cache\org.scala-lang\scala-reflect\jars\scala-reflect-2.10.2.jar;C:\spark\lib\datanucleus-core-3.2.10.jar;C:\spark\lib\datanucleus-rdbms-3.2.9.jar;C:\spark\lib\spark-1.6.1-yarn-shuffle.jar;C:\spark\lib\datanucleus-api-jdo-3.2.6.jar;C:\spark\lib\spark-assembly-1.6.1-hadoop2.4.0.jar;C:\spark\lib\spark-examples-1.6.1-hadoop2.4.0.jar;C:\Program Files (x86)\JetBrains\IntelliJ IDEA Community Edition 2016.2.3\lib\idea_rt.jar" com.intellij.rt.execution.application.AppMain WordsCount
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/spark/lib/spark-assembly-1.6.1-hadoop2.4.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/spark/lib/spark-examples-1.6.1-hadoop2.4.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
16/09/14 16:03:21 INFO SparkContext: Running Spark version 1.6.1
16/09/14 16:03:22 WARN NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
16/09/14 16:03:22 INFO SecurityManager: Changing view acls to: Administrator
16/09/14 16:03:22 INFO SecurityManager: Changing modify acls to: Administrator
16/09/14 16:03:22 INFO SecurityManager: SecurityManager: authentication disabled; ui acls disabled; users with view permissions: Set(Administrator); users with modify permissions: Set(Administrator)
16/09/14 16:03:23 INFO Utils: Successfully started service 'sparkDriver' on port 50762.
16/09/14 16:03:23 ERROR SparkContext: Error initializing SparkContext.
java.lang.NoSuchMethodException: akka.remote.RemoteActorRefProvider.<init> (java.lang.String, akka.actor.ActorSystem$Settings, akka.event.EventStream, akka.actor.Scheduler, akka.actor.DynamicAccess)
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getDeclaredConstructor(Class.java:2178)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:77)
at scala.util.Try$.apply(Try.scala:161)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:74)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:85)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:85)
at scala.util.Success.flatMap(Try.scala:200)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:85)
at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:546)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:104)
at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)
at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1988)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1979)
at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:266)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:457)
at WordsCount$.main(WordsCount.scala:7)
at WordsCount.main(WordsCount.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
16/09/14 16:03:23 INFO SparkContext: Successfully stopped SparkContext
Exception in thread "main" java.lang.NoSuchMethodException: akka.remote.RemoteActorRefProvider.<init>(java.lang.String, akka.actor.ActorSystem$Settings, akka.event.EventStream, akka.actor.Scheduler, akka.actor.DynamicAccess)
at java.lang.Class.getConstructor0(Class.java:3082)
at java.lang.Class.getDeclaredConstructor(Class.java:2178)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$2.apply(DynamicAccess.scala:77)
at scala.util.Try$.apply(Try.scala:161)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:74)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:85)
at akka.actor.ReflectiveDynamicAccess$$anonfun$createInstanceFor$3.apply(DynamicAccess.scala:85)
at scala.util.Success.flatMap(Try.scala:200)
at akka.actor.ReflectiveDynamicAccess.createInstanceFor(DynamicAccess.scala:85)
at akka.actor.ActorSystemImpl.<init>(ActorSystem.scala:546)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:111)
at akka.actor.ActorSystem$.apply(ActorSystem.scala:104)
at org.apache.spark.util.AkkaUtils$.org$apache$spark$util$AkkaUtils$$doCreateActorSystem(AkkaUtils.scala:121)
at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:53)
at org.apache.spark.util.AkkaUtils$$anonfun$1.apply(AkkaUtils.scala:52)
at org.apache.spark.util.Utils$$anonfun$startServiceOnPort$1.apply$mcVI$sp(Utils.scala:1988)
at scala.collection.immutable.Range.foreach$mVc$sp(Range.scala:141)
at org.apache.spark.util.Utils$.startServiceOnPort(Utils.scala:1979)
at org.apache.spark.util.AkkaUtils$.createActorSystem(AkkaUtils.scala:55)
at org.apache.spark.SparkEnv$.create(SparkEnv.scala:266)
at org.apache.spark.SparkEnv$.createDriverEnv(SparkEnv.scala:193)
at org.apache.spark.SparkContext.createSparkEnv(SparkContext.scala:288)
at org.apache.spark.SparkContext.<init>(SparkContext.scala:457)
at WordsCount$.main(WordsCount.scala:7)
at WordsCount.main(WordsCount.scala)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:147)
Process finished with exit code 1
I have been struggling for a couple of hours to solve this problem but couldn't figure out. Can someone help me out? Thanks!
Try installing the Java 7 JDK and you may have better luck. Although the release notes for Scala 2.10.2 seem to be missing, 2.10.4 doesn't support Java 8:
New ByteCode emitter based on ASM
Can target JDK 1.5, 1.6 and 1.7
Emits 1.6 bytecode by default
Old 1.5 backend is deprecated
...and even 2.11.x only has experimental Java 8 support:
The Scala 2.11.x series targets Java 6, with (evolving) experimental support for Java 8. In 2.11.1, Java 8 support is mostly limited to reading Java 8 bytecode and parsing Java 8 source. Stay tuned for more complete (experimental) Java 8 support. The next major release, 2.12, will most likely target Java 8 by default.
Also see: Can I use java 8 in an mixed scala 2.10 / java project built by sbt?

Categories