We Keep Coding

Scala Spark MLLib NoClassDefFoundError

I am learning ALS in Spark mllib
bu when i try to create Rating to passing to ALS.train I got this error
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/mllib/recommendation/Rating
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
at java.lang.Class.getMethod0(Class.java:2866)
at java.lang.Class.getMethod(Class.java:1676)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:126)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.mllib.recommendation.Rating
at java.net.URLClassLoader$1.run(URLClassLoader.java:359)
at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 5 more
Process finished with exit code 1
I use
sbt 0.13.15,
Scala 2.10.4,
spark-core 2.10,
spark-mllib 2.10,
tried on both java 7 and 8
Did I do something wrong?

I found the solution for my case
the thing is i try to run this spark with IntelliJ IDE and I found that in my Build.sbt i have something like this to use dependencies
libraryDependencies ++= Seq( "org.apache.spark" % "spark-core_2.10" % "2.1.0" , "org.apache.spark" % "spark-mllib_2.10" % "2.1.0" % "provided" )
the problem is at "provided" i should change this to "compile" when i try to run with IntelliJ otherwise i'll see error like that

Yep, just remove the provided will solve the problem
libraryDependencies ++= Seq( "org.apache.spark" % "spark-core_2.10" % "2.1.0" ,
"org.apache.spark" % "spark-mllib_2.10" % "2.1.0" )

java.lang.UnsupportedClassVersionError: com/typesafe/config/ConfigException : Unsupported major.minor version 52.0 in play framework project [duplicate]

This question already has answers here:
How to fix java.lang.UnsupportedClassVersionError: Unsupported major.minor version
(51 answers)
Closed 5 years ago.
Hi Everyone i was facing the below error
[error] java.lang.UnsupportedClassVersionError: com/typesafe/config/ConfigException : Unsupported major.minor version 52.0
i was using play framework of version 2.5.9.
and java of version 1.7
please find my build and plugins file :
build.sbt
name := """getbike2"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayJava, PlayEbean)
scalaVersion := "2.11.7"
libraryDependencies ++= Seq(
javaJdbc,
cache,
javaWs,
"org.mockito" % "mockito-core" % "2.2.22",
"com.google.code.gson" % "gson" % "1.7.1",
"commons-io" % "commons-io" % "2.4",
"org.apache.directory.studio" % "org.apache.commons.io" % "2.4",
"org.apache.poi" % "poi" %"3.9",
"org.apache.poi" % "poi-ooxml" % "3.9"
)
jacoco.settings
parallelExecution in jacoco.Config := false
testOptions += Tests.Argument(TestFrameworks.JUnit, "-v", "-q", "-a")
plugins.sbt:
// The Play plugin
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.5.9")
// Web plugins
addSbtPlugin("com.typesafe.sbt" % "sbt-coffeescript" % "1.0.0")
addSbtPlugin("com.typesafe.sbt" % "sbt-less" % "1.1.0")
addSbtPlugin("com.typesafe.sbt" % "sbt-jshint" % "1.0.4")
addSbtPlugin("com.typesafe.sbt" % "sbt-rjs" % "1.0.8")
addSbtPlugin("com.typesafe.sbt" % "sbt-digest" % "1.1.1")
addSbtPlugin("com.typesafe.sbt" % "sbt-mocha" % "1.1.0")
addSbtPlugin("org.irundaia.sbt" % "sbt-sassify" % "1.4.6")
// Play enhancer - this automatically generates getters/setters for public fields
// and rewrites accessors of these fields to use the getters/setters. Remove this
// plugin if you prefer not to have this feature, or disable on a per project
// basis using disablePlugins(PlayEnhancer) in your build.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-play-enhancer" % "1.1.0")
// Play Ebean support, to enable, uncomment this line, and enable in your build.sbt using
// enablePlugins(PlayEbean).
addSbtPlugin("com.typesafe.sbt" % "sbt-play-ebean" % "3.0.2")
addSbtPlugin("de.johoop" % "jacoco4sbt" % "2.2.0")
Terminal log:
vave#vave-Lenovo-G500:~/Desktop/getbike/getbike$ activator
[info] Loading project definition from /home/vave/Desktop/getbike/getbike/project
[info] Set current project to getbike2 (in build file:/home/vave/Desktop/getbike/getbike/)
java.lang.UnsupportedClassVersionError: com/typesafe/config/ConfigException : Unsupported major.minor version 52.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClass(ClassLoader.java:803)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:142)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:442)
at java.net.URLClassLoader.access$100(URLClassLoader.java:64)
at java.net.URLClassLoader$1.run(URLClassLoader.java:354)
at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
at com.typesafe.sbt.web.SbtWeb$$anonfun$com$typesafe$sbt$web$SbtWeb$$load$1.apply(SbtWeb.scala:559)
at com.typesafe.sbt.web.SbtWeb$$anonfun$com$typesafe$sbt$web$SbtWeb$$load$1.apply(SbtWeb.scala:559)
at scala.Option.fold(Option.scala:157)
at com.typesafe.sbt.web.SbtWeb$.com$typesafe$sbt$web$SbtWeb$$load(SbtWeb.scala:573)
at com.typesafe.sbt.web.SbtWeb$$anonfun$globalSettings$1$$anonfun$apply$1.apply(SbtWeb.scala:143)
at com.typesafe.sbt.web.SbtWeb$$anonfun$globalSettings$1$$anonfun$apply$1.apply(SbtWeb.scala:143)
at scala.Function1$$anonfun$andThen$1.apply(Function1.scala:55)
at sbt.Project$.setProject(Project.scala:319)
at sbt.BuiltinCommands$.doLoadProject(Main.scala:503)
at sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:484)
at sbt.BuiltinCommands$$anonfun$loadProjectImpl$2.apply(Main.scala:484)
at sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:59)
at sbt.Command$$anonfun$applyEffect$1$$anonfun$apply$2.apply(Command.scala:59)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$$anonfun$applyEffect$2$$anonfun$apply$3.apply(Command.scala:61)
at sbt.Command$.process(Command.scala:93)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1$$anonfun$apply$1.apply(MainLoop.scala:96)
at sbt.State$$anon$1.process(State.scala:184)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.MainLoop$$anonfun$1.apply(MainLoop.scala:96)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.MainLoop$.next(MainLoop.scala:96)
at sbt.MainLoop$.run(MainLoop.scala:89)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:68)
at sbt.MainLoop$$anonfun$runWithNewLog$1.apply(MainLoop.scala:63)
at sbt.Using.apply(Using.scala:24)
at sbt.MainLoop$.runWithNewLog(MainLoop.scala:63)
at sbt.MainLoop$.runAndClearLast(MainLoop.scala:46)
at sbt.MainLoop$.runLoggedLoop(MainLoop.scala:30)
at sbt.MainLoop$.runLogged(MainLoop.scala:22)
at sbt.StandardMain$.runManaged(Main.scala:54)
at sbt.xMain.run(Main.scala:29)
at xsbt.boot.Launch$$anonfun$run$1.apply(Launch.scala:109)
at xsbt.boot.Launch$.withContextLoader(Launch.scala:128)
at xsbt.boot.Launch$.run(Launch.scala:109)
at xsbt.boot.Launch$$anonfun$apply$1.apply(Launch.scala:35)
at xsbt.boot.Launch$.launch(Launch.scala:117)
at xsbt.boot.Launch$.apply(Launch.scala:18)
at xsbt.boot.Boot$.runImpl(Boot.scala:41)
at xsbt.boot.Boot$.main(Boot.scala:17)
at xsbt.boot.Boot.main(Boot.scala)
[error] java.lang.UnsupportedClassVersionError: com/typesafe/config/ConfigException : Unsupported major.minor version 52.0
[error] Use 'last' for the full log.
Project loading failed: (r)etry, (q)uit, (l)ast, or (i)gnore?
So please help me how to fix this issue.
Thanks in advance

The Play framework requires Java 8 from version 2.4.0 and higher. You should update your Java to version 8 if you want to keep using Play 2.5.9.
https://www.playframework.com/changelog
Play 2.4.0 “Damiya”
Released 26 May 2015
Dependency injection out of the box.
Testing is easier thanks to better support for mocking.
It is now straightforward to embed Play in your application.
You can now aggregate reverse routers from multiple projects.
More Java 8 integration —- Java 8 is now required.
Choice of standard project layout.
Many new anorm features. Anorm is now its own project!
Upgraded to Ebean 4. Ebean is (also) its own project!
HikariCP is the default connection pool
WS supports Server Name Identification (SNI).

Configuration in Java Play - ebean and postgresql

I can't make my Play Framework project run properly. I want to use a postgresql db and ebean, but there's something wrong with the configuration and I keep getting compilation errors. Could you have a look and check what might be the reason?
My build.sbt file:
import play.sbt.PlayJava
name := """play-java-intro"""
version := "1.0-SNAPSHOT"
lazy val root = (project in file(".")).enablePlugins(PlayJava)
scalaVersion := "2.11.7"
libraryDependencies ++= Seq(
"org.postgresql" % "postgresql" % "9.4-1206-jdbc42",
"javax.persistence" % "persistence-api" % "1.0.2"
)
fork in run := true
plugins.sbt:
// The Play plugin
addSbtPlugin("com.typesafe.play" % "sbt-plugin" % "2.5.13")
// Web plugins
addSbtPlugin("com.typesafe.sbt" % "sbt-coffeescript" % "1.0.0")
addSbtPlugin("com.typesafe.sbt" % "sbt-less" % "1.1.0")
addSbtPlugin("com.typesafe.sbt" % "sbt-jshint" % "1.0.4")
addSbtPlugin("com.typesafe.sbt" % "sbt-rjs" % "1.0.8")
addSbtPlugin("com.typesafe.sbt" % "sbt-digest" % "1.1.1")
addSbtPlugin("com.typesafe.sbt" % "sbt-mocha" % "1.1.0")
addSbtPlugin("org.irundaia.sbt" % "sbt-sassify" % "1.4.6")
addSbtPlugin("com.typesafe.sbt" % "sbt-play-ebean" % "3.0.0")
addSbtPlugin("com.typesafe.sbt" % "sbt-play-enhancer" % "1.1.0")
My model class:
package models;
import java.util.*;
import com.avaje.ebean.Finder;
import com.avaje.ebean.Model;
import play.data.format.*;
import play.data.validation.*;
import javax.persistence.Entity;
import javax.persistence.Id;
#Entity
public class Person extends Model {
#Id
private Long id;
#Constraints.Required
public String name;
#Constraints.Required
public String surname;
#Formats.DateTime(pattern="dd/MM/yyyy")
public Date date;
public static Finder<Long,Person> finder = new Finder<Long, Person>(Person.class);
}
And the error message:
[error] com.avaje.ebean.Finder
[error] /home/guras/play-java-intro/app/models/Person.java:6: package com.avaje.ebean does not exist
[error] /home/guras/play-java-intro/app/models/Person.java:7: package com.avaje.ebean does not exist
[error] com.avaje.ebean.Model
[error] /home/guras/play-java-intro/app/models/Person.java:15: cannot find symbol
[error] symbol: class Model
[error] Model
[error] /home/guras/play-java-intro/app/models/Person.java:28: cannot find symbol
[error] symbol: class Finder
[error] location: class models.Person
[error] Finder
[error] /home/guras/play-java-intro/app/models/Person.java:28: cannot find symbol
[error] symbol: class Finder
[error] location: class models.Person
[error] Finder
[info] play.forkrun.protocol.Serializers$playExceptionPickler$$anon$1: Compilation error[package com.avaje.ebean does not exist]
[info] at play.forkrun.protocol.Serializers$playExceptionPickler$.unpickle(Serializers.scala:159)
[info] at play.forkrun.protocol.Serializers$PlayRunsupportReloaderCompileFailureUnpickler$macro$56$2$.unpickle(Serializers.scala:175)
[info] at play.forkrun.protocol.Serializers$PlayRunsupportReloaderCompileResultUnpickler$macro$66$2$.unpickle(Serializers.scala:181)
[info] at scala.pickling.Unpickler$class.unpickleEntry(Pickler.scala:79)
[info] at play.forkrun.protocol.Serializers$PlayRunsupportReloaderCompileResultUnpickler$macro$66$2$.unpickleEntry(Serializers.scala:181)
[info] at scala.pickling.functions$.unpickle(functions.scala:11)
[info] at sbt.serialization.JsonValue$$anonfun$parse$1.apply(SerializedValue.scala:104)
[info] at scala.util.Try$.apply(Try.scala:192)
[info] at sbt.serialization.JsonValue.parse(SerializedValue.scala:104)
[info] at sbt.protocol.BuildValue.value(Values.scala:11)

You have to add ebean jars into your class path as dependency. The imports are failing because You don't have the dependency jars containing those packages in your class path.
You can find the jars in Maven centeral repository here

On you plugins.sbt
addSbtPlugin("com.typesafe.sbt" % "sbt-play-ebean" % "3.1.0")
On your build.sbt
lazy val root = (project in file(".")).enablePlugins(PlayJava, PlayEbean)
libraryDependencies ++= Seq(
javaJdbc,
javaCore,
"org.postgresql" % "postgresql" % "9.4.1212"
)
You have to enable ebean plugin in order to work with play framework.

SBT Unresolved Dependency Error - Elasticsearch Test Framework in a Scala Project

I am doing a Scala project, built with SBT and trying to follow the documentation in:
Elasticsearch website, about their test framework: https://www.elastic.co/guide/en/elasticsearch/reference/current/using-elasticsearch-test-classes.html
There, they talk about using the same versions of ES, which in my case would be ES: 2.2.0 and lucene: 5.4.1
However, I am not able to build my project, due to dependencies issues with the test framework artifact and version.
I have added:
"org.elasticsearch.test" % "framework" % "2.2.0"
But SBT fails with:
Error:Error while importing SBT project:<br/>...<br/><pre>[info]
Resolving commons-io#commons-io;2.4 ... [info] Resolving io.spray#spray-
json_2.12;1.3.2 ... [info] Resolving com.typesafe.akka#akka-http-
testkit_2.12;10.0.1 ... [info] Resolving com.typesafe.akka#akka-stream-
testkit_2.12;2.4.16 ... [info] Resolving com.typesafe.akka#akka-
testkit_2.12;2.4.16 ... [info] Resolving jline#jline;2.14.1 ... [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
org.elasticsearch.test#framework;2.2.0: not found [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note:
Unresolved dependencies path: [warn]
org.elasticsearch.test:framework:2.2.0
(/Users/filipemiranda/datawerks/scala-query-engine/build.sbt#L13-30)
[warn] +- default:scala-query-engine_2.12:1.0 [trace] Stack trace
suppressed: run 'last *:update' for the full output. [trace] Stack trace
suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency:
org.elasticsearch.test#framework;2.2.0: not found [error]
(*:ssExtractDependencies) sbt.ResolveException: unresolved dependency:
org.elasticsearch.test#framework;2.2.0: not found [error] Total time: 6 s,
completed Jan 21, 2017 8:20:44 PM</pre><br/>See complete log in <a
href="/Users/filipemiranda/Library/Logs/IdeaIC2016.1/sbt.last.log">/Users/
filipemiranda/Library/Logs/IdeaIC2016.1/sbt.last.log</a>
I ave looked at: https://oss.sonatype.org/content/repositories/releases/org/elasticsearch/test/framework/
And I did not find the same version there.
Has anyone faced similar issues, how to solve this?
Here is my entire build.sbt
name := "project-name"
version := "1.0"
lazy val scalaV = "2.12.1"
scalaVersion := scalaV
lazy val akkaHttpV = "10.0.1"
lazy val esVersion = "2.2.0"
lazy val luceneVersion = "5.4.1"
lazy val scalatestV = "3.0.1"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-compiler" % scalaV,
"org.scalatest" %% "scalatest" % scalatestV,
"com.typesafe.akka" %% "akka-http-core" % akkaHttpV,
"com.typesafe.akka" %% "akka-http" % akkaHttpV,
"com.typesafe.akka" %% "akka-stream" % "2.4.16",
"com.storm-enroute" %% "scalameter" % "0.8.2" % "test",
"io.searchbox" % "jest" % "2.0.3",
"org.elasticsearch" % "elasticsearch" % "2.2.0" withSources,
"com.twitter" %% "util-eval" % "6.40.0",
"org.apache.lucene" % "lucene-test-framework" % "5.4.1",
"org.elasticsearch.test" % "framework" % "2.2.0",
"com.storm-enroute" %% "scalameter" % "0.8.2" % "test",
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpV % "test"
)
How can I work this around???

Unable to resolve mailer and util plugin [2.2 java ]

I am new to play 2.2 framework. I am getting started with secure social plugin and I have unresolved dependencies during compilation.
sbt.ResolveException: unresolved dependency: com.typesafe#play-plugins-util_2.10;2.2.0: not found
[error] unresolved dependency: com.typesafe#play-plugins-mailer_2.10;2.2.0: not found
The content of my build.sbt is:
libraryDependencies ++= Seq(
javaJdbc,
javaEbean,
cache,
"mysql" % "mysql-connector-java" % "5.1.18",
"com.typesafe" %% "play-plugins-util" % "2.1",
"com.typesafe" %% "play-plugins-mailer" % "2.1",
"securesocial" %% "securesocial" % "master-SNAPSHOT"
)
resolvers += Resolver.url("sbt-plugin-releases",
new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases"))(Resolver.ivyStylePatterns)
resolvers += Resolver.url("typesafe plugins", new URL("http://repo.typesafe.com/typesafe/releases/"))(Resolver.ivyStylePatterns)
resolvers += Resolver.url("sbt-plugin-snapshots", new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots/"))(Resolver.ivyStylePatterns)
and play.plugins is
1000:org.jooq.play.JooqPlugin
150000:com.typesafe.plugin.CommonsMailerPlugin
9994:securesocial.core.DefaultAuthenticatorStore
9995:securesocial.core.DefaultIdGenerator
9996:securesocial.core.providers.utils.DefaultPasswordValidator
9997:securesocial.controllers.DefaultTemplatesPlugin
9998:your.user.Service.Implementation <-- Important: You need to change this
9999:securesocial.core.providers.utils.BCryptPasswordHasher
10004:securesocial.core.providers.UsernamePasswordProvider
I have also tried to download these jars and put them in lib folder. Is this a known issue? It would be great if someone can help me out on this. Also, like in rails, how do we force installing the plugins to a given path like bundle install --path=.path ?

The following build.sbt should work:
name := """hello-play"""
version := "1.0-SNAPSHOT"
resolvers += Resolver.url("sbt-plugin-snapshots", new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots/"))(Resolver.ivyStylePatterns)
libraryDependencies ++= Seq(
javaJdbc,
javaEbean,
cache,
"mysql" % "mysql-connector-java" % "5.1.18",
"com.typesafe" %% "play-plugins-util" % "2.2.0",
"com.typesafe" %% "play-plugins-mailer" % "2.2.0",
"securesocial" %% "securesocial" % "master-SNAPSHOT"
)
play.Project.playScalaSettings