We Keep Coding

sbt run works fine but running jar fails

While I am running from sbt
(sbt run)
my akka http service is running fine, however gives the below error on running using jar
(java -jar myservice.jar)
created by publishLocal
(sbt publishLocal)
Exception in thread "main" java.lang.NoClassDefFoundError: org/slf4j/LoggerFactory
at com.rogers.profileauthn.StartProfileAuthService.<clinit>(StartProfileAuthService.java:15)
Caused by: java.lang.ClassNotFoundException: org.slf4j.LoggerFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
Dependencies in sbt file are below
val akkaVersion = "2.5.11"
val akkaHttpVersion = "10.0.11"
libraryDependencies ++= Seq(
"ch.qos.logback" % "logback-classic" % "1.2.3",
"com.typesafe.akka" %% "akka-http" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-stream" % akkaVersion,
"com.typesafe.akka" %% "akka-http-jackson" % akkaHttpVersion,
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpVersion % Test,
"com.typesafe.akka" %% "akka-testkit" % akkaVersion % Test,
"junit" % "junit" % "4.12" % Test,
"com.novocode" % "junit-interface" % "0.10" % Test,
"com.typesafe.play" %% "play-json" % "2.6.9",
"com.github.swagger-akka-http" %% "swagger-akka-http" % "0.11.0",
"com.google.inject" % "guice" % "4.1.0",
"com.google.guava" % "guava" % "18.0",
"org.mockito" % "mockito-core" % "2.11.0"
)
and the logger I am using is below
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
...
private static final Logger LOGGER = LoggerFactory.getLogger(StartService.class);
LOGGER.info("**** StartService.createAndStartActorSystem Starts ****");

I think thats because the jar did not come packaged with your dependency. Can you try creating a fat jar with sbt assembly? https://github.com/sbt/sbt-assembly. The publishLocal command is used for deploying to an ivy repository and not create jars to run like that i believe. (add plugin and run sbt assembly)

Adding specific version of scala-xml in fat jar

I would like to use the scala-xml jar with my project, I have the following configuration for the fat jar.
In a previous project, I was able to use XML locally using the version:
lazy val scalaXml = "org.scala-lang" % "scala-xml" % "2.11.0-M4"
but when I try and use that version with my fat jar, I get the error from my deployed jar( however, my tests still run correctly on my local machine):
scala/Product$class: java.lang.NoClassDefFoundError
java.lang.NoClassDefFoundError: scala/Product$class
at scala.xml.NamespaceBinding.<init>(NamespaceBinding.scala:23)
at scala.xml.TopScope$.<init>(TopScope.scala:17)
at scala.xml.TopScope$.<clinit>(TopScope.scala)
at example.models.soap11.SoapEnvelope$.toXML(SoapEnvelope.scala:9)
at example.AWSLambdaRequestHandler.execute(RequestHandler.scala:73)
scala-xml related
lazy val scalaXml = "org.scala-lang.modules" %% "scala-xml" % "1.0.6"
Error(tl;dr)
org.scala-lang#scala-library;2.12: not found
[warn] :: org.scala-lang#scala-reflect;2.12: not found
[warn] :: org.scala-lang#scala-compiler;2.12: not found
I have gone through this issue where the solution recommended was to do sbt update but that did not resolve my issue. Instead it resulted in the Error message above.
The build.sbt file of the project here seems to use
libraryDependencies += ("org.scala-lang" % "scala-compiler" % scalaVersion.value % "test").exclude("org.scala-lang.modules", s"scala-xml_${scalaVersion.value}")
as a test dependency. There are 2 other versions defined there:
version := "1.1.0-SNAPSHOT",
mimaPreviousVersion := Some("1.0.6"),
I have also tried the following configurations:
lazy val scalaXml = "org.scala-lang.modules" % "scala-xml_2.12" % "1.0.6"
lazy val scalaXml = "org.scala-lang.modules" %% "scala-xml_2.12" % "1.0.6"
lazy val scalaXml = "org.scala-lang" %% "scala-xml" % "1.0.6"
Detailed config and Error message:
build.sbt(Entire file)
import sbt.Keys.{libraryDependencies, _}
import sbt._
val lambdaVersion = "1.2.0"
val awsVersion = "1.11.22"
val json4sVersion = "3.6.0-M2"
val sttpVersion = "1.1.1"
lazy val scalaXml = "org.scala-lang.modules" %% "scala-xml" % "1.0.6"
lazy val sttp = Seq(
"com.softwaremill.sttp" %% "core",
"com.softwaremill.sttp" %% "json4s"
).map(_ % sttpVersion)
lazy val json4s = Seq(
"org.json4s" %% "json4s-native",
"org.json4s" %% "json4s-ext"
).map(_ % json4sVersion)
lazy val awsLambda = Seq(
"com.amazonaws" % "aws-lambda-java-events",
"com.amazonaws" % "aws-lambda-java-core"
).map(_ % lambdaVersion)
lazy val aws = Seq(
"com.amazonaws" % "aws-java-sdk-core",
"com.amazonaws" % "aws-java-sdk-s3"
).map (_ % awsVersion)
javacOptions ++= Seq("-source", "1.8", "-target", "1.8", "-Xlint")
lazy val root = (project in file(".")).
settings(
inThisBuild(List(
organization := "example",
scalaVersion := "2.12",
version := "0.1.0-SNAPSHOT",
retrieveManaged := true
)),
name := "aws-scala-lambda-test",
libraryDependencies += scalaXml,
libraryDependencies ++= json4s,
libraryDependencies ++= awsLambda,
libraryDependencies ++= aws,
libraryDependencies ++= sttp,
libraryDependencies += "org.scalatest" %% "scalatest" % "3.0.3" % Test,
libraryDependencies += "org.mockito" % "mockito-all" % "1.10.19" % Test
)
assemblyJarName in assembly := "my-example.jar"
assemblyMergeStrategy in assembly := {
case PathList("META-INF", _ # _*) => MergeStrategy.discard
case _ => MergeStrategy.first
}
Error
$ sbt
> update
[info] Resolving org.scala-lang#scala-reflect;2.12 ...
[info] Resolving org.scala-lang#scala-reflect;2.12 ...
[warn] module not found: org.scala-lang#scala-reflect;2.12
[warn] ==== local: tried
[warn] /Users/localuser/.ivy2/local/org.scala-lang/scala-reflect/2.12/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/scala-lang/scala-reflect/2.12/scala-reflect-2.12.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/localuser/.sbt/preloaded/org.scala-lang/scala-reflect/2.12/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/localuser/.sbt/preloaded/org/scala-lang/scala-reflect/2.12/scala-reflect-2.12.pom
[info] Resolving org.scala-lang.modules#scala-parser-combinators_2.12;1.0.4 ...
[info] Resolving org.mockito#mockito-all;1.10.19 ...
[info] Resolving org.scala-lang#scala-compiler;2.12 ...
[info] Resolving org.scala-lang#scala-compiler;2.12 ...
[warn] module not found: org.scala-lang#scala-compiler;2.12
[warn] ==== local: tried
[warn] /Users/localuser/.ivy2/local/org.scala-lang/scala-compiler/2.12/ivys/ivy.xml
[warn] ==== public: tried
[warn] https://repo1.maven.org/maven2/org/scala-lang/scala-compiler/2.12/scala-compiler-2.12.pom
[warn] ==== local-preloaded-ivy: tried
[warn] /Users/localuser/.sbt/preloaded/org.scala-lang/scala-compiler/2.12/ivys/ivy.xml
[warn] ==== local-preloaded: tried
[warn] file:////Users/localuser/.sbt/preloaded/org/scala-lang/scala-compiler/2.12/scala-compiler-2.12.pom
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: org.scala-lang#scala-library;2.12: not found
[warn] :: org.scala-lang#scala-reflect;2.12: not found
[warn] :: org.scala-lang#scala-compiler;2.12: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn]

Scala Spark MLLib NoClassDefFoundError

I am learning ALS in Spark mllib
bu when i try to create Rating to passing to ALS.train I got this error
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/spark/mllib/recommendation/Rating
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2625)
at java.lang.Class.getMethod0(Class.java:2866)
at java.lang.Class.getMethod(Class.java:1676)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:126)
Caused by: java.lang.ClassNotFoundException: org.apache.spark.mllib.recommendation.Rating
at java.net.URLClassLoader$1.run(URLClassLoader.java:359)
at java.net.URLClassLoader$1.run(URLClassLoader.java:348)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:347)
at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312)
at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
... 5 more
Process finished with exit code 1
I use
sbt 0.13.15,
Scala 2.10.4,
spark-core 2.10,
spark-mllib 2.10,
tried on both java 7 and 8
Did I do something wrong?

I found the solution for my case
the thing is i try to run this spark with IntelliJ IDE and I found that in my Build.sbt i have something like this to use dependencies
libraryDependencies ++= Seq( "org.apache.spark" % "spark-core_2.10" % "2.1.0" , "org.apache.spark" % "spark-mllib_2.10" % "2.1.0" % "provided" )
the problem is at "provided" i should change this to "compile" when i try to run with IntelliJ otherwise i'll see error like that

Yep, just remove the provided will solve the problem
libraryDependencies ++= Seq( "org.apache.spark" % "spark-core_2.10" % "2.1.0" ,
"org.apache.spark" % "spark-mllib_2.10" % "2.1.0" )

SBT Unresolved Dependency Error - Elasticsearch Test Framework in a Scala Project

I am doing a Scala project, built with SBT and trying to follow the documentation in:
Elasticsearch website, about their test framework: https://www.elastic.co/guide/en/elasticsearch/reference/current/using-elasticsearch-test-classes.html
There, they talk about using the same versions of ES, which in my case would be ES: 2.2.0 and lucene: 5.4.1
However, I am not able to build my project, due to dependencies issues with the test framework artifact and version.
I have added:
"org.elasticsearch.test" % "framework" % "2.2.0"
But SBT fails with:
Error:Error while importing SBT project:<br/>...<br/><pre>[info]
Resolving commons-io#commons-io;2.4 ... [info] Resolving io.spray#spray-
json_2.12;1.3.2 ... [info] Resolving com.typesafe.akka#akka-http-
testkit_2.12;10.0.1 ... [info] Resolving com.typesafe.akka#akka-stream-
testkit_2.12;2.4.16 ... [info] Resolving com.typesafe.akka#akka-
testkit_2.12;2.4.16 ... [info] Resolving jline#jline;2.14.1 ... [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
UNRESOLVED DEPENDENCIES :: [warn] :::::::::::::::::::::::::::::::::::::::::::::: [warn] ::
org.elasticsearch.test#framework;2.2.0: not found [warn]
:::::::::::::::::::::::::::::::::::::::::::::: [warn] [warn] Note:
Unresolved dependencies path: [warn]
org.elasticsearch.test:framework:2.2.0
(/Users/filipemiranda/datawerks/scala-query-engine/build.sbt#L13-30)
[warn] +- default:scala-query-engine_2.12:1.0 [trace] Stack trace
suppressed: run 'last *:update' for the full output. [trace] Stack trace
suppressed: run 'last *:ssExtractDependencies' for the full output.
[error] (*:update) sbt.ResolveException: unresolved dependency:
org.elasticsearch.test#framework;2.2.0: not found [error]
(*:ssExtractDependencies) sbt.ResolveException: unresolved dependency:
org.elasticsearch.test#framework;2.2.0: not found [error] Total time: 6 s,
completed Jan 21, 2017 8:20:44 PM</pre><br/>See complete log in <a
href="/Users/filipemiranda/Library/Logs/IdeaIC2016.1/sbt.last.log">/Users/
filipemiranda/Library/Logs/IdeaIC2016.1/sbt.last.log</a>
I ave looked at: https://oss.sonatype.org/content/repositories/releases/org/elasticsearch/test/framework/
And I did not find the same version there.
Has anyone faced similar issues, how to solve this?
Here is my entire build.sbt
name := "project-name"
version := "1.0"
lazy val scalaV = "2.12.1"
scalaVersion := scalaV
lazy val akkaHttpV = "10.0.1"
lazy val esVersion = "2.2.0"
lazy val luceneVersion = "5.4.1"
lazy val scalatestV = "3.0.1"
libraryDependencies ++= Seq(
"org.scala-lang" % "scala-compiler" % scalaV,
"org.scalatest" %% "scalatest" % scalatestV,
"com.typesafe.akka" %% "akka-http-core" % akkaHttpV,
"com.typesafe.akka" %% "akka-http" % akkaHttpV,
"com.typesafe.akka" %% "akka-stream" % "2.4.16",
"com.storm-enroute" %% "scalameter" % "0.8.2" % "test",
"io.searchbox" % "jest" % "2.0.3",
"org.elasticsearch" % "elasticsearch" % "2.2.0" withSources,
"com.twitter" %% "util-eval" % "6.40.0",
"org.apache.lucene" % "lucene-test-framework" % "5.4.1",
"org.elasticsearch.test" % "framework" % "2.2.0",
"com.storm-enroute" %% "scalameter" % "0.8.2" % "test",
"com.typesafe.akka" %% "akka-http-testkit" % akkaHttpV % "test"
)
How can I work this around???

Unresolved dependency error with SecureSocial

I require Secure Social Plugin [http://securesocial.ws/guide/installation.html ] for my project. When I run the play 2 app I am getting the following error.
I also referred the following post: Why is the Unresolved Dependencies error with SecureSocial and Play 2.3.2?
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: UNRESOLVED DEPENDENCIES ::
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
[warn] :: ws.securesocial#securesocial;2.1.4: not found
[warn] ::::::::::::::::::::::::::::::::::::::::::::::
sbt.ResolveException: unresolved dependency:ws.securesocial#securesocial;2.1.4: not found
at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:217)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:126)
at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:125)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:115)
at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:115)
at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:103)
at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:48)
at sbt.IvySbt$$anon$3.call(Ivy.scala:57)
at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:93)
at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:81)
at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:102)
at xsbt.boot.Using$.withResource(Using.scala:11)
at xsbt.boot.Using$.apply(Using.scala:10)
at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:62)
at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:52)
at xsbt.boot.Locks$.apply0(Locks.scala:31)
at xsbt.boot.Locks$.apply(Locks.scala:28)
at sbt.IvySbt.withDefaultLogger(Ivy.scala:57)
at sbt.IvySbt.withIvy(Ivy.scala:98)
at sbt.IvySbt.withIvy(Ivy.scala:94)
at sbt.IvySbt$Module.withModule(Ivy.scala:115)
at sbt.IvyActions$.update(IvyActions.scala:125)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1223)
at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1221)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$74.apply(Defaults.scala:1244)
at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$74.apply(Defaults.scala:1242)
at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1246)
at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1241)
at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
at sbt.Classpaths$.cachedUpdate(Defaults.scala:1249)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1214)
at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1192)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
at sbt.std.Transform$$anon$4.work(System.scala:64)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
at sbt.Execute.work(Execute.scala:244)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
[error] (*:update) sbt.ResolveException: unresolved dependency: ws.securesocial#securesocial;2.1.4: not found
[error] Total time: 14 s, completed 8 Mar, 2015 9:22:50 AM
Process finished with exit code 1
This is my build.sbt file
name := "testing"
version := "0.1"
lazy val `testing` = (project in file(".")).enablePlugins(PlayJava)
scalaVersion := "2.11.1"
resolvers ++= Seq(
Resolver.sonatypeRepo("snapshots"),
Resolver.url("SecureSocial Repository", url("http://securesocial.ws/repository/snapshots/"))(Resolver.ivyStylePatterns)
)
libraryDependencies ++= Seq( javaJdbc , javaEbean , cache , javaWs,
"ws.securesocial" % "securesocial" % "2.1.4",
"org.mongodb" % "mongo-java-driver" % "2.12.1",
"org.jongo" % "jongo" % "1.1")
unmanagedResourceDirectories in Test <+= baseDirectory ( _ /"target/web/public/test" )
Am I missing something? Can anyone help me out with this?

Changed "ws.securesocial" % "securesocial" % "2.1.4" to "ws.securesocial" % "securesocial_2.11" % "3.0-M3"
name := "testing"
version := "0.1"
lazy val `testing` = (project in file(".")).enablePlugins(PlayJava)
scalaVersion := "2.11.1"
resolvers += (
Resolver.url("SecureSocial Repository", url("http://securesocial.ws/repository/snapshots/"))(Resolver.ivyStylePatterns)
)
libraryDependencies ++= Seq(
"ws.securesocial" % "securesocial_2.11" % "3.0-M3",
"org.mongodb" % "mongo-java-driver" % "2.12.1",
"org.jongo" % "jongo" % "1.1")
unmanagedResourceDirectories in Test <+= baseDirectory ( _ /"target/web/public/test" )
Check this link for the sample implementation in java and scala.
https://github.com/jaliss/securesocial/tree/master/samples