We Keep Coding

How to get data inside a Throwable Scala object?

How I get information inside a Throwable Scala object?
The code is an example about throwable.getMessage.
JsResultException(errors:List((,List(JsonValidationError(List('eoh' is undefined on object: {"store":"8767565","sku":"1983278364782364782"}),WrappedArray())))))
I need to extract JsResultException, JsonValidationError as string, message 'eoh' is undefined on object message and JSON before object:.
Is this for make graceful log.

Consider converting JsResultException.errors which is
Seq[(JsPath, Seq[JsonValidationError])]
where JsonValidationError.errors is yet another sequence Seq[String], into a simpler tuple
Seq[(JsPath, String)]
like so
case JsResultException(errors) =>
errors.map { case (path, validationErrors) => path -> validationErrors.map(_.messages.mkString(",")).mkString(",") }
This would produce a more managable structure similar to
List((/id,error.path.missing), (/name,error.path.missing))
instead of
List((/id,List(JsonValidationError(List(error.path.missing),WrappedArray()))), (/name,List(JsonValidationError(List(error.path.missing),WrappedArray())))))]
Here is a working example
case class User(name: String, id: Int)
object User {
implicit val formats = Json.format[User]
}
val raw = """{ "nam": "mario", "i": 5 }"""
try {
Json.parse(raw).as[User]
} catch {
case JsResultException(errors) =>
errors.map { case (path, validationErrors) => path -> validationErrors.map(_.messages.mkString(",")).mkString(",") }
}
Also consider using validation to avoid throwing exceptions like so
Json.parse(raw).validate[User] match {
case s: JsSuccess[User] => s
case JsError(errors) =>
errors.map { case (path, validationErrors) => path -> validationErrors.map(_.messages.mkString(",")).mkString(",") }
}

You can always use the scala.util.Try and pattern match the Failure.
import scala.util._
def someExceptionThrowingMethod(): T = ???
Try(someExceptionThrowingMethod()) match {
case Success(t: T) => ???
case Failure(exception: Throwable) => exception match {
case JsResultException((_, JsonValidationError(headMessage :: _) :: _, _) :: _) =>
//here headMessage is the 'eoh' is undefined on object: {"store":"8767565","sku":"1983278364782364782"} message you wrote above
case otherException: Throwable => ???
}
}

JUnit5: How to repeat failed test?

One of the practice many companies follow is to repeat unstable test until is passes x times (in a row or in total). If it is executed n times and fail to pass at least x times it is marked as failed.
TestNG supports that with the following annotation:
#Test(invocationCount = 5, successPercentage = 40)
How do I realize similar functionality with JUnit5?
There's similar annotation in JUnit5, called #RepeatedTest(5) but it is not executed conditionally.

Ok, I took a little bit of time to whip together a little example of how to do this using the TestTemplateInvocationContextProvider, ExecutionCondition, and TestExecutionExceptionHandler extension points.
The way I was able to handle failing tests was to mark them as "aborted" rather than let them flat out fail (so that the entire test execution does not consider it a failure) and only fail tests when we can't get the minimum amount of successful runs. If the minimum amount of tests has already succeeded, then we also mark the remaining tests as "disabled". The test failures are tracked in a ExtensionContext.Store so that the state can be looked up at each place.
This is a very rough example that definitely has a few problems but can hopefully serve as an example of how to compose different annotations. I ended up writing it in Kotlin:
#Retry-esque annotation loosely based on the TestNG example:
import org.junit.jupiter.api.TestTemplate
import org.junit.jupiter.api.extension.ExtendWith
#TestTemplate
#Target(AnnotationTarget.FUNCTION)
#ExtendWith(RetryTestExtension::class)
annotation class Retry(val invocationCount: Int, val minSuccess: Int)
TestTemplateInvocationContext used by templatized tests:
import org.junit.jupiter.api.extension.Extension
import org.junit.jupiter.api.extension.TestTemplateInvocationContext
class RetryTemplateContext(
private val invocation: Int,
private val maxInvocations: Int,
private val minSuccess: Int
) : TestTemplateInvocationContext {
override fun getDisplayName(invocationIndex: Int): String {
return "Invocation number $invocationIndex (requires $minSuccess success)"
}
override fun getAdditionalExtensions(): MutableList<Extension> {
return mutableListOf(
RetryingTestExecutionExtension(invocation, maxInvocations, minSuccess)
)
}
}
TestTemplateInvocationContextProvider extension for the #Retry annotation:
import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.ExtensionContextException
import org.junit.jupiter.api.extension.TestTemplateInvocationContext
import org.junit.jupiter.api.extension.TestTemplateInvocationContextProvider
import org.junit.platform.commons.support.AnnotationSupport
import java.util.stream.IntStream
import java.util.stream.Stream
class RetryTestExtension : TestTemplateInvocationContextProvider {
override fun supportsTestTemplate(context: ExtensionContext): Boolean {
return context.testMethod.map { it.isAnnotationPresent(Retry::class.java) }.orElse(false)
}
override fun provideTestTemplateInvocationContexts(context: ExtensionContext): Stream<TestTemplateInvocationContext> {
val annotation = AnnotationSupport.findAnnotation(
context.testMethod.orElseThrow { ExtensionContextException("Must be annotated on method") },
Retry::class.java
).orElseThrow { ExtensionContextException("${Retry::class.java} not found on method") }
checkValidRetry(annotation)
return IntStream.rangeClosed(1, annotation.invocationCount)
.mapToObj { RetryTemplateContext(it, annotation.invocationCount, annotation.minSuccess) }
}
private fun checkValidRetry(annotation: Retry) {
if (annotation.invocationCount < 1) {
throw ExtensionContextException("${annotation.invocationCount} must be greater than or equal to 1")
}
if (annotation.minSuccess < 1 || annotation.minSuccess > annotation.invocationCount) {
throw ExtensionContextException("Invalid ${annotation.minSuccess}")
}
}
}
Simple data class representing the retry (injected into test cases in this example using ParameterResolver).
data class RetryInfo(val invocation: Int, val maxInvocations: Int)
Exception used for representing failed retries:
import java.lang.Exception
internal class RetryingTestFailure(invocation: Int, cause: Throwable) : Exception("Failed test execution at invocation #$invocation", cause)
Main extension implementing ExecutionCondition, ParameterResolver, and TestExecutionExceptionHandler.
import org.junit.jupiter.api.extension.ConditionEvaluationResult
import org.junit.jupiter.api.extension.ExecutionCondition
import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.ParameterContext
import org.junit.jupiter.api.extension.ParameterResolver
import org.junit.jupiter.api.extension.TestExecutionExceptionHandler
import org.opentest4j.TestAbortedException
internal class RetryingTestExecutionExtension(
private val invocation: Int,
private val maxInvocations: Int,
private val minSuccess: Int
) : ExecutionCondition, ParameterResolver, TestExecutionExceptionHandler {
override fun evaluateExecutionCondition(
context: ExtensionContext
): ConditionEvaluationResult {
val failureCount = getFailures(context).size
// Shift -1 because this happens before test
val successCount = (invocation - 1) - failureCount
when {
(maxInvocations - failureCount) < minSuccess -> // Case when we cannot hit our minimum success
return ConditionEvaluationResult.disabled("Cannot hit minimum success rate of $minSuccess/$maxInvocations - $failureCount failures already")
successCount < minSuccess -> // Case when we haven't hit success threshold yet
return ConditionEvaluationResult.enabled("Have not ran $minSuccess/$maxInvocations successful executions")
else -> return ConditionEvaluationResult.disabled("$minSuccess/$maxInvocations successful runs have already ran. Skipping run $invocation")
}
}
override fun supportsParameter(
parameterContext: ParameterContext,
extensionContext: ExtensionContext
): Boolean = parameterContext.parameter.type == RetryInfo::class.java
override fun resolveParameter(
parameterContext: ParameterContext,
extensionContext: ExtensionContext
): Any = RetryInfo(invocation, maxInvocations)
override fun handleTestExecutionException(
context: ExtensionContext,
throwable: Throwable
) {
val testFailure = RetryingTestFailure(invocation, throwable)
val failures: MutableList<RetryingTestFailure> = getFailures(context)
failures.add(testFailure)
val failureCount = failures.size
val successCount = invocation - failureCount
if ((maxInvocations - failureCount) < minSuccess) {
throw testFailure
} else if (successCount < minSuccess) {
// Case when we have still have retries left
throw TestAbortedException("Aborting test #$invocation/$maxInvocations- still have retries left",
testFailure)
}
}
private fun getFailures(context: ExtensionContext): MutableList<RetryingTestFailure> {
val namespace = ExtensionContext.Namespace.create(
RetryingTestExecutionExtension::class.java)
val store = context.parent.get().getStore(namespace)
#Suppress("UNCHECKED_CAST")
return store.getOrComputeIfAbsent(context.requiredTestMethod.name, { mutableListOf<RetryingTestFailure>() }, MutableList::class.java) as MutableList<RetryingTestFailure>
}
}
And then, the test consumer:
import org.junit.jupiter.api.DisplayName
internal class MyRetryableTest {
#DisplayName("Fail all retries")
#Retry(invocationCount = 5, minSuccess = 3)
internal fun failAllRetries(retryInfo: RetryInfo) {
println(retryInfo)
throw Exception("Failed at $retryInfo")
}
#DisplayName("Only fail once")
#Retry(invocationCount = 5, minSuccess = 4)
internal fun succeedOnRetry(retryInfo: RetryInfo) {
if (retryInfo.invocation == 1) {
throw Exception("Failed at ${retryInfo.invocation}")
}
}
#DisplayName("Only requires single success and is first execution")
#Retry(invocationCount = 5, minSuccess = 1)
internal fun firstSuccess(retryInfo: RetryInfo) {
println("Running: $retryInfo")
}
#DisplayName("Only requires single success and is last execution")
#Retry(invocationCount = 5, minSuccess = 1)
internal fun lastSuccess(retryInfo: RetryInfo) {
if (retryInfo.invocation < 5) {
throw Exception("Failed at ${retryInfo.invocation}")
}
}
#DisplayName("All required all succeed")
#Retry(invocationCount = 5, minSuccess = 5)
internal fun allRequiredAllSucceed(retryInfo: RetryInfo) {
println("Running: $retryInfo")
}
#DisplayName("Fail early and disable")
#Retry(invocationCount = 5, minSuccess = 4)
internal fun failEarly(retryInfo: RetryInfo) {
throw Exception("Failed at ${retryInfo.invocation}")
}
}
And the test output in IntelliJ looks like:
I don't know if throwing a TestAbortedException from the TestExecutionExceptionHandler.handleTestExecutionException is supposed to abort the test, but I am using it here.

U can try this extension for junit 5.
<dependency>
<groupId>io.github.artsok</groupId>
<artifactId>rerunner-jupiter</artifactId>
<version>LATEST</version>
</dependency>
Examples:
/**
* Repeated three times if test failed.
* By default Exception.class will be handled in test
*/
#RepeatedIfExceptionsTest(repeats = 3)
void reRunTest() throws IOException {
throw new IOException("Error in Test");
}
/**
* Repeated two times if test failed. Set IOException.class that will be handled in test
* #throws IOException - error occurred
*/
#RepeatedIfExceptionsTest(repeats = 2, exceptions = IOException.class)
void reRunTest2() throws IOException {
throw new IOException("Exception in I/O operation");
}
/**
* Repeated ten times if test failed. Set IOException.class that will be handled in test
* Set formatter for test. Like behavior as at {#link org.junit.jupiter.api.RepeatedTest}
* #throws IOException - error occurred
*/
#RepeatedIfExceptionsTest(repeats = 10, exceptions = IOException.class,
name = "Rerun failed test. Attempt {currentRepetition} of {totalRepetitions}")
void reRunTest3() throws IOException {
throw new IOException("Exception in I/O operation");
}
/**
* Repeated 100 times with minimum success four times, then disabled all remaining repeats.
* See image below how it works. Default exception is Exception.class
*/
#DisplayName("Test Case Name")
#RepeatedIfExceptionsTest(repeats = 100, minSuccess = 4)
void reRunTest4() {
if(random.nextInt() % 2 == 0) {
throw new RuntimeException("Error in Test");
}
}
View at IDEA:
With minimum success four times then disables all other:
You can also mix #RepeatedIfExceptionsTest with #DisplayName
source -> github

if you are running tests via Maven, with Surefire you care re-run failing tests automatically by using rerunFailingTestsCount.
However, as of 2.21.0, that does not work for JUnit 5 (only 4.x). But hopefully it will be supported in the next releases.

If you happen to be running your tests using the Gradle build tool, you can use the Test Retry Gradle plugin. This will rerun each failed test a certain number of times, with the option of failing the build if too many failures have occurred overall.
plugins {
id 'org.gradle.test-retry' version '1.2.0'
}
test {
retry {
maxRetries = 3
maxFailures = 20 // Optional attribute
}
}

Parsing ASCII communication with Hardware using Java/Scala?

I am writing software that will interface external device via exchange of ASCII commands. Example:
POS? 1 2
=>1=-1158.4405
=>2=+0000.0000
where above we ask for position of motorised microscope stage for the 1-st and 2-nd axes. It responds with positions in um. More details and examples.
My Question: is there a library that would ease parsing such string outputs, and/or would help generate queries?
Otherwise, what are the best practises for parsing and communicating with hardware using Java/Scala?

Trying to cope with following syntax (see 12.1 Format):
Reply syntax:
[<argument>[{SP<argument>}]"="]<value>LF
Multi-line reply syntax:
{[<argument>[{SP<argument>}]"="]<value>SP LF}
[<argument>[{SP<argument>}]"="]<value>LF for the last line!
This is my code:
import scala.util.parsing.combinator._
case class Result(argument: String, value: Float)
class ReplyParser extends RegexParsers{
override def skipWhitespace = false
private def floatingPointNumber: Parser[String] =
"""(-|\+)?(\d+(\.\d*)?|\d*\.\d+)""".r
private def value: Parser[Float] = floatingPointNumber ^^ (_.toFloat)
private def argument: Parser[String] = "[^= \n]+".r
private def arguments: Parser[List[String]] = rep1sep(argument," ") <~ "="
private def result: Parser[List[Result]] = arguments.? ~ value ^^ {
case arguments ~ value =>
arguments.getOrElse(List("")).map {
Result(_,value)
}
}
def reply: Parser[List[Result]] = rep1sep(result, " \n".r) <~ " " ^^ {
case result => result.flatten
}
}
object Parsing extends ReplyParser {
def main(args: Array[String]) {
val result = parseAll(reply,"a=+000.123 \nc d=-999.567 \n789 ")
println(s"$result")
}
}

JSON4s can't find constructor w/spark

I've run into an issue with attempting to parse json in my spark job. I'm using spark 1.1.0, json4s, and the Cassandra Spark Connector, with DSE 4.6. The exception thrown is:
org.json4s.package$MappingException: Can't find constructor for BrowserData org.json4s.reflect.ScalaSigReader$.readConstructor(ScalaSigReader.scala:27)
org.json4s.reflect.Reflector$ClassDescriptorBuilder.ctorParamType(Reflector.scala:108)
org.json4s.reflect.Reflector$ClassDescriptorBuilder$$anonfun$6.apply(Reflector.scala:98)
org.json4s.reflect.Reflector$ClassDescriptorBuilder$$anonfun$6.apply(Reflector.scala:95)
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
My code looks like this:
case class BrowserData(navigatorObjectData: Option[NavigatorObjectData],
flash_version: Option[FlashVersion],
viewport: Option[Viewport],
performanceData: Option[PerformanceData])
.... other case classes
def parseJson(b: Option[String]): Option[String] = {
implicit val formats = DefaultFormats
for {
browserDataStr <- b
browserData = parse(browserDataStr).extract[BrowserData]
navObject <- browserData.navigatorObjectData
userAgent <- navObject.userAgent
} yield (userAgent)
}
def getJavascriptUa(rows: Iterable[com.datastax.spark.connector.CassandraRow]): Option[String] = {
implicit val formats = DefaultFormats
rows.collectFirst { case r if r.getStringOption("browser_data").isDefined =>
parseJson(r.getStringOption("browser_data"))
}.flatten
}
def getRequestUa(rows: Iterable[com.datastax.spark.connector.CassandraRow]): Option[String] = {
rows.collectFirst { case r if r.getStringOption("ua").isDefined =>
r.getStringOption("ua")
}.flatten
}
def checkUa(rows: Iterable[com.datastax.spark.connector.CassandraRow], sessionId: String): Option[Boolean] = {
for {
jsUa <- getJavascriptUa(rows)
reqUa <- getRequestUa(rows)
} yield (jsUa == reqUa)
}
def run(name: String) = {
val rdd = sc.cassandraTable("beehive", name).groupBy(r => r.getString("session_id"))
val counts = rdd.map(r => (checkUa(r._2, r._1)))
counts
}
I use :load to load the file into the REPL, and then call the run function. The failure is happening in the parseJson function, as far as I can tell. I've tried a variety of things to try to get this to work. From similar posts, I've made sure my case classes are in the top level in the file. I've tried compiling just the case class definitions into a jar, and including the jar in like this: /usr/bin/dse spark --jars case_classes.jar
I've tried adding them to the conf like this: sc.getConf.setJars(Seq("/home/ubuntu/case_classes.jar"))
And still the same error. Should I compile all of my code into a jar? Is this a spark issue or a JSON4s issue? Any help at all appreciated.

Is there a good library to embed a command prompt in a scala (or java) application

I have an application that I'd like to have a prompt in. If it helps, this is a graph database implementation and I need a prompt just like any other database client (MySQL, Postgresql, etc.).
So far I have my own REPL like so:
object App extends Application {
REPL ! Read
}
object REPL extends Actor {
def act() {
loop {
react {
case Read => {
print("prompt> ")
var message = Console.readLine
this ! Eval(message)
}
case More(sofar) => {
//Eval didn't see a semicolon
print(" --> ")
var message = Console.readLine
this ! Eval(sofar + " " + message)
}
case Eval(message) => {
Evaluator ! Eval(message)
}
case Print(message) => {
println(message)
//And here's the loop
this ! Read
}
case Exit => {
exit()
}
case _ => {
println("App: How did we get here")
}
}
}
}
this.start
}
It works, but I would really like to have something with history. Tab completion is not necessary.
Any suggestions on a good library? Scala or Java works.
Just to be clear I don't need an REPL to evaluate my code (I get that with scala!), nor am I looking to call or use something from the command line. I'm looking for a prompt that is my user experience when my client app starts up.

Scala itself, and lots of programs out there, uses a readline-like library for its REPL. Specifically, JLine.
I found another question about this, for which the answers don't seem promising.

BeanShell does some of what you want: http://www.beanshell.org/

I got it. these two blogs really helped.
http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html
http://danielwestheide.com/blog/2013/01/16/the-neophytes-guide-to-scala-part-9-promises-and-futures-in-practice.html
def interprete(code: String) : Future[String] = {
val p = Promise[String]()
Future {
var result = reader.readLine()
p.success(result)
}
writer.write(code + "\n")
writer.flush()
p.future
}
for (ln <- io.Source.stdin.getLines){
val f = interprete(ln)
f.onComplete {
case Success(s) =>
println("future returned: " + s)
case Failure(ex) =>
println(s"interpreter failed due to ${ex.getMessage}")
}
}