Im using https://github.com/AsyncHttpClient/async-http-client this library in my scala project, and im performing some http calls with it, but now on some http calls I need to retry a call if I dont get my expected result for 3 times.
How should I implement something like this?
thaknks
This is an example of retry function based on Future.recoverWith
If you run it you can see that it prints "run process" until the Future is successful but not more than 'times' times
object X extends App{
type Request = String
type Response = String
import scala.concurrent.ExecutionContext.Implicits.global
def retry(request: Request, process: Request => Future[Response], times: Int): Future[Response] ={
val responseF = process(request)
if(times > 0)
responseF.recoverWith{
case ex => println("fail")
retry(request, process, times - 1)
}
else
responseF
}
def process(s: Request): Future[Response] = {
println("run process")
if(Random.nextBoolean()) Future.successful("good") else Future.failed(new Exception)
}
val result = retry("", process, 3)
import scala.concurrent.duration._
println(Await.result(result, 1.second))
}
Related
We created a rest API on python and it is locally running. And the 'http://127.0.0.1:5002/business' API is showing contents {"business name": "something"} if I open it on google chrome. However, when we call this API in nodejs, it always gives me the error. But if I use another API(exactly same code but different api in nodejs), it is working.
async function get_recommend_initial(){
//https://ViolaS.api.stdlib.com/InitialRecommendation#dev/
// // agent.add('providing recommendations...');
const options = {
method: 'GET'
,uri: 'http://127.0.0.1:5002/business'
// ,uri:'https://ViolaS.api.stdlib.com/InitialRecommendation#dev/'
// ,json: true
};
// return request(options).then(response => {
// console.log(response)
// return (response)
// }).catch(function (err) {
// console.log('No recommend data');
// console.log(err);
// });
return requestAPI(options).then(function(data)
{
let initial_recommendation = JSON.parse(data);
console.log(initial_recommendation);
//return initial_recommendation.information[0].name;
}).catch(function (err) {
console.log('No recommend data');
console.log(err);
});
}
1
The API that is created by python file which is running locally. You can see the API code figure by moving your mouth above 1. Thanks!!!
The python code is as follows:
app = Flask(__name__)
#Add resources to be much cleaner
api = Api(app)
features = {}
class Business(Resource):
def get(self):
return {'business name': 'something'} # Fetches first column that is Employee ID
def post(self):
some_json = request.get_json()
print(some_json)
countNumber = features.get('count',0) + 1
features['count'] = countNumber
return {'You sent': some_json,
'Count:':countNumber}, 201
def put(self):
some_json = request.get_json()
print(some_json)
#record the count number
countNumber = features.get('count',0) + 1
features['count'] = countNumber
features['ok'] = 'yes'
return {'You sent': some_json,
'Count:':countNumber,
'Ok:': features['ok']}, 201
api.add_resource(Business, '/business') # Route_1
if __name__ == '__main__':
app.run(port='5002')
The error is as following:
dialogflowFirebaseFulfillment
Error: Unknown response type: "undefined" at WebhookClient.addResponse_ (/srv/node_modules/dialogflow-fulfillment/src/dialogflow-fulfillment.js:277:13) at WebhookClient.add (/srv/node_modules/dialogflow-fulfillment/src/dialogflow-fulfillment.js:245:12) at Sys_Recommend (/srv/index.js:31:11) at <anonymous>
And the log is:
No recommend data
One of the practice many companies follow is to repeat unstable test until is passes x times (in a row or in total). If it is executed n times and fail to pass at least x times it is marked as failed.
TestNG supports that with the following annotation:
#Test(invocationCount = 5, successPercentage = 40)
How do I realize similar functionality with JUnit5?
There's similar annotation in JUnit5, called #RepeatedTest(5) but it is not executed conditionally.
Ok, I took a little bit of time to whip together a little example of how to do this using the TestTemplateInvocationContextProvider, ExecutionCondition, and TestExecutionExceptionHandler extension points.
The way I was able to handle failing tests was to mark them as "aborted" rather than let them flat out fail (so that the entire test execution does not consider it a failure) and only fail tests when we can't get the minimum amount of successful runs. If the minimum amount of tests has already succeeded, then we also mark the remaining tests as "disabled". The test failures are tracked in a ExtensionContext.Store so that the state can be looked up at each place.
This is a very rough example that definitely has a few problems but can hopefully serve as an example of how to compose different annotations. I ended up writing it in Kotlin:
#Retry-esque annotation loosely based on the TestNG example:
import org.junit.jupiter.api.TestTemplate
import org.junit.jupiter.api.extension.ExtendWith
#TestTemplate
#Target(AnnotationTarget.FUNCTION)
#ExtendWith(RetryTestExtension::class)
annotation class Retry(val invocationCount: Int, val minSuccess: Int)
TestTemplateInvocationContext used by templatized tests:
import org.junit.jupiter.api.extension.Extension
import org.junit.jupiter.api.extension.TestTemplateInvocationContext
class RetryTemplateContext(
private val invocation: Int,
private val maxInvocations: Int,
private val minSuccess: Int
) : TestTemplateInvocationContext {
override fun getDisplayName(invocationIndex: Int): String {
return "Invocation number $invocationIndex (requires $minSuccess success)"
}
override fun getAdditionalExtensions(): MutableList<Extension> {
return mutableListOf(
RetryingTestExecutionExtension(invocation, maxInvocations, minSuccess)
)
}
}
TestTemplateInvocationContextProvider extension for the #Retry annotation:
import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.ExtensionContextException
import org.junit.jupiter.api.extension.TestTemplateInvocationContext
import org.junit.jupiter.api.extension.TestTemplateInvocationContextProvider
import org.junit.platform.commons.support.AnnotationSupport
import java.util.stream.IntStream
import java.util.stream.Stream
class RetryTestExtension : TestTemplateInvocationContextProvider {
override fun supportsTestTemplate(context: ExtensionContext): Boolean {
return context.testMethod.map { it.isAnnotationPresent(Retry::class.java) }.orElse(false)
}
override fun provideTestTemplateInvocationContexts(context: ExtensionContext): Stream<TestTemplateInvocationContext> {
val annotation = AnnotationSupport.findAnnotation(
context.testMethod.orElseThrow { ExtensionContextException("Must be annotated on method") },
Retry::class.java
).orElseThrow { ExtensionContextException("${Retry::class.java} not found on method") }
checkValidRetry(annotation)
return IntStream.rangeClosed(1, annotation.invocationCount)
.mapToObj { RetryTemplateContext(it, annotation.invocationCount, annotation.minSuccess) }
}
private fun checkValidRetry(annotation: Retry) {
if (annotation.invocationCount < 1) {
throw ExtensionContextException("${annotation.invocationCount} must be greater than or equal to 1")
}
if (annotation.minSuccess < 1 || annotation.minSuccess > annotation.invocationCount) {
throw ExtensionContextException("Invalid ${annotation.minSuccess}")
}
}
}
Simple data class representing the retry (injected into test cases in this example using ParameterResolver).
data class RetryInfo(val invocation: Int, val maxInvocations: Int)
Exception used for representing failed retries:
import java.lang.Exception
internal class RetryingTestFailure(invocation: Int, cause: Throwable) : Exception("Failed test execution at invocation #$invocation", cause)
Main extension implementing ExecutionCondition, ParameterResolver, and TestExecutionExceptionHandler.
import org.junit.jupiter.api.extension.ConditionEvaluationResult
import org.junit.jupiter.api.extension.ExecutionCondition
import org.junit.jupiter.api.extension.ExtensionContext
import org.junit.jupiter.api.extension.ParameterContext
import org.junit.jupiter.api.extension.ParameterResolver
import org.junit.jupiter.api.extension.TestExecutionExceptionHandler
import org.opentest4j.TestAbortedException
internal class RetryingTestExecutionExtension(
private val invocation: Int,
private val maxInvocations: Int,
private val minSuccess: Int
) : ExecutionCondition, ParameterResolver, TestExecutionExceptionHandler {
override fun evaluateExecutionCondition(
context: ExtensionContext
): ConditionEvaluationResult {
val failureCount = getFailures(context).size
// Shift -1 because this happens before test
val successCount = (invocation - 1) - failureCount
when {
(maxInvocations - failureCount) < minSuccess -> // Case when we cannot hit our minimum success
return ConditionEvaluationResult.disabled("Cannot hit minimum success rate of $minSuccess/$maxInvocations - $failureCount failures already")
successCount < minSuccess -> // Case when we haven't hit success threshold yet
return ConditionEvaluationResult.enabled("Have not ran $minSuccess/$maxInvocations successful executions")
else -> return ConditionEvaluationResult.disabled("$minSuccess/$maxInvocations successful runs have already ran. Skipping run $invocation")
}
}
override fun supportsParameter(
parameterContext: ParameterContext,
extensionContext: ExtensionContext
): Boolean = parameterContext.parameter.type == RetryInfo::class.java
override fun resolveParameter(
parameterContext: ParameterContext,
extensionContext: ExtensionContext
): Any = RetryInfo(invocation, maxInvocations)
override fun handleTestExecutionException(
context: ExtensionContext,
throwable: Throwable
) {
val testFailure = RetryingTestFailure(invocation, throwable)
val failures: MutableList<RetryingTestFailure> = getFailures(context)
failures.add(testFailure)
val failureCount = failures.size
val successCount = invocation - failureCount
if ((maxInvocations - failureCount) < minSuccess) {
throw testFailure
} else if (successCount < minSuccess) {
// Case when we have still have retries left
throw TestAbortedException("Aborting test #$invocation/$maxInvocations- still have retries left",
testFailure)
}
}
private fun getFailures(context: ExtensionContext): MutableList<RetryingTestFailure> {
val namespace = ExtensionContext.Namespace.create(
RetryingTestExecutionExtension::class.java)
val store = context.parent.get().getStore(namespace)
#Suppress("UNCHECKED_CAST")
return store.getOrComputeIfAbsent(context.requiredTestMethod.name, { mutableListOf<RetryingTestFailure>() }, MutableList::class.java) as MutableList<RetryingTestFailure>
}
}
And then, the test consumer:
import org.junit.jupiter.api.DisplayName
internal class MyRetryableTest {
#DisplayName("Fail all retries")
#Retry(invocationCount = 5, minSuccess = 3)
internal fun failAllRetries(retryInfo: RetryInfo) {
println(retryInfo)
throw Exception("Failed at $retryInfo")
}
#DisplayName("Only fail once")
#Retry(invocationCount = 5, minSuccess = 4)
internal fun succeedOnRetry(retryInfo: RetryInfo) {
if (retryInfo.invocation == 1) {
throw Exception("Failed at ${retryInfo.invocation}")
}
}
#DisplayName("Only requires single success and is first execution")
#Retry(invocationCount = 5, minSuccess = 1)
internal fun firstSuccess(retryInfo: RetryInfo) {
println("Running: $retryInfo")
}
#DisplayName("Only requires single success and is last execution")
#Retry(invocationCount = 5, minSuccess = 1)
internal fun lastSuccess(retryInfo: RetryInfo) {
if (retryInfo.invocation < 5) {
throw Exception("Failed at ${retryInfo.invocation}")
}
}
#DisplayName("All required all succeed")
#Retry(invocationCount = 5, minSuccess = 5)
internal fun allRequiredAllSucceed(retryInfo: RetryInfo) {
println("Running: $retryInfo")
}
#DisplayName("Fail early and disable")
#Retry(invocationCount = 5, minSuccess = 4)
internal fun failEarly(retryInfo: RetryInfo) {
throw Exception("Failed at ${retryInfo.invocation}")
}
}
And the test output in IntelliJ looks like:
I don't know if throwing a TestAbortedException from the TestExecutionExceptionHandler.handleTestExecutionException is supposed to abort the test, but I am using it here.
U can try this extension for junit 5.
<dependency>
<groupId>io.github.artsok</groupId>
<artifactId>rerunner-jupiter</artifactId>
<version>LATEST</version>
</dependency>
Examples:
/**
* Repeated three times if test failed.
* By default Exception.class will be handled in test
*/
#RepeatedIfExceptionsTest(repeats = 3)
void reRunTest() throws IOException {
throw new IOException("Error in Test");
}
/**
* Repeated two times if test failed. Set IOException.class that will be handled in test
* #throws IOException - error occurred
*/
#RepeatedIfExceptionsTest(repeats = 2, exceptions = IOException.class)
void reRunTest2() throws IOException {
throw new IOException("Exception in I/O operation");
}
/**
* Repeated ten times if test failed. Set IOException.class that will be handled in test
* Set formatter for test. Like behavior as at {#link org.junit.jupiter.api.RepeatedTest}
* #throws IOException - error occurred
*/
#RepeatedIfExceptionsTest(repeats = 10, exceptions = IOException.class,
name = "Rerun failed test. Attempt {currentRepetition} of {totalRepetitions}")
void reRunTest3() throws IOException {
throw new IOException("Exception in I/O operation");
}
/**
* Repeated 100 times with minimum success four times, then disabled all remaining repeats.
* See image below how it works. Default exception is Exception.class
*/
#DisplayName("Test Case Name")
#RepeatedIfExceptionsTest(repeats = 100, minSuccess = 4)
void reRunTest4() {
if(random.nextInt() % 2 == 0) {
throw new RuntimeException("Error in Test");
}
}
View at IDEA:
With minimum success four times then disables all other:
You can also mix #RepeatedIfExceptionsTest with #DisplayName
source -> github
if you are running tests via Maven, with Surefire you care re-run failing tests automatically by using rerunFailingTestsCount.
However, as of 2.21.0, that does not work for JUnit 5 (only 4.x). But hopefully it will be supported in the next releases.
If you happen to be running your tests using the Gradle build tool, you can use the Test Retry Gradle plugin. This will rerun each failed test a certain number of times, with the option of failing the build if too many failures have occurred overall.
plugins {
id 'org.gradle.test-retry' version '1.2.0'
}
test {
retry {
maxRetries = 3
maxFailures = 20 // Optional attribute
}
}
I'm having a spring boot application which is calling iteratively a mockserver instance via a hystrix command, with a fallback method.
The mockserver is configured to allways respond with status code 500. When running without having circuitBreaker.sleepWindowInMilliseconds, everything works fine, the call is done to the mockserver and then the fallback method is invoked.
After configuring circuitBreaker.sleepWindowInMilliseconds value to 5 minutes or so, I would expect that during 5 minutes no calls are done to the mockserver all the calls being directed to the fallback method, but that's not the case.
It looks like the circuitBreaker.sleepWindowInMilliseconds configuration is ignored.
For instance if I reconfigure the mockservice to reply with status code 200 while the iteration is still running, it would imediately print the "mockservice response",without waiting 5 minutes.
in the spring boot main application class:
#RequestMapping("/iterate")
public void iterate() {
for (int i = 1; i<100; i++ ) {
try {
System.out.println(bookService.readingMockService());
Thread.sleep(3000);
} catch (Exception e) {
System.out.println(e.getMessage());
}
}
}
in the spring boot service :
#HystrixCommand(groupKey = "ReadingMockService", commandKey = "ReadingMockService", threadPoolKey = "ReadingMockService", fallbackMethod = "reliableMock", commandProperties = {
#HystrixProperty(name ="circuitBreaker.sleepWindowInMilliseconds", value = "300000") })
public String readingMockService() {
URI uri = URI.create("http://localhost:1080/askmock");
return this.restTemplate.getForObject(uri, String.class);
}
also the mock server is running on the same machine, being configured like :
new MockServerClient("127.0.0.1", 1080).reset();
new MockServerClient("127.0.0.1", 1080)
.when(request("/askmock"))
.respond(response()
.withStatusCode(500)
.withBody("mockservice response")
.applyDelay());
Found the problem :
This property (...circuitBreaker.sleepWindowInMilliseconds ) works together with another one (...circuitBreaker.requestVolumeThreshold ).
If not specifically set this defaults to 20, meaning that first hystrix will try to connect 20 times the usual way and only afterwards the sleepWindowInMilliseconds will get activated and will go to fallback only.
Also the circuit break opens only if the percentage of failed calls exceeds circuitBreaker.errorThresholdPercentage
and in the same time the total number of failed calls exceeds circuitBreaker.requestVolumeThreshold, all within a window of metrics.rollingStats.timeInMilliseconds
From the docs:
https://github.com/Netflix/Hystrix/wiki/configuration#circuitBreaker.sleepWindowInMilliseconds
and by looking at the source code:
https://github.com/Netflix/Hystrix/blob/master/hystrix-core/src/main/java/com/netflix/hystrix/HystrixCommandProperties.java
Using
#HystrixProperty(name="hystrix.command.ReadingMockService.circuitBreaker.sleepWindowInMilliseconds"
should work.
I want to do some parallel computation and I'm getting a really strange java.lang.NullPointerException on calling ANY functions outside the object I have.
Take a look:
case class Return(session: String, job: Int)
case class Ready(n: Int)
case class DoJob(session: String, job: Int)
case class NotReady
object Notifications extends Controller with Secure {
class AtorMeio extends Actor {
import scala.collection.mutable.{Map => MMap}
val job: MMap[(String, Int), Option[Int]] = MMap()
def act {
loop {
react {
case DoJob(session, jobn) =>
if (job.get((session, jobn)).isEmpty) {
jobn match {
case 1 =>
job.put((session, jobn), None)
val n = Messaging.oi //Messaging.retrieveNumberOfMessages(new FlagTerm(new Flags(Flags.Flag.SEEN), false))
job.put((session, jobn), Some(n))
case 2 =>
// do!
}
}
case Return(session, jobn) =>
if (job.get((session, jobn)).isDefined && job.get((session, jobn)).get.isDefined) {
val ret = job.get((session, jobn)).get.get
job.remove((session, jobn))
reply(Ready(ret))
}
else
reply(NotReady)
}
}
}
}
private var meuator: AtorMeio = null
lazy val ator = {
if (Option(meuator).isEmpty) {
meuator = new AtorMeio
meuator.start
}
meuator
}
def pendingNotifications = {
ator ! DoJob(session.getId, 1)
ator !? Return(session.getId, 1) match {
case Ready(ret) =>
if (ret.toString != Option[String](params.get("current")).getOrElse("-1")) "true" else Suspend("80s")
case _ =>
}
}
}
I'm getting an error in executing Messaging.oi which is basically an object with:
def oi = 4
Here is the stacktrace:
controllers.Notifications$AtorMeio#1889d53: caught java.lang.NullPointerException
java.lang.NullPointerException
at controllers.Messaging$.oi(Messaging.scala:108)
at controllers.Notifications$AtorMeio$$anonfun$act$1$$anonfun$apply$1.apply(Notifications.scala:38)
at controllers.Notifications$AtorMeio$$anonfun$act$1$$anonfun$apply$1.apply(Notifications.scala:31) at scala.actors.ReactorTask.run(ReactorTask.scala:34)
at scala.actors.ReactorTask.compute(ReactorTask.scala:66)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:147)
at scala.concurrent.forkjoin.ForkJoinTask.quietlyExec(ForkJoinTask.java:422)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.mainLoop(ForkJoinWorkerThread.java:340)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:325)
Line 108 is exactly this oneliner def. Ahh entrance point is def pendingNotifications.
Anyone can help? Thanks a lot!
Have you tried replacing
private var meuator: AtorMeio = null
by either:
private var meuator: AtorMeio = None
Configure your breakpoints view in your debugger to halt/break on NullPointerExceptions ...
And: you did see you set this to null here:
private var meuator: AtorMeio = null
?? Or?
Ok people, after digging a lot I discovered the problem: Somehow, somewhere if you have the "Controller" class from play framework mixed in, it crashes mercifully. So I just wrapped this thing into a 'clean' class and it worked.
I have an application that I'd like to have a prompt in. If it helps, this is a graph database implementation and I need a prompt just like any other database client (MySQL, Postgresql, etc.).
So far I have my own REPL like so:
object App extends Application {
REPL ! Read
}
object REPL extends Actor {
def act() {
loop {
react {
case Read => {
print("prompt> ")
var message = Console.readLine
this ! Eval(message)
}
case More(sofar) => {
//Eval didn't see a semicolon
print(" --> ")
var message = Console.readLine
this ! Eval(sofar + " " + message)
}
case Eval(message) => {
Evaluator ! Eval(message)
}
case Print(message) => {
println(message)
//And here's the loop
this ! Read
}
case Exit => {
exit()
}
case _ => {
println("App: How did we get here")
}
}
}
}
this.start
}
It works, but I would really like to have something with history. Tab completion is not necessary.
Any suggestions on a good library? Scala or Java works.
Just to be clear I don't need an REPL to evaluate my code (I get that with scala!), nor am I looking to call or use something from the command line. I'm looking for a prompt that is my user experience when my client app starts up.
Scala itself, and lots of programs out there, uses a readline-like library for its REPL. Specifically, JLine.
I found another question about this, for which the answers don't seem promising.
BeanShell does some of what you want: http://www.beanshell.org/
I got it. these two blogs really helped.
http://danielwestheide.com/blog/2013/01/09/the-neophytes-guide-to-scala-part-8-welcome-to-the-future.html
http://danielwestheide.com/blog/2013/01/16/the-neophytes-guide-to-scala-part-9-promises-and-futures-in-practice.html
def interprete(code: String) : Future[String] = {
val p = Promise[String]()
Future {
var result = reader.readLine()
p.success(result)
}
writer.write(code + "\n")
writer.flush()
p.future
}
for (ln <- io.Source.stdin.getLines){
val f = interprete(ln)
f.onComplete {
case Success(s) =>
println("future returned: " + s)
case Failure(ex) =>
println(s"interpreter failed due to ${ex.getMessage}")
}
}