We Keep Coding

How to prevent IllegalStateException in reactor when you need to block

We have a synchronous process that needs to call two REST endpoints, whereas the result of the first is needed for the second. Using Springs WebClient the .block() causes the following exception:
java.lang.IllegalStateException: block()/blockFirst()/blockLast() are blocking, which is not supported in thread parallel-2
How can this be prevented?
Here is a simplified code snippet:
var job = webClient.createJob().block();
if (job == null || StringUtils.isBlank(job.getId())) {
throw new Exception("WebClient did not return with a job id");
}
batchRecords(job.getId(), records);// does some additional calls to the webClient
This works in the unit test, but when called through a #RestController the above exception is thrown.
EDIT:
The batchRecords method currently also has blocking Monos in it, so we can have a delay in between:
public void batchRecords(final String jobId, final List<InventoryRecord> records)
var recordCount = 0;
var inventoryPositions = new ArrayList<InventoryPosition>();
var recordIterator = records.iterator();
while (recordIterator != null && recordIterator.hasNext()) {
var inventoryRecord = recordIterator.next();
inventoryPositions.add(mapInventoryPosition(inventoryRecord));
recordCount++;
if (inventoryPositions.size() == batchSize) {
var response = createBatch(jobId, inventoryPositions);
Thread.sleep(sleepTime);
response.block();
inventoryPositions = new ArrayList<>();
}
}
}

You should do it reactively without blocking:
webClient.createJob()
.filter(job -> !StringUtils.isBlank(job.getId()))
.flatMap(job -> batchRecords(job.getId(), records))
.switchIfEmpty(Mono.error(new Exception("WebClient did not return with a job id")));
As soon as the createJob operation is finished, the result is filtered and provided to the flatMap operator. In case of an empty response (Mono.empty()) an exception is thrown.

How to implement simple retry using AsyncHttpClient and scala

Im using https://github.com/AsyncHttpClient/async-http-client this library in my scala project, and im performing some http calls with it, but now on some http calls I need to retry a call if I dont get my expected result for 3 times.
How should I implement something like this?
thaknks

This is an example of retry function based on Future.recoverWith
If you run it you can see that it prints "run process" until the Future is successful but not more than 'times' times
object X extends App{
type Request = String
type Response = String
import scala.concurrent.ExecutionContext.Implicits.global
def retry(request: Request, process: Request => Future[Response], times: Int): Future[Response] ={
val responseF = process(request)
if(times > 0)
responseF.recoverWith{
case ex => println("fail")
retry(request, process, times - 1)
}
else
responseF
}
def process(s: Request): Future[Response] = {
println("run process")
if(Random.nextBoolean()) Future.successful("good") else Future.failed(new Exception)
}
val result = retry("", process, 3)
import scala.concurrent.duration._
println(Await.result(result, 1.second))
}

Delete directory recursively in Scala

I am writing the following (with Scala 2.10 and Java 6):
import java.io._
def delete(file: File) {
if (file.isDirectory)
Option(file.listFiles).map(_.toList).getOrElse(Nil).foreach(delete(_))
file.delete
}
How would you improve it ? The code seems working but it ignores the return value of java.io.File.delete. Can it be done easier with scala.io instead of java.io ?

With pure scala + java way
import scala.reflect.io.Directory
import java.io.File
val directory = new Directory(new File("/sampleDirectory"))
directory.deleteRecursively()
deleteRecursively() Returns false on failure

Try this code that throws an exception if it fails:
def deleteRecursively(file: File): Unit = {
if (file.isDirectory) {
file.listFiles.foreach(deleteRecursively)
}
if (file.exists && !file.delete) {
throw new Exception(s"Unable to delete ${file.getAbsolutePath}")
}
}
You could also fold or map over the delete if you want to return a value for all the deletes.

Using scala IO
import scalax.file.Path
val path = Path.fromString("/tmp/testfile")
try {
path.deleteRecursively(continueOnFailure = false)
} catch {
case e: IOException => // some file could not be deleted
}
or better, you could use a Try
val path: Path = Path ("/tmp/file")
Try(path.deleteRecursively(continueOnFailure = false))
which will either result in a Success[Int] containing the number of files deleted, or a Failure[IOException].

From
http://alvinalexander.com/blog/post/java/java-io-faq-how-delete-directory-tree
Using Apache Common IO
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.filefilter.WildcardFileFilter;
public void deleteDirectory(String directoryName)
throws IOException
{
try
{
FileUtils.deleteDirectory(new File(directoryName));
}
catch (IOException ioe)
{
// log the exception here
ioe.printStackTrace();
throw ioe;
}
}
The Scala one can just do this...
import org.apache.commons.io.FileUtils
import org.apache.commons.io.filefilter.WildcardFileFilter
FileUtils.deleteDirectory(new File(outputFile))
Maven Repo Imports

Using the Java NIO.2 API:
import java.nio.file.{Files, Paths, Path, SimpleFileVisitor, FileVisitResult}
import java.nio.file.attribute.BasicFileAttributes
def remove(root: Path): Unit = {
Files.walkFileTree(root, new SimpleFileVisitor[Path] {
override def visitFile(file: Path, attrs: BasicFileAttributes): FileVisitResult = {
Files.delete(file)
FileVisitResult.CONTINUE
}
override def postVisitDirectory(dir: Path, exc: IOException): FileVisitResult = {
Files.delete(dir)
FileVisitResult.CONTINUE
}
})
}
remove(Paths.get("/tmp/testdir"))
Really, it's a pity that the NIO.2 API is with us for so many years and yet few people are using it, even though it is really superior to the old File API.

Expanding on Vladimir Matveev's NIO2 solution:
object Util {
import java.io.IOException
import java.nio.file.{Files, Paths, Path, SimpleFileVisitor, FileVisitResult}
import java.nio.file.attribute.BasicFileAttributes
def remove(root: Path, deleteRoot: Boolean = true): Unit =
Files.walkFileTree(root, new SimpleFileVisitor[Path] {
override def visitFile(file: Path, attributes: BasicFileAttributes): FileVisitResult = {
Files.delete(file)
FileVisitResult.CONTINUE
}
override def postVisitDirectory(dir: Path, exception: IOException): FileVisitResult = {
if (deleteRoot) Files.delete(dir)
FileVisitResult.CONTINUE
}
})
def removeUnder(string: String): Unit = remove(Paths.get(string), deleteRoot=false)
def removeAll(string: String): Unit = remove(Paths.get(string))
def removeUnder(file: java.io.File): Unit = remove(file.toPath, deleteRoot=false)
def removeAll(file: java.io.File): Unit = remove(file.toPath)
}

Using java 6 without using dependencies this is pretty much the only way to do so.
The problem with your function is that it return Unit (which I btw would explicit note it using def delete(file: File): Unit = {
I took your code and modify it to return map from file name to the deleting status.
def delete(file: File): Array[(String, Boolean)] = {
Option(file.listFiles).map(_.flatMap(f => delete(f))).getOrElse(Array()) :+ (file.getPath -> file.delete)
}

To add to Slavik Muz's answer:
def deleteFile(file: File): Boolean = {
def childrenOf(file: File): List[File] = Option(file.listFiles()).getOrElse(Array.empty).toList
#annotation.tailrec
def loop(files: List[File]): Boolean = files match {
case Nil ⇒ true
case child :: parents if child.isDirectory && child.listFiles().nonEmpty ⇒
loop((childrenOf(child) :+ child) ++ parents)
case fileOrEmptyDir :: rest ⇒
println(s"deleting $fileOrEmptyDir")
fileOrEmptyDir.delete()
loop(rest)
}
if (!file.exists()) false
else loop(childrenOf(file) :+ file)
}

This one uses java.io but one can delete directories matching it with wildcard string which may or may not contain any content within it.
for (file <- new File("<path as String>").listFiles;
if( file.getName() matches("[1-9]*"))) FileUtils.deleteDirectory(file)
Directory structure e.g.
* A/1/, A/2/, A/300/ ... thats why the regex String: [1-9]*, couldn't find a File API in scala which supports regex(may be i missed something).

Getting little lengthy, but here's one that combines the recursivity of Garrette's solution with the npe-safety of the original question.
def deleteFile(path: String) = {
val penultimateFile = new File(path.split('/').take(2).mkString("/"))
def getFiles(f: File): Set[File] = {
Option(f.listFiles)
.map(a => a.toSet)
.getOrElse(Set.empty)
}
def getRecursively(f: File): Set[File] = {
val files = getFiles(f)
val subDirectories = files.filter(path => path.isDirectory)
subDirectories.flatMap(getRecursively) ++ files + penultimateFile
}
getRecursively(penultimateFile).foreach(file => {
if (getFiles(file).isEmpty && file.getAbsoluteFile().exists) file.delete
})
}

This is recursive method that clean all in directory, and return count of deleted files
def cleanDir(dir: File): Int = {
#tailrec
def loop(list: Array[File], deletedFiles: Int): Int = {
if (list.isEmpty) deletedFiles
else {
if (list.head.isDirectory && !list.head.listFiles().isEmpty) {
loop(list.head.listFiles() ++ list.tail ++ Array(list.head), deletedFiles)
} else {
val isDeleted = list.head.delete()
if (isDeleted) loop(list.tail, deletedFiles + 1)
else loop(list.tail, deletedFiles)
}
}
}
loop(dir.listFiles(), 0)
}

What I ended up with
def deleteRecursively(f: File): Boolean = {
if (f.isDirectory) f.listFiles match {
case files: Array[File] => files.foreach(deleteRecursively)
case null =>
}
f.delete()
}

os-lib makes it easy to delete recursively with a one-liner:
os.remove.all(os.pwd/"dogs")
os-lib uses java.nio under the hood, just doesn't expose all the Java ugliness. See here for more info on how to use the library.

You can do this by excute external system commands.
import sys.process._
def delete(path: String) = {
s"""rm -rf ${path}""".!!
}

Grails paginate not working

I have this list code in my controller that uses dynamic finders
def listPurchaseRequest(Integer max){
params.max = Math.min(max ?: 5, 100)
def purchaseRequestList = PurchaseRequest.list (params)
if ( params.query) {
purchaseRequestList = PurchaseRequest.findAllByRequestByLikeOrRequestNumberLike("%${params.query}%", "%${params.query}%", params)
}
[purchaseRequestInstanceList: purchaseRequestList,
purchaseRequestInstanceTotal: //this]
}
My search and list is working except for my pagination.
<g:paginate total="${purchaseRequestInstanceTotal}" params="${params}" maxsteps="3" prev="«" next="»" />
if i use purchaseRequestList.totalCount it works with the default list but when the result is displayed after i search it gives me a Exception evaluating property 'totalCount' for java.util.ArrayList, Reason: groovy.lang.MissingPropertyException: No such property: totalCount for class: rms.PurchaseRequest error
if i use purchaseRequestList.count() it gives me this Could not find which method count() to invoke from this list: public java.lang.Number java.util.Collection#count(groovy.lang.Closure) public java.lang.Number java.util.Collection#count(java.lang.Object) error

You need to use the CountBy* methods for the search. Try this
def listPurchaseRequest(Integer max){
params.max = Math.min(max ?: 5, 100)
def purchaseRequestList, count
if (params.query) {
purchaseRequestList = PurchaseRequest.findAllByRequestByLikeOrRequestNumberLike("%${params.query}%", "%${params.query}%", params)
count = PurchaseRequest.countByRequestByLikeOrRequestNumberLike("%${params.query}%", "%${params.query}%")
} else {
purchaseRequestList = PurchaseRequest.list (params)
count = purchaseRequestList.totalCount
}
[purchaseRequestInstanceList: purchaseRequestList,
purchaseRequestInstanceTotal: count]
}
FYI, I moved the .list() into an else clause to save you calling both list and findBy when params.query is set.

Strange NullPointerException in Actors in scala using play framework

I want to do some parallel computation and I'm getting a really strange java.lang.NullPointerException on calling ANY functions outside the object I have.
Take a look:
case class Return(session: String, job: Int)
case class Ready(n: Int)
case class DoJob(session: String, job: Int)
case class NotReady
object Notifications extends Controller with Secure {
class AtorMeio extends Actor {
import scala.collection.mutable.{Map => MMap}
val job: MMap[(String, Int), Option[Int]] = MMap()
def act {
loop {
react {
case DoJob(session, jobn) =>
if (job.get((session, jobn)).isEmpty) {
jobn match {
case 1 =>
job.put((session, jobn), None)
val n = Messaging.oi //Messaging.retrieveNumberOfMessages(new FlagTerm(new Flags(Flags.Flag.SEEN), false))
job.put((session, jobn), Some(n))
case 2 =>
// do!
}
}
case Return(session, jobn) =>
if (job.get((session, jobn)).isDefined && job.get((session, jobn)).get.isDefined) {
val ret = job.get((session, jobn)).get.get
job.remove((session, jobn))
reply(Ready(ret))
}
else
reply(NotReady)
}
}
}
}
private var meuator: AtorMeio = null
lazy val ator = {
if (Option(meuator).isEmpty) {
meuator = new AtorMeio
meuator.start
}
meuator
}
def pendingNotifications = {
ator ! DoJob(session.getId, 1)
ator !? Return(session.getId, 1) match {
case Ready(ret) =>
if (ret.toString != Option[String](params.get("current")).getOrElse("-1")) "true" else Suspend("80s")
case _ =>
}
}
}
I'm getting an error in executing Messaging.oi which is basically an object with:
def oi = 4
Here is the stacktrace:
controllers.Notifications$AtorMeio#1889d53: caught java.lang.NullPointerException
java.lang.NullPointerException
at controllers.Messaging$.oi(Messaging.scala:108)
at controllers.Notifications$AtorMeio$$anonfun$act$1$$anonfun$apply$1.apply(Notifications.scala:38)
at controllers.Notifications$AtorMeio$$anonfun$act$1$$anonfun$apply$1.apply(Notifications.scala:31) at scala.actors.ReactorTask.run(ReactorTask.scala:34)
at scala.actors.ReactorTask.compute(ReactorTask.scala:66)
at scala.concurrent.forkjoin.RecursiveAction.exec(RecursiveAction.java:147)
at scala.concurrent.forkjoin.ForkJoinTask.quietlyExec(ForkJoinTask.java:422)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.mainLoop(ForkJoinWorkerThread.java:340)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:325)
Line 108 is exactly this oneliner def. Ahh entrance point is def pendingNotifications.
Anyone can help? Thanks a lot!

Have you tried replacing
private var meuator: AtorMeio = null
by either:
private var meuator: AtorMeio = None

Configure your breakpoints view in your debugger to halt/break on NullPointerExceptions ...
And: you did see you set this to null here:
private var meuator: AtorMeio = null
?? Or?

Ok people, after digging a lot I discovered the problem: Somehow, somewhere if you have the "Controller" class from play framework mixed in, it crashes mercifully. So I just wrapped this thing into a 'clean' class and it worked.