We Keep Coding

Scala, Akka-Http and mysql - Too slow

I tried to connect MySQL to my scala-Akka code. In the receive method, I have 3 URL's and for these URL's, I implemented a simple hit counter.
So whenever I visit that particular URL, I insert a row into a table which will be the (URL, count, its timestamp). But I noticed its too slow. Completing only 180 requests in 10 seconds! What can I do to improve performance? Please help!
import akka.actor.{Actor, ActorLogging, ActorSystem, Props}
import akka.http.scaladsl.Http
import akka.http.scaladsl.model.{ContentTypes, HttpEntity}
import akka.http.scaladsl.server.Directives._
import akka.pattern.ask
import akka.stream.ActorMaterializer
import akka.util.Timeout
import scala.concurrent.duration._
import scala.io.StdIn
import java.sql.{Connection, DriverManager, PreparedStatement, Timestamp}
import java.time.{LocalDateTime, LocalTime}
import java.util.Date
import akka.http.javadsl.server.Route
object DBCP3 {
final case class fetch1(param:String)
val sql = "INSERT INTO URLInfo (url, time, count)" + "VALUES (?, ?, ?)"
val url = "jdbc:mysql://127.0.0.1/url"
val driver = "com.mysql.jdbc.Driver"
val username = "root"
val password = "SomePassword"
Class.forName(driver)
var connection = DriverManager.getConnection(url, username, password)
val date = new Date
private var number1: Int = 0
private var number3: Int = 0
private var number2: Int = 0
def insertIntoDB(path: String, num: Int) = {
val stm: PreparedStatement = connection.prepareStatement(sql)
stm.setString(1, path)
stm.setTimestamp(2, new Timestamp(date.getTime))
stm.setInt(3, num)
stm.execute()
}
class ActorA extends Actor with ActorLogging {
def receive = {
case fetch1(param) =>
if(param=="path1") {
number1+=1
insertIntoDB("http://localhost:8080/path1",number1)
context.sender() ! number1
}
if(param=="path2") {
number2+=1
insertIntoDB("http://localhost:8080/path2",number2)
context.sender() ! number2
}
if(param=="path3") {
number3+=1
insertIntoDB("http://localhost:8080/path3",number3)
context.sender() ! number3
}
}
}
def main(args: Array[String]) {
implicit val system = ActorSystem("my-system")
implicit val materializer = ActorMaterializer()
implicit val executionContext = system.dispatcher
implicit val timeout: Timeout = 1.seconds
val actor1 = system.actorOf(Props[ActorA], "SimpleActor1")
val route = concat(
path("path1") {
get {
onComplete((actor1 ? fetch1("path1")).mapTo[Int])
{
number => complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<h1>You visited $number times</h1>"))
}
}
},
path("path2") {
onComplete((actor1 ? fetch1("path2")).mapTo[Int])
{
number => complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<h1>You visited $number times</h1>"))
}
},
path("path3") {
onComplete((actor1 ? fetch1("path3")).mapTo[Int])
{
number => complete(HttpEntity(ContentTypes.`text/html(UTF-8)`, s"<h1>You visited $number times</h1>"))
}
}
)
val bindingFuture = Http().bindAndHandle(route, "localhost", 8080)
println(s"Server online at http://localhost:8080/\nPress RETURN to stop...")
StdIn.readLine() // let it run until user presses return
val _ = bindingFuture
.flatMap(_.unbind()) // trigger unbinding from the port
connection.close()
}
}

Testing Grails App Generating pdf file Cannot get property 'config' on null object

I am trying to test a service class that will generate a pdf file but this error occurs:
[getDocument] EXCEPTION: variables: [input, input, input], message:
Cannot get property 'config' on null object
My Service class is:
class TemplateService {
static transactional = false
def grailsApplication
def getDocument(inputs, idTemp) {
def result
if(inputs) {
long dateBeginTransaction = System.currentTimeMillis()
try {
def http = new HTTPBuilder(grailsApplication.config.tempdoc.url?.replace("COI", idTemp))
http.auth.basic grailsApplication.config.rest.login, grailsApplication.config.rest.password
http.request(POST,JSON) { req ->
headers.'Accept' = 'application/json'
headers.'Content-type' = 'application/json'
body = [
inputs: inputs
]
response.success = { resp, json ->
log.info "[getDocument] time: " + (System.currentTimeMillis() - dateBeginTransaction) / 1000 + " ms"
result = json?.pdf
}
response.failure = { resp, reader ->
log.error "[getDocument] inputs: " + inputs + ", response: " + resp + ", message: " + reader?.message
}
}
} catch (Exception e) {
log.error "[getDocument] EXCEPTION: inputs: " + inputs + ", message: " + e.message
}
} else {
log.error "[getDocument] params sense valors"
}
result
}
}
This is my Test:
*Note inputs is an arraylist
void "generate document"() {
given: "generate document"
def TemplateService = new TemplateService()
when:
def result = TemplateService.getDocument(inputs, idTemp)
then:
result != null
result.size() > 0
where:
inputs = [ "input", "input", "input"]
idTemp = "12AD"
}

At the very least, you'll have to mock the config in your test. In Grails 3.3.5:
class PlaServiceSpec extends Specification implements ServiceUnitTest<Plaservice>, DataTest {
Closure doWithConfig() {{ config ->
config.plantidoc.url = 'my url'
}}
void "generate document"() {
given:
def variables = ["input:input", "input:input"]
def idPlantitlla = "12AD"
when:
def result = service.getDocument(variables, idPlantitlla)
then:
// test the result
}
}

you can make your test an Integration Test (move it to test/integration-test folder) and in this way grailsApplication will be injected into your service

How to add Status = KO in Gatling script?

Is it possible to fail my request?
I would like to put Status = KO in asLongAs() section. My condition is like, if I get WorkflowFailed = True or Count > 8 then I want to fail that request using Status = KO.
I have seen somewhere about session.markAsFailed but how and where to use this?
Thanks.
Here is the code,
class LaunchResources extends Simulation {
val scenarioRepeatCount = Integer.getInteger("scenarioRepeatCount", 1).toInt
val userCount = Integer.getInteger("userCount", 1).toInt
val UUID = System.getProperty("UUID", "24d0e03")
val username = System.getProperty("username", "p1")
val password = System.getProperty("password", "P12")
val testServerUrl = System.getProperty("testServerUrl", "https://someurl.net")
val count = new java.util.concurrent.atomic.AtomicInteger(0)
val httpProtocol = http
.baseURL(testServerUrl)
.basicAuth(username, password)
.connection("""keep-alive""")
.contentTypeHeader("""application/vnd+json""")
val headers_0 = Map(
"""Cache-Control""" -> """no-cache""",
"""Origin""" -> """chrome-extension://fdmmgasdw1dojojpjoooidkmcomcm""")
val scn = scenario("LaunchAction")
.repeat (scenarioRepeatCount) {
exec(http("LaunchAResources")
.post( """/api/actions""")
.headers(headers_0)
.body(StringBody(s"""{"UUID": "$UUID", "stringVariables" : {"externalFilePath" : "/Test.mp4"}}"""))
.check(jsonPath("$.id").saveAs("WorkflowID")))
.exec(http("SaveWorkflowStatus")
.get("""/api/actions/{$WorkflowID}""")
.headers(headers_0)
.check(jsonPath("$.status").saveAs("WorkflowStatus")))
}
.asLongAs(session => session.attributes("WorkflowStatus") != "false" && count.getAndIncrement() < 8) {
doIf(session => session("WorkflowFailed").validate[String].map(WorkflowFailed => !WorkflowFailed.contains("true")).recover(true))
{
pause(pauseTime)
.exec(http("SaveWorkflowStatus")
.get("""/api/actions/${WorkflowID}""")
.headers(headers_0)
.check(jsonPath("$.running").saveAs("WorkflowStatus"))
.check(jsonPath("$.failed").saveAs("WorkflowFailed")))
.exec(session => {
val wflowStatus1 = session.get("WorkflowStatus").asOption[String]
val wflowFailed1 = session.get("WorkflowFailed").asOption[String]
println("Inner Loop Workflow Status: ========>>>>>>>> " + wflowStatus1.getOrElse("COULD NOT FIND STATUS"))
println("Inner Loop Workflow Failed?? ========>>>>>>>> " + wflowFailed1.getOrElse("COULD NOT FIND STATUS"))
println("Count =====>> " + count)
session})
}
}
setUp(scn.inject(atOnceUsers(userCount))).protocols(httpProtocol)
}

there's a method available on the session
exec(session => session.markAsFailed)

Scala/Java reflective programming: invoke constructor by typecasted arguments

In the library json4s, I intend to write a weakly typed deserializer for some malformed data (mostly the result of XML -> JSON conversions)
I want the dynamic program to get the type information of a given constructor (easy, e.g. 'Int'), apply it on a parsed string (e.g. "12.51"), automatically convert string into the type (in this case 12.51 should be typecasted to 13), then call the constructor.
I come up with the following implementation:
import org.json4s.JsonAST.{JDecimal, JDouble, JInt, JString}
import org.json4s._
import scala.reflect.ClassTag
object WeakNumDeserializer extends Serializer[Any] {
def cast[T](cc: Class[T], v: Any): Option[T] = {
implicit val ctg: ClassTag[T] = ClassTag(cc)
try {
Some(v.asInstanceOf[T])
}
catch {
case e: Throwable =>
None
}
}
override def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Any] = Function.unlift{
tuple: (TypeInfo, JValue) =>
tuple match {
case (TypeInfo(cc, _), JInt(v)) =>
cast(cc, v)
case (TypeInfo(cc, _), JDouble(v)) =>
cast(cc, v)
case (TypeInfo(cc, _), JDecimal(v)) =>
cast(cc, v)
case (TypeInfo(cc, _), JString(v)) =>
cast(cc, v.toDouble)
case _ =>
None
}
}
}
However executing the above code on a real Double => Int case always yield IllegalArgumentException. Debugging reveals that the line:
v.asInstanceOf[T]
does not convert Double type to Int in memory, it remains as a Double number after type erasure, and after it is used in reflection to call the constructor it triggers the error.
How do I bypass this and make the reflective function figuring this out?
Is there a way to tell the Java compiler to actually convert it into an Int type?
UPDATE: to help validating your answer I've posted my test cases:
case class StrStr(
a: String,
b: String
)
case class StrInt(
a: String,
b: Int
)
case class StrDbl(
a: String,
b: Double
)
case class StrIntArray(
a: String,
b: Array[Int]
)
case class StrIntSeq(
a: String,
b: Seq[Int]
)
case class StrIntSet(
a: String,
b: Set[Int]
)
class WeakSerializerSuite extends FunSuite with TestMixin{
implicit val formats = DefaultFormats ++ Seq(StringToNumberDeserializer, ElementToArrayDeserializer)
import org.json4s.Extraction._
test("int to String") {
val d1 = StrInt("a", 12)
val json = decompose(d1)
val d2 = extract[StrStr](json)
d2.toString.shouldBe("StrStr(a,12)")
}
test("string to int") {
val d1 = StrStr("a", "12")
val json = decompose(d1)
val d2 = extract[StrInt](json)
d2.toString.shouldBe("StrInt(a,12)")
}
test("double to int") {
val d1 = StrDbl("a", 12.51)
val json = decompose(d1)
val d2 = extract[StrInt](json)
d2.toString.shouldBe("StrInt(a,12)")
}
test("int to int array") {
val d1 = StrInt("a", 12)
val json = decompose(d1)
val d2 = extract[StrIntArray](json)
d2.copy(b = null).toString.shouldBe("StrIntArray(a,null)")
}
test("int to int seq") {
val d1 = StrInt("a", 12)
val json = decompose(d1)
val d2 = extract[StrIntSeq](json)
d2.toString.shouldBe("StrIntSeq(a,List(12))")
}
test("int to int set") {
val d1 = StrInt("a", 12)
val json = decompose(d1)
val d2 = extract[StrIntSet](json)
d2.toString.shouldBe("StrIntSet(a,Set(12))")
}
test("string to int array") {
val d1 = StrStr("a", "12")
val json = decompose(d1)
val d2 = extract[StrIntArray](json)
d2.copy(b = null).toString.shouldBe("StrIntArray(a,null)")
}
test("string to int seq") {
val d1 = StrStr("a", "12")
val json = decompose(d1)
val d2 = extract[StrIntSeq](json)
d2.toString.shouldBe("StrIntSeq(a,List(12))")
}
test("string to int set") {
val d1 = StrStr("a", "12")
val json = decompose(d1)
val d2 = extract[StrIntSet](json)
d2.toString.shouldBe("StrIntSet(a,Set(12))")
}

I've found the first solution, TL:DR: its totally absurd & illogical, and absolutely full of boilerplates for a established strongly typed language. Please post your answer deemed any better:
abstract class WeakDeserializer[T: Manifest] extends Serializer[T] {
// final val tpe = implicitly[Manifest[T]]
// final val clazz = tpe.runtimeClass
// cannot serialize
override def serialize(implicit format: Formats): PartialFunction[Any, JValue] = PartialFunction.empty
}
object StringToNumberDeserializer extends WeakDeserializer[Any] {
override def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Any] = {
case (TypeInfo(cc, _), JString(v)) =>
cc match {
case java.lang.Byte.TYPE => v.toByte
case java.lang.Short.TYPE => v.toShort
case java.lang.Character.TYPE => v.toInt.toChar
case java.lang.Integer.TYPE => v.toInt
case java.lang.Long.TYPE => v.toLong
case java.lang.Float.TYPE => v.toFloat
case java.lang.Double.TYPE => v.toDouble
case java.lang.Boolean.TYPE => v.toBoolean
//TODO: add boxed type
}
}
}
object ElementToArrayDeserializer extends WeakDeserializer[Any] {
val listClass = classOf[List[_]]
val seqClass = classOf[Seq[_]]
val setClass = classOf[Set[_]]
val arrayListClass = classOf[java.util.ArrayList[_]]
override def deserialize(implicit format: Formats): PartialFunction[(TypeInfo, JValue), Any] = {
case (ti# TypeInfo(this.listClass, _), jv) =>
List(extractInner(ti, jv, format))
case (ti# TypeInfo(this.seqClass, _), jv) =>
Seq(extractInner(ti, jv, format))
case (ti# TypeInfo(this.setClass, _), jv) =>
Set(extractInner(ti, jv, format))
case (ti# TypeInfo(this.arrayListClass, _), jv) =>
import scala.collection.JavaConverters._
new java.util.ArrayList[Any](List(extractInner(ti, jv, format)).asJava)
case (ti# TypeInfo(cc, _), jv) if cc.isArray =>
val a = Array(extractInner(ti, jv, format))
mkTypedArray(a, firstTypeArg(ti))
}
def mkTypedArray(a: Array[_], typeArg: ScalaType) = {
import java.lang.reflect.Array.{newInstance => newArray}
a.foldLeft((newArray(typeArg.erasure, a.length), 0)) { (tuple, e) => {
java.lang.reflect.Array.set(tuple._1, tuple._2, e)
(tuple._1, tuple._2 + 1)
}}._1
}
def extractInner(ti: TypeInfo, jv: JValue, format: Formats): Any = {
val result = extract(jv, firstTypeArg(ti))(format)
result
}
def firstTypeArg(ti: TypeInfo): ScalaType = {
val tpe = ScalaType.apply(ti)
val firstTypeArg = tpe.typeArgs.head
firstTypeArg
}
}

Scala how to inject mock object to ScalatraFlatSpec

I stuck with Unit test in Scala for many days. I cannot inject mock object to Unit test. The ScalatraFlatSpec call to the actual database not my mock variable and i have no idea to do.
This is my API
class Dashboard extends Servlet {
get("/:brand_code") {
val start = System.currentTimeMillis
val brandCode = params.get("brand_code").get
var brandId = 0;
val sqlFind = "SELECT DISTINCT(id) FROM brands WHERE brand_code=?"
val found:List[Map[String, Any]] = ConnectionModel.getExecuteQuery(sqlFind, List(brandCode))
if(found.isEmpty){
halt(404, send("error", s"brand_code [$brandCode] not found."))
}else{
brandId = found(0).getOrElse("id", 0).toString.toInt
send("Yeah55", brandId)
}
}
And this is Servlet
abstract class Servlet extends ScalatraServlet with CorsSupport with JacksonJsonSupport {
protected implicit lazy val jsonFormats: Formats = DefaultFormats.withBigDecimal
protected override def transformResponseBody(body: JValue): JValue = body.underscoreKeys
protected lazy val body = parsedBody.extract[Map[String, Any]]
protected def send(message: String, data: Any = None) = Map("message" -> message, "data" -> data)
options("/*") {
response.setHeader(
"Access-Control-Allow-Headers", request.getHeader("Access-Control-Request-Headers")
)
}
before() {
contentType = formats("json")
}
}
And this is ConnectionModel and ConnectionModelAble
trait ConnectionModelAble {
def getExecuteQuery(sql: String, parameters: List[Any]): List[Map[String, Any]]
}
object ConnectionModel extends ConnectionModelAble{
var connection:Connection = {
val url = "jdbc:mysql://localhost:3306/db"
val username = "root"
val password = ""\
Class.forName("com.mysql.jdbc.Driver")
DriverManager.getConnection(url, username, password)
}
def getExecuteQuery(sql: String, parameters: List[Any]): List[Map[String, Any]]= {
try {
val statement = connection.createStatement()
var preparedStatement: PreparedStatement = connection.prepareStatement(sql);
var formatDate: DateFormat = new SimpleDateFormat("yyyy/MM/dd HH:mm:ss");
// Do some execute
for (i <- 0 until parameters.size) {
parameters(i) match {
case _: Int => preparedStatement.setInt(i + 1, parameters(i).toString.toInt)
case _: Double => preparedStatement.setDouble(i + 1, parameters(i).toString.toDouble)
case _: Date => preparedStatement.setDate(i + 1, new java.sql.Date(formatDate.parse(parameters(i).toString).getTime))
case default => preparedStatement.setString(i + 1, parameters(i).toString)
}
}
val resultSet = preparedStatement.executeQuery()
val metaData: ResultSetMetaData = resultSet.getMetaData();
val columnCount = metaData.getColumnCount();
var ret: List[Map[String, Any]] = List();
while (resultSet.next()) {
var row: Map[String, Any] = Map[String, Any]();
for (i <- 1 to columnCount) {
val columnName = metaData.getColumnName(i);
var obj = resultSet.getObject(i);
row += columnName -> obj
}
ret = ret :+ row
}
ret
}catch {
case e: Exception => {
e.printStackTrace();
List()
}
}
}
And this is my unit test
class DashboardSpec extends ScalatraFlatSpec with MockitoSugar {
addServlet(new Dashboard, "/v1/dashboard/*")
it should "return get dashboard correctly" in {
val brandCode = "APAAA"
val brandId = 157
get("/v1/dashboard/APAAA") {
val connectModel = mock[ConnectionModelAble]
val sqlFind = "SELECT DISTINCT(id) FROM brands WHERE brand_code=?"
Mockito.when(connectModel.getExecuteQuery(sqlFind, List(brandCode))).thenReturn(
List(Map("id" -> 150))
)
assert(status == 200)
println(connectModel.getExecuteQuery(sqlFind, List(brandCode)))
println(body)
}
}
}
I found that body from unit test is not from my mock data, it's from real database. What should i do.
Thank you.

You aren't injecting your mock into the Dashboard, so the Connection you're seeing in getExecuteQuery is the one provided by ConnectionModel.connection. You probably want to use a dependency injection framework or something like the Cake pattern to make sure your Dashboard is referring to your mock instance.