I'm trying to find out why my jHipster app is so slow when querying a database.
This is one of my services using spring-data's PagingAndSortingRepository
#Transactional(readOnly = true)
public Page<Center> findAll(Pageable pageable) {
log.debug("Request to get all Centers");
return centerRepository.findAll(pageable);
}
I've used JHipsters' LoggingAspect and added a timer to log the performance of each method.
#Around("loggingPointcut()")
public Object logAround(ProceedingJoinPoint joinPoint) throws Throwable {
Stopwatch stopwatch = null;
if (log.isDebugEnabled()) {
log.debug("Enter: {}.{}() with argument[s] = {}", joinPoint.getSignature().getDeclaringTypeName(),
joinPoint.getSignature().getName(), Arrays.toString(joinPoint.getArgs()));
stopwatch = Stopwatch.createStarted();
}
try {
Object result = joinPoint.proceed();
if (log.isDebugEnabled()) {
log.debug("Exit: {}.{}() [took {} ms] with result = {}", joinPoint.getSignature().getDeclaringTypeName(),
joinPoint.getSignature().getName(), stopwatch.elapsed(MILLISECONDS), result);
}
return result;
} catch (IllegalArgumentException e) {
log.error("Illegal argument: {} in {}.{}()", Arrays.toString(joinPoint.getArgs()),
joinPoint.getSignature().getDeclaringTypeName(), joinPoint.getSignature().getName());
throw e;
}
}
I configured hibernate to generate the statictics:
spring.jpa.properties.hibernate.generate_statistics=true
If I change the log-levels of org.hibernate.stat.internal.ConcurrentStatisticsImpl and org.hibernate.engine.internal.StatisticalLoggingSessionEventListener I see the following logs:
2016-10-13 11:00:00,640 DEBUG [http-nio-8080-exec-8] LoggingAspect: Enter: com.fluidda.broncholab.service.CenterService.findAll() with argument[s] = [Page request [number: 0, size 20, sort: id: ASC]]
2016-10-13 11:00:00,643 DEBUG [http-nio-8080-exec-8] CenterService: Request to get all Centers
2016-10-13 11:00:02,238 DEBUG [http-nio-8080-exec-8] ConcurrentStatisticsImpl: HHH000117: HQL: select count(generatedAlias0) from Center as generatedAlias0, time: 1ms, rows: 1
2016-10-13 11:00:02,241 DEBUG [http-nio-8080-exec-8] ConcurrentStatisticsImpl: HHH000117: HQL: select generatedAlias0 from Center as generatedAlias0 order by generatedAlias0.id asc, time: 2ms, rows: 3
2016-10-13 11:00:02,242 DEBUG [http-nio-8080-exec-8] LoggingAspect: Exit: com.fluidda.broncholab.service.CenterService.findAll() [took 1601 ms] with result = Page 1 of 1 containing com.fluidda.broncholab.domain.Center instances
2016-10-13 11:00:02,243 INFO [http-nio-8080-exec-8] StatisticalLoggingSessionEventListener: Session Metrics {
568512 nanoseconds spent acquiring 1 JDBC connections;
0 nanoseconds spent releasing 0 JDBC connections;
92324 nanoseconds spent preparing 2 JDBC statements;
992105 nanoseconds spent executing 2 JDBC statements;
0 nanoseconds spent executing 0 JDBC batches;
34717 nanoseconds spent performing 3 L2C puts;
0 nanoseconds spent performing 0 L2C hits;
0 nanoseconds spent performing 0 L2C misses;
0 nanoseconds spent executing 0 flushes (flushing a total of 0 entities and 0 collections);
2943 nanoseconds spent executing 2 partial-flushes (flushing a total of 0 entities and 0 collections)
If you take a look at the timings (not the timestamp when the item was logged), you'll see a big difference between:
ConcurrentStatisticsImpl: time: 2ms, rows: 3
LoggingAspect: [took 1601 ms]
StatisticalLoggingSessionEventListener: 992105 nanoseconds spent executing 2 JDBC statements;
The strange is, that these performance issue's do not occur all the time! This is the dropwizard statistic:
Service name Count Mean Min p50 p75 p95 p99 Max
....web.rest.CenterResource.getAllCenters 5 13 10 16 16 16 16 1,612
Does anyone know what may cause these performance drops?
Does anyone know how I can investigate any further?
After adding extra logging, and not using an ASYNC logger, I've find out, the server was losing time parsing the HQL.
I've created another question about this: Hibernate QueryTranslatorImpl HQL AST parsing performance
Related
I have a table for metrics in Oracle DB where one of the columns is the timestamp. I need to read the metrics from the DB and group them into given intervals of any length( 2 Month or 3 Hour or 1 Days or 2 years etc) between a starting timestamp and ending timestamp. The timestamp will be of format
2020-05-24T18:51:10.018-07:00
I know I can read all the entries from the table and sort them and group them into intervals by converting them all into seconds, but is there a better way to do it ?
You may use match_recognize for this.
with t(ts) as (
select
current_timestamp
+ interval '3' minute
* dbms_random.value(0, level)
from dual
connect by level < 20
)
select *
from t
match_recognize(
order by ts asc
measures
match_number() as grp
all rows per match
pattern(a w5min*)
define
/*Rows within 5 minutes after the first row in bucket*/
w5min as ts - first(ts) < interval '5' minute
)
TS | GRP
:------------------ | --:
2021-11-03 06:20:40 | 1
2021-11-03 06:20:56 | 1
2021-11-03 06:23:27 | 1
2021-11-03 06:23:49 | 1
2021-11-03 06:25:23 | 1
2021-11-03 06:25:36 | 1
2021-11-03 06:32:14 | 2
2021-11-03 06:34:38 | 2
2021-11-03 06:36:29 | 2
2021-11-03 06:36:59 | 2
2021-11-03 06:39:29 | 3
2021-11-03 06:40:17 | 3
2021-11-03 06:41:07 | 3
2021-11-03 06:47:14 | 4
2021-11-03 06:48:31 | 4
2021-11-03 06:52:29 | 5
2021-11-03 06:59:22 | 6
2021-11-03 07:02:05 | 6
2021-11-03 07:04:54 | 7
db<>fiddle here
Whether it’s the best way, who can tell? If you want a Java solution, I suggest this one.
OffsetDateTime[] timestamps = {
OffsetDateTime.parse("2020-05-24T18:51:10.018-07:00"),
OffsetDateTime.parse("2020-03-07T23:45:02.399-08:00"),
OffsetDateTime.parse("2020-05-24T20:01:11.442-07:00"),
OffsetDateTime.parse("2020-03-08T01:03:05.079-08:00"),
OffsetDateTime.parse("2020-05-24T19:32:34.461-07:00"),
};
TemporalAmount intervalLength = Duration.ofHours(2);
List<OffsetDateTime> list = new ArrayList<>(Arrays.asList(timestamps));
list.sort(Comparator.naturalOrder());
List<List<OffsetDateTime>> groups = new ArrayList<>();
if (! list.isEmpty()) {
List<OffsetDateTime> currentGroup = new ArrayList<>();
Iterator<OffsetDateTime> itr = list.iterator();
OffsetDateTime first = itr.next();
currentGroup.add(first);
OffsetDateTime groupEnd = first.plus(intervalLength);
while (itr.hasNext()) {
OffsetDateTime current = itr.next();
if (current.isBefore(groupEnd)) {
currentGroup.add(current);
} else { // new group
groups.add(currentGroup);
currentGroup = new ArrayList<>();
groupEnd = current.plus(intervalLength);
currentGroup.add(current);
}
}
// remember to add last group
groups.add(currentGroup);
}
groups.forEach(System.out::println);
Output:
[2020-03-07T23:45:02.399-08:00, 2020-03-08T01:03:05.079-08:00]
[2020-05-24T18:51:10.018-07:00, 2020-05-24T19:32:34.461-07:00, 2020-05-24T20:01:11.442-07:00]
The advantage of declaring intervalLength a TemporalAmount is you are free to assign either a time-based Duration to it (as above) or a date-based Period.
TemporalAmount intervalLength = Period.ofMonths(3);
In this case the result is just one group:
[2020-03-07T23:45:02.399-08:00, 2020-03-08T01:03:05.079-08:00,
2020-05-24T18:51:10.018-07:00, 2020-05-24T19:32:34.461-07:00,
2020-05-24T20:01:11.442-07:00]
Link
Oracle tutorial: Date Time explaining how to use java.time.
I can see in the logs that Spring retry is sending 2 requests to the remote server and both requests return successful responses.
I am not able to get the reason behind the same.
Code:
Class StatusClient{
#CircuitBreaker(maxAttemptsExpression = "#{${remote.broadridge.circuitBreaker.maxAttempts}}",
openTimeoutExpression = "#{${remote.broadridge.circuitBreaker.openTimeout}}", resetTimeoutExpression = "#{${remote.broadridge.circuitBreaker.resetTimeout}}")
public Optional<JobStatusResponseDTO> getStatus(String account, String jobNumber) {
client.post()
.uri(PATH)
.body(BodyInserters.fromValue(request))
.exchangeToMono(response -> {
if (response.statusCode() == HttpStatus.NO_CONTENT) {
return Mono.empty();
} else if (isClientOrServerError(response)) {
return Mono.error(new RemoteClientException(String.format("status is not received: %s", response.statusCode())));
}
stopWatch.stop();
log.info("time taken by the getStatus=[{}] for {}", (stopWatch.getTotalTimeMillis()), request);
return response.bodyToMono(JobStatusResponseDTO.class);
})
.block();
return Optional.ofNullable(block);}
}
Class status{
#Retryable(maxAttemptsExpression = "#{${remote.retry.maxAttempts}}", backoff = #Backoff(delayExpression = "#{${remote.retry.delay}}"))
public Optional<JobStatusResponseDTO> getStatus(String jobNumber, String accountNumber) {
return statusClient.getStatus(accountNumber, jobNumber);
}
}
Config in application.yml
circuitBreaker:
maxAttempts: 3 # defalut 3
openTimeout: 5000 # defalut 5000
resetTimeout: 20000 # defalut 20000
retry:
maxAttempts: 3 # defalut 3
delay: 1000 # defalut 1000
Logs:
792 <14>1 2021-10-26T16:26:32.978917+00:00 - 2021-10-26 16:26:32.978 INFO [batch,ec40b8fe1f6a4cfb,06052e092b3f8e66] : time taken by the getStatus=[582] for JobStatusRequestDTO(account=12
456, jobNumber=S123456)
792 <14>1 2021-10-26T16:26:18.263121+00:00 2021-10-26 16:26:18.262 INFO [batch,ec40b8fe1f6a4cfb,21202725a0002bde] : time taken by the getStatus=[592] for JobStatusRequestDTO(account=12
456, jobNumber=S123456)
Both the request are a few seconds apart.
Edit 1:
changed circuit breaker to the max attempt to 1. Now it is retrying 3 times. There is still an issue. It seems it is calling the remote server only once and not calling after.
The remote call is wrapped in a circuit breaker.
1st Attempt log:
status is not received: 503 SERVICE_UNAVAILABLE
2nd Attempt log:
org.springframework.retry.ExhaustedRetryException: Retry exhausted after last attempt with no recovery path;
3rd Attempt log:
org.springframework.retry.ExhaustedRetryException: Retry exhausted after last attempt with no recovery path;
circuitBreaker:
maxAttempts: 1
openTimeout: 5000 # defalut 5000
resetTimeout: 20000 # defalut 20000
retry:
maxAttempts: 3 # defalut 3
delay: 1000 # defalut 1000
This is because you have placed the default retry.maxAttempts to 3 with a delay of 1000ms. Spring will auto-retry if there is no response within the mentioned delay time. So, replace the retry.maxAttemps to 2 then it won't give multiple responses.
you can simply paste below lines in application.properties.
retry.maxAttempts=2
retry.maxDelay=100
Also, I suggest you go through this.
I'm using Optaplanner for a hospital bed allocation project. I'm working with spring boot, spring JPA and postgres as a DB. I've set all classes with #PlanningEntity and #PlanningSolution annotations, set
#PlanningVariable, write the constraints with drool and configure it with '.xml' file .After solving the value of planning variable, which in my cas"bed" , doesn't change!!
this is the main class:
#SpringBootApplication(scanBasePackages = { "com.asma.optaplanner.demo" })
public class OptaPlannerDemoApplication {
public static void main(String[] args) {
SpringApplication.run(OptaPlannerDemoApplication.class, args);
}
#Bean
public CommandLineRunner demoData(PatientAdmissionScheduleHelper helper) {
return args -> {
SolverFactory<PatientAdmissionSchedule> solverfactory = SolverFactory.createFromXmlResource("Solver_Config.xml");
helper.initilizeDataBase();
System.out.println("before solving");
Solver<PatientAdmissionSchedule> solver = solverfactory.buildSolver();
PatientAdmissionSchedule unsolvedSchedule = helper.getSchedule();
unsolvedSchedule.getAdmissionList().forEach(admission -> {
System.out.println(admission.toString());
});
solver.solve(unsolvedSchedule);
PatientAdmissionSchedule solvedSchedule = solver.getBestSolution();
System.out.println("after solving");
solvedSchedule.getAdmissionList().forEach(admission -> {
System.out.println(admission.toString());
});
};
}
the result:
before solving
2020-06-03 21:33:05.256 WARN 9228 --- [ main] o.d.c.kie.builder.impl.KieBuilderImpl : File 'Constraint.drl' is in folder '' but declares package 'com.asma.optaplanner.demo'. It is advised to have a correspondance between package and folder names.
PatientName patient1bed=Bed [externalCode= bed11, room= Room [name=room1, capacity=2], indexInRoom=1], From 1, To5
PatientName patient2bed=Bed [externalCode= bed12, room= Room [name=room1, capacity=2], indexInRoom=2], From 2, To4
2020-06-03 21:33:06.208 INFO 9228 --- [ main] o.o.core.impl.solver.DefaultSolver : Solving started: time spent (77), best score (-80hard/0soft), environment mode (REPRODUCIBLE), random (JDK with seed 0).
2020-06-03 21:37:06.210 INFO 9228 --- [ main] o.o.c.i.l.DefaultLocalSearchPhase : Local Search phase (0) ended: time spent (240079), best score (0hard/0soft), score calculation speed (38961/sec), step total (418).
2020-06-03 21:37:06.222 INFO 9228 --- [ main] .c.i.c.DefaultConstructionHeuristicPhase : Construction Heuristic phase (1) ended: time spent (240091), best score (0hard/0soft), score calculation speed (1222/sec), step total (2).
2020-06-03 21:37:06.222 INFO 9228 --- [ main] o.o.core.impl.solver.DefaultSolver : Solving ended: time spent (240091), best score (0hard/0soft), score calculation speed (38946/sec), phase total (2), environment mode (REPRODUCIBLE).
after solving
2020-06-03 21:37:06.226 INFO 9228 --- [ main] ConditionEvaluationReportLoggingListener :
Error starting ApplicationContext. To display the conditions report re-run your application with 'debug' enabled.
2020-06-03 21:37:06.238 ERROR 9228 --- [ main] o.s.boot.SpringApplication : Application run failed
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:798) [spring-boot-2.3.1.BUILD-SNAPSHOT.jar:2.3.1.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:779) [spring-boot-2.3.1.BUILD-SNAPSHOT.jar:2.3.1.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) [spring-boot-2.3.1.BUILD-SNAPSHOT.jar:2.3.1.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) [spring-boot-2.3.1.BUILD-SNAPSHOT.jar:2.3.1.BUILD-SNAPSHOT]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.3.1.BUILD-SNAPSHOT.jar:2.3.1.BUILD-SNAPSHOT]
at com.asma.optaplanner.demo.OptaPlannerDemoApplication.main(OptaPlannerDemoApplication.java:18) [classes/:na]
Caused by: java.lang.NullPointerException: null
at com.asma.optaplanner.demo.model.Admission.toString(Admission.java:120) ~[classes/:na]
at com.asma.optaplanner.demo.OptaPlannerDemoApplication.lambda$2(OptaPlannerDemoApplication.java:40) [classes/:na]
at java.util.ArrayList.forEach(Unknown Source) ~[na:1.8.0_171]
at com.asma.optaplanner.demo.OptaPlannerDemoApplication.lambda$0(OptaPlannerDemoApplication.java:39) [classes/:na]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:795) [spring-boot-2.3.1.BUILD-SNAPSHOT.jar:2.3.1.BUILD-SNAPSHOT]
... 5 common frames omitted
Constraint.drl :
package com.asma.optaplanner.demo;
//list any import classes here.
dialect "java"
import com.asma.optaplanner.demo.model.Admission ;
import com.asma.optaplanner.demo.model.AdmissionDemand ;
import com.asma.optaplanner.demo.model.Room ;
import com.asma.optaplanner.demo.model.Bed ;
import org.optaplanner.core.api.score.buildin.hardsoft.HardSoftScoreHolder;
//declare any global variables here
global HardSoftScoreHolder scoreHolder;
//Hard Constraints
//same gender at the same room in the same night
rule "SameRoomGenderconstraint"
when
$leftAdmission: Admission(
bed != null,
$room : Room,
$leftFrom : DateFromIndex,
$leftTo : DateToIndex,
$leftGender : gender)
$rightAdmission : Admission(
room == $room,
DateToIndex >= $leftFrom ,
DateFromIndex <= $leftTo ,
$rightFrom : DateFromIndex,
$rightTo : dateToIndex,
gender == $leftGender)
then
scoreHolder.addHardConstraintMatch(kcontext,
-10 * (1 + Math.max($leftTo, $rightTo) - Math.max($leftFrom, $rightFrom)));
end
rule "2PatientInTheSameBed"
//include attributes such as "salience" here...
when
$leftAdmission: Admission(
bed != null,
$bed : bed,
$leftFrom : dateFromIndex,
$leftTo : dateToIndex,
$leftId : id)
$rightAdmission: Admission(
bed == $bed,
dateToIndex >= $leftFrom ,
dateFromIndex <= $leftTo ,
$rightFrom : dateFromIndex,
$rightTo : dateToIndex,
id != $leftId)
then
scoreHolder.addHardConstraintMatch(kcontext,
-5 * (1 + Math.max($leftTo, $rightTo) - Math.max($leftFrom, $rightFrom)));
end
In case your #PlanningSolution.PlanningEntity :(#PlanningEntityCollectionProperty / #PlanningEntityProperty) don't change, you should review your drool file, it might be a bit hard to debug .drl files, u might try ConstraintProvider intreface via java, it will be easier to understand solving routine/rule.
Plus, the change is related to "ranged properties" annotated "#ValueRangeProvider", from which you can plan/optimize your solution.
As you're creating a SolverFactory in Spring Boot manually (instead of autowiring it with the optaplanner-spring-boot-starter), do pass the ClassLoader parameter to avoid common issues.
If you copied optaplanner-example's PatientAdmissionSchedule domain classes, note that it has nullable=true on the #PlanningVariable, so it can return unassigned entities. In fact, if you don't have a constraint (typically a medium constraint) to minimize that, all entities are likely to be assigned to null.
we have a 15000000 entry database and executing a SELECT like
SELECT technical_id, attribute_a, attribute_b, attribute_c from test WHERE ( attribute_a_fltr #> (?::character varying[]) OR attribute_a_fltr = '{n/a}') AND ( attribute_b_fltr #> (?::character varying[]) OR attribute_b_fltr = '{n/a}') AND ( attribute_c = ? OR attribute_c = 0)
will slow down dramatically after 9 tries. Here are the results
Iteration: 0 Entries: 593 time :6931
Iteration: 1 Entries: 593 time :7879
Iteration: 2 Entries: 593 time :8721
Iteration: 3 Entries: 593 time :9490
Iteration: 4 Entries: 593 time :10240
Iteration: 5 Entries: 593 time :11016
Iteration: 6 Entries: 593 time :11736
Iteration: 7 Entries: 593 time :12461
Iteration: 8 Entries: 593 time :13168
Iteration: 9 Entries: 593 time :152329
Iteration: 10 Entries: 593 time :290717
Iteration: 11 Entries: 593 time :435933
Iteration: 12 Entries: 593 time :567401
Iteration: 13 Entries: 593 time :695307
Iteration: 14 Entries: 593 time :835853
Here comes the Java code
Connection connection = DriverManager.getConnection("jdbc:postgresql://localhost:5432/test-db", props);
PreparedStatement prepStatement = connection.prepareStatement(FILTER_7);
prepStatement.setString(1,"{AAAAAAIAAAAAIAAAAAAAAgAAAAAgAAAAAAACAAAAACAAAAAAAAIAAgAAIAAgAAAAAgACAAAgACAAAAAAAAIA}");
prepStatement.setString(2,"{gAAAAAAQAAAAAAIAAABAAAAAAAgAAAAAAQAAACAAAAAABAAAAIAAAAAAEAAAAAACAAAAQAAAAgAIAEAAAAAA}");
prepStatement.setInt(3, 1979);
long t0 = System.currentTimeMillis();
long iter = 0;
while (true) {
ResultSet resultSet = prepStatement.executeQuery();
long count = 0;
while(resultSet.next()) {
++count;
}
System.out.println("Iteration: "+iter +" Entries: "+ count + " time :" + (System.currentTimeMillis() - t0));
++iter;
}
I have a system with HTTP POST requests and it runs with Spring 5 (standalone tomcat). In short it looks like this:
client (Apache AB) ----> micro service (java or golang) --> RabbitMQ --> Core(spring + tomcat).
The thing is, when I use my Java (Spring) service, it is ok. AB shows this output:
ab -n 1000 -k -s 2 -c 10 -s 60 -p test2.sh -A 113:113 -T 'application/json' https://127.0.0.1:8449/SecureChat/chat/v1/rest-message/send
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)
Completed 100 requests
...
Completed 1000 requests
Finished 1000 requests
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8449
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256
Document Path: /rest-message/send
Document Length: 39 bytes
Concurrency Level: 10
Time taken for tests: 434.853 seconds
Complete requests: 1000
Failed requests: 0
Keep-Alive requests: 0
Total transferred: 498000 bytes
Total body sent: 393000
HTML transferred: 39000 bytes
Requests per second: 2.30 [#/sec] (mean)
Time per request: 4348.528 [ms] (mean)
Time per request: 434.853 [ms] (mean, across all concurrent
requests)
Transfer rate: 1.12 [Kbytes/sec] received
0.88 kb/s sent
2.00 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 4 14 7.6 17 53
Processing: 1110 4317 437.2 4285 8383
Waiting: 1107 4314 437.2 4282 8377
Total: 1126 4332 436.8 4300 8403
That is through TLS.
But when I try to use my Golang service I get timeout:
Benchmarking 127.0.0.1 (be patient)...apr_pollset_poll: The timeout specified has expired (70007)
Total of 92 requests completed
And this output:
ab -n 100 -k -s 2 -c 10 -s 60 -p test2.sh -T 'application/json' http://127.0.0.1:8089/
This is ApacheBench, Version 2.3 <$Revision: 1807734 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient)...^C
Server Software:
Server Hostname: 127.0.0.1
Server Port: 8089
Document Path: /
Document Length: 39 bytes
Concurrency Level: 10
Time taken for tests: 145.734 seconds
Complete requests: 92
Failed requests: 1
(Connect: 0, Receive: 0, Length: 1, Exceptions: 0)
Keep-Alive requests: 91
Total transferred: 16380 bytes
Total body sent: 32200
HTML transferred: 3549 bytes
Requests per second: 0.63 [#/sec] (mean)
Time per request: 15840.663 [ms] (mean)
Time per request: 1584.066 [ms] (mean, across all concurrent requests)
Transfer rate: 0.11 [Kbytes/sec] received
0.22 kb/s sent
0.33 kb/s total
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.0 0 0
Processing: 1229 1494 1955.9 1262 20000
Waiting: 1229 1291 143.8 1262 2212
Total: 1229 1494 1955.9 1262 20000
That is through plane tcp.
I guess I have some mistakes in my code. I made it in one file
func initAmqp(rabbitUrl string) {
var err error
conn, err = amqp.Dial(rabbitUrl)
failOnError(err, "Failed to connect to RabbitMQ")
}
func main() {
err := gcfg.ReadFileInto(&cfg, "config.gcfg")
if err != nil {
log.Fatal(err);
}
PrintConfig(cfg)
if cfg.Section_rabbit.RabbitUrl != "" {
initAmqp(cfg.Section_rabbit.RabbitUrl);
}
mux := http.NewServeMux();
mux.Handle("/", NewLimitHandler(1000, newTestHandler()))
server := http.Server {
Addr: cfg.Section_basic.Port,
Handler: mux,
ReadTimeout: 20 * time.Second,
WriteTimeout: 20 * time.Second,
}
defer conn.Close();
log.Println(server.ListenAndServe());
}
func NewLimitHandler(maxConns int, handler http.Handler) http.Handler {
h := &limitHandler{
connc: make(chan struct{}, maxConns),
handler: handler,
}
for i := 0; i < maxConns; i++ {
h.connc <- struct{}{}
}
return h
}
func newTestHandler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
handler(w, r);
})
}
func handler(w http.ResponseWriter, r *http.Request) {
if b, err := ioutil.ReadAll(r.Body); err == nil {
fmt.Println("message is ", string(b));
res := publishMessages(string(b))
w.Write([]byte(res))
w.WriteHeader(http.StatusOK)
counter ++;
}else {
w.WriteHeader(http.StatusInternalServerError)
w.Write([]byte("500 - Something bad happened!"))
}
}
func publishMessages(payload string) string {
ch, err := conn.Channel()
failOnError(err, "Failed to open a channel")
q, err = ch.QueueDeclare(
"", // name
false, // durable
false, // delete when unused
true, // exclusive
false, // noWait
nil, // arguments
)
failOnError(err, "Failed to declare a queue")
msgs, err := ch.Consume(
q.Name, // queue
"", // consumer
true, // auto-ack
false, // exclusive
false, // no-local
false, // no-wait
nil, // args
)
failOnError(err, "Failed to register a consumer")
corrId := randomString(32)
log.Println("corrId ", corrId)
err = ch.Publish(
"", // exchange
cfg.Section_rabbit.RabbitQeue, // routing key
false, // mandatory
false, // immediate
amqp.Publishing{
DeliveryMode: amqp.Transient,
ContentType: "application/json",
CorrelationId: corrId,
Body: []byte(payload),
Timestamp: time.Now(),
ReplyTo: q.Name,
})
failOnError(err, "Failed to Publish on RabbitMQ")
defer ch.Close();
result := "";
for d := range msgs {
if corrId == d.CorrelationId {
failOnError(err, "Failed to convert body to integer")
log.Println("result = ", string(d.Body))
return string(d.Body);
}else {
log.Println("waiting for result = ")
}
}
return result;
}
Can someone help?
EDIT
here are my variables
type limitHandler struct {
connc chan struct{}
handler http.Handler
}
var conn *amqp.Connection
var q amqp.Queue
EDIT 2
func (h *limitHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
select {
case <-h.connc:
fmt.Println("ServeHTTP");
h.handler.ServeHTTP(w, req)
h.connc <- struct{}{}
default:
http.Error(w, "503 too busy", http.StatusServiceUnavailable)
}
}
EDIT 3
func failOnError(err error, msg string) {
if err != nil {
log.Fatalf("%s: %s", msg, err)
panic(fmt.Sprintf("%s: %s", msg, err))
}
}