We Keep Coding

get list from DB and merge the common fields of the list

I have a list as follows:-
List<ScheduleActionDispatchDTO> pendingScheduleActions
=restClientApi.getPendingSchedules();
The above list is coming from DB whose values are like this -
schedule_request_id = 576, user_id = 24,
start_time_utc = '2022-12-16 21:00:00', end_time_utc = '2022-12-17 01:00:00',
request_json = '{"testId": "5", "grade": "A"}'
schedule_request_id = 576, user_id = 24,
start_time_utc = '2022-12-16 21:00:00', end_time_utc = '2022-12-17 01:00:00',
request_json = '{"subjectId": "10", "name": "dictation"}'
schedule_request_id = 577, user_id = 24, start_time_utc = '2022-12-17 21:00:00',
end_time_utc = '2022-12-18 01:00:00', request_json = '{"testId": "5", "grade": "A"}'
Now I want the result to be such that if values of schedule_request_id, user_id, start_time_utc and end_time_utc of any rows are same then merge the values of request_json of those rows together.
So it should become -
schedule_request_id = 576, user_id = 24,
start_time_utc = '2022-12-16 21:00:00', end_time_utc = '2022-12-17 01:00:00',
combinedResult = '[{"testId": "5", "grade": "A"}, {"subjectId": "10", "name": "dictation"}]'
and
schedule_request_id = 577, user_id = 24, start_time_utc = '2022-12-17 21:00:00',
end_time_utc = '2022-12-18 01:00:00', combinedResult = '{"testId": "5", "grade": "A"}'
I tried this -
Map<Long, List<ScheduleActionDispatchDTO>> requestMap = pendingScheduleActions.stream()
.collect(Collectors.groupingBy(
ScheduleActionDispatchDTO::getScheduleRequestId, Collectors.toList()));
for (Map.Entry<Long, List<ScheduleActionDispatchDTO>> entry : requestMap.entrySet()) {
List<ScheduleActionDispatchDTO> sameRequestActions = entry.getValue();
Map<ScheduleActionDispatchPair, ScheduleActionDispatchDTO> schedulePairAction =
sameRequestActions.stream().
collect(Collectors.toMap(
s -> new ScheduleActionDispatchPair(s.getScheduleRequestId(), s.getUserUd(), s.getStartTimeUtc(), s.getEndTimeUtc()),
s -> s));
// iterate and combine but not sure how
}

Well, you could do the following. I assumed you have a record1 of the following form:
public record PendingScheduleAction(
long scheduleRequestId,
int userId,
LocalDateTime startTimeUtc,
LocalDateTime endTimeUtc,
String requestJson
) { }
We could first create a Merger, which is used as grouping-by key. We add an ofPendingScheduleAction method, which is a convenience method to ease the grouping-by. We also add a merge method, which is able to merge two PendingScheduleAction objects.
record Merger(long scheduleRequestId, int userId, LocalDateTime startTimeUtc, LocalDateTime endTimeUtc) {
public static Merger ofPendingScheduleAction(PendingScheduleAction action) {
return new Merger(action.scheduleRequestId(), action.userId(), action.startTimeUtc(), action.endTimeUtc());
}
private static PendingScheduleAction merge(PendingScheduleAction a, PendingScheduleAction b) {
String json = a.requestJson() + '\0' + b.requestJson();
return new PendingScheduleAction(a.scheduleRequestId(), a.userId(), a.startTimeUtc(), a.endTimeUtc(), json);
}
}
Now you can utilize groupingBy, reducing and collectingAndThen in order to achieve the desired result:
list.stream()
.collect(groupingBy(Merger::ofPendingScheduleAction, collectingAndThen(reducing(Merger::merge), optional -> {
var entry = optional.orElseThrow();
String[] jsonLines = entry.requestJson().split("\0");
String json = (jsonLines.length == 1 ? jsonLines[0] : "[" + String.join(",", jsonLines) + "]");
return new PendingScheduleAction(entry.scheduleRequestId(), entry.userId(), entry.startTimeUtc(), entry.endTimeUtc(), json);
})));
What happens here, is that we first define how our PendingScheduleAction instances are grouped. The Javadocs call this the classifier. Now you said "such that if values of schedule_request_id, user_id, start_time_utc and end_time_utc of any rows are same then merge", so we need to have those properties together into an object. The Merger class does this.
Then Collectors.reducing(Merger::merge) takes all objects of a group and merges them using the Merger::merge method. Since reducing returns an Optional, we need to unpack it (with orElseThrow), but we also need to fix the JSON, since we temporarily joined it with a NUL byte.
Then the result is a Map<Merger, PendingScheduleAction>, of which you can call values() to get a list with the merged values.
A few notes:
I used a NUL byte to temporarily separate different JSON parts (which is kinda hacky). This is due to your requirement that you want multiple merged JSON entries to be in a JSON array, but a single entry without an array. I suggest you put all JSON entries into an array, regardless of the number of entries (which may be 1). This is both easier to parse and easier to process.
You could replace all \0s by , in the abovementioned code and the finisher function of the collectingAndThen method would look like this:
var entry = optional.orElseThrow();
return new PendingScheduleAction(entry.scheduleRequestId(), entry.userId(), entry.startTimeUtc(), entry.endTimeUtc(), "[" + entry.requestJson() + "]");
Your start time and end time suggest they are UTC timestamps, but the date strings show a local time. I think your timestamps should look like something like this: 2022-12-16T21:00:00Z.
1 If you don't have access to the records feature, you could easily simulate them with the following:
public class PendingScheduleAction {
private long scheduleRequestId;
private int userId;
private LocalDateTime startTimeUtc;
private LocalDateTime endTimeUtc;
private String requestJson;
// Canonical all-args constructor
public PendingScheduleAction(long scheduleRequestId, int userId, LocalDateTime startTimeUtc, LocalDateTime endTimeUtc, String requestJson) {
this.scheduleRequestId = scheduleRequestId;
this.userId = userId;
this.startTimeUtc = startTimeUtc;
this.endTimeUtc = endTimeUtc;
this.requestJson = requestJson;
}
// Record-style getter (just the name of the field, not starting with 'get'
public long scheduleRequestId() {
return scheduleRequestId;
}
// And the remaining getters
public boolean equals(Object o) {
// A proper equals implementation.
// Don't forget hashCode()
}
}
You could also use Lombok to annotate the class with #AllArgsConstructor, #Getter and #EqualsAndHashCode.

How to get Page as result in Querydsl query with fetch or fetchResults properly?

Hi what i trying to achieve here is, i want to submit Pageable data into QueryDsl query and get the result as Page, how can i do it properly? here is what i do until now :
here is my controller :
#PostMapping("/view-latest-stock-by-product-codes")
public ResponseEntity<RequestResponseDTO<Page<StockAkhirResponseDto>>> findStockByProductCodes(
#RequestBody StockViewByProductCodesDto request) {
Page<StockAkhirResponseDto> stockAkhir = stockService.findByBulkProduct(request);
return ResponseEntity.ok(new RequestResponseDTO<>(PESAN_TAMPIL_BERHASIL, stockAkhir));
}
in my controller i submit StockViewByProductCodesDto which is looked like this :
#Data
public class StockViewByProductCodesDto implements Serializable {
private static final long serialVersionUID = -2530161364843162467L;
#Schema(description = "Kode gudang yang ingin di tampilkan", example = "GBKTJKT1", required = true)
private String warehouseCode;
#Schema(description = "id dari sebuah branch", example = "1", required = true)
private Long branchId;
#Schema(description = "Kode Branch", example = "JKT", required = true)
private String branchCode;
#Schema(description = "Kode Product yang merupakan kode yang di ambil dari master product", example = "[\"MCM-508\",\"TL-101\"]", required = true)
private List<String> productCodes;
#Schema(description = "Size of row per page", example = "15", required = true)
#NotNull
private int size;
#Schema(description = "Page number", example = "1", required = true)
#NotNull
private int page;
#Schema(description = "Sort by", example = "id", required = false)
private String sort;
}
and here is my service :
public Page<StockAkhirResponseDto> findByBulkProduct(StockViewByProductCodesDto request) {
String warehouseCode = request.getWarehouseCode();
Long branchId = request.getBranchId();
String branchCode = request.getBranchCode();
List<String> productCodes = request.getProductCodes();
Set<String> productCodesSet = new HashSet<String>(productCodes);
Pageable pageable = PageUtils.pageableUtils(request);
Page<StockAkhirResponseDto> stockAkhir = iStockQdslRepository.findBulkStockAkhirPage(warehouseCode, branchId, branchCode, productCodesSet, pageable);
return stockAkhir;
}
as you can see, i extract pageable information with PageUtils.pageableUtils(request), here is my pageableUtils function looked like :
public static Pageable pageableUtils(RequestKeyword request) {
int page = 0;
int size = 20;
if (request.getPage() > 0) {
page = request.getPage() - 1;
}
if (request.getSize() > 0) {
size = request.getSize();
}
if (!request.getSort().isEmpty()) {
return PageRequest.of(page, size, Sort.by(request.getSort()).descending());
} else {
return PageRequest.of(page, size);
}
}
after i got the Pageable data, i submit it into my repository, which is looked like this :
public Page<StockAkhirResponseDto> findBulkStockAkhirPage(String warehouseCode, Long branchId, String branchCode,
Set<String> productCodes, Pageable pageable) {
JPQLQuery<Tuple> query = new JPAQuery<>(em);
long offset = pageable.getOffset();
long limit = pageable.getPageSize();
QStock qStock = QStock.stock;
NumberExpression<Integer> totalQty = qStock.qty.sum().intValue();
query = query.select(qStock.productId, qStock.productCode, totalQty).from(qStock)
.where(qStock.warehouseCode.eq(warehouseCode), qStock.productCode.in(productCodes),
qStock.branchCode.eq(branchCode), qStock.branchId.eq(branchId))
.groupBy(qStock.productId, qStock.productCode);
query.limit(limit);
query.offset(offset);
QueryResults<Tuple> result = query.fetchResults();
long total = result.getTotal();
List<Tuple> rows = result.getResults();
List<StockAkhirResponseDto> stockAkhirDto = rows.stream()
.map(t -> new StockAkhirResponseDto(t.get(0, Long.class), t.get(1, String.class), t.get(2, Integer.class)))
.collect(Collectors.toList());
return new PageImpl<>(stockAkhirDto, pageable, total);
}
there is no error in my editor when viewing this my repository and i able to run my project, but when i execute my repository function, i got this error :
"org.hibernate.hql.internal.ast.QuerySyntaxException: expecting CLOSE,
found ',' near line 1, column 38 [select count(distinct
stock.productId, stock.productCode, stock.warehouseId,
stock.warehouseCode, stock.branchCode, stock.branchId)\nfrom
com.bit.microservices.b2b.warehouse.entity.Stock stock\nwhere
stock.warehouseCode = ?1 and stock.productCode in ?2 and
stock.branchCode = ?3 and stock.branchId = ?4]; nested exception is
java.lang.IllegalArgumentException:
org.hibernate.hql.internal.ast.QuerySyntaxException: expecting CLOSE,
found ',' near line 1, column 38 [select count(distinct
stock.productId, stock.productCode, stock.warehouseId,
stock.warehouseCode, stock.branchCode, stock.branchId)\nfrom
com.bit.microservices.b2b.warehouse.entity.Stock stock\nwhere
stock.warehouseCode = ?1 and stock.productCode in ?2 and
stock.branchCode = ?3 and stock.branchId = ?4]"
the problem is here, on this line :
QueryResults<Tuple> result = query.fetchResults();
when i execute that line, it give me that error, i try to get the fetchResult, because i want to get the .getTotal() for the total.
but if i execute the query with .fetch(), it worked fine, like this :
List<StockAkhirResponseDto> stockAkhirDto = query.fetch()
i got my sql result execute correctly, what did i missed here? how do i get Page result correctly?

Your problem could be related with an open QueryDSL issue. The documented issue has to do with the use of fetchCount but I think very likely could be also your case.
Consider the following comment in the mentioned issue:
fetchCount() uses a COUNT function, which is an aggregate function. Your query already has aggregate functions. You cant aggregate aggregate functions, unless a subquery is used (which is not available in JPA). Therefore this use case cannot be supported.
The issue also provides a temporary solution.
Basically, the idea is be able to perform the COUNT by creating a statement over the initial select. AFAIK it is not possible with QueryDsl and this is why in the indicated workarounds they access the underline mechanisms provided by Hibernate.
Perhaps, another thing that you can try to avoid the limitation is to create a database view for your query, the corresponding QueryDsl objects over it, and use these objects to perform the actual computation. I am aware that it is not an ideal solution, but it will bypass this current QueryDsl limitation.

How to add example value for body in swagger 2.0 / OPENAPI 3?

I am following the Swagger 2.0 Annotation docs , but I am unable to add example values for body.
public Response updateUserInfo(#RequestBody(description = "", required = true, content = #Content(mediaType = "application/json")}) final String userInfo)
Is there any equivalent of #ApiImplicitParams or #ExampleProperty for body.
Though the above doc states an example for #ApiResponse :
examples = { #ExampleObject(name = "boo", value = "example",summary = "example of boo", externalValue = "example of external value") }
but this is not working for body.
I have also tried this but the example does not show up:
public Response updateUserInfo(##Parameter(examples = {#ExampleObject(name = "userInfo", value = "some example")})) final String userInfo)

Swagger UI Bad Request URL Generation

I've been using swagger for some time now and I have now ran into a problem where the generated request URL puts my optional parameters in the request. This is not happening on other projects I have that are using the same annotations, swagger version, and swagger maven version.
Here is a code snippet inside my #Controller
private static final String BY_USERNAME = "byUsername";
private static final String BY_NAME = "byName";
private static final String BY_PHONENUMBER = "byPhoneNumber";
private static final String BY_USERTYPENAME = "byUserTypeName";
private static final String BY_ACTIVE = "byActive";
private static final String BY_ACTIVE_AND_AVAILABLE = "byActiveAndAvailable";
#ApiOperation(value = "Read User(s)", notes = "notes removed so post wont be so long", consumes = "application/x-www-form-urlencoded")
#RequestMapping(value = "", method = RequestMethod.GET, consumes = "application/x-www-form-urlencoded")
#ApiImplicitParam(value = "Authorization Token", name = TokenAuthenticationService.HEADER_STRING, dataType = "String", paramType = "header")
#ApiResponses(value = { #ApiResponse(message = "successful read call", code = 200), #ApiResponse(message = "bad parameter provided", code = 400) })
public Page<User> readUser(
#ApiParam(name = "queryType", value = "type of query", allowableValues = BY_USERNAME + "," + BY_NAME + "," + BY_PHONENUMBER + ","
+ BY_USERTYPENAME + "," + BY_ACTIVE + "," + BY_ACTIVE_AND_AVAILABLE) #RequestParam(value = "queryType", required = true) String queryType,
#ApiParam(name = "username", value = "username of user") #RequestParam(value = "username", required = false) String username,
#ApiParam(name = "name", value = "name of user(s)") #RequestParam(value = "name", required = false) String name,
#ApiParam(name = "phoneNumber", value = "phone number of user(s)") #RequestParam(value = "phoneNumber", required = false) String phoneNumber,
#ApiParam(name = "userTypeName", value = "user type name of user(s)") #RequestParam(value = "userTypeName", required = false) String userTypeName,
#ApiParam(name = "active", value = "active status of user(s)") #RequestParam(value = "active", required = false) Boolean active,
#ApiParam(name = "available", value = "available status of user(s)") #RequestParam(value = "available", required = false) Boolean available,
Pageable pageable) {
This is very basic swagger annotations. The problem is the resulting URL in the swagger-ui looks like this:
When I attempt to use it, it puts these inside the REST request.
This of course causes spring to throw exceptions about invalid characters
I tried searching all over the internet, swagger pages, and here on stackoverflow and I could not find any mention of this error. The very weird thing is, if I change the RequestMethod to be POST it works as it should. Seems to be a bug in swagger-ui handling optional parameters. Anyone else ran into this? Is there a fix besides making the request method POST?
EDIT: The "consumes = "application/x-www-form-urlencoded"" was not in my original code. I've been debugging this and adding stuff to try and make it work lol.
Versions used:
springfox-swagger2 version 2.6.1
springfox-swagger-ui version 2.6.1

What version of the swagger-ui are you using?
Since you have enableUrlTemplating(true) in your docket you need to use the appropriate implementation of swagger-ui that supports it.
For Gradle:
compile group: 'io.springfox.ui', name: 'springfox-swagger-ui-rfc6570', version: '1.0.0'
For Maven:
<!-- https://mvnrepository.com/artifact/io.springfox.ui/springfox-swagger-ui-rfc6570 -->
<dependency>
<groupId>io.springfox.ui</groupId>
<artifactId>springfox-swagger-ui-rfc6570</artifactId>
<version>1.0.0</version>
</dependency>
Docket supports url templating (see #21), however it is still in incubation and not officially supported by the swagger 2.0 specification. So the choices you have are:
make enableUrlTemplating(false). In which case everything will work with the exception that if you have multiple paths with different query parameters, only one of them would show up.
e.g. if you have the following urls
http://example.org/customers?firstName=A&lastName=B findCustomersByName
http://example.org/customers?email=a#b.com findCustomersByEmail
Only one of them will show up.
Keep enableUrlTemplating(true) but change the swagger-ui you're using to the one above.

Parsing for a dynamic json

I have json which I am trying to parse using Jackson. JSON looks as -
coupons: {
1: {
title: "Mode von",
description: "",
type: "offer",
code: "-",
expiry: "0000-00-00",
link: ""
},
2: {
title: "Prime 1",
description: "",
type: "offer",
code: "-",
expiry: "0000-00-00",
link: "http://test.com/"
}
}
The number of coupons are not constant here and would vary from response to response.
My dilemma is to create corresponding java class which could hold such object.
I tried with Map as -
public class Coupons {
Map<String, String>coupons = new HashMap<String, String>();
public Map<String, String> getCoupons() {
return coupons;
}
}
But -
System.out.println(coupons.getCoupons().get("type"));
System.out.println(coupons.getCoupons().get("code"));
always get me null. What would be right java class for this json?

your first level of keys are the index numbers 1, 2, 3, etc.
so in order to get the type and code, you have to specificy the key.
you could do this:
var coupons = coupons.getCoupons(); //<--breakpoint to see if this is really populated.
foreach( String key in coupons.Keys ){ //<- pseudo code, iterate over keys
var obj = coupons.get(key);
var type = obj.get("type");
etc..
}
hope this helps you move on