how to logging with a id in quarkus vertx reactive ?
I want to see processing steps from request to response with the same id in the log.
Although each component is a different thread.
I'm afraid there's no built-in concept of a request ID, and you'll have to generate your IDs yourself. One solution could be that you use an AtomicLong instance to generate a request ID for each request.
Then, to store and access the ID, you basically have two options.
First option: you can store that ID in the request's context by having
#Inject
CurrentVertxRequest request;
(...)
request.getCurrent().put("requestId", id);
And then various components that produce logs can access the ID by
request.getCurrent().get("requestId");
and add that to the log message.
Second option: if you want to avoid the mess of having to append the ID in each log message manually, you can add it to the Mapped Diagnostic Context (MDC). The problem with this is that the MDC context is not propagated by default, so to make sure that each thread sees the ID, you'll need a custom ThreadContextProvider like this:
public class MdcContextProvider implements ThreadContextProvider {
#Override
public ThreadContextSnapshot currentContext(Map<String, String> props) {
Map<String, String> propagate = MDC.getCopyOfContextMap();
return () -> {
Map<String, String> old = MDC.getCopyOfContextMap();
MDC.setContextMap(propagate);
return () -> {
MDC.setContextMap(old);
};
};
}
#Override
public ThreadContextSnapshot clearedContext(Map<String, String> props) {
return () -> {
Map<String, String> old = MDC.getCopyOfContextMap();
MDC.clear();
return () -> {
MDC.setContextMap(old);
};
};
}
#Override
public String getThreadContextType() {
return "MDC";
}
}
and add a META-INF/services/org.eclipse.microprofile.context.spi.ThreadContextProvider file containing the qualified name of that class.
Then, store the request ID in the MDC using
MDC.put("rid", requestId);
And change the formatting string of your logs (for example, the quarkus.log.console.format property) to contain a reference to it, which would be %X{rid}, to make sure that this value is added to each log.
With this option you should probably also make sure that the MDC entry gets cleared when the request processing is done.
So this option is unfortunately much more complicated, but will potentially help keep your code cleaner, because you won't have to append the ID to each log.
I am newbie at Caffeine and Spring boot. I am trying to find a solution to cache a list of objects. the DB transaction takes 3 seconds to load the list, so I am looking to cache the resultset using Caffeine. To cache each item individually, I am following the below approach, but it doesn't seem to work.
public List<Item> getAllItemsOnStartUp() {
allItemsList = repo.getAllItems();
for (Item item : allItemsList) {
staticItem = item;
getItem(item.getItmsId());
}
return allItemsList;
}
#CachePut(value = "allItems", key = "#itmsId")
public Item getItem(BigInteger itmsId) {
return item;
}
#Override
#Cacheable(value = "allItems")
public List<Item> getAllItems() {
allItemsList = repo.getAllItems();
return allItemsList;
}
#Override
#Transactional
#CachePut(value = "allItems", key="#a0.itmsId")
public Item insertItem(Item item) {
Item savedItem = rRepo.save(item);
return savedItem;
}
When the server starts up, getAllItemsOnStartUp() must run and populate cache. the app calls getAllItems() to retrieve the list, it is expected to use cache here but every time the app gets data from DB which takes 3 seconds.
I saw this post today but I will answer anyway. Maybe it can help others.
Spring-boot uses by default CaffeineCache to cache service calls and it's very simple to use it. You just have to annotate your service method using #Cacheable. The example below caches the user permissions in a cache named "sso-users-cache" (because I don't want to call the service to check user's permissions all the time), creating an entry using company name (the system is multi-company) and userId, unless the method returns an error (remember: if you don't have a clausule unless, you can cache even an error of your service).
#Cacheable(cacheNames = ["sso-users-cache"], key = "#company.concat('-sso-user-').concat(#userId)", unless = "#result instanceof T(com.github.michaelbull.result.Err)")
fun fetchActionsByOrganizationAndUser(company: String, userId: String): Result<Set<String>, String> {
val response = Unirest
.get(getUserPermitionsUrl(company = company, userId = userId))
.header("Content-Type", "application/json")
.asString()
.ifFailure {
logger.error("SSO Error: Get user permissions failed: context: $company : userId: $userId")
}
return if(response.isSuccess) {
Ok(serializeUtil.asCollection(response.body, String::class.java).toSet())
} else {
Err("SSO Error: Get user permissions failed: context: $company : userId: $userId\"")
}
}
The parameter cacheNames defines an entry in your cache while the key, will tell the name of the entry on cache (it's used because users have different permissions and they need different entries inside the cache.
The parameter unless tells the cache to not cache if my method returns an error (very important!)
About update the cache informations, it's not necessary. What you need is to invalidate the cache information in case user's permission changes. For exemple, the method below add a new permission for the user and I need to invalidate (clear) the cache I have for the specific user:
#CacheEvict(cacheNames = ["sso-user-actions-cache"], key = "#company.concat('-user-actions-').concat(#userId)")
fun addPermissionToUser(company: String, userId: String, permission: String) {
Unirest
.put(addUserPermitionsUrl(company = company, userId = userId, permission = permission))
.header("Content-Type", "application/json")
.asEmpty()
}
Again: it's important to use the property key because I want to clear only the entry for a specific user!!! Otherwise, you will clean the whole permission cache (for all users)
After the #CacheEvict, the next call to check user's permission, spring will realize that the user doesn't have an entry on cache and will call the service and cache the return. That's it.
Suppose I have some service S that receives requests from client C.
S cannot response immediately due to heavy calculations, C also cannot wait until forever and has his own timeout period.
My idea is to implement the server side as described here:
REST and long running jobs, Farazdagi
In my ServerController I have a thread pool for deferred calculations and a concurrent map to store responses.
private final int NUMBER_OF_WORKERS = 10;
private Map<String, ResponseEntity<MathResponse>> responseMap = new ConcurrentHashMap<>();
private ExecutorService executorService = Executors.newFixedThreadPool(NUMBER_OF_WORKERS);
My /calculate mapping submits jobs to the thread pool and returns with 202 (Accepted) HTTP status and puts redirection link to Location header.
#RequestMapping(value = "/calculate", method = RequestMethod.POST)
public ResponseEntity<String> startWorkflow(#RequestBody MathRequest request, UriComponentsBuilder builder) {
UUID uuid = UUID.randomUUID();
executorService.submit(() -> {
// time-consuming calculations here
ResponseEntity<MathResponse>response = HardMath.execute(request)
responseMap.put(uuid.toString(), response);
});
HttpHeaders headers = new HttpHeaders();
UriComponents uriComponents = builder.path("/wf/queue/{id}").buildAndExpand(uuid.toString());
headers.setLocation(uriComponents.toUri());
return new ResponseEntity<>(headers, HttpStatus.ACCEPTED);
}
In /queue/id mapping I return result if it's in the map:
#RequestMapping(value = "/queue/{id}", method = RequestMethod.GET)
public ResponseEntity<MathResponse> getQueueInfo(#PathVariable("id") String id) {
ResponseEntity<MathResponse> defaultQueueResponse = new ResponseEntity<>(new MathResponse(), HttpStatus.OK);
return responseMap.getOrDefault(id, defaultQueueResponse);
}
I suppose that using such low-level things like ConcurrentHashMap is not a good idea. Are there any options in Spring that I could use instead of reinventing the wheel?
There's also the question of resilience; if the results are local to an instance of S (i.e. in an in-process Map) then if that instance of S crashes or is restarted then the results are lost and C would be forced to resubmit its request(s). If the results cache within S was backed by a resilient store then the results could survive a crash/restart of S.
Spring's caching abstraction with a backing store of <insert storage technology here> could help.
For example I have a bean
public class Order
{
int orderID;
String name;
}
And I have a POST operation
#ApiOperation(value = "Insert a new order", response = Order.class)
#RequestMapping(value = "/addOrder", method = RequestMethod.POST)
#ResponseStatus(HttpStatus.CREATED)
#ResponseBody
public Order addOrder(#Valid #RequestBody Order order)
{
//Set random id here
order.id = 'xxxxx';
Order o = orderService.insertOrder(order);
return o;
}
And in Swagger I have the following:
So my question is, how do I hide id on POST but show ID on GET?
Or should I add a description saying that even if you choose to add an ID it wont do anything and just return my random id? Just like in Kubernetes (uid)
And properties like read-only in #ApiModelProperty will solve anything?
A simple approach is to split your bean in two - one for creating a new object, and another one which extends that for data about an existing object.
e.g.
public class IncompleteOrder {
String name;
}
public class ExistingOrder extends IncompleteOrder {
int id;
}
Then have your POST method take an object of IncompleteOrder and return one of ExistingOrder. I'd also delegrate responsibility for assigning a random order id to the underlying service...
public ExistingOrder addOrder(#Valid #RequestBody IncompleteOrder order) {
ExistingOrder o = orderService.insertOrder(order);
return o;
}
The same thing could be achieved by having two completely separate classes with no inheritance relationship, which would probably be appropriate if there was a significant divergence between the information needed to create a new order from the information which is on an existing order.
An alternative is to ask what the id is actually for - why are your clients getting integer id's for anything? Ideally, if they want any information about the order they should be querying the API for the resource, and to do that they need the URI of the order rather than the integer id. So external services communicating about an order should be passing the URIs back and forth rather than ids. Perhaps you could encourage your clients to communicate with each via the URI you return in the Location header from your POST request? Then you could do away with exposing the id on your response and have a purely symmetric request / response body.
We are trying to implement a special partial update function in Spring 3.2. We are using Spring for the backend and have a simple Javascript frontend. I've not been able to find a straight-forward solution to our requirements, which is The update() function should take in any number of field:values and update the persistence model accordingly.
We have in-line editing for all of our fields, so that when the user edits a field and confirms, an id and the modified field get passed to the controller as json. The controller should be able to take in any number of fields from the client (1 to n) and update only those fields.
e.g., when a user with id==1 edits his displayName, the data posted to the server looks like this:
{"id":"1", "displayName":"jim"}
Currently, we have an incomplete solution in the UserController as outlined below:
#RequestMapping(value = "/{id}", method = RequestMethod.POST )
public #ResponseBody ResponseEntity<User> update(#RequestBody User updateUser) {
dbUser = userRepository.findOne(updateUser.getId());
customObjectMerger(updateUser, dbUser);
userRepository.saveAndFlush(updateUuser);
...
}
The code here works, but has some issues: The #RequestBody creates a new updateUser, fills in the id and the displayName. CustomObjectMerger merges this updateUser with the corresponding dbUser from the database, updating the only fields included in updateUser.
The problem is that Spring populates some fields in updateUser with default values and other auto-generated field values, which, upon merging, overwrites valid data that we have in dbUser. Explicitly declaring that it should ignore these fields is not an option, as we want our update to be able to set these fields as well.
I am looking into some way to have Spring automatically merge ONLY the information explicitly sent into the update() function into the dbUser (without resetting default/auto field values). Is there any simple way to do this?
Update: I've already considered the following option which does almost what I'm asking for, but not quite. The problem is that it takes update data in as #RequestParam and (AFAIK) doesn't do JSON strings:
//load the existing user into the model for injecting into the update function
#ModelAttribute("user")
public User addUser(#RequestParam(required=false) Long id){
if (id != null) return userRepository.findOne(id);
return null;
}
....
//method declaration for using #MethodAttribute to pre-populate the template object
#RequestMapping(value = "/{id}", method = RequestMethod.POST )
public #ResponseBody ResponseEntity<User> update(#ModelAttribute("user") User updateUser){
....
}
I've considered re-writing my customObjectMerger() to work more appropriately with JSON, counting and having it take into consideration only the fields coming in from HttpServletRequest. but even having to use a customObjectMerger() in the first place feels hacky when spring provides almost exactly what I am looking, minus the lacking JSON functionality. If anyone knows of how to get Spring to do this, I'd greatly appreciate it!
I've just run into this same problem. My current solution looks like this. I haven't done much testing yet, but upon initial inspection it looks to be working fairly well.
#Autowired ObjectMapper objectMapper;
#Autowired UserRepository userRepository;
#RequestMapping(value = "/{id}", method = RequestMethod.POST )
public #ResponseBody ResponseEntity<User> update(#PathVariable Long id, HttpServletRequest request) throws IOException
{
User user = userRepository.findOne(id);
User updatedUser = objectMapper.readerForUpdating(user).readValue(request.getReader());
userRepository.saveAndFlush(updatedUser);
return new ResponseEntity<>(updatedUser, HttpStatus.ACCEPTED);
}
The ObjectMapper is a bean of type org.codehaus.jackson.map.ObjectMapper.
Hope this helps someone,
Edit:
Have run into issues with child objects. If a child object receives a property to partially update it will create a fresh object, update that property, and set it. This erases all the other properties on that object. I'll update if I come across a clean solution.
We are using #ModelAttribute to achive what you want to do.
Create a method annotated with#modelattribute which loads a user based on a pathvariable throguh a repository.
create a method #Requestmapping with a param #modelattribute
The point here is that the #modelattribute method is the initializer for the model. Then spring merges the request with this model since we declare it in the #requestmapping method.
This gives you partial update functionality.
Some , or even alot? ;) would argue that this is bad practice anyway since we use our DAOs directly in the controller and do not do this merge in a dedicated service layer. But currently we did not ran into issues because of this aproach.
I build an API that merge view objects with entities before call persiste or merge or update.
It's a first version but I think It's a start.
Just use the annotation UIAttribute in your POJO`S fields then use:
MergerProcessor.merge(pojoUi, pojoDb);
It works with native Attributes and Collection.
git: https://github.com/nfrpaiva/ui-merge
Following approach could be used.
For this scenario, PATCH method would be more appropriate since the entity will be partially updated.
In controller method, take the request body as string.
Convert that String to JSONObject. Then iterate over the keys and update matching variable with the incoming data.
import org.json.JSONObject;
#RequestMapping(value = "/{id}", method = RequestMethod.PATCH )
public ResponseEntity<?> updateUserPartially(#RequestBody String rawJson, #PathVariable long id){
dbUser = userRepository.findOne(id);
JSONObject json = new JSONObject(rawJson);
Iterator<String> it = json.keySet().iterator();
while(it.hasNext()){
String key = it.next();
switch(key){
case "displayName":
dbUser.setDisplayName(json.get(key));
break;
case "....":
....
}
}
userRepository.save(dbUser);
...
}
Downside of this approach is, you have to manually validate the incoming values.
I've a customized and dirty solution employs java.lang.reflect package. My solution worked well for 3 years with no problem.
My method takes 2 arguments, objectFromRequest and objectFromDatabase both have the type Object.
The code simply does:
if(objectFromRequest.getMyValue() == null){
objectFromDatabase.setMyValue(objectFromDatabase.getMyValue); //change nothing
} else {
objectFromDatabase.setMyValue(objectFromRequest.getMyValue); //set the new value
}
A "null" value in a field from request means "don't change it!".
-1 value for a reference column which have name ending with "Id" means "Set it to null".
You can also add many custom modifications for your different scenarios.
public static void partialUpdateFields(Object objectFromRequest, Object objectFromDatabase) {
try {
Method[] methods = objectFromRequest.getClass().getDeclaredMethods();
for (Method method : methods) {
Object newValue = null;
Object oldValue = null;
Method setter = null;
Class valueClass = null;
String methodName = method.getName();
if (methodName.startsWith("get") || methodName.startsWith("is")) {
newValue = method.invoke(objectFromRequest, null);
oldValue = method.invoke(objectFromDatabase, null);
if (newValue != null) {
valueClass = newValue.getClass();
} else if (oldValue != null) {
valueClass = oldValue.getClass();
} else {
continue;
}
if (valueClass == Timestamp.class) {
valueClass = Date.class;
}
if (methodName.startsWith("get")) {
setter = objectFromRequest.getClass().getDeclaredMethod(methodName.replace("get", "set"),
valueClass);
} else {
setter = objectFromRequest.getClass().getDeclaredMethod(methodName.replace("is", "set"),
valueClass);
}
if (newValue == null) {
newValue = oldValue;
}
if (methodName.endsWith("Id")
&& (valueClass == Number.class || valueClass == Integer.class || valueClass == Long.class)
&& newValue.equals(-1)) {
setter.invoke(objectFromDatabase, new Object[] { null });
} else if (methodName.endsWith("Date") && valueClass == Date.class
&& ((Date) newValue).getTime() == 0l) {
setter.invoke(objectFromDatabase, new Object[] { null });
}
else {
setter.invoke(objectFromDatabase, newValue);
}
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
In my DAO class, simcardToUpdate comes from http request:
simcardUpdated = (Simcard) session.get(Simcard.class, simcardToUpdate.getId());
MyUtil.partialUpdateFields(simcardToUpdate, simcardUpdated);
updatedEntities = Integer.parseInt(session.save(simcardUpdated).toString());
The main problem lies in your following code:
#RequestMapping(value = "/{id}", method = RequestMethod.POST )
public #ResponseBody ResponseEntity<User> update(#RequestBody User updateUser) {
dbUser = userRepository.findOne(updateUser.getId());
customObjectMerger(updateUser, dbUser);
userRepository.saveAndFlush(updateUuser);
...
}
In the above functions, you call some of your private functions & classes (userRepository, customObjectMerger, ...), but give no explanation how it works or how those functions look like. So I can only guess:
CustomObjectMerger merges this updateUser with the corresponding
dbUser from the database, updating the only fields included in
updateUser.
Here we don't know what happened in CustomObjectMerger (that's your function, and you don't show it). But from what you describe, I can make a guess: you copy all the properties from updateUser to your object at database. This is absolutely a wrong way, since when Spring map the object, it will fill all the data. And you only want to update some specific properties.
There are 2 options in your case:
1) Sending all the properties (including the unchanged properties) to the server. This may cost a little more bandwidth, but you still keep your way
2) You should set some special values as the default value for the User object (for example, id = -1, age = -1...). Then in customObjectMerger you just set the value that is not -1.
If you feel the 2 above solutions aren't satisfied, consider parsing the json request yourself, and don't bother with Spring object mapping mechanism. Sometimes it just confuse a lot.
Partial updates can be solved by using #SessionAttributes functionality, which are made to do what you did yourself with the customObjectMerger.
Look at my answer here, especially the edits, to get you started:
https://stackoverflow.com/a/14702971/272180
I've done this with a java Map and some reflection magic:
public static Entidade setFieldsByMap(Map<String, Object> dados, Entidade entidade) {
dados.entrySet().stream().
filter(e -> e.getValue() != null).
forEach(e -> {
try {
Method setter = entidade.getClass().
getMethod("set"+ Strings.capitalize(e.getKey()),
Class.forName(e.getValue().getClass().getTypeName()));
setter.invoke(entidade, e.getValue());
} catch (Exception ex) { // a lot of exceptions
throw new WebServiceRuntimeException("ws.reflection.error", ex);
}
});
return entidade;
}
And the entry point:
#Transactional
#PatchMapping("/{id}")
public ResponseEntity<EntityOutput> partialUpdate(#PathVariable String entity,
#PathVariable Long id, #RequestBody Map<String, Object> data) {
// ...
return new ResponseEntity<>(obj, HttpStatus.OK);
}