I'm writing an application with Spring boot 2. My method try to generate value until the value will be unique. Each unique generated value is added to the cache. Generally, it should generate value with the first try, but the more application run - more chance that generated value will have duplicates, and it will be required to generate it again.
I want to have the metric, shows the percentile of tryToGenerate values.
Let's say my code looks the following:
public void generateUniqueValue() {
String value;
int tryToGenerate = 0;
do {
tryToGenerate ++
value = generateRandom();
boolean unique = isUniqueValue(value);
} while (!unique);
addGeneratedValueToCache(value);
}
I'm using micrometer, but have no idea what should I start with. Becase in order to calculate percentile I need to store not the single value, but the array of the values
To determine the result of isUniqueValue(..), you're already storing the used values already. You can use that storage to create a proper metric.
Let's assume that you're storing each used value within a list called usedIds. Additionally, let's say you're generating a unique number between 0 and 1000.
In that case, you could write a method like this:
public float getPercentageUsed() {
return (float) usedIds.size() / 1000;
}
Now you can create a Gauge to add this value to the metrics. Let's say that the class that's used to generate unique values is called UniqueValueService, and it's a proper Spring bean. In that case, you could create a Gauge bean like this:
#Bean
public Gauge uniqueValueUsedGauge(MeterRegistry registry, UniqueIdService service) {
return Gauge
.builder("unique-values-used", service::getPercentageUsed)
.baseUnit("%")
.description("Percentage of possible unique values that have been used")
.register(registry);
}
Edit: It seems I've misunderstood your question. If you want to get a histogram of tryToGenerate, to see how many attempts succeed within the first, second or nth attempt, you can use a DistributionSummary. For example:
#Bean
public DistributionSummary summary(MeterRegistry registry) {
return DistributionSummary
.builder("unique-value-attempts")
.sla(1, 5, 10)
.publishPercentileHistogram()
.register(registry);
}
In this example, it will count how many calls succeeded within the 1st, 5th or 10th attempt.
In your service, you can now autowire the DistributionSummary and at the end of the loop, you can use it like this:
do {
tryToGenerate++
value = generateRandom();
boolean unique = isUniqueValue(value);
} while (!unique);
distributionSummary.record(tryToGenerate); // Add this
Now you can use /actuator/metrics/unique-value-attempts.histogram?tag=le:1 to see how many calls succeeded within the first try. The same can be done with the 5th and 10th try.
Related
i have spring boot and java application. at particular time of the day it should execute a method. so in the production server we have kept 4 dyno's and because of 4 dyno's it is executing 4 times per day. so i have used cache and getting the date . based on the date i try to execute the method only once .still it is executing 4 times.
#Scheduled(zone = "${scheduler.zone}", cron = "${scheduler.cron.job}")
public void processScheduled() {
synchronized(this) {
LocalDate localDate = redisTemplate.getValue("DATE");
if (localDate == null || LocalDate.now().isAfter(localDate)) {
log.info("Entered process in SchedulerConfig");
redisTemplate.putValue("DATE", LocalDate.now());
schedulerService.processScheduled();
}
}
}
the above code is written in a config java class.
the schedulerService.processScheduled(); should be triggered only once a day irrespective of no of the dyno's.
can anyone help me on this ?
I am assuming you are using Heroku Dynos, so there are 4 separate instances of your app running in production. Since there are 4 separate instances, your use of synchronized will be of little use here. All 4 of your instances are getting invoked at the same time, so there is a chance that, all of them will get the value of redisTemplate.getValue("DATE") as not null. You need an atomic operation with an centralized entity. Redis does serve as an centralized entity, but redisTemplate.getValue("DATE") and then redisTemplate.putValue("DATE", LocalDate.now()) is not an atomic operation. So it is very possible that 4 instances call redisTemplate.getValue("DATE") and get the real date. Since all 4 of them are not null, they will all update the value and then process your operation.
You should try using redisTemplate.delete(key). It is an atomic operation which returns true if the key is deleted else returns false. So your code may look like this.
#Scheduled(zone = "${scheduler.zone}", cron = "${scheduler.cron.job}")
public void processScheduled() {
boolean isDeleted= redisTemplate.delete("DATE"); // only one will get true, other will get false.
if (isDeleted) {
log.info("Entered process in SchedulerConfig");
schedulerService.processScheduled();
// sleep for some time and then update the date value
redisTemplate.putValue("DATE", LocalDate.now());
}
}
Or you can update the Date value after some time of the invocation with #Scheduled
#Scheduled(zone = "${scheduler.zone}", cron = "${scheduler.flag.update.job}")
public void updateDateFlag() {
redisTemplate.putValue("DATE", LocalDate.now());
}
I am working on automation in jbehave. I have scenario in which i have multiple steps. Now out of that 2 steps are having same input parameter name. For one i am passing value from examples table and for other i am passing value in statement itself even though it is taking value from example table only.
I know i can change the input parameter name in java/jbehave statement but that is not possible because both the statements are used at multiple times and for one scenario i can not break all existing scenarios.
Any clue on this or anyone who faced similar situation should also comment on this.
code snippet
story file
Scenario :
Given I pass value '1'
And I take value from temp file '<value>'
Examples
value |
2
java file :
#Given("I pass value '$value'")
public void fn(#Named("value") int value)
{
------
}
#Given("I take value from temp '$value'")
public void fn2(#Named("value") int value)
{
-----
}
Now the issue here is that for first statement even though i am passing value '1' from statement it is taking value from example table only which is '2'.
I tested a similar scenario on versions 3.9.5 and 4.0.4 of JBehave, and it worked fine on both versions.
This is my scenario:
Scenario: some scenario
Given I pass value '1'
Given I take value from temp file '<value>'
Examples:
|value|
|2|
|3|
and here is my java code:
public class MySteps {
#Given("I pass value '$value'")
public void fn(int value)
{
System.out.println("GIVEN : I pass value = " + value);
}
#Given("I take value from temp file '$value'")
public void fn2( #Named("value") int value)
{
System.out.println("GIVEN : I take value from temp = " + value);
}
}
I have created a project in Eclipse using maven jbehave-simple-archetype.
You can clone this project into Eclipse from this link: https://github.com/kordirko/jb_test
To change a version just edit it in pom.xml => <jbehave.core.version>3.9.5</jbehave.core.version>
========== EDIT ================
Consider setting useDelimiterNamedParameters(true) option in JBehve configuration
http://jbehave.org/reference/stable/parametrised-scenarios.html
They write in documentation, that starting from version 4.0.4 this option is enabled by default.
We can configure JBehave to interpret the name contained between the delimiters as the parameter name and look it up in the parameters provided by the examples table. The default behaviour of parameter lookup is overridden via the ParameterControls:
new MostUsefulConfiguration()
.useParameterControls(new ParameterControls().useDelimiterNamedParameters(true));
In this mode, the step method would look much simplified:
#Given("a stock of $symbol and a $threshold")
public void aStock(String symbol, double threshold) {
// ...
}
Starting from version 4.0, the use of delimiter named parameters is the default behaviour.
I tested this option and it seemed to work both in 3.9.5 and 4.0.4 versions.
This is a test scenario:
Scenario: some scenario
Given I pass value '1'
Given I pass value '<value>'
........
Examples:
|value|
|2|
|3|
and a java method without #Named annotation:
#Given("I pass value '$value'")
public void fn(int value)
{
System.out.println("GIVEN : I pass value = " + value);
}
The first step Given I pass value '1' is taking 1 as a parameter.
The second step Given I pass value '<value>' matches <value> by name with a column named <value> from the example table, and is taking values from this table.
I'm trying to convert a large Map> to some JavaBean. The key of map corresponds to some property of JavaBean, and the value somehow is decoded to property value. So I decided to use some util for that, but don't know what will work. There are some requirements I have to this util (or framework):
all configuration must be in separate files
should be a possibility to map dynamic quantity of keys:
there is a map:
key | value
quan | n
key_1| value_1
key_2| value_2
........ | .............
key_n| value_n
where n - is any number
and the JavaBean has a List of some beans. They have a property. value_1, value_2, ... must be mapped in this property, and in the end there must be so much beans in list, as these keys and values in map.
3.. should be a possibility to set up custom decoder for property mapping, because in most cases the value in map is a List with 1 value, so I need to get the first item of list (if it's not empty).
4.. should be a possibility run some script to execute extraordinary mappings, for example:
there is a map, that is described in 2d point.
and the JavaBean has a property of type HashMap, where value_1 is mapped to Bean1 and some analogous value from input map is mapped to Bean2.
I've tried to use smooks, but when I've started, all these requirements were not clear yet and the smooks was something new, I haven't worked with it until now. So the smooks config doesn't contain the whole business-logic (because of second req.) and looks ugly, I don't like that. I can show the most ugliest fragment for 2d point:
<jb:bean beanId="javaBean" class="com.example.JavaBean" createOnElement="map">
<jb:wiring property="someBeans" beanIdRef="someBeanItems"/>
</jb:bean>
<jb:bean beanId="someBeanItems" class="java.util.ArrayList" createOnElement="map/entry">
<jb:wiring beanIdRef="someBeanItem"/>
</jb:bean>
<jb:bean beanId="someBeanItem" class="com.example.someBeanItem" createOnElement="map/entry">
<condition>map.quan[0]>0</condition>
<jb:expression property="property1">
index = map.quan[0]-1;
value = additionalProperties.property1_List[index];
map.quan[0] = map.quan[0] - 1;
return value;
</jb:expression>
</jb:bean>
Here "property1_List" is builded before executing smooks.
Now I look for something more nice and need your help: maybe you know how to make that better using smooks? Or what another frameworks for mapping can you recommend for my issue?
I am designing a RESTful API.
One service should offer query functionality for multiple key-value pairs. For example, the client can query with one HTTP GET request for different products and the associated quantity.
The client wants to query product 1 with the amount 44 AND product 2 with the amount 55.
I actually don't want my URI to look like this:
/produkt?productId1=1&productquantity1=44&productId2=2&productquantity2=55
because I don't know how many products are queried.
Or like this:
/produkt?product=1,44&product=2,55
because how can the client know that before the comma there is the productId and after the comma the quantity.
Does anyone have other solutions? Or is it not RESTful to offer the possibility to query multiple products with one request? Is it better to offer the possibility to query just one product with the associated quantity and if the client wants to query more products, they should send more requests?
Here is one idea to pass a parameter:
/products?productDetail=[{"key":"key0","value":"key1"},{"key":"key2","value":"key2"},{"key":"key3","value":"key3"}]
where
[{"key":"key0","value":"key1"},{"key":"key2","value":"key2"},{"key":"key3","value":"key3"}]
is a JSON representation of the List<kv> class
class kv {
String key;
String value;
public kv(String key, String value) {
super();
this.key = key;
this.value = value;
}
public String getKey() {
return key;
}
public void setKey(String key) {
this.key = key;
}
public String getValue() {
return value;
}
public void setValue(String value) {
this.value = value;
}
}
so you can easily convert query parameter productDetail in to List<kv> using
new Gson().fromJson(productDetail,kv.class);
than you can easily iterate all elements.
Another suggestion is, if you don't know how many products are queried then use a POST request for this.
I would expand upon your second suggestion a little by adding explicit names for the parts of your product, and using semicolons in place of commas to separate product attributes, since the order does not matter*.
/products?id=1;qty=44&qty=55;id=2
Note how id and qty are switched around for the second product, because the order of attributes does not matter.
* There is a convention to use commas when the order is important, and semicolons when the order is not important.
The most idiomatic HTTP GET way I can think to do it:
/produkt?productId=1&productquantity=44&productId=2&productquantity=55
I ran into a similar situation recently where Clients needed to be able to search by "Product Attributes" which are defined arbitrarily for each Product. Clients also needed to be able to create these links easily and pass them around via email, etc. I didn't want to create a custom query string because I didn't want to create a custom parser and it adds an extra step where the Client could make a mistake. And I didn't want to pass json as it relied on the Client to generate that JSON in some cases.
While the HTTP Spec doesn't take a hard stance on multiple values per parameter, every example I can find indicates that order is preserved. Java example:
String[] productIds = this.request.getParameterValues("productId");
String[] productQuantities = this.request.getParameterValues("productquantity");
productIds[0]; // 1
productQuantities[0]; // 44
productIds[1]; // 2
productQuantities[1]; // 55
I'll leave error and index range checking as an exercise for the reader.
Hy,
Hbase allows a column family to have different qualifiers in different rows. In my case a column family has the following specification
abc[cnt] # where cnt is an integer that can be any positive integer
what I want to achieve is to get all the data from a different column family, only if the value of the described qualifier (in a different column family) matches.
for narrowing the Scan down I just add those two families I need for the query. but that is as far as I could get for now.
I already achieved the same behaviour with a SingleColumnValueFilter, but then the qualifier was known in advance. but for this one the qualifier can be abc1, abc2 ... there would be too many options, thus too many SingleColumnValueFilter's.
Then I tried using the ValueFilter, but this filter only returns those columns that match the value, thus the wrong column family.
Can you think of any way to achieve my goal, querying for a value within a dynamically created qualifier in a column family and returning the contents of the column family and another column family (as specified when creating the Scan)? preferably only querying once.
Thanks in advance for any input.
UPDATE: (for clarification as discussed in the comments)
in a more graphical way, a row may have the following:
colfam1:aaa
colfam1:aab
colfam1:aac
colfam2:abc1
colfam2:abc2
whereas I want to get all of the family colfam1 if any value of colfam2 has e.g. the value x, with regard to the fact that colfam2:abc[cnt] is dynamically created with cnt being any positive integer
I see two approaches for this: client-side filtering or server-side filtering.
Client-side filtering is more straightforward. The Scan adds only the two families "colfam1" and "colfam2". Then, for each Result you get from scanner.next(), you must filter according to the qualifiers in "colfam2".
byte[] queryValue = Bytes.toBytes("x");
Scan scan = new Scan();
scan.addFamily(Bytes.toBytes("colfam1");
scan.addFamily(Bytes.toBytes("colfam2");
ResultScanner scanner = myTable.getScanner(scan);
Result res;
while((res = scanner.next()) != null) {
NavigableMap<byte[],byte[]> colfam2 = res.getFamilyMap(Bytes.toBytes("colfam2"));
boolean foundQueryValue = false;
SearchForQueryValue: while(!colfam2.isEmpty()) {
Entry<byte[], byte[]> cell = colfam2.pollFirstEntry();
if( Bytes.equals(cell.getValue(), queryValue) ) {
foundQueryValue = true;
break SearchForQueryValue;
}
}
if(foundQueryValue) {
NavigableMap<byte[],byte[]> colfam1 = res.getFamilyMap(Bytes.toBytes("colfam1"));
LinkedList<KeyValue> listKV = new LinkedList<KeyValue>();
while(!colfam1.isEmpty()) {
Entry<byte[], byte[]> cell = colfam1.pollFirstEntry();
listKV.add(new KeyValue(res.getRow(), Bytes.toBytes("colfam1"), cell.getKey(), cell.getValue());
}
Result filteredResult = new Result(listKV);
}
}
(This code was not tested)
And then finally filteredResult is what you want. This approach is not elegant and might also give you performance issues if you have a lot of data in those families. If "colfam1" has a lot of data, you don't want to transfer it to the client if it will end up not being used if value "x" is not in a qualifier of "colfam2".
Server-side filtering. This requires you to implement your own Filter class. I believe you cannot use the provided filter types to do this. Implementing your own Filter takes some work, you also need to compile it as a .jar and make it available to all RegionServers. But then, it helps you to avoid sending loads of data of "colfam1" in vain.
It is too much work for me to show you how to custom implement a Filter, so I recommend reading a good book (HBase: The Definitive Guide for example). However, the Filter code will look pretty much like the client-side filtering I showed you, so that's half of the work done.