I am using the otel java agent above multimple spring boot microservices.
I want to achieve a simple scenario. Let's suppose that my system is called system A. An arbitrary System B, which is probably not based on otel, makes a request to my system with a generated traceId of it's own attached to request headers. I want somehow to keep logging this traceId inside my distributed logging.
What i have tried so far :
Option 1 -> Write an extension for the java agent and add a customIdGenerator which reads the id produced from System B from a flat source. [traceId should be 32d long, the traceId from System b cannot guarantee that].
public class DemoIdGenerator implements IdGenerator {
private static final AtomicLong traceId = new AtomicLong(0);
private static final AtomicLong spanId = new AtomicLong(0);
#Override
public String generateSpanId() {
return String.format("%016d", spanId.incrementAndGet());
}
// has to be 32d
#Override
public String generateTraceId() {
return String.format("%032d", traceId.incrementAndGet());
}
}
Option 2 -> Write a custom SpanProcessor, add the id as span attribute and propagate the attribute while overriding the onStart(..) SpanProcessor method. - [cannot retrieve attributes since spans are immutable].
public class DemoSpanProcessor implements SpanProcessor {
#Override
public void onStart(Context parentContext, ReadWriteSpan span) {
// cannot retrieve parent context span attributes
span.setAttribute("parent_external_id", "id_from_parent");
}
...
}
Another problem is that when i will try to attach SpanContext to logger pattern, the context does not contain span attributes either. so I can't use them for logging with OpenTelemtryAppender.
final class AutoValue_ImmutableSpanContext extends ImmutableSpanContext {
private final String traceId;
private final String spanId;
private final TraceFlags traceFlags;
private final TraceState traceState;
private final boolean remote;
private final boolean valid;
...
Any idea? Has anyone tried to achieve something similar using Baggage interface maybe?
I found a suitable way using Baggage api propagation. I wrote a custom span processor so i achieved propagating custom attributes through Baggage.current() and populate them on spans while overriding the onStart() method of span processor.
https://github.com/open-telemetry/opentelemetry-java-instrumentation/discussions/7396#discussioncomment-4539912
Related
Looking for the name of the design pattern by which you have a POJO with public final "properties" where the property acts as a holder/wrapper for a certain type of value and contains the getter/setter for that value as well as potentially some additional logic.
This differs from the "Property Container" design pattern where you have a single properties container which contains many types, as this only holds a single value and can thus enjoy the benefits of remaining strongly typed.
An example use:
public class User extends Entity<User> {
private static final Structure<User> STRUCTURE = Structure.of(User.class, User::new)
.addPrimaryKey("user_id", UUID).property((e) -> e.userID).setDefault(() -> UUID()).build()
.addIndex("username", VCHARS_50).property((e) -> e.username).build()
.addIndex("email", VCHARS_255).property((e) -> e.email).build()
.add("password", VCHARS_255).property((e) -> e.passwordHash).build()
.add("privacy_policy_accepted", EPOCH).property((e) -> e.ppAccepted).setDefault(() -> now()).build()
.add("tos_accepted", EPOCH).property((e) -> e.tosAccepted).setDefault(() -> now()).build()
.add("registration_date", EPOCH).property((e) -> e.registrationDate).setDefault(() -> now()).build()
.buildFor(Schema.MASTER);
public final Property<UUID> userID = new Property<>();
public final Property<String> username = new Property<>();
public final Property<String> email = new Property<>();
public final Property<String> passwordHash = new Property<>();
public final Property<Long> ppAccepted = new Property<>();
public final Property<Long> tosAccepted = new Property<>();
public final Property<Long> registrationDate = new Property<>();
public User() {
super(STRUCTURE);
}
public void hashAndSetPassword(String password) {
this.passwordHash.set(Argon2Factory.create(Argon2Types.ARGON2id).hash(3, 102800, 1, password.toCharArray()));
}
public boolean verifyPassword(String attempt) {
return Argon2Factory.create(Argon2Types.ARGON2id).verify(passwordHash.get(), attempt.toCharArray());
}
}
With each entity property using the following:
public class Property<T> {
private T currentValue;
public void set(T newValue) {
this.currentValue = newValue;
}
public T get() {
return this.currentValue;
}
#Override
public boolean equals(Object o) {
return Objects.equals(currentValue, o);
}
#Override
public int hashCode() {
return Objects.hashCode(currentValue);
}
#Override
public String toString() {
return String.valueOf(currentValue);
}
}
We can extend or modify this Properties class and make it do more useful stuff for us, like have it record an original value, provided on creation (pulled from a database) and allow it to self-report on whether the current value of the property differs from what it was originally. Useful for determining which columns need to be updated in a database.
Most notably, this eliminates the need to create getters and setters for every new property because the Property has that functionality already. Moreover, the getters/setters are able to be overridden per-property if additional logic is needed.
I naturally ended up using this design while aiming for a more broad goal of eliminating the use of reflection/annotation processors and other black magic from my web framework. However, I’m having difficulty finding it on the internet so that I might be able to research its potential deficiencies.
This kind of wrapper "variable" is used for Observable properties like StringProperty and such. Its primary use is to hold state and have change listeners, binding in general.
It is fruitfully used, like in JavaFX. And as you mentioned, in entity frameworks. But it definitely is stateful, non-functional, mutable.
A pattern name I cannot find, and I think the gang of 4 would haunt one, if calling this a pattern, other than State.
Credit to #Michael and #Kayaman for answering in the comments: This is not a known design pattern, contrary to my expectations.
In other words, there is not a name by which people generally know to refer to what I’m calling a "Property" nor the design I’m suggesting which assumes public getters and setters are desired and uses public final fields to expose a wrapper which provides them.
This is likely because, as Kayaman pointed out, it’s pretty heavy while being not terribly useful.
TLDR;
Does my DailyRecordDataManager class have a code smell? Is it a 'God Class'? and how can I improve the structure?
Hi,
I'm working on my first project with Spring. It's going to fetch covid-19 data from the Madrid (where I live) government website, organise it by locality, and serve it up through an API.
Here is a sample of the JSON data I'm consuming.
{
"codigo_geometria": "079603",
"municipio_distrito": "Madrid-Retiro",
"tasa_incidencia_acumulada_ultimos_14dias": 23.4668991007149,
"tasa_incidencia_acumulada_total": 1417.23308497532,
"casos_confirmados_totales": 1691,
"casos_confirmados_ultimos_14dias": 28,
"fecha_informe": "2020/07/01 09:00:00"
}
Each JSON object is a a record of cases and the infection rate on a specific date and for a specific municipal district.
After fetching the data the program: parses it, filters it, trims/rounds some properties, maps it by locality, uses it to create an object for each locality (DistrictData), and writes the locality DistrictData objects to a MonoDB instance.
At the moment I have split each of these steps in the process separate classes, as per the single responsibility principle. As can be seen in the linked screenshot:
screenshot of intellij package structure
My problem is I don't know how to link these multiple classes together.
At the moment I have a Manager class which smells a bit like a God Class to me:
#Service
public class DailyRecordDataManager implements DataManager {
private final Logger logger = LoggerFactory.getLogger(DailyRecordDataManager.class);
private final DailyRecordDataCollector<String> dataCollector;
private final DataVerifier<String> dataVerifier;
private final JsonParser<DailyRecord> dataParser;
private final DataFilter<List<DailyRecord>> dataFilter;
private final DataTrimmer<List<DailyRecord>> dataTrimmer;
private final DataSorter<List<DailyRecord>> dataSorter;
private final DataMapper<List<DailyRecord>> dataMapper;
private final DataTransformer dataTransformer;
private final DistrictDataService districtDataService;
public DailyRecordDataManager(DailyRecordDataCollector<String> collector,
DataVerifier<String> verifier,
JsonParser<DailyRecord> parser,
DataFilter<List<DailyRecord>> dataFilter,
DataTrimmer<List<DailyRecord>> dataTrimmer,
DataSorter<List<DailyRecord>> dataSorter,
DataMapper dataMapper,
DataTransformer dataConverter,
DistrictDataService districtDataService) {
this.dataCollector = collector;
this.dataVerifier = verifier;
this.dataParser = parser;
this.dataFilter = dataFilter;
this.dataTrimmer = dataTrimmer;
this.dataSorter = dataSorter;
this.dataMapper = dataMapper;
this.dataTransformer = dataConverter;
this.districtDataService = districtDataService;
}
#Override
public boolean newData() {
String data = dataCollector.collectData();
if (!dataVerifier.verifyData(data)) {
logger.debug("Data is not new.");
return false;
}
List<DailyRecord> parsedData = dataParser.parse(data);
if (parsedData.size() == 0) {
return false;
}
List<DailyRecord> filteredData = dataFilter.filter(parsedData);
List<DailyRecord> trimmedData = dataTrimmer.trim(filteredData);
List<DailyRecord> sortedData = dataSorter.sort(trimmedData);
Map<String, List<DailyRecord>> mappedData = dataMapper.map(sortedData);
List<DistrictData> convertedData = dataTransformer.transform(mappedData);
districtDataService.save(convertedData);
return true;
}
}
I also thought about linking all of the involved classes together in a chain of Injected Dependencies -> so each class has the next class in the process as a dependency and, provided nothing goes wrong with the data, calls that next class in the chain when it's time.
I do also however feel that there must be a design pattern that solves the problem I have!
Thanks!
For anyone who finds this and wonders what I ended up opting for the Pipeline pattern.
It allowed me to easily organise all of the individual classes I was using into one clean workflow. It also made each stage of the process very easy to test. As well as the pipeline class itself!
I highly recommend anyone interested in the patter in Java to check out this article, which I used extensively.
I have a pretty simple interface which manages the update of business proposals, specifically during a nightly batch process each record is submitted here (but it might be used in other scenarios).
This interface is used inside an EJB 2.0 Bean, which fetches records and "cycles" them.
Beware names are translated from Italian to English so pardon possible errors. I also simplified some concepts.
public interface ProposalUpdateService {
void updateProposal(final ProposalFirstType proposal);
void updateProposal(final ProposalSecondType proposal);
}
The implementation of this interface has quite a lot of dependencies:
public class ProposalUpdateDefaultService implements ProposalUpdateService {
private final ComplexService complexService;
private final OtherComplexService otherComplexService;
private final ProposalStep<Proposal> stepOne;
private final ProposalStep<Proposal> stepTwo;
private final ProposalStep<ProposalTypeTwo> stepThree;
private final ProposalStep<Proposal> stepFour;
public ProposalUpdateDefaultService(
final ComplexService complexService,
final OtherComplexService otherComplexService,
final YetAnotherComplexService yetAnotherComplexService,
final SimpleService simpleService,
final OtherSimpleService otherSimpleService,
final YetAnotherSimpleService yetAnotherSimpleService,
final Converter<ProposalTypeOne, ComplexServiceType> converterProposalTypeOne,
final Converter<ProposalTypeTwo, OtherComplexServiceType> converterProposalTypeTwo) {
this.complexService = complexService;
this.otherComplexService = otherComplexService;
stepOne = new StepOne(yetAnotherComplexService);
stepTwo =
new StepTwo(
complexService,
otherComplexService,
yetAnotherComplexService,
converterProposalTypeOne,
converterProposalTypeTwo);
stepThree =
new StepThree(
simpleService,
otherSimpleService,
yetAnotherSimpleService);
stepFour = new StepFour();
}
...
As you can see this class encapsulate the update of a Proposal object, and this process is splitted in four phases, each representing a single concept (such as, "should this proposal be expired?" or "should I advance its state?"). Those four phases may be arranged differently between different types of Proposal.
Here is the highly simplified implementation of those two updateProposal methods:
#Override
public void updateProposal(final ProposalTypeOne proposal) {
stepOne.process(proposal);
stepTwo.process(proposal);
if (...) {
stepFour.process(proposal);
}
}
#Override
public void updateProposal(final ProposalTypeTwo proposal) {
stepOne.process(proposal);
stepTwo.process(proposal);
stepThree.process(proposal);
stepFour.process(proposal);
}
The two private fields
private final ComplexService complexService;
private final OtherComplexService otherComplexService;
are used for helper private methods.
As you can see this class just organize and delegate work, however, it does depend on too many other classes. The same could be said for certain ProposalStep(s).
The *Service(s) are used inside each step to retrieve details from the database, to update dependent entries, etc.
Would you accept this number of dependencies?
How would you refactor to simplify?
I've read about the Facade Service concept as a way to reduce dependencies, and how I should group cluster of dependencies together, but here I don't really understand what to do.
I may group the Converter(s) and the Service(s) which uses them, but they'll be too many anyway.
Let me know if other details are needed.
The issue I can see is ProposalUpdateDefaultService doing too many things and know too much. It accepts a lot of services, creates steps and executes the steps instead it should only accept a single parameter object and update without knowing what are the steps.
First I would try to reduce the parameters from the constructor ProposalUpdateDefaultService by creating a separate class which will contain the services and converters.
public class ServicesAndConverters {
ComplexService complexService;
OtherComplexService otherComplexService
//...
}
In that way the code can be much cleaner
public class ProposalUpdateDefaultService implements ProposalUpdateService {
ServiceAndConverters serviceAndConvert;
public ProposalUpdateDefaultService(final ServiceAndConverters serviceAndConverters) {
this.serviceAndConvert = serviceAndConverters; //maybe group them in two different class??
}
}
Now the second issue I can see to create steps in the ProposalUpdateDefaultService itself. This should be responsibility of different class. Something like below
public class ProposalUpdateDefaultService implements ProposalUpdateService {
ServiceAndConverters serviceAndConvert;
StepCreator stepCreator = new StepCreator();
public ProposalUpdateDefaultService(final ServiceAndConverters serviceAndConverters) {
this.serviceAndConvert = serviceAndConverters;
stepCreator.createSteps(this.serviceAndConverter);
}
}
And the StepCreator class should look like this
public class StepCreator implements ProposalUpdateService {
private final ProposalStep<Proposal> stepOne;
private final ProposalStep<Proposal> stepTwo;
private final ProposalStep<ProposalTypeTwo> stepThree;
private final ProposalStep<Proposal> stepFour;
public void createSteps(ServiceAndConverters s) {
// do the step processing here
}
}
Now ProposalUpdateDefaultService can execute the steps without knowing what is the steps and which service need to execute
#Override
public void updateProposal(final ProposalTypeOne proposal) {
stepCreator.getStepOne().process(proposal);
stepCreator.getStepTwo().process(proposal);
if (...) {
stepCreator.getStepFour().process(proposal);
}
}
The solution that I found more convenient is just removing the ProposalUpdateService abstraction, and letting the EJB Bean manage the various steps.
This abstraction layer was unnecessary as of now, and each step is still usable individually. Both ProposalUpdateService method invocations become private methods in the EJB Bean.
In project I am working on we have a bunch of commonly used helpers. Consider the following example:
public class ServiceHelper {
public HttpServletRequest() getRequest() { ... }
public Model getModel() { ... }
public UserCache getUserCache() { ... }
public ComponentContainer getComponentContainer() { ... }
}
Imagine this helper is being used across the whole application by every web service we have. Then, in order to test these services I need to mock it. Each time. But what if I create a factory of some kind instead, something like:
public class ServiceHelperMockStore {
public static ServiceHelper create() {
return init();
}
public static ServiceHelper create(final Model model) {
final ServiceHelper helper = init();
when(helper.getModel()).thenReturn(model);
return helper;
}
private static ServiceHelper init() {
final ServiceHelper helper = mock(ServiceHelper.class);
final HttpServletRequest request = mock(HttpServletRequest.class);
final Model model = mock(Model.class);
final UserCache userCache = mock(UserCache.class);
final ComponentContainer container = mock(ComponentContainer.class);
final BusinessRules businessRules= mock(BusinessRules.class);
final ModelTransformer modelTransformer = mock(ModelTransformer.class);
when(helper.getRequest()).thenReturn(request);
when(helper.getModel()).thenReturn(model);
when(helper.getUserCache()).thenReturn(userCache);
when(helper.getComponentContainer()).thenReturn(container);
when(container.getComponent(BusinessRules.class)).thenReturn(businessRules);
when(componentContainer.getComponent(ModelTransformer.class)).thenReturn(modelTransformer);
return helper;
}
}
This factory nicely fit my purposes and oftentimes I can completely avoid using 'mock' and 'when' in the actual test suites. Instead, I can do the following:
#RunWith(MockitoJUnitRunner.Silent.class)
public class ModelServiceTest {
private final Model model = new Model();
private final ServiceHelper serviceHelper = ServiceHelperMockStore.create(model);
private final BusinessRules businessRules = serviceHelper.getComponentContainer().getComponent(BusinessRules.class);
private final ModelType modelType1 = new ModelType();
private final ModelType modelType2 = new ModelType();
private final ModelService modelService = new ModelService(serviceHelper);
#Before
public void setUp() {
modelType1.setItemId("item1");
modelType2.setItemId("item2");
model.setTypes(modelType1, modelType2);
when(businessRules.get("type")).thenReturn(modelType1);
}
...tests...
}
So instead of creating a lot of mocks in the ModelServiceTest, I can just access the predefined ones, like:
BusinessRules businessRules = serviceHelper.getComponentContainer().getComponent(BusinessRules.class);
and this even reflect my helper's API. Also, I can provide my own mock or stub passing parameters to my factory method or using some different approach.
The only problem I have is UnnecessaryStubbingException being thrown by Mockito as normally I don't use all those stubbings I've created per each test file. So I have to use MockitoJUnitRunner.Silent runner to silent the error and according to the mockito api docs it is not recommended.
So I am seeking for an advice what kind of approach must be chosen in this case. Am I doing it right or there is some other way? Or, maybe, using such kind of factories is a bad style of programming in relation to unit tests as it hides some initialization and makes happening things less evident so I must do just a plain copy of my code between test suits?
The fact that you need this identical complex mock configuration at different places shows that your code violates the Law of Demeter (Don't talk to strangers).
A unit should only get dependencies it actually interacts with (other than only to getting another dependency from it).
So instead of creating a lot of mocks in the ModelServiceTest, I can just access the predefined ones,
You Unittests are not only verification of correct behavior but also minimal examples how to use the CUT (Code under test).
The configuration of the CUTs dependencies is an essential part of that example and should be easily accessible to the reader of the tests.
I'd strongly discourage from "factories for mocks" especially it they were moved to other classes (in the test folder).
We are using Guice in our project for DI. Currently we have some configurations(properties) that we load a t server startup from a file. These are then bound to all the components & used for all the requests.
But now, we have multiple property files & load them at startup. These configurations can be different per REST(Jersey) request as they depend on the input.
So, we need to bind these configurations dynamically for each request. I looked into Guice API for #RequestScoped, but did not find anything specificallyu helpful.
There are few questions similar to this, but no luck yet. Can you please help me with this.
I'm providing 2 ways of doing this and both are request scoped.
Using HttpServletRequest, for classes where you can Inject request object.
Using ThreadLocal, Generic way. It can be used in any class.
(NOTE: This method wouldn't work if your creating new threads in your code and want to access the value. In which case you'll have to pass the values through Objects to those threads)
I meant something like this:
public class RequestFilter implements ContainerRequestFilter {
#Context
private HttpServletRequest request;
#Override
public void filter(ContainerRequestContext requestContext) throws IOException {
List listOfConfig = //load Config;
request.setAttribute("LOADED_CONFIG",listOfConfig);
// If you want to access this value at some place where Request object cannot be injected (like in service layers, etc.) Then use below ThreadLocals.
ThreadLocalWrapper.getInstance().get().add("adbc"); // In general add your config here, instead of abdc.
}
}
My ThreadLocalWrapper looks like this:
public class ThreadLocalWrapper {
private static ThreadLocal<List<String>> listOfStringLocals; // You can modify this to a list of Object or an Object by itself.
public static synchronized ThreadLocal<List<String>> getInstance() {
if (listOfStringLocals == null) {
listOfStringLocals = new ThreadLocal<List<String>>() {
#Override
protected List<String> initialValue() {
return new ArrayList<String>();
}
};
}
return listOfStringLocals;
}
}
To Access the value:
In Controller - Inject HttpServletRequest Object and do getAttribute() to get the value. Since HttpServletRequest Object is requestScoped, you can set the loaded config. into this and access it in your controller's using request Object again.
In Any other part of the code - If HttpServletRequest is not available then you can always use the ThreadLocal example shown. To access this value.
public class GuiceTransactionImpl implements GuiceTransaction {
private String value = "";
public GuiceTransactionImpl(String text) {
value = text;
}
#Override
public String returnSuccess() {
return value + " Thread Local Value " + ThreadLocalWrapper.getInstance().get();
}
}