We Keep Coding

Disable Kafka connection in SpringBoot tests

I'm working on a springboot project following a microservice architecture and I use Kafka as an event bus to exchange data between some of them. I also have Junit tests which test some part of my application which doesn't require the bus and others that require it by using an embedded Kafka broker.
The problem I have is when I launch all my tests, they take so much time and they fail because each of then is trying to connect to the embedded Kafka broker (connection not available) whereas they don't need Kafka bus in order to achieve their task.
Is it possible to disable the loading of Kafka components for these tests and only allow them for the ones that require it ?

This is how I usually write my JUnit tester classes, that usually wont connect to KAFKA Brokers for each test.
Mocking the REST API, if your KAFKA Client (Producer/Consumer) integrated with a REST API
public class MyMockedRESTAPI {
public MyMockedRESTAPI() {
}
public APIResponseWrapper apiResponseWrapper(parameters..) throws RestClientException {
if (throwException) {
throw new RestClientException(....);
}
return new APIResponseWrapper();
}
}
A factory class to generate an incoming KAFKA Event and REST API request and response wrappers
public class mockFactory {
private static final Gson gson = new Gson();
public static KAKFAEvent generateKAFKAEvent() {
KAKFAEvent kafkaEvent = new KAKFAEvent();
kafkaEvent.set...
kafkaEvent.set...
kafkaEvent.set...
return KAKFAEvent;
}
public static ResponseEntity<APIResponse> createAPIResponse() {
APIResponse response = new APIResponse();
return new ResponseEntity<>(response, HttpStatus.OK);
}
}
A Test Runner Class
#RunWith(SpringJUnit4ClassRunner.class)
public class KAFKAJUnitTest {
Your assertion should be declared here
}
You can also refer : https://www.baeldung.com/spring-boot-kafka-testing

a good practice would be to avoid sending messages to Kafka while testing code in your isolated microservice scope. but when you need to make an integration test ( many microservices in the same time ) sometimes you need to activate Kafka messages.
So my purpose is :
1- Activate/Deactivate loding Kafka configuration as required
#ConditionalOnProperty(prefix = "my.kafka.consumer", value = "enabled", havingValue = "true", matchIfMissing = false)
#Configuration
public class KafkaConsumerConfiguration {
...
}
#ConditionalOnProperty(prefix = "my.kafka.producer", value = "enabled", havingValue = "true", matchIfMissing = false)
#Configuration
public class KafkaProducerConfiguration {
...
}
and then u will be able to activate/deactivate loading consumer and producer as you need...
Examples :
#SpringBootApplication
#Import(KafkaConsumerConfiguration.class)
public class MyMicroservice_1 {
public static void main(String[] args) {
SpringApplication.run(MyMicroservice_1.class, args);
}
}
or
#SpringBootApplication
#Import(KafkaProducerConfiguration.class)
public class MyMicroservice_2 {
public static void main(String[] args) {
SpringApplication.run(MyMicroservice_2.class, args);
}
}
or maybe a microservice that need both of configurations
#SpringBootApplication
#Import(value = { KafkaProducerConfiguration.class, KafkaConsumerConfiguration.class })
public class MyMicroservice_3 {
public static void main(String[] args) {
SpringApplication.run(MyMicroservice_3.class, args);
}
}
2 - You need also to make sending messages depending on the current spring profile. To do that u can override the send method of the Kafka template object:
#ConditionalOnProperty(prefix = "my.kafka.producer", value = "enabled", havingValue = "true", matchIfMissing = false)
#Configuration
public class KafkaProducerConfiguration {
...
#Resource
Environment environment;
#Bean
public KafkaTemplate<String, String> kafkaTemplate() {
return new KafkaTemplate<>(producerFactory()) {
#Override
protected ListenableFuture<SendResult<String, String>> doSend(ProducerRecord<String, String> producerRecord) {
if (Arrays.asList(environment.getActiveProfiles()).contains("test")) {
return null;
}
return super.doSend(producerRecord);
}
};
}
#Bean
public ProducerFactory<String, String> producerFactory() {
Map<String, Object> props = new HashMap<>();
...
return new DefaultKafkaProducerFactory<>(props);
}
}

Spring AMQP detect similar payload types to use in #RabbitHandler

I used to have a #RabbitListener that works fine like this:
#Component
public class AppointmentMessageListener {
#RabbitListener(queues = "${rabbitmq.immediateQueueName}")
public void receiveImmediateAppointmentMessage(AppointmentMessage appointmentMessage) {
// Do something
}
}
Now I want to have a different type of message on the same queue, and I tried it like the docs said:
#Component
#RabbitListener(queues = "${rabbitmq.immediateQueueName}")
public class AppointmentMessageListener {
#RabbitHandler
public void receiveImmediateAppointmentMessage(AppointmentMessage appointmentMessage) {
// Do something
}
#RabbitHandler
public void receiveImmediateAppointmentMessage(AppointmentDeleteMessage appointmentDeleteMessage) {
// Do something else
}
}
This doesn't work, and I get org.springframework.amqp.AmqpException: No method found for class java.util.LinkedHashMap.
The JSON looks like below, and the difference between the 2 types is only in the object structure. I don't control the structure of the message. Is there any way I can detect the correct type to use in my multimethod listener?
{
"key":"event-message-resource.immediate",
"creationTimestamp":1643804135376,
"object": {
// here is the object I am interested in
}
}
I use the following configuration besides the exchange, queue and routing key declarables.:
#Bean
public Jackson2JsonMessageConverter jsonMessageConverter() {
ObjectMapper objectMapper = Jackson2ObjectMapperBuilder.json().build();
return new Jackson2JsonMessageConverter(objectMapper);
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(SimpleRabbitListenerContainerFactoryConfigurer configurer,
ConnectionFactory connectionFactory,
AuthenticationRabbitListenerAdvice rabbitListenerAdvice) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
factory.setAdviceChain(combineAdvice(factory.getAdviceChain(), rabbitListenerAdvice));
return factory;
}

The framework can't infer the target type when using #RabbitHandler because it uses the type to select the method. The JSON has to be converted before the method selection is performed.
See https://docs.spring.io/spring-amqp/docs/current/reference/html/#Jackson2JsonMessageConverter-from-message
You need to use something in the message to determine which type to create.

Based on the answer from the revered Gary Russell and his answer on his post here, I managed to get it to work by subclassing the ClassMapper and deciding on the type based on the routing key.
#Component
#RequiredArgsConstructor
public class CustomClassMapper extends DefaultJackson2JavaTypeMapper {
#Value("${rabbitmq.deleteRoutingKey}")
private final String deleteRoutingKey;
#NonNull
#Override
public Class<?> toClass(MessageProperties properties) {
String routingKey = properties.getReceivedRoutingKey();
if (deleteRoutingKey.equals(routingKey)) {
return AppointmentDeleteMessage.class;
}
return AppointmentMessage.class;
}
}
Then I set the classMapper in the messageConverter.
#Bean
public Jackson2JsonMessageConverter jsonMessageConverter(CustomClassMapper customClassMapper) {
ObjectMapper objectMapper = Jackson2ObjectMapperBuilder.json().build();
Jackson2JsonMessageConverter messageConverter = new Jackson2JsonMessageConverter(objectMapper);
messageConverter.setClassMapper(customClassMapper);
return messageConverter;
}

How can I do integration flow to be invoked when I pass File dir to messageChannel and InboundFileAdapter reading files from it?

I have integration flow that reads files from specific dir, transform it to pojo and save in list.
Config class:
#Configuration
#ComponentScan
#EnableIntegration
#IntegrationComponentScan
public class IntegrationConfig {
#Bean
public MessageChannel fileChannel(){
return new DirectChannel();
}
#Bean
public MessageSource<File> fileMessageSource(){
FileReadingMessageSource readingMessageSource = new FileReadingMessageSource();
CompositeFileListFilter<File> compositeFileListFilter= new CompositeFileListFilter<>();
compositeFileListFilter.addFilter(new SimplePatternFileListFilter("*.csv"));
compositeFileListFilter.addFilter(new AcceptOnceFileListFilter<>());
readingMessageSource.setFilter(compositeFileListFilter);
readingMessageSource.setDirectory(new File("myFiles"));
return readingMessageSource;
}
#Bean
public CSVToOrderTransformer csvToOrderTransformer(){
return new CSVToOrderTransformer();
}
#Bean
public IntegrationFlow convert(){
return IntegrationFlows.from(fileMessageSource(),source -> source.poller(Pollers.fixedDelay(500)))
.channel(fileChannel())
.transform(csvToOrderTransformer())
.handle("loggerOrderList","processOrders")
.channel(MessageChannels.queue())
.get();
}
}
Transformer:
public class CSVToOrderTransformer {
#Transformer
public List<Order> transform(File file){
List<Order> orders = new ArrayList<>();
Pattern pattern = Pattern.compile("(?m)^(\\d*);(WAITING_FOR_PAYMENT|PAYMENT_COMPLETED);(\\d*)$");
Matcher matcher = null;
try {
matcher = pattern.matcher(new String(Files.readAllBytes(file.toPath()), StandardCharsets.UTF_8));
} catch (IOException e) {
e.printStackTrace();
}
while (!matcher.hitEnd()){
if(matcher.find()){
Order order = new Order();
order.setOrderId(Integer.parseInt(matcher.group(1)));
order.setOrderState(matcher.group(2).equals("WAITING_FOR_PAYMENT")? OrderState.WAITING_FOR_PAYMENT:OrderState.PAYMENT_COMPLETED);
order.setOrderCost(Integer.parseInt(matcher.group(3)));
orders.add(order);
}
}
return orders;
}
}
OrderState enum :
public enum OrderState {
CANCELED,
WAITING_FOR_PAYMENT,
PAYMENT_COMPLETED
}
Order :
public class Order {
private int orderId;
private OrderState orderState;
private int orderCost;
}
LoggerOrderList service:
#Service
public class LoggerOrderList {
private static final Logger LOGGER = LogManager.getLogger(LoggerOrderList.class);
public List<Order> processOrders(List<Order> orderList){
orderList.forEach(LOGGER::info);
return orderList;
}
}
1)How can I do that flow starts when I pass invoke gateway method?
2)How can I read passed message in inbound-channel-adapter(in my case is FileReadingMessageSource)?

The FileReadingMessageSource is based on the polling the provided in the configuration directory. This is the beginning of the flow and it cannot be used in the middle of some logic.
You didn’t explain what is your gateway is, but probably you would like to have similar logic to get content if the dir passed as a payload of sent message. However such a logic doesn’t look like a fit for that message source anyway. It’s goal is to poll the dir all the time for new content. If you want something similar for several dirs, you may consider to have dynamically registered flows for provided dirs: https://docs.spring.io/spring-integration/docs/current/reference/html/dsl.html#java-dsl-runtime-flows.
Otherwise you need to consider to have a plain services activator which would call listFiles() on the provided dir. just because without “wait for new content “ feature it does not make sense to abuse FileReeadingMessageSource

How do I test a Service with calls to an external API in Spring

Okay, so I am pretty new to testing and Spring Boot in general, so please correct me if I am doing something completely wrong here in the first place.
As a project my team and I are making a Web Application using Spring Boot where we are making calls to the Microsoft Graph API in some of our services. See this service for cancelling an event in a user's calendar:
import com.microsoft.graph.authentication.IAuthenticationProvider;
import com.microsoft.graph.models.extensions.IGraphServiceClient;
import com.microsoft.graph.requests.extensions.GraphServiceClient;
import org.springframework.stereotype.Service;
#Service
public class CancelEventService {
public void cancelEvent(final String token, final String id) {
IAuthenticationProvider authenticationProvider = iHttpRequest -> iHttpRequest.addHeader("Authorization", "Bearer " + token);
IGraphServiceClient graphServiceClient = GraphServiceClient.builder().authenticationProvider(authenticationProvider).buildClient();
graphServiceClient.me().events(id)
.buildRequest()
.delete();
}
}
This is working great, but I have been struggling for a couple of days with how to write unit tests for this. The way I see it, I want to either make mocks of the GraphServiceClient, or use a tool like WireMock to make the request go to a mockserver and return some values I configure.
I've tried doing both, but I can't make mocks of GraphServiceClient because it is not a Bean in my project, so I can't figure out how I should proceed to make an implementation I can autowire in to my Service.
When it comes to WireMock I am not even sure I understand if it is capable of doing what I want to, and if it is, I sure haven't been able to configure it correctly (note that we are using JUnit 5 for this project). A simple example where you make a GET-request to Google.com and return "Hello" via WireMock would suffice to get me going here.
Any concrete examples of what I should do, or even just a nod in the right direction would be well appreciated.

Well, I cannot assure you that it will work but it will give you a better landscape of the situation:
1) So first, we need to make a slight change on your service.
Need to extract IGraphServiceClient from the method so we can mock it later, see:
#Service
public class CancelEventService {
private IGraphServiceClient graphServiceClient;
#Autowired
public CancelEventService(IGraphServiceClient graphServiceClient){
this.graphServiceClient = graphServiceClient;
}
public void cancelEvent(final String token, final String id) {
IAuthenticationProvider authenticationProvider = iHttpRequest -> iHttpRequest.addHeader("Authorization", "Bearer " + token);
graphServiceClient = GraphServiceClient.builder().authenticationProvider(authenticationProvider).buildClient();
graphServiceClient.me().events(id)
.buildRequest()
.delete();
}
}
2) The test would look like this:
(Notice that all we are using here is included in spring boot test module, so you shouldn't need to add anything to the project dependencies)
#RunWith(SpringRunner.class)
public class CancelEventServiceTest {
private IGraphServiceClient graphServiceClientMock;
private CancelEventService serviceToBeTested;
#Before
public void setUp(){
graphServiceClientMock = Mockito.mock(IGraphServiceClient.class, RETURNS_DEEP_STUBS);
serviceToBeTested = new CancelEventService(graphServiceClientMock);
}
#Test
public void test_1() {
serviceToBeTested.cancelEvent("token", "id");
verify(graphServiceClientMock, times(1)).me().events("id").buildRequest()
.delete();
}
}
Hope it helps!

As a follow up on this, we found a solution to the problem by creating a class
GraphServiceClientImpl with a method that returns an instantiated GraphServiceClient.
#Component
public class GraphServiceClientImpl {
public IGraphServiceClient instantiateGraphServiceClient(final String token) {
IAuthenticationProvider authenticationProvider = iHttpRequest -> iHttpRequest.addHeader("Authorization", "Bearer " + token);
return GraphServiceClient.builder().authenticationProvider(authenticationProvider).buildClient();
}
}
#Service
public class CancelEventService{
private GraphServiceClientImpl graphServiceClientImpl;
public CancelEventService(final GraphServiceClientImpl graphServiceClientImpl) {
this.graphServiceClientImpl = graphServiceClientImpl;
}
public void cancelEvent(final String token, final String id) {
IGraphServiceClient graphServiceClient = graphServiceClientImpl.instantiateGraphServiceClient(token);
graphServiceClient
.me()
.events(id)
.buildRequest()
.delete();
}
}
Then, our test:
#ExtendWith(MockitoExtension.class)
class CancelEventServiceTest {
#Mock
private GraphServiceClientImpl graphServiceClientImpl;
#InjectMocks
private CancelEventService cancelEventService;
#BeforeEach
void setUp() {
MockitoAnnotations.initMocks(this);
}
#Test
void cancelEvent_Successful() {
//given
IGraphServiceClient serviceClientMock = mock(IGraphServiceClient.class, RETURNS_DEEP_STUBS);
given(graphServiceClientImpl.instantiateGraphServiceClient(anyString())).willReturn(serviceClientMock);
//when
cancelEventService.cancelBooking("Token", "1");
//then
verify(serviceClientMock, times(1)).me();
}
}
Probably not the optimal solution, but it works. Any other takes on this would be welcome!

how to make spring boot mvc understand observables?

i want to return Observable from spring's mvc controller. it works with Single:
#GetMapping Object a() {return Single.just(1);}
as expected i get 1 when i query the server. but when i do the same with Observable:
#GetMapping Object a() {return Observable.just(1);}
the answer i got is {}. spring-mvc doesn't subscribe to returned Observable but simply serializes it to json. can spring-mvc understand Observable out of the box and i just messed up with some configuration? or do i have to register my custom handlers or install some plugins?

You can use Spring MVC Reactive (but it's not currently released as final version). It works with Reactor AND RxJava. You'll be able to write this sort of controller :
#Test
class ExampleController {
#RequestMapping("/hello")
public Single<String> hello() { return Single.just("world"); }
}
or You can write your own class adapter and transform a Single as a Spring DeferredResult (see this example)
This example cames from an Spring Boot Starter that you may wants to directly use.

There is a return value handler in spring-cloud-netflix. Below pom dependencies worked for me.
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-netflix-core</artifactId>
</dependency>
I have tested this with rx 1.x
<dependency>
<groupId>io.reactivex</groupId>
<artifactId>rxjava</artifactId>
<version>1.1.10</version>
</dependency>

If you can't upgrade to the spring-reactive, you can use the org.springframework.web.context.request.async.DeferredResult
public class ExampleController {
#RequestMapping("/hello")
public DeferredResult<ResponseEntity<String>> hello() {
DeferredResult<ResponseEntity<String>> deferredResult = new DeferredResult<>();
// your observable
Observable.just("world")
.subscribe(
text -> deferredResult.setResult(
ResponseEntity.accepted()
.contentType(MediaType.TEXT_PLAIN)
.body(text)),
error -> deferredResult.setResult(
ResponseEntity.status(HttpStatus.I_AM_A_TEAPOT)
.build()
);
return deferredResult;
}
}

If you are on springboot 1.5
and want to return Observable from your restControllers
and avoid this message :
"No converter found for return value of type: class io.reactivex.internal.operators.observable.ObservableMap"
you have to add two class
#EnableWebMvc
#Configuration
#ComponentScan(basePackages = {"PACKAGE CONTROLLER"})
public class DispatcherContextConfig extends WebMvcConfigurerAdapter {
#Override
public void addReturnValueHandlers(List<HandlerMethodReturnValueHandler> returnValueHandlers) {
returnValueHandlers.add(new ObservableReturnValueHandler());
}
}
and
public class ObservableReturnValueHandler implements AsyncHandlerMethodReturnValueHandler {
#Override
public boolean isAsyncReturnValue(Object returnValue, MethodParameter returnType) {
return returnValue != null && supportsReturnType(returnType);
}
#Override
public boolean supportsReturnType(MethodParameter returnType) {
return Observable.class.isAssignableFrom(returnType.getParameterType());
}
#Override
public void handleReturnValue(Object returnValue, MethodParameter returnType, ModelAndViewContainer mavContainer,
NativeWebRequest webRequest) throws Exception {
if (returnValue == null) {
mavContainer.setRequestHandled(true);
return;
}
final Observable<?> observable = Observable.class.cast(returnValue);
WebAsyncUtils.getAsyncManager(webRequest)
.startDeferredResultProcessing(new ObservableAdapter<>(observable), mavContainer);
}
public class ObservableAdapter<T> extends DeferredResult<T> {
public ObservableAdapter(Observable<T> observable) {
observable.subscribe(this::setResult, this::setErrorResult);
}
}
}
then
#GetMapping(path = "{idDocument}/ropObs")
public Observable<DocumentDto> getDocumentRopObs(#PathVariable String idDocument) {
DocumentDto r = documentService.getDocumentRopInfo(idDocument)
.map(dtoMapper::documentBusinessToDto)
.doOnError(error -> {throw new ApiErrorServerError(error.getMessage());})
.blockingSingle();
return Observable.just(r);
}
and a better practice
#GetMapping(path = "{idDocument}/ropObs2")
public Observable<DocumentDto> getDocumentRopObs2(#PathVariable String idDocument) {
return documentService.getDocumentRopInfo(idDocument).map(dtoMapper::documentBusinessToDto);
}