I have the below route. In unit test, since I doesn't have the FTP server available, I'd like to use camel's test support and send an invalid message to "ftp://hostname/input" and verify that it failed and routed to "ftp://hostname/error".
I gone through the documentation which mainly talks about using the "mock:" endpoint but I am not sure how to use it in this scenario.
public class MyRoute extends RouteBuilder
{
#Override
public void configure()
{
onException(EdiOrderParsingException.class).handled(true).to("ftp://hostname/error");
from("ftp://hostname/input")
.bean(new OrderEdiTocXml())
.convertBodyTo(String.class)
.convertBodyTo(Document.class)
.choice()
.when(xpath("/cXML/Response/Status/#text='OK'"))
.to("ftp://hostname/valid").otherwise()
.to("ftp://hostname/invalid");
}
}
As Ben says you can either setup a FTP server and use the real components. The FTP server can be embedded, or you can setup a FTP server in-house. The latter is more like an integration testing, where you may have a dedicated test environment.
Camel is very flexible in its test kit, and if you want to build an unit test that do not use the real FTP component, then you can replace that before the test. For example in your example you can replace the input endpoint of a route to a direct endpoint to make it easier to send a message to the route. Then you can use an interceptor to intercept the sending to the ftp endpoints, and detour the message.
The advice with part of the test kit offers these capabilities: http://camel.apache.org/advicewith.html. And is also discussed in chapter 6 of the Camel in action book, such as section 6.3, that talks about simulating errors.
In your example you could do something a like
public void testSendError() throws Exception {
// first advice the route to replace the input, and catch sending to FTP servers
context.getRouteDefinitions().get(0).adviceWith(context, new AdviceWithRouteBuilder() {
#Override
public void configure() throws Exception {
replaceFromWith("direct:input");
// intercept valid messages
interceptSendToEndpoint("ftp://hostname/valid")
.skipSendToOriginalEndpoint()
.to("mock:valid");
// intercept invalid messages
interceptSendToEndpoint("ftp://hostname/invalid")
.skipSendToOriginalEndpoint()
.to("mock:invalid");
}
});
// we must manually start when we are done with all the advice with
context.start();
// setup expectations on the mocks
getMockEndpoint("mock:invalid").expectedMessageCount(1);
getMockEndpoint("mock:valid").expectedMessageCount(0);
// send the invalid message to the route
template.sendBody("direct:input", "Some invalid content here");
// assert that the test was okay
assertMockEndpointsSatisfied();
}
From Camel 2.10 onwards we will make the intercept and mock a bit easier when using advice with. As well we are introducing a stub component. http://camel.apache.org/stub
Have a look at MockFtPServer!
<dependency>
<groupId>org.mockftpserver</groupId>
<artifactId>MockFtpServer</artifactId>
<version>2.2</version>
<scope>test</scope>
</dependency>
With this one you can simulate all sorts of behaviors like permission problems, etc:
Example:
fakeFtpServer = new FakeFtpServer();
fakeFtpServer.setServerControlPort(FTPPORT);
FileSystem fileSystem = new UnixFakeFileSystem();
fileSystem.add(new DirectoryEntry(FTPDIRECTORY));
fakeFtpServer.setFileSystem(fileSystem);
fakeFtpServer.addUserAccount(new UserAccount(USERNAME, PASSWORD, FTPDIRECTORY));
...
assertTrue("Expected file to be transferred", fakeFtpServer.getFileSystem().exists(FTPDIRECTORY + "/" + FILENAME));
take a look at this unit test and those in the same directory...they'll show you how to standup a local FTP server for testing and how to use CamelTestSupport to validate scenarios against it, etc...
example unit test...
https://svn.apache.org/repos/asf/camel/trunk/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FromFileToFtpTest.java
which extends this test support class...
https://svn.apache.org/repos/asf/camel/trunk/components/camel-ftp/src/test/java/org/apache/camel/component/file/remote/FtpsServerTestSupport.java
In our project we do not create mock FTP Server to test the route but we use properties that can be replaced by a file Camel Component for the local development and unit testing.
Your code would look like this:
public class MyRoute extends RouteBuilder
{
#Override
public void configure()
{
onException(EdiOrderParsingException.class)
.handled(true)
.to("{{myroute.error}}");
from("{{myroute.input.endpoint}}")
.bean(new OrderEdiTocXml())
.convertBodyTo(String.class)
.convertBodyTo(Document.class)
.choice()
.when(xpath("/cXML/Response/Status/#text='OK'"))
.to("{{myroute.valid.endpoint}}}")
.otherwise()
.to("{{myroute.invalid.endpoint}}");
}
}
And locally and for system test we use a file endpoint declared in the property file:
myroute.input.endpoint=file:/home/user/myproject/input
myroute.valid.endpoint=file:/home/user/myproject/valid
myroute.invalid.endpoint=file:/home/user/myproject/invalid
myroute.error=file:/home/user/myproject/error
or in a JUnit CamelTestSupport you can use the useOverridePropertiesWithPropertiesComponent method to set the properties you want to overrides.
As an alternative you can also use a "direct" route instead but you can miss some File options that can be tested by the unit test.
And we only test the FTP connection with the real system by setting the properties like this:
myroute.input.endpoint=ftp://hostname/input
myroute.valid.endpoint=ftp://hostname/valid
myroute.invalid.endpoint=ftp://hostname/invalid
myroute.error=ftp://hostname/error
With this you can also have different configuration for e.g production server that will differentiate from the Integration Test Environment.
Example of Properties for Production environment:
myroute.input.endpoint=ftp://hostname-prod/input
myroute.valid.endpoint=ftp://hostname-prod/valid
myroute.invalid.endpoint=ftp://hostname-prod/invalid
myroute.error=ftp://hostname-prod/error
In my opinion it is totally acceptable to use file endpoint to simplify the JUnit code and it will test the route only and not the connection.
Testing the connection is more like an Integration Test and should be executed on the real server connected with the real external system (in your case FTP servers, but can be other endpoints/systems as well).
By using properties you can also configure different URL's per environment (For example: we have 3 testing environments and one production environment, all with different endpoints).
Related
I am using Karate to do integration testing for a Spring Boot app. The app consumes two other services (Service A and Service B) . I have written Karate mocks for Service A and B. My integration tests are written as two different feature files. One of the feature is tested using Mocks for service A and B. Another feature uses Mocks for service B and Spring Contract Stubs for service A.
Feature with only mocks
Feature: Test Some feature
Background:
* configure headers = { Content-Type: 'application/json' }
* url baseUrl
#Start up the mocks
Scenario: Start the Mocks
* karate.start({ mock: '../mock/service/service-a.feature', port: 9000})
* karate.start({ mock: '../mock/service/service-b.feature', port: 9001})
Feature with mocks and Spring Contract stubs
Feature: Test Some more features
Background:
* configure headers = { Content-Type: 'application/json' }
* url baseUrl
#Start up the mocks
Scenario: Start the Mocks
# Service A will use Spring Contract stubrunner.
* karate.start({ mock: '../mock/service/service-b.feature', port: 9001})
Now, when we run the tests, the second one fails saying that the port is already used. I tried using karate.stop(9000) at the end of the feature file which is running first, but it does not help. Also, I am not sure about the behaviour of this stop method. Any suggestions to solve this problem ? Why the mock service is still running even after the tests are done ?
As far as I know, the mock should stop when the JVM exits - so I can't explain what's going on in your case. So maybe you should create a way to replicate and file a bug: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I personally recommend starting your mocks from the unit-test Java code (typically the JUnit class) which means you can keep a reference to the mock and then call stop() on it. Even here, normally it is not mandatory, as the mock should terminate along with the JVM. Read the docs here: https://github.com/intuit/karate/tree/develop/karate-netty#embedding
Then, note that best-practice is to dynamically provision a port and then pass that to the test or any other consumers. Especially in CI-CD pipelines, this avoids the problem of the port in use or taking too much time to de-allocate.
And maybe the direct answer you are looking for. When you call karate.start() you get an object back which is of type MockServer. So you can keep a reference to it and call stop() on it when needed: https://github.com/intuit/karate/tree/develop/karate-netty#within-a-karate-test
For example:
* def server1 = karate.start('mock1.feature')
* def port1 = server1.port
# do some tests
* server1.stop()
I have recently started out with Spring and am unsure about how to approach this issue. I have a Spring boot program which makes calls to remote REST APIs. For example an AddressService class with getAddress(String user) method, which makes a HTTP call and returns a JSON response. I would like to set up Spring profiles for development purposes local, dev, uat, prod.
When the program is running with the local profile, I would like to "mock" these external API calls with an expected JSON response, so I can just test logic, but when it is run in any of the other profiles I would like to make the actual calls. How can I go about doing this? From what I read, there's many ways people approach this, using WireMock, RestTemplate, Mockito etc. I'm confused about which is the way to go.
Any advice would be greatly appreciated. Thanks.
WireMock,Mockit is for unittest, to mock the real request. Example here:
How do I mock a REST template exchange?
When you need a running implementation with a mock, i think the easiest way is that you have a interface
public interface AdressAdapter {
public List<Adress> getAddress(String name);
}
And two different implementations depending on the profile.
#Profile("local")
public class DummyAdress implements AdressAdapter{
#Override
public List<Adress> getAddress(String name) {
//Mock here something
return null;
}
}
! means NOT locale profile in this case.
#Profile("!local")
public class RealAdress implements AdressAdapter{
#Override
public List<Adress> getAddress(String name) {
//Make Restcall
return null;
}
}
What you could do is use different application.properties files depending on your profile. That way, you just change the url to a mock server for your local profile.
So what you have to do is :
Create another application.properties in your resources folder named : application-local.properties.
Change the url of the desired service.
Start your application with the VM option -Dspring.profiles.active=local.
Here is a link that describe well what you want to achieve.
For your mock server, you could use Wiremock, Mountebank, Postman,... that can be start separately and mock specific endpoints to return what you want.
We use the Apache Camel's Netty4 HTTP Component for almost everything on the artifact I have a problem with right now.
The issue is that, a proxy is now required to make external/outbound calls to the Internet...and eventually this is not working. All the calls are getting blocked/rejected — and just for the record, we had the same issue in a different artifact and we were able to circumvent it by using the JVM proxy settings, but this one is using the Async Http Client.
I've tried setting the proxy at the JVM level by using http.proxyHost, http.proxyPort, etc., but it didn't work this time.
Question(s): Is there a way to configure any proxy settings within this component? I've been digging inside org.apache.camel.component.netty4.http.NettyHttpConfiguration to see if there are any relevant settings I can change/use — I'm not completely sure what HTTP client it's used behind scenes and I'm guessing it might be Netty O:)
If relevant, (one of) our routes looks like:
#Component
public final class Route extends AbstractRouteBuilder {
#Override
public void configure() throws Exception {
super.configure();
from("{{route.inbound.reports}}") // netty4-http:https://hostname.tld/api/v1/reports
.choice()
.when(header(Exchange.HTTP_METHOD).isEqualToIgnoreCase(HttpMethod.GET))
.toD("seda:get")
.choice()
.when(AbstractHelper::isOk)
.setProperty("source", constant("user"))
.to("seda:retrieve?timeout={{async.timeout:4500}}")
.setBody(simple("${property.results}"))
.marshal().json(JsonLibrary.Jackson)
.end()
.endChoice()
.otherwise()
.toD("{{route.artifact.reports}}");
from("seda:get")
.toD("{{route.artifact.reports}}")
.unmarshal().json(JsonLibrary.Jackson)
.to(exec("analyze"));
from("seda:retrieve")
.filter(PredicateBuilder.and(header(Key.ACCOUNT_ID).isNotNull()))
.setHeader(Exchange.HTTP_METHOD, constant(HttpMethod.GET))
.setHeader(Header.API_KEY, simple("{{vendor.api-key}}"))
.toD("{{route.outbound.reports}}") // netty4-http:https://external-hostname.tld/api/client/reports
.unmarshal().json(JsonLibrary.Jackson)
.choice()
.when(AbstractHelper::isOk)
.to(exec("aggregate"))
.otherwise()
.to(exec("handleFailure"))
.end();
}
// ...
}
I guess there is no way around this one...or at least, not something I could find within the allowed timeframe.
We ended-up using a different component for all external/outbound calls (that obey the proxy rules); specifically:
AHC / camel-ahc
HTTP4 / camel-http4
I've got a series of "pipelined" components that all communicate through ActiveMQ message queues. Each component uses Camel to treat each of these queues as an Endpoint. Each component uses the same basic pattern:
Where each component consumes messages off of an input queue, processes the message(s), and then places 1+ messages on an outbound/output queue. The "output" queue then becomes the "input" queue for the next component in the chain. Pretty basic.
I am now trying to roll up my sleeves and provide unit testing for each component using the MockEndpoints provided by Camel's test API. I have been pouring over the javadocs and the few examples on Camel's website, but am having difficulty connecting all the dots.
It seems to me that, for each component, a portion of my unit testing is going to want to accomplish the following three things:
Test to see if there are messages waiting on a particular "input" queue
Pull those messages down and process them
Push new messages to an "output" queue and verify that they made it there
I believe I need to create MockEndpoints for each queue like so:
#EndpointInject(uri = "mock:inputQueue")
protected MockEndpoint intputQueue;
#EndpointInject(uri = "mock:outputQueue")
protected MockEndpoint outputQueue;
So now, in my JUnit test methods, I can set up expectations and interact with these endpoints:
#Test
public final void processMethodShouldSendToOutputQueue()
{
Component comp = new Component();
comp.process();
outputQueue.assertIsSatisfied();
}
I'm just not understanding how to wire everything up correctly:
How do I connect comp to the inputQueue and outputQueue MockEndpoints?
For each MockEndpoint, how do I set up expectations so that assertIsSatisfied() checks that a message is present inside a particular queue, or that a particular queue contains messages?
Adam, there are several ways to do this.
For POJO components, blackbox test them separately from any Camel context/routing to focus on business logic.
If you want to do end-to-end testing of the routes, consider using one of these approaches to validate that each step in the route is satisfied.
use NotifyBuilder to build Exchange validation expressions (somewhat complex to get your head around)
use AdviceWith to dynamically change the route before its run (add Log/Mock endpoints, etc)
I prefer AdviceWith because its very flexible and leverages the familiar MockEndpoints. For a complete example of this, see this unit test
In short, you will create a unit test to inject MockEndpoints into your route and then validate against them as usual...
context.getRouteDefinition("myRouteId").adviceWith(context, new AdviceWithRouteBuilder() {
#Override
public void configure() throws Exception {
// mock all endpoints
mockEndpoints();
}
});
getMockEndpoint("mock:direct:start").expectedBodiesReceived("Hello World");
template.sendBody("direct:start", "Hello World");
I have a EJB to send a message to JMS queue and wait the reply from it. I want to test the EJB, it's easy to use OpenEJB to do the JUnit test of the EJB. But the problem is this EJB will wait the JMS response to continue process.
Although I can send message in my junit code, but because the EJB is still on-going, I cannot run it before the EJB is completed.
2nd solution is I can initialize a MDB to listen and reply the JMS message form the EJB, but the problem is the MDB must in src\main\java and cannot in src\test\java. The problem is this is just a test code and I should not package it to production environment. (I use Maven)
Or should I use mock object ?
You're on the right track. There area few ways to handle this. Here are a couple tips for unit testing with OpenEJB and Maven.
Test beans
You can write all sorts of EJBs and other testing utilities and have them deployed. All you need is a ejb-jar.xml for the test code like so:
src/main/resources/ejb-jar.xml (the normal one)
src/test/resources/ejb-jar.xml (the testing beans)
As usual the ejb-jar.xml file only needs to contain <ejb-jar/> and nothing more. Its existence simply tells OpenEJB to inspect that part of the classpath and scan it for beans. Scanning the entire classpath is very slow, so this is just convention to speed that up.
TestCase injection
With the above src/test/resources/ejb-jar.xml you could very easily add that test-only MDB and have it setup to process the request in a way that the TestCase needs. But the src/test/resources/ejb-jar.xml also opens up some other interesting functionality.
You could have the TestCase itself do it by declaring references to whatever JMS resources you need and have them injected.
import org.apache.openejb.api.LocalClient;
#LocalClient
public class ChatBeanTest extends TestCase {
#Resource
private ConnectionFactory connectionFactory;
#Resource(name = "QuestionBean")
private Queue questionQueue;
#Resource(name = "AnswerQueue")
private Queue answerQueue;
#EJB
private MyBean myBean;
#Override
protected void setUp() throws Exception {
Properties p = new Properties();
p.put(Context.INITIAL_CONTEXT_FACTORY, "org.apache.openejb.client.LocalInitialContextFactory");
InitialContext initialContext = new InitialContext(p);
initialContext.bind("inject", this); // here's the magic!
}
}
Now you're just one thread away from being able to respond to the JMS message the testcase itself. You can launch off a little runnable that will read a single message, send the response you want, then exit.
Maybe something like:
public void test() throws Exception {
final Thread thread = new Thread() {
#Override
public void run() {
try {
final Connection connection = connectionFactory.createConnection();
connection.start();
final Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
final MessageConsumer incoming = session.createConsumer(requestQueue);
final String text = ((TextMessage) incoming.receive(1000)).getText();
final MessageProducer outgoing = session.createProducer(responseQueue);
outgoing.send(session.createTextMessage("Hello World!"));
} catch (JMSException e) {
e.printStackTrace();
}
}
};
thread.setDaemon(true);
thread.start();
myBean.doThatThing();
// asserts here...
}
See
Alternate Descriptors
If you did want to use the MDB solution and only wanted to enable it for just the one test and not all tests, you could define it in a special src/test/resources/mockmdb.ejb-jar.xml file and enable it in the specific test case(s) where it is needed.
See this doc for more information on how to enable that descriptor and the various options of alternate descriptors.
I think you should use mocks for this. If you're sending messages to a real JMS server, listening for them, replying to them, etc. then you're doing something other than a unit test. I'm not going to get into the argument about what that should be called, but I think it's pretty well universally accepted that a unit-test shouldn't be talking to live databases, message queues, etc.
If I've understood your question correct - It's a bad design to have an EJB send a JMS message and then await a response, in fact contradictory to the whole idea of EJB.
You send a JMS message, and then forget about it. You have an MDB to receive the message. If the EJB depends on a response, JMS is not the way to go, but rather use another EJB.
To test the sending, mock the JMS classes, test the MDB separately.
EJB's are designed for synchronous tasks, JMS for asynchronous tasks - if you have to do asynchronous communication to an external system, I suggest you design your system after that, and do proper asynchronous flows. An EJB that sits and waits for a JMS reply is at best an ugly hack, and will not add any good to your system design.
Thanks for David's answer, it's what I want. I know unit test should not depend on other external resource like JMS server. But if I use Maven + OpenEJB, I still can let the test code in a closed environment. It can help to do automatically test with external resource dependency, especially for some old programs which not easy to refactor.
And if you see the following error message in initialContext.bind("inject", this)
Ensure that class was annotated with #org.apache.openejb.api.LocalClient and was successfully discovered and deployed.
One reference is http://openejb.apache.org/3.0/local-client-injection.html, but add "openejb.tempclassloader.skip=annotations" doesn't work for me. Please check this doc OpenEJB Local Client Injection Fails. There is already a patch for it, I think it will be fixed in OpenEJB 3.1.5 or 4.0
Also I've found it is best practice to actually break out your logic in your MDB to a different class. This isolates your business logic from being in an MDB and allows you to expose your logic as more than one way (MDB, EJB, Web Service, POJO, etc.). It also allows you to more easily test your business logic without the need to test the protocol (JMS in this case).
As for testing JMS, mocking may be the better choice. Or if you really need to test the protocol "in container" look at using something like the JBoss Microcontainer (I believe you can get this packaged with some of the JBoss projects like Seam). Then you can fire up a mini-container for testing things like EJB and JMS.
But overall, it is best to avoid having to need a container unless absolutely necessary. That's why separating your business logic from your implementation logic (even if you don't use mocks) is a good practice.