We Keep Coding

Mockserver fails to read request: unknown message format

I need to test some REST client. For that purpose I'm using org.mockserver.integration.ClientAndServer
I start my server. Create some expectation. After that I mock my client. Run this client. But when server receives request I see in logs:
14:00:13.511 [MockServer-EventLog0] INFO org.mockserver.log.MockServerEventLog - received binary request:
505249202a20485454502f322e300d0a0d0a534d0d0a0d0a00000604000000000000040100000000000408000000000000ff0001
14:00:13.511 [MockServer-EventLog0] INFO org.mockserver.log.MockServerEventLog - unknown message format
505249202a20485454502f322e300d0a0d0a534d0d0a0d0a00000604000000000000040100000000000408000000000000ff0001
This is my test:
#RunWith(PowerMockRunner.class)
#PrepareForTest({NrfClient.class, SnefProperties.class})
#PowerMockIgnore({"javax.net.ssl.*"})
#TestPropertySource(locations = "classpath:test.properties")
public class NrfConnectionTest {
private String finalPath;
private UUID uuid = UUID.fromString("8d92d4ac-be0e-4016-8b2c-eff2607798e4");
private ClientAndServer mockServer;
#Before
public void startMockServer() {
mockServer = startClientAndServer(8888);
}
#After
public void stopServer() {
mockServer.stop();
}
#Test
public void NrfRegisterTest() throws Exception {
//create some expectation
new MockServerClient("127.0.0.1", 8888)
.when(HttpRequest.request()
.withMethod("PUT")
.withPath("/nnrf-nfm/v1/nf-instances/8d92d4ac-be0e-4016-8b2c-eff2607798e4"))
.respond(HttpResponse.response().withStatusCode(201));
//long preparations and mocking the NrfClient (client that actually make request)
//NrfClient is singleton, so had to mock a lot of methods.
PropertiesConfiguration config = new PropertiesConfiguration();
config.setAutoSave(false);
File file = new File("test.properties");
if (!file.exists()) {
String absolutePath = file.getAbsolutePath();
finalPath = absolutePath.substring(0, absolutePath.length() - "test.properties".length()) + "src\\test\\resources\\test.properties";
file = new File(finalPath);
}
try {
config.load(file);
config.setFile(file);
} catch(ConfigurationException e) {
LogUtils.warn(NrfConnectionTest.class, "Failed to load properties from file " + "classpath:test.properties", e);
}
SnefProperties spyProperties = PowerMockito.spy(SnefProperties.getInstance());
PowerMockito.doReturn(finalPath).when(spyProperties, "getPropertiesFilePath");
PowerMockito.doReturn(config).when(spyProperties, "getProperties");
PowerMockito.doReturn(config).when(spyProperties, "getLastUpdatedProperties");
NrfConfig nrfConfig = getNrfConfig();
NrfClient nrfClient = PowerMockito.spy(NrfClient.getInstance());
SnefAddressInfo snefAddressInfo = new SnefAddressInfo("127.0.0.1", "8080");
PowerMockito.doReturn(nrfConfig).when(nrfClient, "loadConfiguration", snefAddressInfo);
PowerMockito.doReturn(uuid).when(nrfClient, "getUuid");
Whitebox.setInternalState(SnefProperties.class, "instance", spyProperties);
nrfClient.initialize(snefAddressInfo);
//here the client makes request
nrfClient.run();
}
private NrfConfig getNrfConfig() {
NrfConfig nrfConfig = new NrfConfig();
nrfConfig.setNrfDirectConnection(true);
nrfConfig.setNrfAddress("127.0.0.1:8888");
nrfConfig.setSnefNrfService(State.ENABLED);
nrfConfig.setSmpIp("127.0.0.1");
nrfConfig.setSmpPort("8080");
return nrfConfig;
}
}
Looks like I miss some server configuration, or use it in wrong way.
Or, maybe the reason is in powermock: could it be that mockserver is incompatible with powermock or PowerMockRunner?

How to get the error message in Controller or Route after handling exception in errorHandler?

I had to customized Sftp Inbound default handler LoggingHandler and using my own CustomizedErrorHandler which extends ErrorHandler. But I can't return any message to my controller after handling exceptions.
I was researching couple of days and I found nothing to show my customized message to my UI using Controller. Below are some code snippet from my CustomizedErrorHandler, SftpInboundConfiguration.
SftpInboundConfiguration
public IntegrationFlow fileFlow() {
SftpInboundChannelAdapterSpec spec = Sftp
.inboundAdapter(getSftpSessionFactory())
.preserveTimestamp(true)
.remoteDirectory(getSourceLocation())
.autoCreateLocalDirectory(true)
.deleteRemoteFiles(false)
.localDirectory(new File(getDestinationLocation()));
return IntegrationFlows
.from(spec, e -> e.id(BEAN_ID)
.autoStartup(false)
.poller(sftpPoller())
)
.channel(sftpReceiverChannel())
.handle(sftpInboundMessageHandler())
.get();
}
... ... ...
public PollerMetadata sftpPoller() {
PollerMetadata pollerMetadata = new PollerMetadata();
List<Advice> adviceChain = new ArrayList<>();
pollerMetadata.setErrorHandler(customErrorMessageHandler());
pollerMetadata.setTrigger(new PeriodicTrigger(5000));
return pollerMetadata;
}
... ... ...
private CustomErrorMessageHandler customErrorMessageHandler() {
return new CustomErrorMessageHandler(
controlChannel(),
BEAN_ID
);
}
CustomErrorMessageHandler
public class CustomErrorMessageHandler implements ErrorHandler {
private final MessageChannel CONTROL_CHANNEL;
private final String BEAN_ID;
public CustomErrorMessageHandler(
MessageChannel controlChannel,
String beanID
) {
this.CONTROL_CHANNEL = controlChannel;
this.BEAN_ID = beanID;
}
public void handleError(#NotNull Throwable throwable) {
final Throwable rootCause = ExceptionUtils.getRootCause(throwable);
if (rootCause instanceof MessagingException) {
log.error("MessagingException : {} ", rootCause.getMessage());
} else if (rootCause instanceof SftpException) {
log.error("SftpException : {}", rootCause.getMessage());
} ... ... ...
else {
log.error("Unknown : Cause : {} , Error : {}",
rootCause, rootCause.getMessage());
}
log.info("Stopping SFTP Inbound");
boolean is_stopped = CONTROL_CHANNEL.send(
new GenericMessage<>("#" + BEAN_ID + ".stop()"));
if (is_stopped) {
log.info("SFTP Inbound Stopped.");
} else {
log.info("SFTP Inbound Stop Failed.");
}
}
}
Now I want to save some customized message from if-else statements and need to show it in UI. Is there any way to save the message and show it using Route or Controller ?

Don't customize the error handler, use poller.errorChannel("myErrorChannel") instead.
Then add an error channel flow
#Bean
IntegrationFlow errors() {
return IntegrationFLows.from("myErrorChannel")
.handle(...)
...
.get();
The message sent to the handler is an ErrorMessage with a MessagingException payload, with cause and failedMessage which was the message at the point of the failure and originalMessage which is the original message emitted by the adapter.
After handling the exception, you can simply call a method on your controller to tell it the state.

Remove file via SftpOutboundGateway after processing file

I'm using Spring Integration to read files from a SFTP Server and everything works fine using an InboundChannelAdapter with Java Configuration.
Now, i want to modify my process in order to remove all processed files from SFTP Server. Therefore I want to use an SFTP OutboundGateway with Java Configuration. This is my new code with few modifications based on https://docs.spring.io/spring-integration/docs/5.0.0.BUILD-SNAPSHOT/reference/html/sftp.html#sftp-outbound-gateway:
#Configuration
public class SftpConfiguration {
#Value("${sftp.host}")
String sftpHost = "";
#Value("${sftp.user}")
String sftpUser = "";
#Value("${sftp.pass}")
String sftpPass = "";
#Value("${sftp.port}")
Integer sftpPort = 0;
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(sftpHost);
factory.setPort(sftpPort);
factory.setUser(sftpUser);
factory.setPassword(sftpPass);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
#Bean
public SftpInboundFileSynchronizer sftpInboundFileSynchronizer() {
SftpInboundFileSynchronizer fileSynchronizer = new SftpInboundFileSynchronizer(sftpSessionFactory());
fileSynchronizer.setDeleteRemoteFiles(false);
fileSynchronizer.setRemoteDirectory("/upload/");
return fileSynchronizer;
}
#Bean
#InboundChannelAdapter(channel = "sftpChannel", poller = #Poller(cron = "0 * * * * ?"))
public MessageSource<File> sftpMessageSource() {
SftpInboundFileSynchronizingMessageSource source =
new SftpInboundFileSynchronizingMessageSource(sftpInboundFileSynchronizer());
source.setLocalDirectory(new File("sftp-folder"));
source.setAutoCreateLocalDirectory(true);
return source;
}
#Bean
#ServiceActivator(inputChannel = "sftpChannel")
MessageHandler handler() {
return new MessageHandler() {
#Override
public void handleMessage(Message<?> message) throws MessagingException {
File f = (File) message.getPayload();
try {
myProcessingClass.processFile(f);
SftpOutboundGateway sftpOG = new SftpOutboundGateway(sftpSessionFactory(), "rm",
"'/upload/" + f.getName() + "'");
} catch(QuoCrmException e) {
logger.error("File [ Process with errors, file won't be deleted: " + e.getMessage() + "]");
}
}
};
}
}
Modifications are:
I defined my fileSynchronizer to setDeleteRemoteFiles(false), in order to remove files manually according to my process.
In my MessageHandler, I added my SFTPOutboundGateway and if there is no exception, it means the processing was successful and removes the file (but if there is an exception won't delete the file).
This code is not removing any file. Any suggestions?

You shouldn't create a new gateway for each request (which is what you are doing).
You are not doing anything with sftpOG after you create it anyway; you need to send a message to the gateway.
You can create a reply-producing handler and wire it's output channel to the gateway (which should be its own #Bean).
Or, you can simply use an SftpRemoteFileTemplate to remove the file - but again, you only need one, you don't need to create a new one for each request.

Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: Resource not found

I try to write a test of pubsub:
#Test
public void sendTopic() throws Exception {
CustomSubscriber customSubscriber = new CustomSubscriber();
customSubscriber.startAndWait();
CustomPublisher customPublisher = new CustomPublisher();
customPublisher.publish("123");
}
and:
public CustomSubscriber() {
this.subscriptionName = SubscriptionName.create(SdkServiceConfig.s.GCP_PROJECT_ID, SdkServiceConfig.s.TOPIC_ID );
this.receiveMsgAction = (message, consumer) -> {
// handle incoming message, then ack/nack the received message
System.out.println("Id : " + message.getMessageId());
System.out.println("Data : " + message.getData().toStringUtf8());
consumer.ack();
};
this.afterStopAction = new ApiFutureEmpty();
}
// [TARGET startAsync()]
public void startAndWait() throws Exception {
Subscriber subscriber = createSubscriberWithCustomCredentials();
subscriber.startAsync();
// Wait for a stop signal.
afterStopAction.get();
subscriber.stopAsync().awaitTerminated();
}
and:
public ApiFuture<String> publish(String message) throws Exception {
ByteString data = ByteString.copyFromUtf8(message);
PubsubMessage pubsubMessage = PubsubMessage.newBuilder().setData(data).build();
ApiFuture<String> messageIdFuture = publisher.publish(pubsubMessage);
ApiFutures.addCallback(messageIdFuture, new ApiFutureCallback<String>() {
public void onSuccess(String messageId) {
System.out.println("published with message id: " + messageId);
}
public void onFailure(Throwable t) {
System.out.println("failed to publish: " + t);
}
});
return messageIdFuture;
}
/**
* Example of creating a {#code Publisher}.
*/
// [TARGET newBuilder(TopicName)]
// [VARIABLE "my_project"]
// [VARIABLE "my_topic"]
public void createPublisher(String projectId, String topicId) throws Exception {
TopicName topic = TopicName.create(projectId, topicId);
try {
publisher = createPublisherWithCustomCredentials(topic);
} finally {
// When finished with the publisher, make sure to shutdown to free up resources.
publisher.shutdown();
}
}
When i run the code i get this error:
Caused by: io.grpc.StatusRuntimeException: NOT_FOUND: Resource not found (resource=add-partner-request).
What am i missing?

Whatever entity is named "add-partner-request" was not successfully created or does not belong to the project. If "add-partner-request" is the topic, then you actually need to create the topic; the line TopicName.create(projectId, topicId) is not sufficient for creating the topic itself. Typically, one would create the topic in the Cloud Pub/Sub portion of the Cloud console or via a gcloud command, e.g.,
gcloud pubsub topics create add-partner-request
Ensure that the project you are logged into in the console is the one used in the code. You should also set the project explicitly when creating the topic via the --project flag or verify that the default project is the correct one:
gcloud config list --format='text(core.project)'
For tests, it is typical to create and delete in code. For example, to create a topic:
Topic topic = null;
ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
TopicAdminClient topicAdminClient = TopicAdminClient.create();
try {
topic = topicAdminClient.createTopic(topicName);
} catch (APIException e) {
System.out.println("Issue creating topic!");
}
If "add-partner-request" is the subscription name, then the same things apply. The gcloud command would change a bit:
gcloud pubsub subscriptions create add-partner-request --topic=<topic name>
The command to create the subscription in Java would be as follows:
Subscription subscription = null;
ProjectTopicName topicName = ProjectTopicName.of(projectId, topicId);
ProjectSubscriptionName subscriptionName = ProjectSubscriptionName.of(projectId, subscriptionId);
SubscriptionAdminClient subscriptionAdminClient = SubscriptionAdminClient.create();
try {
subscription = subscriptionAdminClient.createSubscription(
subscriptionName, topicName, PushConfig.newBuilder().build(), 600);
} catch (APIException e) {
System.out.println("Issue creating topic!");
}

I'm assuming TOPIC_ID is the name of your topic; you actually need to reference a subscription. You can easily create a subscription from the GCP console, then reference that name in the SubscriptionName.create(project,yoursubscriptionname)

I think that you forget to create a topic inside your project with the following name "add-partner-request".
You can create it using the following code:
try (TopicAdminClient topicAdminClient = TopicAdminClient.create()) {
// projectId <= unique project identifier, eg. "my-project-id"
TopicName topicName = TopicName.create(projectId, "add-partner-request");
Topic topic = topicAdminClient.createTopic(topicName);
return topic;
}

Spring SFTP Outbound Gateway: How to close the session after GET in Java Config?

I have written a piece of code which uses Spring SFTP Outbound gateway and performs a GET operation. The whole configuration is in JAVA (no XML).
I have made a caching session factory which allows a maximum of 10 sessions. Due to which after multiple GET request when it exceeds 10, GET request start failing.
I read the docs and it was written to close the session after operation but i'm unable to figure out as to how to close this session in JAVA Configuration?
#org.springframework.integration.annotation.MessagingGateway
public interface FileOperationGateway {
#Gateway(requestChannel = "sftpChannelDownload")
InputStream downloadFromSftp(Message<Boolean> message);
}
#Bean
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
factory.setHost(SFTP_HOST);
factory.setPort(SFTP_PORT);
factory.setUser(SFTP_USERNAME);
factory.setPassword(SFTP_PASSWORD);
factory.setAllowUnknownKeys(true);
return new CachingSessionFactory<LsEntry>(factory);
}
/**
* Bean for Caching the session
*
*/
#Bean
#Autowired
public CachingSessionFactory<LsEntry> cachingSessionFactory(SessionFactory<LsEntry> sftpSessionFactory) {
CachingSessionFactory<LsEntry> cachingSessionFactory = new CachingSessionFactory<>(sftpSessionFactory, 10);
cachingSessionFactory.setSessionWaitTimeout(SFTP_SESSION_TIMEOUT);
return cachingSessionFactory;
}
/**
* Bean for Remote File Template
*
* #return
* #throws Exception
*/
#Bean
#Autowired
public RemoteFileTemplate<LsEntry> remoteFileTemplateDesigner(CachingSessionFactory<LsEntry> csf) throws Exception {
ExpressionParser expressionParser = new SpelExpressionParser();
Expression expression = expressionParser.parseExpression("'" + SFTP_LOCATION + "'");
SftpRemoteFileTemplate rft = new SftpRemoteFileTemplate(csf);
rft.setRemoteDirectoryExpression(expression);
rft.setRemoteFileSeparator("/");
rft.setFileNameGenerator((msg) -> {
Timestamp timestamp = new Timestamp(System.currentTimeMillis());
Instant instant = timestamp.toInstant();
String fileNameFromHeader = msg.getHeaders().get(FileOperationConstants.FILE_HEADER_KEY).toString();
String newFileName;
if (fileNameFromHeader.lastIndexOf("/") != -1) {
newFileName = fileNameFromHeader.substring(fileNameFromHeader.lastIndexOf("/"));
} else if (fileNameFromHeader.lastIndexOf("\\") != -1) {
newFileName = fileNameFromHeader.substring(fileNameFromHeader.lastIndexOf("\\"));
} else
newFileName = fileNameFromHeader;
String fileNameOnly = newFileName.substring(0, newFileName.lastIndexOf("."));
String fileType = newFileName.substring(newFileName.lastIndexOf(".") + 1);
return (fileNameOnly + "__" + instant.toString() + "." + fileType);
});
rft.afterPropertiesSet();
return rft;
}
#Bean
#Autowired
#ServiceActivator(inputChannel = "sftpChannelDownload")
public SftpOutboundGatewaySpec downloadHandler(RemoteFileTemplate<LsEntry> rft) {
SftpOutboundGatewaySpec sogs = Sftp.outboundGateway(rft, FileOperationConstants.FILE_DOWNLOAD_COMMAND,
FileOperationConstants.FILE_DOWNLOAD_EXPRESSION);
sogs.options(Option.STREAM);
return sogs;
}
******UPDATE:******
I created a new class with #messageEndpoint and placed the closeable session code in it. I then called this handler from my service class (where i was consuming the stream)This worked:
#MessageEndpoint
public class FileOperationCloseSessionMessageHandler {
#ServiceActivator(inputChannel = "sftpCloseSession")
public void closeSession(Message<Boolean> msg) throws IOException {
Closeable closeable = new IntegrationMessageHeaderAccessor(msg).getCloseableResource();
if (closeable != null) {
closeable.close();
}
}
}
Placed this line in #MessagingGateway annotated class
#Gateway(requestChannel = "sftpCloseSession")
void closeSession(Message<InputStream> msg);
And then called the gateway method from service class:
Message<InputStream> msg = msgGateway.downloadFromSftp(message);
InputStream is = msg.getPayload();
msgGateway.closeSession(msg);

sogs.options(Option.STREAM);
When you stream the file, you are responsible for closing the session after you have finished streaming. This is explained in the documentation.
When consuming remote files as streams, the user is responsible for closing the Session after the stream is consumed. For convenience, the Session is provided in the IntegrationMessageHeaderAccessor.CLOSEABLE_RESOURCE header, a convenience method is provided on the IntegrationMessageHeaderAccessor:
Closeable closeable = new IntegrationMessageHeaderAccessor(message).getCloseableResource();
if (closeable != null) {
closeable.close();
}
Framework components such as the File Splitter and Stream Transformer will automatically close the session after the data is transferred.