How to send a POST request that contains byte array by WebClient? - java

I want to send a request of a file through WebClient by POST method, and I need to send the file as byte[] to get right response.
I made MultipartFile file to byte[], and then I thought I need to use BodyInserters
to make this body contains byte[] but I don't know how to make that request body.
How to send a POST request that contains byte array by WebClient?
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.reactive.function.BodyInserters;
import org.springframework.web.reactive.function.client.WebClient;
#RestController
public class ApiController {
#PostMapping(value = "/update")
public String update(#RequestParam("file") MultipartFile file, #RequestParam("uri") String uri ) {
String result = "{error : error}";
byte[] byteArr;
BodyInserters byteArrInserter;
try {
byteArr = file.getBytes();
A? publisher = B?.C?; // I don't know what will be right for those A?, B?, C?
byteArrInserter = BodyInserters.fromDataBuffers(publisher); // In fact, I'm not sure this method will be good for this situation, either.
} catch (Exception e) {
System.out.println(e);
}
WebClient client = WebClient.builder()
.codecs(configurer -> configurer.defaultCodecs().maxInMemorySize( 1024*1024*1024 * 2)) // 2GB
.build();
try {
result = client
.post()
.uri(uri)
.body(byteArrInserter)
.retrieve().bodyToMono(String.class).block();
} catch (Exception e) {
System.out.println(e);
}
return result;
}
}


I don't know how to send binary content with WebClient, but there are some other options. First there is SpringBoot own RestTemplate class which is an alternative to WebClient. Here is an comparative article about them both: Spring WebClient vs. RestTemplate. If you wish to use third party Http clients that definitely can send binary content you can look at 2 popular choices:
Apache Http Client
OK Http Client
And finally, I wrote my own Http client that is part of Open-source MgntUtils library written and maintained by me. This Http client is very simplistic but does the job. I am actually at the process of adding the feature that allows to upload binary info. I haven't published it yet, but you can download the branch or look at the code how I do it there. The branch is located here The HttpClient class is here. See method public String sendHttpRequest(String requestUrl, HttpMethod callMethod, ByteBuffer data) throws IOException at line 162. The method with which I tested this method looks like this:
private static void testHttpClientBinaryUpload() {
try {
byte[] content = Files.readAllBytes(Paths.get("C:\\Michael\\Personal\\pics\\testPic.jpg"));
HttpClient client = new HttpClient();
Integer length = content.length;
Files.write(Paths.get("C:\\Michael\\tmp\\testPicOrig.jpg"), content);
client.setRequestHeader("Content-Length", length.toString());
String result = client.sendHttpRequest("http://localhost:8080/upload", HttpMethod.POST, ByteBuffer.wrap(content));
System.out.println(result);
System.out.println("HTTP " + client.getLastResponseCode() + " " + client.getLastResponseMessage());
} catch (Exception e) {
System.out.println(TextUtils.getStacktrace(e, "com.mgnt."));
}
}
And server side Springboot rest controller that receives the request looks like this:
#RestController
#RequestMapping("/upload")
public class UploadTestController {
#PostMapping
public ResponseEntity<String> uploadTest(HttpServletRequest request) {
try {
String lengthStr = request.getHeader("content-length");
int length = TextUtils.parseStringToInt(lengthStr, -1);
if(length > 0) {
byte[] buff = new byte[length];
ServletInputStream sis =request.getInputStream();
int counter = 0;
while(counter < length) {
int chunkLength = sis.available();
byte[] chunk = new byte[chunkLength];
sis.read(chunk);
for(int i = counter, j= 0; i < counter + chunkLength; i++, j++) {
buff[i] = chunk[j];
}
counter += chunkLength;
if(counter < length) {
TimeUtils.sleepFor(5, TimeUnit.MILLISECONDS);
}
}
Files.write(Paths.get("C:\\Michael\\tmp\\testPic.jpg"), buff);
}
} catch (Exception e) {
System.out.println(TextUtils.getStacktrace(e));
}
return ResponseEntity.ok("Success");
}
}
I am planning to publish the new version of the library in few days and this feature will be included. In any case here is the link to Maven Artifacts, Github and Javadoc

Related

How to Invoke Multiple webmethod URLs with different filename at once in spring boot

I am very new to Spring Boot project.
I am writing backend code where I have a webmethod url which can download only one file at a time based on fileNo. It is invoked from the front-end when user enters fileNo and submits.
User can enter maximum 5 fileNo(comma-separated) at one time.
In that case I have to take each file no and and set it into my url, invoke it which will download 5 files and put it in one common folder.
Below code is working for one FileNo and downloading the file,
Is there anyway where I can set and invoke all 5 URLs concurrently, download all 5 files and put it in a same folder.
Or If I have to set it one by one in my URL then how to do it. What is the best way to do this. (went through few similar posts but couldn't fine anything for my solution). Looking for a solution here. Thanks
#SneakyThrows
#RequestMapping(value = "/{fileName:.+}", method = RequestMethod.GET)
#ApiResponses(value = {
#ApiResponse(code = 200, message = "Success"),
#ApiResponse(code = 500, message = "Server Error")
})
public ResponseEntity getCollateralDownloadData(#PathVariable("fileName") String fileName) {
String wmURL = "https://www.qa.referencesite.com/invoke/File_appDesigns.process:processTpfQuery?prdType=PTO&tpf_aif=one&SALESNO=&PRODNO=&OASN="+fileName;
try {
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM));
HttpEntity<String> entity = new HttpEntity<>(headers);
ResponseEntity<byte[]> response = restTemplate.build()
.exchange(wmURL, HttpMethod.GET, entity, byte[].class);
return ResponseEntity.ok()
.contentType(MediaType.parseMediaType("application/octet-stream"))
.header(HttpHeaders.CONTENT_DISPOSITION, "attachment; filename=\"" + fileName + "\"")
.body(response.getBody());
} catch (Exception e) {
e.printStackTrace();
}
return ResponseEntity.status(HttpStatus.OK).body("Collateral Download Created successfully");
}

There are 2 approaches depending on your need :
The client may trigger requests concurrently, so the client has to send a request for each fileNo and you backend deals with it
The client may trigger only one request, so your backend should be modified to trigger subsequent calls, wait for result and build a common response. You may use Spring WebFlux facilities or Java ExecutorService.invokeAll() method to manage subsequent parallels requests.
Regards.

Here is complete Flow of code what I have done.
#RestController
public class DownloadOAFileController {
#Autowired
private RestTemplateBuilder restTemplate;
private static final int MYTHREADS = 30;
#SneakyThrows
#RequestMapping(value = "/downloadzippedFile/{fileName}", method = RequestMethod.GET)
#ApiResponses(value = {
#ApiResponse(code = 200, message = "Success"),
#ApiResponse(code = 500, message = "Server Error")
})
public ResponseEntity<Object> downloadzipFiles(#PathVariable("fileName") String fileName) throws IOException, ExecutionException, InterruptedException {
downloadMultipleFilesToRemoteDiskUsingThread(fileName);
String zipFilename = "/Users/kumars22/Downloads/Files.zip";
File file = new File(zipFilename);
InputStreamResource resource = new InputStreamResource(new FileInputStream(file));
HttpHeaders headers = new HttpHeaders();
headers.add("Content-Disposition",
String.format("attachment; filename=\"%s\"",file.getName()));
headers.add("Cache-Control","no-cache, no-store, no-revalidate");
headers.add("pragma","no-cache");
headers.add("Expires","0");
ResponseEntity<Object> responseEntity = ResponseEntity.ok().headers(headers)
.contentLength(file.length())
.contentType(MediaType.parseMediaType("application/octet-stream"))
.body(resource);
return responseEntity;
}
//Created a Directory where all files will be downloaded (Got some space in AWS for actual implementation.
public static void createDirectory() {
File file = new File("/Users/kumars22/Downloads/Files");
if (!file.exists()) {
file.mkdir();
}
}
public static void downloadMultipleFilesToRemoteDiskUsingThread( String fileName ) throws ExecutionException, InterruptedException, IOException {
createDirectory();
ExecutorService executor = Executors.newFixedThreadPool(MYTHREADS);
String[] serials = fileName.split(",");
String[] wmURLs = new String[serials.length];
for (int i =0;i<serials.length;i++) {
wmURLs[i] = "https://www.qa.referencesite.com/invoke/File_appDesigns.process:processTpfQuery?prdType=PTO&tpf_aif=one&SALESNO=&PRODNO=&OASN="+serials[i];
}
for (int i = 0; i < wmURLs.length; i++) {
String url = wmURLs[i];
Runnable worker = new MultipleCallThreadController(url,fileName,"James");
executor.execute(worker);
}
executor.shutdown();
// Wait until all threads are finish
while (!executor.isTerminated()) {
}
zipFiles("/Users/kumars22/Downloads/Files","/Users/kumars22/Downloads/Files.zip");
System.out.println("\nFinished all threads");
}
//Zip the Folder having all files
public static void zipFiles(String sourceDirPath, String zipFilePath) throws IOException {
Path p = Files.createFile(Paths.get(zipFilePath));
Path pp = Paths.get(sourceDirPath);
try (ZipOutputStream zs = new ZipOutputStream(Files.newOutputStream(p));
Stream<Path> paths = Files.walk(pp)) {
paths
.filter(path -> !Files.isDirectory(path))
.forEach(path -> {
ZipEntry zipEntry = new ZipEntry(pp.relativize(path).toString());
try {
zs.putNextEntry(zipEntry);
Files.copy(path, zs);
zs.closeEntry();
} catch (IOException e) {
System.err.println(e);
}
});
}
}

Spring Boot + Video error "video playback aborted due to a network error"

So I've designed an application which has a reactjs frontend serving up content from a spring boot backend. I have a rest controller which serves up video. It works really well for about 10 minutes, and then I get "video playback aborted due to a network error" in my browser. Is there something I'm doing wrong here? Should this be done on another thread or async or something? I imagine that's all done by spring.
#RestController
#CrossOrigin
public class VideoDirectoryController {
#Autowired
private ConfigManager<LibraryConfig> librariesConf;
/**
*
* #param libraryName
* #param fileName
* #return
*/
#GetMapping(value = "/getVideo", produces = "video/mp4")
public byte[] getVideo(#RequestParam(value = "library") String libraryName,
#RequestParam(value = "fileName") String fileName) {
Library lib = librariesConf.getConfig().getLibraries().get(libraryName.toLowerCase());
Video video = lib.getVideoFiles().get(fileName);
lib.getRecentlyViewed().add(video);
librariesConf.getConfig().getLibraries().keySet().forEach((libr) -> System.out.println("LIBR: :" + libr));
try (FileInputStream out = new FileInputStream(lib.getFileDirectory(fileName))) {
return IOUtils.toByteArray(out);
} catch (IOException e) {
e.printStackTrace();
}
return null;
}
As I said this works fine for about 10 minutes. It does it both if I use :3000 (the frontend) and if I directly hit the URL fetching the video (:8080/getVideo)
Any help is appreciated I've never worked with multimedia in Spring Boot before.

I would suggest using the StreamingResponseBody or to directly write to the OutputStream of the HttpServletRequest. Together with setting the content type and size of the response.
#GetMapping(value = "/getVideo", produces = "video/mp4")
public void getVideo(#RequestParam(value = "library") String libraryName,
#RequestParam(value = "fileName") String fileName, HttpServletResponse response) {
Library lib = librariesConf.getConfig().getLibraries().get(libraryName.toLowerCase());
Video video = lib.getVideoFiles().get(fileName);
lib.getRecentlyViewed().add(video);
librariesConf.getConfig().getLibraries().keySet().forEach((libr) -> System.out.println("LIBR: :" + libr));
try (FileInputStream out = new FileInputStream(lib.getFileDirectory(fileName))) {
return StreamUtils.copy(out, response.getOutputStream());
} catch (IOException e) {
e.printStackTrace();
}
}
Or the same using a StreamingResponseBody
#GetMapping(value = "/getVideo", produces = "video/mp4")
public StreamingResponseBody getVideo(#RequestParam(value = "library") String libraryName,
#RequestParam(value = "fileName") String fileName) {
Library lib = librariesConf.getConfig().getLibraries().get(libraryName.toLowerCase());
Video video = lib.getVideoFiles().get(fileName);
lib.getRecentlyViewed().add(video);
librariesConf.getConfig().getLibraries().keySet().forEach((libr) -> System.out.println("LIBR: :" + libr));
return new StreamingResponseBody() {
public void write(OutputStream out2) throws IOException {
try (FileInputStream out = new FileInputStream(lib.getFileDirectory(fileName))) {
StreamUtils.copy(out, out2);
}
}
};
}

Slack request verification: Can't compute matching request digest using signed secret

I'm implementing interactive messages on Slack, which contains some action buttons. Using Slack App I'm able to handle Slack users clicking the buttons on my Java Springboot API.
To this moment, everything is fine. However, I struggle to compute matching request signature (digest) to verify, that it actually comes from Slack. I read all the documentation for that on Slack verification documentation page.
The page decribes, that the signature has to be computed as a HMAC SHA256 hash, using Signing Secret as a key and content as concatenation of slack version, timestamp and request body, for example:
v0:123456789:command=/weather&text=94070
On the page is stated:
...Evaluate only the raw HTTP request body when computing signatures.
... so I'm not encoding/deserializing the request before hash computing (I've attached my received request from Slack below)
To compute the hash I use the code found on StackOverflow:
private String computeMessageDigest(String content) {
final String ALGORITHM = "HmacSHA256";
final String UTF_8 = "UTF-8";
try {
Key signingKey = new SecretKeySpec(signingSecret.getBytes(UTF_8), ALGORITHM);
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(signingKey);
return Hex.encodeHexString(mac.doFinal(content.getBytes(UTF_8)));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
I tried also this online hash generator to compare the results, and they were the same.
The request received from Slack looks like this:
{
"headers": {
"x-forwarded-for": ["::ffff:52.72.111.29"],
"x-forwarded-proto": ["https"],
"x-pagekite-port": ["443"],
"host": ["inqool.pagekite.me"],
"user-agent": ["Slackbot 1.0 (+https://api.slack.com/robots)"],
"accept-encoding": ["gzip,deflate"],
"accept": ["application/json,*/*"],
"x-slack-signature": ["v0=87fbffb089501ba823991cc20058df525767a8a2287b3809f9afff3e3b600dd8"],
"x-slack-request-timestamp": ["1531221943"],
"content-length": ["2731"],
"Content-Type": ["application/x-www-form-urlencoded;charset=UTF-8"]
},
"body": "payload=%7B%22type%22%3A%22interactive_message%22%2C%22actions%22%3A%5B%7B%22name%22%3A%22reject_btn%22%2C%22type%22%3A%22button%22%2C%22value%22%3A%22false%22%7D%5D%2C%22callback_id%22%3A%22artwork%3D40d7a87f-466c-4fc9-b454-09ce020d4465%22%2C%22team%22%3A%7B%22id%22%3A%22T03NP6SA7%22%2C%22domain%22%3A%22artstaq%22%7D%2C%22channel%22%3A%7B%22id%22%3A%22G8F2WR4FJ%22%2C%22name%22%3A%22privategroup%22%7D%2C%22user%22%3A%7B%22id%22%3A%22U66T9QX60%22%2C%22name%22%3A%22majo%22%7D%2C%22action_ts%22%3A%221531221943.512498%22%2C%22message_ts%22%3A%221531221198.000225%22%2C%22attachment_id%22%3A%221%22%2C%22token%22%3A%22ZABrZDXgJCOOLNau5mXnfNQR%22%2C%22is_app_unfurl%22%3Afalse%2C%22original_message%22%3A%7B%22text%22%3A%22User+just+put+item+on+*EXCHANGE*.%22%2C%22bot_id%22%3A%22BBM1W4QEL%22%2C%22attachments%22%3A%5B%7B%22author_name%22%3A%22Slack+Test%3B+slack%40test.com%22%2C%22callback_id%22%3A%22artwork%3D40d7a87f-466c-4fc9-b454-09ce020d4465%22%2C%22fallback%22%3A%22Slack+Test%3B+%3Cmailto%3Aslack%40test.com%7Cslack%40test.com%3E+just+put+item+Panenka+%5C%2F+Doll+by+artist+Jaroslav+Vale%5Cu010dka+into+ON+REQUEST+mode%22%2C%22text%22%3A%22%3Chttp%3A%5C%2F%5C%2Flocalhost%3A8080%5C%2Fartist%5C%2F609cd328-d533-4ab0-b982-ec2f104476f2%7CJaroslav+Vale%5Cu010dka%3E%22%2C%22title%22%3A%22Panenka+%5C%2F+Doll%22%2C%22footer%22%3A%22ARTSTAQ+Slack+Reporter%22%2C%22id%22%3A1%2C%22title_link%22%3A%22http%3A%5C%2F%5C%2Flocalhost%3A8080%5C%2Fartwork%5C%2F40d7a87f-466c-4fc9-b454-09ce020d4465%22%2C%22color%22%3A%22f0d0ad%22%2C%22fields%22%3A%5B%7B%22title%22%3A%22Trading+type%22%2C%22value%22%3A%22ON+REQUEST%22%2C%22short%22%3Atrue%7D%5D%2C%22actions%22%3A%5B%7B%22id%22%3A%221%22%2C%22name%22%3A%22approve_btn%22%2C%22text%22%3A%22APPROVE%22%2C%22type%22%3A%22button%22%2C%22value%22%3A%22true%22%2C%22style%22%3A%22primary%22%2C%22confirm%22%3A%7B%22text%22%3A%22Do+you+really+want+to+approve+this+artwork%3F%22%2C%22title%22%3A%22Approve+artwork%22%2C%22ok_text%22%3A%22Yes%22%2C%22dismiss_text%22%3A%22Cancel%22%7D%7D%2C%7B%22id%22%3A%222%22%2C%22name%22%3A%22reject_btn%22%2C%22text%22%3A%22REJECT%22%2C%22type%22%3A%22button%22%2C%22value%22%3A%22false%22%2C%22style%22%3A%22danger%22%2C%22confirm%22%3A%7B%22text%22%3A%22Do+you+really+want+to+reject+this+artwork%3F%22%2C%22title%22%3A%22Reject+artwork%22%2C%22ok_text%22%3A%22Yes%22%2C%22dismiss_text%22%3A%22Cancel%22%7D%7D%5D%7D%5D%2C%22type%22%3A%22message%22%2C%22subtype%22%3A%22bot_message%22%2C%22ts%22%3A%221531221198.000225%22%7D%2C%22response_url%22%3A%22https%3A%5C%2F%5C%2Fhooks.slack.com%5C%2Factions%5C%2FT03NP6SA7%5C%2F395760858899%5C%2FGlP9jsNQak7FqEciEHhscx4L%22%2C%22trigger_id%22%3A%22395632563524.3771230347.851ab60578de033398338a9faeb41a15%22%7D"
}
When I computed the HMAC SHA256 hash, I got 561034bb6860c07a6b4eaf245b6da3ea869c7806c7f7be20b1a830b6d25c54c8 but I should get 87fbffb089501ba823991cc20058df525767a8a2287b3809f9afff3e3b600dd8, as in the request header.
I also tried to compute the hash from the URL decoded body, but still not be able to get the matching signature.
Am I doing something wrong? Thanks for the answers/hints.
EDIT: here's the whole source code of my REST controller and request verifier:
package com.artstaq.resource;
import com.artstaq.integration.slack.SlackRequestVerifier;
import org.springframework.http.HttpEntity;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import javax.inject.Inject;
#RestController
#RequestMapping("/content_admin")
public class ContentAdminResource {
private SlackRequestVerifier slackVerifier;
#RequestMapping(value = "/slack/artwork/resolve", method = RequestMethod.POST, consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
public void resolve(HttpEntity<String> request) {
slackVerifier.verifySlackRequest(request);
}
#Inject
public void setSlackVerifier(SlackRequestVerifier slackVerifier) {
this.slackVerifier = slackVerifier;
}
}
package com.artstaq.integration.slack;
import com.artstaq.exception.SignatureVerificationException;
import com.artstaq.exception.TimestampTooOldException;
import org.apache.commons.codec.binary.Hex;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpEntity;
import org.springframework.stereotype.Component;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.security.Key;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
/**
* Class providing request verification received from Slack
*/
#Component
public class SlackRequestVerifier {
#Value("${integration.slack.version:v0}")
private String version;
#Value("${integration.slack.signingSecret}")
private String signingSecret;
/**
* Verifies the integrity of received Slack request.
*/
public void verifySlackRequest(HttpEntity<String> request) {
String timestamp = request.getHeaders().getFirst(SlackHeaders.TIMESTAMP);
Instant timeInstant = Instant.ofEpochSecond(Long.valueOf(timestamp));
if (timeInstant.plus(5, ChronoUnit.MINUTES).compareTo(Instant.now()) < 0) {
throw new TimestampTooOldException(timeInstant);
}
String expectedDigest = request.getHeaders().getFirst(SlackHeaders.SIGNATURE);
String basestring = String.join(":", version, timestamp, request.getBody());
String computedDigest = version + "=" + computeMessageDigest(basestring);
if (!computedDigest.equals(expectedDigest)) {
throw new SignatureVerificationException(expectedDigest, computedDigest);
}
}
/**
* Compute HMAC SHA256 digest for given content using defined slack signing secret
*/
private String computeMessageDigest(String content) {
final String ALGORITHM = "HmacSHA256";
final String UTF_8 = "UTF-8";
try {
Key signingKey = new SecretKeySpec(signingSecret.getBytes(UTF_8), ALGORITHM);
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(signingKey);
return Hex.encodeHexString(mac.doFinal(content.getBytes(UTF_8)));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private static class SlackHeaders {
private static final String TIMESTAMP = "X-Slack-Request-Timestamp";
private static final String SIGNATURE = "X-Slack-Signature";
}
}

I stumbled upon the very same problem on a Node.js implementation, and found this Medium article which states the following:
Note: We cannot use the built-in querystring Node package because it only supports RFC3986 space encoding and Slack requires us to implement RFC1738 space encoding.
What's the difference between both encodings? The way spaces are parsed:
RFC3986 will convert " " to "%20"
RFC1738 will convert " " to "+"
For Node.js, it suggests to install qs and to use it like this:
qs.stringify(req.body, { format : 'RFC1738' });

The following worked for us:
public enum SigningVerification {
VERIFIED,
DENIED
}
public SigningVerification verify(ImmutableSigningSecretRequest request) {
String basestring = String.join(":", "v0", request.timestamp(), request.body());
SecretKeySpec secret_key = new SecretKeySpec(signingSecret.getBytes(), "HmacSHA256");
Mac sha256_HMAC = Try.of(() -> Mac.getInstance("HmacSHA256")).getOrElseThrow((SupplierRuntimeException) RuntimeException::new);
Try.run(() -> sha256_HMAC.init(secret_key));
String hash = "v0=" + Hex.encodeHexString(sha256_HMAC.doFinal(basestring.getBytes()));
return hash.equals(request.verificationSignature()) ? VERIFIED : DENIED;
}
Controller:
#PostMapping("/command")
public RichMessage postCommand(#RequestHeader(value = "X-Slack-Request-Timestamp") String timestamp,
#RequestHeader(value = "X-Slack-Signature") String signature,
#RequestParam(value = "text", required = false) String message,
#RequestBody String body) {
SigningSecretVerification.SigningVerification verification = verifier.verify(ImmutableSigningSecretRequest
.builder()
.timestamp(timestamp)
.verificationSignature(signature)
.body(body)
.build()
);
return new RichMessage(message);
}
We basically just followed the steps in the Slack doc and it works fine.

I had the same problem, using Spring's #RequestBody.
After going to the trouble of setting up a mitmproxy between Slack and my Spring app in order to compare request bodies, it turned out that Spring was decoding e.g. asterisk characters instead of leaving them as %2A.
The fix for me was to switch to asking for a [HttpServletRequest](https://github.com/boclips/terry/commit/c51382a5a6a9e96d5b19e22b038654bfb19b65b0#diff-79f3c274c9fa96261f8c9e09306a088bR37) (doesn't need a Spring annotation) and reading the raw body from it using `request.reader.use { it.readText() }` (using Kotlin's `use` to close the reader object after reading).
EDIT: the above technique doesn't work, and getting the raw request body from Spring is a mission in itself! Ongoing.

we just stumbled along with the exact same problem.
Your tip about the asterisk decoding helped us a lot!
I don't know if you already solved your issue with caching the request, but maybe you want to take a look on our open source SlackBot SDK for Spring boot, where we were able to solve that issue: https://github.com/kreait/slack-spring-boot-starter/blob/master/starter/slack-spring-boot/src/main/kotlin/io/olaph/slack/broker/security/VerificationMethodArgumentResolver.kt
This VerificationMethodArgumentResolver basically receives the request, wraps it in a ContentCachingRequestWrapper, and invokes the internalResolveArgument of the normal ArgumentResolvers and verifies the request using the cached request. The tricky part here is, that the cache is empty until you've requested its parameterMap. So it is important to validate the signing after you've consumed the request.

I also got bitten by this. Using #RequestBody does not give you back the original body as #andrew-bruce also noted. Specifically for me it failed on the original %2A ending up as an unencoded * when getting the body like that. Obviously that fails the verification.
I ended up with this solution:
a filter combined with a HttpServletRequest wrapper that allows to read the body multiple times:
import org.springframework.web.filter.OncePerRequestFilter
import javax.servlet.FilterChain
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse
/**
* To verify if requests are coming from Slack we need to implement this:
* https://api.slack.com/authentication/verifying-requests-from-slack. Luckily the Bolt framework already implements
* this for us, however we need to provide it with a body that is unaltered. Somewhere in Springs filterchain Spring
* will already have consumed the [HttpServletRequest#inputstream], so we cannot get it from the [HttpServletRequest]
* directly. Spring obviously provides a [org.springframework.web.bind.annotation.RequestBody] annotation, but this is
* slightly different from the original body. This servlet filter will be put as the very first in
* the chain (see [SlackConfig#multiReadRequestFilter] should make sure
* that we can re-read it and construct the raw body, so that the verification doesn't fail.
*/
class MultiReadHttpServletFilter : OncePerRequestFilter() {
override fun doFilterInternal(request: HttpServletRequest, response: HttpServletResponse, filterChain: FilterChain) {
val multiReadHttpServletRequest = MultiReadHttpServletRequest(request)
filterChain.doFilter(multiReadHttpServletRequest, response)
}
}
import org.apache.commons.io.IOUtils
import java.io.ByteArrayInputStream
import java.io.IOException
import javax.servlet.ReadListener
import javax.servlet.ServletInputStream
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletRequestWrapper
class MultiReadHttpServletRequest(request: HttpServletRequest) : HttpServletRequestWrapper(request) {
private var body: ByteArray = IOUtils.toByteArray(request.inputStream)
#Throws(IOException::class)
override fun getInputStream(): ServletInputStream {
return object : ServletInputStream() {
val bais = ByteArrayInputStream(body)
override fun isReady(): Boolean = true
override fun isFinished(): Boolean = bais.available() == 0
override fun read(): Int = bais.read()
override fun setReadListener(readListener: ReadListener) {
throw NotImplementedError("Not implemented!")
}
}
}
}
Configure it as first in the chain. Because this filter actually consumes things in-memory I've specifically targeted a specific path, so it's only for the incoming Slack events in this case.
#Bean
fun multiReadRequestFilter(): FilterRegistrationBean<MultiReadHttpServletFilter> {
// this needs to match the path(s) of the controller SlackAppController
val urlPatterns = slacks.allByKey().keys.map { "/v2/$it/slack/events" }.toTypedArray()
val registrationBean = FilterRegistrationBean<MultiReadHttpServletFilter>()
registrationBean.filter = MultiReadHttpServletFilter()
registrationBean.addUrlPatterns(*urlPatterns)
registrationBean.order = Ordered.HIGHEST_PRECEDENCE
return registrationBean
}
Now I can use it to retrieve the 'real' raw body:
#RestController
#RequestMapping("/v2/{slackId}/slack/events")
class SlackAppController() {
// ...
#PostMapping
fun handle(
#PathVariable("slackId") slackId: String,
httpServletRequest: HttpServletRequest,
#RequestParam queryStringParams: MultiValueMap<String, String>,
#RequestHeader headers: MultiValueMap<String, String>): ResponseEntity<*> {
val body = IOUtils.toString(httpServletRequest.inputStream, StandardCharsets.UTF_8)
// ...
}
}

I was having the same problem, In my controller I was receiving the body of the request as a Map, I received all the values but when I was calculating the hash I saw that the slack-signature and my hash wasn't the same.
I tried to receive the request body as a String just like the #Stefan solution and that works for me, so, instead using HttpEntity<String> in your controller, you must receive the body as plain String with #RequestBody String body in your method argument, the reason is that slack sends encoded values in the request, %2F or %3A, with HttpEntity or Map, spring interprets that values as / and : and this is the reason why your hash is not equals as slack signature.
Hope this help you.

Here's what I found out on the topic:
Spring MVC will read the body of the request and will return a different body, where the order of parameters is changed. The reading part usually happens inside one of the first filters on the chain HiddenHttpMethodFilter and this is the main reason why my signature verification failed.
The request is "reconstructed" incorrectly here ServletServerHttpRequest I'm not sure if this should be filed as a bug or not, but it's definitely messed up.
If you use an injected HttpEntity<String> or #RequestBody String body you will receive the wrong body, not the raw content but the "reconstructed" one
And now the solution:
Create a filter for verifying Slack signatures and register it with the highest priority so it's on to top of the filter chain:
#Bean
public FilterRegistrationBean<SlackVerificationFilter> slackVerificationFilterRegistrationBean() {
String path = "/slack";
FilterRegistrationBean<SlackVerificationFilter> frb = new FilterRegistrationBean<>(new SlackVerificationFilter());
frb.setDispatcherTypes(EnumSet.of(DispatcherType.REQUEST));
frb.setName("csrfFilter");
frb.setAsyncSupported(true);
frb.addUrlPatterns(path);
frb.setMatchAfter(false);
frb.setEnabled(true);
frb.setOrder(Ordered.HIGHEST_PRECEDENCE);
return frb;
}
Inside the filter, wrap the request with some sort of HttpServletRequestWrapper like this:
public class SlackVerificationFilter extends GenericFilterBean {
public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
final BufferedRequestWrapper request = new BufferedRequestWrapper((HttpServletRequest) req);
final HttpServletResponse response = (HttpServletResponse) res;
String rawBody = IOUtils.toString(request.getInputStream(), "UTF-8");
// do signature verification here
chain.doFilter(request, response);
}
}
I won't go into details about the request wrapper. There are lots of examples of it on this site and elsewhere.
Your HttpServletRequestWrapper must implement the following methods:
public ServletInputStream getInputStream();
public BufferedReader getReader() throws IOException;
public Map<String, String[]> getParameterMap();
public String getParameter(String name);
After this you should no longer have problems verifying Slack signatures.
In my case, I didn't have issues with any of the encoded characters mentioned above (%20, %2A, etc). I only had problems verifying signatures of slash command requests. Message action request were verified correctly because they only had 1 request parameter in the body (payload).

I had this same issue using #RequestBody and not being able to validate the request. Here's how I fixed it in Kotlin. It should translate to Java fairly easily.
import org.springframework.web.bind.annotation.*
import org.apache.commons.io.IOUtils
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import javax.servlet.http.HttpServletRequest
import org.apache.commons.codec.binary.Hex
fun validateSlackWebhook(
#RequestHeader("X-Slack-Request-Timestamp") slackRequestTimestamp: String,
#RequestHeader("X-Slack-Signature") slackSignature: String,
request: HttpServletRequest,
) {
val body = IOUtils.toString(request.reader)
val basestring = "v0:$slackRequestTimestamp:$body"
val sha256Hmac = Mac.getInstance("HmacSHA256")
val secretKey = SecretKeySpec(slackSigningSecret.toByteArray(), "HmacSHA256")
sha256Hmac.init(secretKey)
val finalHex = "v0=${Hex.encodeHexString(sha256Hmac.doFinal(basestring.toByteArray()))}"
val validated = finalHex.compareTo(slackSignature) == 0
if (!validated) {
// Code to run if request was not validated
return
}
// Code to run if request was validated
}

I finally solved it by looking at the official SDK source code of Slack!
https://github.com/slackapi/java-slack-sdk/blob/f283e45601157a0d2483ea3d3e8074e80b81a0e6/slack-app-backend/src/main/java/com/slack/api/app_backend/SlackSignature.java#L88-L122
public String generate(String slackRequestTimestamp, String requestBody) {
if (slackRequestTimestamp == null) {
return null;
}
// 1) Retrieve the X-Slack-Request-Timestamp header on the HTTP request, and the body of the request.
// "slackRequestTimestamp" here
// 2) Concatenate the version number, the timestamp, and the body of the request to form a basestring.
// Use a colon as the delimiter between the three elements.
// For example, v0:123456789:command=/weather&text=94070. The version number right now is always v0.
String baseString = "v0:" + slackRequestTimestamp + ":" + requestBody;
// 3) With the help of HMAC SHA256 implemented in your favorite programming, hash the above basestring,
// using the Slack Signing Secret as the key.
SecretKeySpec sk = new SecretKeySpec(slackSigningSecret.getBytes(), ALGORITHM);
try {
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(sk);
byte[] macBytes = mac.doFinal(baseString.getBytes());
StringBuilder hashValue = new StringBuilder(2 * macBytes.length);
for (byte macByte : macBytes) {
hashValue.append(String.format("%02x", macByte & 0xff));
}
return "v0=" + hashValue.toString();
// 4) Compare this computed signature to the X-Slack-Signature header on the request.
} catch (NoSuchAlgorithmException | InvalidKeyException e) {
log.error("Failed to hash the base string value with HMAC-SHA256 because {}", e.getMessage(), e);
return null;
}
}

How to get a file content from WebDAV via Apache Jackrabbit library?

There is a good example showing how to put a file onto WebDAV server:
Java: How to upload a file to a WebDAV server from a servlet?
But how can I get a file content?
There is MethodPut class for the PUT command, but there is no appropriate GetMethod (although enum DavMethods.METHOD_GET is presented).

I solved the task 1) implementing my own class for the GET method 2) reading response bytes which represents file's content. I'd prefer to find a simpler solution in Jackrabbit still.
public class MyGetMethod extends DavMethodBase {
public MyGetMethod(String uri) {
super(uri);
}
public String getName() {
return DavMethods.METHOD_GET;
}
public boolean isSuccess(int statusCode) {
return statusCode == 200;
}
}
static void jackrabbitGet() throws Exception {
HttpClient client = new HttpClient();
Credentials creds = new UsernamePasswordCredentials("user", "pass");
client.getState().setCredentials(AuthScope.ANY, creds);
MyGetMethod method = new MyGetMethod(url goes here);
client.executeMethod(method);
if (method.isSuccess(method.getStatusCode())) {
byte[] resp = method.getResponseBody();
System.out.println("Got response: " + resp.length + " bytes");
}
}

Limit output payload response in CXF JAX-RS based service

I have multiple jax-rs services built using cxf/spring. I want to control the output payload response size of all services. For simplicity sake, let's say none of api's in any of the services should ever return a JSON response payload more than 500 characters and I want to control this in one place instead of relying on individual services to adhere to this requirement. (We already have other features built into the custom framework/base component that all services depend on).
I have tried implementing this using JAX-RS's WriterInterceptor, ContainerResponseFilter and CXF's Phase Interceptor, but none of the approaches seem to be completely satisfy my requirement. More details on what I've done so far:
Option 1: (WriterInteceptor) In the overridden method, I get the ouputstream and set the max size of the cache to 500. When I invoke an api that returns more than 500 characters in the response payload, I get an HTTP 400 Bad Request status, but the response body contains the entire JSON payload.
#Provider
public class ResponsePayloadInterceptor implements WriterInterceptor {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
#Override
public void aroundWriteTo(WriterInterceptorContext context) throws IOException, WebApplicationException {
final OutputStream outputStream = context.getOutputStream();
CacheAndWriteOutputStream cacheAndWriteOutputStream = new CacheAndWriteOutputStream(outputStream);
cacheAndWriteOutputStream.setMaxSize(500);
context.setOutputStream(cacheAndWriteOutputStream);
context.proceed();
}
}
Option 2a: (CXF Phase Inteceptor) In the overridden method, I get the response as String from the ouputstream and check it's size. If it's greater than 500, I create a new Response object with only the data Too much data and set it in the message. Even if the response is > 500 characters, I get an HTTP 200 OK status with the entire JSON. Only when I use the phase as POST_MARSHAL or a later phase, I'm able to get hold of the JSON response and check it's length, but by that time the response has already been streamed to the client.
#Provider
public class ResponsePayloadInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
public ResponsePayloadInterceptor() {
super(Phase.POST_MARSHAL);
}
#Override
public void handleMessage(Message message) throws Fault {
LOGGER.info("handleMessage() - Response intercepted");
try {
OutputStream outputStream = message.getContent(OutputStream.class);
...
CachedOutputStream cachedOutputStream = (CachedOutputStream) outputStream;
String responseBody = IOUtils.toString(cachedOutputStream.getInputStream(), "UTF-8");
...
LOGGER.info("handleMessage() - Response: {}", responseBody);
LOGGER.info("handleMessage() - Response Length: {}", responseBody.length());
if (responseBody.length() > 500) {
Response response = Response.status(Response.Status.BAD_REQUEST)
.entity("Too much data").build();
message.getExchange().put(Response.class, response);
}
} catch (IOException e) {
LOGGER.error("handleMessage() - Error");
e.printStackTrace();
}
}
}
Option 2b: (CXF Phase Inteceptor) Same as above, but only the contents of if block is changed. If response length is greater than 500, I create a new output stream with the string Too much data and set it in message. But if the response payload is > 500 characters, I still get an HTTP 200 OK status with an invalid JSON response (entire JSON + additional text) i.e., the response looks like this: [{"data":"", ...}, {...}]Too much data (the text 'Too much data' is appended to the JSON)
if (responseBody.length() > 500) {
InputStream inputStream = new ByteArrayInputStream("Too much data".getBytes("UTF-8"));
outputStream.flush();
IOUtils.copy(inputStream, outputStream);
OutputStream out = new CachedOutputStream();
out.write("Too much data".getBytes("UTF-8"));
message.setContent(OutputStream.class, out);
}
Option 3: (ContainerResponseFilter) Using the ContainerResponseFilter, I added a Content-Length response header with value as 500. If response length is > 500, I get an HTTP 200 OK status with an invalid JSON response (truncated to 500 characters). If the response length is < 500, still get an HTTP 200 OK status, but the client waits for more data to be returned by the server (as expected) and times out, which isn't a desirable solution.
#Provider
public class ResponsePayloadFilter implements ContainerResponseFilter {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadFilter.class);
#Override
public void filter(ContainerRequestContext requestContext, ContainerResponseContext responseContext) throws IOException {
LOGGER.info("filter() - Response intercepted");
CachedOutputStream cos = (CachedOutputStream) responseContext.getEntityStream();
StringBuilder responsePayload = new StringBuilder();
ByteArrayOutputStream out = new ByteArrayOutputStream();
if (cos.getInputStream().available() > 0) {
IOUtils.copy(cos.getInputStream(), out);
byte[] responseEntity = out.toByteArray();
responsePayload.append(new String(responseEntity));
}
LOGGER.info("filter() - Content: {}", responsePayload.toString());
responseContext.getHeaders().add("Content-Length", "500");
}
}
Any suggestions on how I can tweak the above approaches to get what I want or any other different pointers?

I resolved this partially using help from this answer. I say partially because I'm successfully able to control the payload, but the not the response status code. Ideally, if the response length is greater than 500 and I modify the message content, I would like to send a different response status code (other than 200 OK). But this is a good enough solution for me to proceed at this point. If I figure out how to update the status code as well, I'll come back and update this answer.
import org.apache.commons.io.IOUtils;
import org.apache.cxf.interceptor.Fault;
import org.apache.cxf.io.CachedOutputStream;
import org.apache.cxf.message.Message;
import org.apache.cxf.phase.AbstractPhaseInterceptor;
import org.apache.cxf.phase.Phase;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
public class ResponsePayloadInterceptor extends AbstractPhaseInterceptor<Message> {
private static final Logger LOGGER = LoggerFactory.getLogger(ResponsePayloadInterceptor.class);
public ResponsePayloadInterceptor() {
super(Phase.PRE_STREAM);
}
#Override
public void handleMessage(Message message) throws Fault {
LOGGER.info("handleMessage() - Response intercepted");
try {
OutputStream outputStream = message.getContent(OutputStream.class);
CachedOutputStream cachedOutputStream = new CachedOutputStream();
message.setContent(OutputStream.class, cachedOutputStream);
message.getInterceptorChain().doIntercept(message);
cachedOutputStream.flush();
cachedOutputStream.close();
CachedOutputStream newCachedOutputStream = (CachedOutputStream) message.getContent(OutputStream.class);
String currentResponse = IOUtils.toString(newCachedOutputStream.getInputStream(), "UTF-8");
newCachedOutputStream.flush();
newCachedOutputStream.close();
if (currentResponse != null) {
LOGGER.info("handleMessage() - Response: {}", currentResponse);
LOGGER.info("handleMessage() - Response Length: {}", currentResponse.length());
if (currentResponse.length() > 500) {
InputStream replaceInputStream = IOUtils.toInputStream("{\"message\":\"Too much data\"}", "UTF-8");
IOUtils.copy(replaceInputStream, outputStream);
replaceInputStream.close();
message.setContent(OutputStream.class, outputStream);
outputStream.flush();
outputStream.close();
} else {
InputStream replaceInputStream = IOUtils.toInputStream(currentResponse, "UTF-8");
IOUtils.copy(replaceInputStream, outputStream);
replaceInputStream.close();
message.setContent(OutputStream.class, outputStream);
outputStream.flush();
outputStream.close();
}
}
} catch (IOException e) {
LOGGER.error("handleMessage() - Error", e);
throw new RuntimeException(e);
}
}

Categories