I am using URL class to read an InputStream from it. Is there any way I can use RestTemplate for this?
InputStream input = new URL(url).openStream();
JsonReader reader = new JsonReader(new InputStreamReader(input, StandardCharsets.UTF_8.displayName()));
How can I get InputStream with RestTemplate instead of using URL?
The previous answers are not wrong, but they don't go into the depth that I like to see. There are cases when dealing with low level InputStream is not only desirable, but necessary, the most common example being streaming a large file from source (some web server) to destination (a database). If you try to use a ByteArrayInputStream, you will be, not so surprisingly, greeted with OutOfMemoryError. Yes, you can roll your own HTTP client code, but you'll have to deal with erroneous response codes, response converters etc. If you are already using Spring, looking to RestTemplate is a natural choice.
As of this writing, spring-web:5.0.2.RELEASE has a ResourceHttpMessageConverter that has a boolean supportsReadStreaming, which if set, and the response type is InputStreamResource, returns InputStreamResource; otherwise it returns a ByteArrayResource. So clearly, you're not the only one that asked for streaming support.
However, there is a problem: RestTemplate closes the response soon after the HttpMessageConverter runs. Thus, even if you asked for InputStreamResource, and got it, it's no good, because the response stream has been closed. I think this is a design flaw that they overlooked; it should've been dependent on the response type. So unfortunately, for reading, you must consume the response fully; you can't pass it around if using RestTemplate.
Writing is no problem though. If you want to stream an InputStream, ResourceHttpMessageConverter will do it for you. Under the hood, it uses org.springframework.util.StreamUtils to write 4096 bytes at a time from the InputStream to the OutputStream.
Some of the HttpMessageConverter support all media types, so depending on your requirement, you may have to remove the default ones from RestTemplate, and set the ones you need, being mindful of their relative ordering.
Last but not the least, implementations of ClientHttpRequestFactory has a boolean bufferRequestBody that you can, and should, set to false if you are uploading a large stream. Otherwise, you know, OutOfMemoryError. As of this writing, SimpleClientHttpRequestFactory (JDK client) and HttpComponentsClientHttpRequestFactory (Apache HTTP client) support this feature, but not OkHttp3ClientHttpRequestFactory. Again, design oversight.
Edit:
Filed ticket SPR-16885.
Spring has a org.springframework.http.converter.ResourceHttpMessageConverter. It converts Spring's org.springframework.core.io.Resource class.
That Resource class encapsulates a InputStream, which you can obtain via someResource.getInputStream().
Putting this all together, you can actually get an InputStream via RestTemplate out-of-the-box by specifying Resource.class as your RestTemplate invocation's response type.
Here is an example using one of RestTemplate's exchange(..) methods:
import org.springframework.web.client.RestTemplate;
import org.springframework.http.HttpMethod;
import org.springframework.core.io.Resource;
ResponseEntity<Resource> responseEntity = restTemplate.exchange( someUrlString, HttpMethod.GET, someHttpEntity, Resource.class );
InputStream responseInputStream;
try {
responseInputStream = responseEntity.getBody().getInputStream();
}
catch (IOException e) {
throw new RuntimeException(e);
}
// use responseInputStream
You should not get the InputStream directly. RestTemplate is meant to encapsulate processing the response (and request) content. Its strength is handling all the IO and handing you a ready-to-go Java object.
One of RestTemplate's original authors, Brian Clozel, has stated:
RestTemplate is not meant to stream the response body; its contract
doesn't allow it, and it's been around for so long that changing such
a basic part of its behavior cannot be done without disrupting many
applications.
You'll need to register appropriate HttpMessageConverter objects. Those will have access to the response's InputStream, through an HttpInputMessage object.
As Abdull suggests, Spring does come with an HttpMessageConverter implementation for Resource which itself wraps an InputStream, ResourceHttpMessageConverter. It doesn't support all Resource types, but since you should be programming to interfaces anyway, you should just use the superinterface Resource.
The current implementation (4.3.5), will return a ByteArrayResource with the content of the response stream copied to a new ByteArrayInputStream which you can access.
You don't have to close the stream. The RestTemplate takes care of that for you. (This is unfortunate if you try to use a InputStreamResource, another type supported by the ResourceHttpMessageConverter, because it wraps the underlying response's InputStream but is closed before it can be exposed to your client code.)
I encountered the same issue and solved it by extending RestTemplate and closing the connection only after the stream is read.
you can see the code here: https://github.com/ItamarBenjamin/stream-rest-template
Thanks to Abhijit Sarkar's answer for leading the way.
I needed to download a heavy JSON stream and break it into small streamable manageable pieces of data.
The JSON is composed of objects that have big properties: such big properties can be serialized to a file, and thus removed from the unmarshalled JSON object.
Another use case is to download a JSON stream object by object, process it like a map/reduce algorythm and produce a single output without having to load the whole stream in memory.
Yet another use case is to read a big JSON file and only pick a few objects based on a condition, while unmarshalling to Plain Old Java Objects.
Here is an example: we'd like to stream a very huge JSON file that is an array, and we'd like to retrieve only the first object in the array.
Given this big file on a server, available at http://example.org/testings.json :
[
{ "property1": "value1", "property2": "value2", "property3": "value3" },
{ "property1": "value1", "property2": "value2", "property3": "value3" },
... 1446481 objects => a file of 104 MB => take quite long to download...
]
Each row of this JSON array can be parsed as this object:
#lombok.Data
public class Testing {
String property1;
String property2;
String property3;
}
You need this class make the parsing code reusable:
import com.fasterxml.jackson.core.JsonParser;
import java.io.IOException;
#FunctionalInterface
public interface JsonStreamer<R> {
/**
* Parse the given JSON stream, process it, and optionally return an object.<br>
* The returned object can represent a downsized parsed version of the stream, or the result of a map/reduce processing, or null...
*
* #param jsonParser the parser to use while streaming JSON for processing
* #return the optional result of the process (can be {#link Void} if processing returns nothing)
* #throws IOException on streaming problem (you are also strongly encouraged to throw HttpMessageNotReadableException on parsing error)
*/
R stream(JsonParser jsonParser) throws IOException;
}
And this class to parse:
import com.fasterxml.jackson.core.JsonFactory;
import com.fasterxml.jackson.core.JsonParser;
import lombok.AllArgsConstructor;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.HttpOutputMessage;
import org.springframework.http.MediaType;
import org.springframework.http.converter.HttpMessageConverter;
import java.io.IOException;
import java.util.Collections;
import java.util.List;
#AllArgsConstructor
public class StreamingHttpMessageConverter<R> implements HttpMessageConverter<R> {
private final JsonFactory factory;
private final JsonStreamer<R> jsonStreamer;
#Override
public boolean canRead(Class<?> clazz, MediaType mediaType) {
return MediaType.APPLICATION_JSON.isCompatibleWith(mediaType);
}
#Override
public boolean canWrite(Class<?> clazz, MediaType mediaType) {
return false; // We only support reading from an InputStream
}
#Override
public List<MediaType> getSupportedMediaTypes() {
return Collections.singletonList(MediaType.APPLICATION_JSON);
}
#Override
public R read(Class<? extends R> clazz, HttpInputMessage inputMessage) throws IOException {
try (InputStream inputStream = inputMessage.getBody();
JsonParser parser = factory.createParser(inputStream)) {
return jsonStreamer.stream(parser);
}
}
#Override
public void write(R result, MediaType contentType, HttpOutputMessage outputMessage) {
throw new UnsupportedOperationException();
}
}
Then, here is the code to use to stream the HTTP response, parse the JSON array and return only the first unmarshalled object:
// You should #Autowire these:
JsonFactory jsonFactory = new JsonFactory();
ObjectMapper objectMapper = new ObjectMapper();
RestTemplateBuilder restTemplateBuilder = new RestTemplateBuilder();
// If detectRequestFactory true (default): HttpComponentsClientHttpRequestFactory will be used and it will consume the entire HTTP response, even if we close the stream early
// If detectRequestFactory false: SimpleClientHttpRequestFactory will be used and it will close the connection as soon as we ask it to
RestTemplate restTemplate = restTemplateBuilder.detectRequestFactory(false).messageConverters(
new StreamingHttpMessageConverter<>(jsonFactory, jsonParser -> {
// While you use a low-level JsonParser to not load everything in memory at once,
// you can still profit from smaller object mapping with the ObjectMapper
if (!jsonParser.isClosed() && jsonParser.nextToken() == JsonToken.START_ARRAY) {
if (!jsonParser.isClosed() && jsonParser.nextToken() == JsonToken.START_OBJECT) {
return objectMapper.readValue(jsonParser, Testing.class);
}
}
return null;
})
).build();
final Testing firstTesting = restTemplate.getForObject("http://example.org/testings.json", Testing.class);
log.debug("First testing object: {}", firstTesting);
You can pass in your own response extractor. Here is an example where I write out the json to disk in a streaming fashion -
RestTemplate restTemplate = new RestTemplateBuilder().basicAuthentication("user", "their_password" ).build();
int responseSize = restTemplate.execute(uri,
HttpMethod.POST,
(ClientHttpRequest requestCallback) -> {
requestCallback.getHeaders().setContentType(MediaType.APPLICATION_JSON);
requestCallback.getBody().write(body.getBytes());
},
responseExtractor -> {
FileOutputStream fos = new FileOutputStream(new File("out.json"));
return StreamUtils.copy(responseExtractor.getBody(), fos);
}
)
Very simple, yet efficient solution would be using ResponseExtractor. It's especially useful when you want to operate on very large InputStream and your RAM is limited.
Here is how you should be implementing it:
public void consumerInputStreamWithoutBuffering(String url, Consumer<InputStream> streamConsumer) throws IOException {
final ResponseExtractor responseExtractor =
(ClientHttpResponse clientHttpResponse) -> {
streamConsumer.accept(clientHttpResponse.getBody());
return null;
};
restTemplate.execute(url, HttpMethod.GET, null, responseExtractor);
}
And then, invoke the method anywhere you need:
Consumer<InputStream> doWhileDownloading = inputStream -> {
//Use inputStream for your business logic...
};
consumerInputStreamWithoutBuffering("https://localhost.com/download", doWhileDownloading);
Please, be aware of the following common pitfall:
public InputStream getInputStreamFromResponse(String url) throws IOException {
final ResponseExtractor<InputStream> responseExtractor =
clientHttpResponse -> clientHttpResponse.getBody();
return restTemplate.execute(url, HttpMethod.GET, null, responseExtractor);
}
Here InputStream will be closed before you can access it
I solve it by doing that.
I hope it will help you all.
#GetMapping("largeFile")
public ResponseEntity<InputStreamResource> downloadLargeFile(
#RequestParam("fileName") String fileName
) throws IOException {
RestTemplate restTemplate = new RestTemplate();
// Optional Accept header
RequestCallback requestCallback = request -> request.getHeaders()
.setAccept(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.ALL));
// Streams the response instead of loading it all in memory
ResponseExtractor<InputStreamResource> responseExtractor = response -> {
// Here I write the response to a file but do what you like
Path path = Paths.get("tmp/" + fileName);
Files.copy(response.getBody(), path, StandardCopyOption.REPLACE_EXISTING);
return new InputStreamResource(new FileInputStream(String.format("tmp/%s", fileName)));
};
InputStreamResource response = restTemplate.execute(
String.format("http://%s:%s/file/largeFileRestTemplate?fileName=%s", host, "9091", fileName),
HttpMethod.GET,
requestCallback,
responseExtractor
);
return ResponseEntity
.ok()
.header(HttpHeaders.CONTENT_DISPOSITION, String.format("attachment; filename=%s", fileName))
.body(response);
}
As a variant you can consume response as bytes and than convert to stream
byte data[] = restTemplate.execute(link, HttpMethod.GET, null, new BinaryFileExtractor());
return new ByteArrayInputStream(data);
Extractor is
public class BinaryFileExtractor implements ResponseExtractor<byte[]> {
#Override
public byte[] extractData(ClientHttpResponse response) throws IOException {
return ByteStreams.toByteArray(response.getBody());
}
}
Related
I'm implementing interactive messages on Slack, which contains some action buttons. Using Slack App I'm able to handle Slack users clicking the buttons on my Java Springboot API.
To this moment, everything is fine. However, I struggle to compute matching request signature (digest) to verify, that it actually comes from Slack. I read all the documentation for that on Slack verification documentation page.
The page decribes, that the signature has to be computed as a HMAC SHA256 hash, using Signing Secret as a key and content as concatenation of slack version, timestamp and request body, for example:
v0:123456789:command=/weather&text=94070
On the page is stated:
...Evaluate only the raw HTTP request body when computing signatures.
... so I'm not encoding/deserializing the request before hash computing (I've attached my received request from Slack below)
To compute the hash I use the code found on StackOverflow:
private String computeMessageDigest(String content) {
final String ALGORITHM = "HmacSHA256";
final String UTF_8 = "UTF-8";
try {
Key signingKey = new SecretKeySpec(signingSecret.getBytes(UTF_8), ALGORITHM);
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(signingKey);
return Hex.encodeHexString(mac.doFinal(content.getBytes(UTF_8)));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
I tried also this online hash generator to compare the results, and they were the same.
The request received from Slack looks like this:
{
"headers": {
"x-forwarded-for": ["::ffff:52.72.111.29"],
"x-forwarded-proto": ["https"],
"x-pagekite-port": ["443"],
"host": ["inqool.pagekite.me"],
"user-agent": ["Slackbot 1.0 (+https://api.slack.com/robots)"],
"accept-encoding": ["gzip,deflate"],
"accept": ["application/json,*/*"],
"x-slack-signature": ["v0=87fbffb089501ba823991cc20058df525767a8a2287b3809f9afff3e3b600dd8"],
"x-slack-request-timestamp": ["1531221943"],
"content-length": ["2731"],
"Content-Type": ["application/x-www-form-urlencoded;charset=UTF-8"]
},
"body": "payload=%7B%22type%22%3A%22interactive_message%22%2C%22actions%22%3A%5B%7B%22name%22%3A%22reject_btn%22%2C%22type%22%3A%22button%22%2C%22value%22%3A%22false%22%7D%5D%2C%22callback_id%22%3A%22artwork%3D40d7a87f-466c-4fc9-b454-09ce020d4465%22%2C%22team%22%3A%7B%22id%22%3A%22T03NP6SA7%22%2C%22domain%22%3A%22artstaq%22%7D%2C%22channel%22%3A%7B%22id%22%3A%22G8F2WR4FJ%22%2C%22name%22%3A%22privategroup%22%7D%2C%22user%22%3A%7B%22id%22%3A%22U66T9QX60%22%2C%22name%22%3A%22majo%22%7D%2C%22action_ts%22%3A%221531221943.512498%22%2C%22message_ts%22%3A%221531221198.000225%22%2C%22attachment_id%22%3A%221%22%2C%22token%22%3A%22ZABrZDXgJCOOLNau5mXnfNQR%22%2C%22is_app_unfurl%22%3Afalse%2C%22original_message%22%3A%7B%22text%22%3A%22User+just+put+item+on+*EXCHANGE*.%22%2C%22bot_id%22%3A%22BBM1W4QEL%22%2C%22attachments%22%3A%5B%7B%22author_name%22%3A%22Slack+Test%3B+slack%40test.com%22%2C%22callback_id%22%3A%22artwork%3D40d7a87f-466c-4fc9-b454-09ce020d4465%22%2C%22fallback%22%3A%22Slack+Test%3B+%3Cmailto%3Aslack%40test.com%7Cslack%40test.com%3E+just+put+item+Panenka+%5C%2F+Doll+by+artist+Jaroslav+Vale%5Cu010dka+into+ON+REQUEST+mode%22%2C%22text%22%3A%22%3Chttp%3A%5C%2F%5C%2Flocalhost%3A8080%5C%2Fartist%5C%2F609cd328-d533-4ab0-b982-ec2f104476f2%7CJaroslav+Vale%5Cu010dka%3E%22%2C%22title%22%3A%22Panenka+%5C%2F+Doll%22%2C%22footer%22%3A%22ARTSTAQ+Slack+Reporter%22%2C%22id%22%3A1%2C%22title_link%22%3A%22http%3A%5C%2F%5C%2Flocalhost%3A8080%5C%2Fartwork%5C%2F40d7a87f-466c-4fc9-b454-09ce020d4465%22%2C%22color%22%3A%22f0d0ad%22%2C%22fields%22%3A%5B%7B%22title%22%3A%22Trading+type%22%2C%22value%22%3A%22ON+REQUEST%22%2C%22short%22%3Atrue%7D%5D%2C%22actions%22%3A%5B%7B%22id%22%3A%221%22%2C%22name%22%3A%22approve_btn%22%2C%22text%22%3A%22APPROVE%22%2C%22type%22%3A%22button%22%2C%22value%22%3A%22true%22%2C%22style%22%3A%22primary%22%2C%22confirm%22%3A%7B%22text%22%3A%22Do+you+really+want+to+approve+this+artwork%3F%22%2C%22title%22%3A%22Approve+artwork%22%2C%22ok_text%22%3A%22Yes%22%2C%22dismiss_text%22%3A%22Cancel%22%7D%7D%2C%7B%22id%22%3A%222%22%2C%22name%22%3A%22reject_btn%22%2C%22text%22%3A%22REJECT%22%2C%22type%22%3A%22button%22%2C%22value%22%3A%22false%22%2C%22style%22%3A%22danger%22%2C%22confirm%22%3A%7B%22text%22%3A%22Do+you+really+want+to+reject+this+artwork%3F%22%2C%22title%22%3A%22Reject+artwork%22%2C%22ok_text%22%3A%22Yes%22%2C%22dismiss_text%22%3A%22Cancel%22%7D%7D%5D%7D%5D%2C%22type%22%3A%22message%22%2C%22subtype%22%3A%22bot_message%22%2C%22ts%22%3A%221531221198.000225%22%7D%2C%22response_url%22%3A%22https%3A%5C%2F%5C%2Fhooks.slack.com%5C%2Factions%5C%2FT03NP6SA7%5C%2F395760858899%5C%2FGlP9jsNQak7FqEciEHhscx4L%22%2C%22trigger_id%22%3A%22395632563524.3771230347.851ab60578de033398338a9faeb41a15%22%7D"
}
When I computed the HMAC SHA256 hash, I got 561034bb6860c07a6b4eaf245b6da3ea869c7806c7f7be20b1a830b6d25c54c8 but I should get 87fbffb089501ba823991cc20058df525767a8a2287b3809f9afff3e3b600dd8, as in the request header.
I also tried to compute the hash from the URL decoded body, but still not be able to get the matching signature.
Am I doing something wrong? Thanks for the answers/hints.
EDIT: here's the whole source code of my REST controller and request verifier:
package com.artstaq.resource;
import com.artstaq.integration.slack.SlackRequestVerifier;
import org.springframework.http.HttpEntity;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RestController;
import javax.inject.Inject;
#RestController
#RequestMapping("/content_admin")
public class ContentAdminResource {
private SlackRequestVerifier slackVerifier;
#RequestMapping(value = "/slack/artwork/resolve", method = RequestMethod.POST, consumes = MediaType.APPLICATION_FORM_URLENCODED_VALUE)
public void resolve(HttpEntity<String> request) {
slackVerifier.verifySlackRequest(request);
}
#Inject
public void setSlackVerifier(SlackRequestVerifier slackVerifier) {
this.slackVerifier = slackVerifier;
}
}
package com.artstaq.integration.slack;
import com.artstaq.exception.SignatureVerificationException;
import com.artstaq.exception.TimestampTooOldException;
import org.apache.commons.codec.binary.Hex;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpEntity;
import org.springframework.stereotype.Component;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import java.security.Key;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
/**
* Class providing request verification received from Slack
*/
#Component
public class SlackRequestVerifier {
#Value("${integration.slack.version:v0}")
private String version;
#Value("${integration.slack.signingSecret}")
private String signingSecret;
/**
* Verifies the integrity of received Slack request.
*/
public void verifySlackRequest(HttpEntity<String> request) {
String timestamp = request.getHeaders().getFirst(SlackHeaders.TIMESTAMP);
Instant timeInstant = Instant.ofEpochSecond(Long.valueOf(timestamp));
if (timeInstant.plus(5, ChronoUnit.MINUTES).compareTo(Instant.now()) < 0) {
throw new TimestampTooOldException(timeInstant);
}
String expectedDigest = request.getHeaders().getFirst(SlackHeaders.SIGNATURE);
String basestring = String.join(":", version, timestamp, request.getBody());
String computedDigest = version + "=" + computeMessageDigest(basestring);
if (!computedDigest.equals(expectedDigest)) {
throw new SignatureVerificationException(expectedDigest, computedDigest);
}
}
/**
* Compute HMAC SHA256 digest for given content using defined slack signing secret
*/
private String computeMessageDigest(String content) {
final String ALGORITHM = "HmacSHA256";
final String UTF_8 = "UTF-8";
try {
Key signingKey = new SecretKeySpec(signingSecret.getBytes(UTF_8), ALGORITHM);
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(signingKey);
return Hex.encodeHexString(mac.doFinal(content.getBytes(UTF_8)));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private static class SlackHeaders {
private static final String TIMESTAMP = "X-Slack-Request-Timestamp";
private static final String SIGNATURE = "X-Slack-Signature";
}
}
I stumbled upon the very same problem on a Node.js implementation, and found this Medium article which states the following:
Note: We cannot use the built-in querystring Node package because it only supports RFC3986 space encoding and Slack requires us to implement RFC1738 space encoding.
What's the difference between both encodings? The way spaces are parsed:
RFC3986 will convert " " to "%20"
RFC1738 will convert " " to "+"
For Node.js, it suggests to install qs and to use it like this:
qs.stringify(req.body, { format : 'RFC1738' });
The following worked for us:
public enum SigningVerification {
VERIFIED,
DENIED
}
public SigningVerification verify(ImmutableSigningSecretRequest request) {
String basestring = String.join(":", "v0", request.timestamp(), request.body());
SecretKeySpec secret_key = new SecretKeySpec(signingSecret.getBytes(), "HmacSHA256");
Mac sha256_HMAC = Try.of(() -> Mac.getInstance("HmacSHA256")).getOrElseThrow((SupplierRuntimeException) RuntimeException::new);
Try.run(() -> sha256_HMAC.init(secret_key));
String hash = "v0=" + Hex.encodeHexString(sha256_HMAC.doFinal(basestring.getBytes()));
return hash.equals(request.verificationSignature()) ? VERIFIED : DENIED;
}
Controller:
#PostMapping("/command")
public RichMessage postCommand(#RequestHeader(value = "X-Slack-Request-Timestamp") String timestamp,
#RequestHeader(value = "X-Slack-Signature") String signature,
#RequestParam(value = "text", required = false) String message,
#RequestBody String body) {
SigningSecretVerification.SigningVerification verification = verifier.verify(ImmutableSigningSecretRequest
.builder()
.timestamp(timestamp)
.verificationSignature(signature)
.body(body)
.build()
);
return new RichMessage(message);
}
We basically just followed the steps in the Slack doc and it works fine.
I had the same problem, using Spring's #RequestBody.
After going to the trouble of setting up a mitmproxy between Slack and my Spring app in order to compare request bodies, it turned out that Spring was decoding e.g. asterisk characters instead of leaving them as %2A.
The fix for me was to switch to asking for a [HttpServletRequest](https://github.com/boclips/terry/commit/c51382a5a6a9e96d5b19e22b038654bfb19b65b0#diff-79f3c274c9fa96261f8c9e09306a088bR37) (doesn't need a Spring annotation) and reading the raw body from it using `request.reader.use { it.readText() }` (using Kotlin's `use` to close the reader object after reading).
EDIT: the above technique doesn't work, and getting the raw request body from Spring is a mission in itself! Ongoing.
we just stumbled along with the exact same problem.
Your tip about the asterisk decoding helped us a lot!
I don't know if you already solved your issue with caching the request, but maybe you want to take a look on our open source SlackBot SDK for Spring boot, where we were able to solve that issue: https://github.com/kreait/slack-spring-boot-starter/blob/master/starter/slack-spring-boot/src/main/kotlin/io/olaph/slack/broker/security/VerificationMethodArgumentResolver.kt
This VerificationMethodArgumentResolver basically receives the request, wraps it in a ContentCachingRequestWrapper, and invokes the internalResolveArgument of the normal ArgumentResolvers and verifies the request using the cached request. The tricky part here is, that the cache is empty until you've requested its parameterMap. So it is important to validate the signing after you've consumed the request.
I also got bitten by this. Using #RequestBody does not give you back the original body as #andrew-bruce also noted. Specifically for me it failed on the original %2A ending up as an unencoded * when getting the body like that. Obviously that fails the verification.
I ended up with this solution:
a filter combined with a HttpServletRequest wrapper that allows to read the body multiple times:
import org.springframework.web.filter.OncePerRequestFilter
import javax.servlet.FilterChain
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse
/**
* To verify if requests are coming from Slack we need to implement this:
* https://api.slack.com/authentication/verifying-requests-from-slack. Luckily the Bolt framework already implements
* this for us, however we need to provide it with a body that is unaltered. Somewhere in Springs filterchain Spring
* will already have consumed the [HttpServletRequest#inputstream], so we cannot get it from the [HttpServletRequest]
* directly. Spring obviously provides a [org.springframework.web.bind.annotation.RequestBody] annotation, but this is
* slightly different from the original body. This servlet filter will be put as the very first in
* the chain (see [SlackConfig#multiReadRequestFilter] should make sure
* that we can re-read it and construct the raw body, so that the verification doesn't fail.
*/
class MultiReadHttpServletFilter : OncePerRequestFilter() {
override fun doFilterInternal(request: HttpServletRequest, response: HttpServletResponse, filterChain: FilterChain) {
val multiReadHttpServletRequest = MultiReadHttpServletRequest(request)
filterChain.doFilter(multiReadHttpServletRequest, response)
}
}
import org.apache.commons.io.IOUtils
import java.io.ByteArrayInputStream
import java.io.IOException
import javax.servlet.ReadListener
import javax.servlet.ServletInputStream
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletRequestWrapper
class MultiReadHttpServletRequest(request: HttpServletRequest) : HttpServletRequestWrapper(request) {
private var body: ByteArray = IOUtils.toByteArray(request.inputStream)
#Throws(IOException::class)
override fun getInputStream(): ServletInputStream {
return object : ServletInputStream() {
val bais = ByteArrayInputStream(body)
override fun isReady(): Boolean = true
override fun isFinished(): Boolean = bais.available() == 0
override fun read(): Int = bais.read()
override fun setReadListener(readListener: ReadListener) {
throw NotImplementedError("Not implemented!")
}
}
}
}
Configure it as first in the chain. Because this filter actually consumes things in-memory I've specifically targeted a specific path, so it's only for the incoming Slack events in this case.
#Bean
fun multiReadRequestFilter(): FilterRegistrationBean<MultiReadHttpServletFilter> {
// this needs to match the path(s) of the controller SlackAppController
val urlPatterns = slacks.allByKey().keys.map { "/v2/$it/slack/events" }.toTypedArray()
val registrationBean = FilterRegistrationBean<MultiReadHttpServletFilter>()
registrationBean.filter = MultiReadHttpServletFilter()
registrationBean.addUrlPatterns(*urlPatterns)
registrationBean.order = Ordered.HIGHEST_PRECEDENCE
return registrationBean
}
Now I can use it to retrieve the 'real' raw body:
#RestController
#RequestMapping("/v2/{slackId}/slack/events")
class SlackAppController() {
// ...
#PostMapping
fun handle(
#PathVariable("slackId") slackId: String,
httpServletRequest: HttpServletRequest,
#RequestParam queryStringParams: MultiValueMap<String, String>,
#RequestHeader headers: MultiValueMap<String, String>): ResponseEntity<*> {
val body = IOUtils.toString(httpServletRequest.inputStream, StandardCharsets.UTF_8)
// ...
}
}
I was having the same problem, In my controller I was receiving the body of the request as a Map, I received all the values but when I was calculating the hash I saw that the slack-signature and my hash wasn't the same.
I tried to receive the request body as a String just like the #Stefan solution and that works for me, so, instead using HttpEntity<String> in your controller, you must receive the body as plain String with #RequestBody String body in your method argument, the reason is that slack sends encoded values in the request, %2F or %3A, with HttpEntity or Map, spring interprets that values as / and : and this is the reason why your hash is not equals as slack signature.
Hope this help you.
Here's what I found out on the topic:
Spring MVC will read the body of the request and will return a different body, where the order of parameters is changed. The reading part usually happens inside one of the first filters on the chain HiddenHttpMethodFilter and this is the main reason why my signature verification failed.
The request is "reconstructed" incorrectly here ServletServerHttpRequest I'm not sure if this should be filed as a bug or not, but it's definitely messed up.
If you use an injected HttpEntity<String> or #RequestBody String body you will receive the wrong body, not the raw content but the "reconstructed" one
And now the solution:
Create a filter for verifying Slack signatures and register it with the highest priority so it's on to top of the filter chain:
#Bean
public FilterRegistrationBean<SlackVerificationFilter> slackVerificationFilterRegistrationBean() {
String path = "/slack";
FilterRegistrationBean<SlackVerificationFilter> frb = new FilterRegistrationBean<>(new SlackVerificationFilter());
frb.setDispatcherTypes(EnumSet.of(DispatcherType.REQUEST));
frb.setName("csrfFilter");
frb.setAsyncSupported(true);
frb.addUrlPatterns(path);
frb.setMatchAfter(false);
frb.setEnabled(true);
frb.setOrder(Ordered.HIGHEST_PRECEDENCE);
return frb;
}
Inside the filter, wrap the request with some sort of HttpServletRequestWrapper like this:
public class SlackVerificationFilter extends GenericFilterBean {
public void doFilter(ServletRequest req, ServletResponse res, FilterChain chain) throws IOException, ServletException {
final BufferedRequestWrapper request = new BufferedRequestWrapper((HttpServletRequest) req);
final HttpServletResponse response = (HttpServletResponse) res;
String rawBody = IOUtils.toString(request.getInputStream(), "UTF-8");
// do signature verification here
chain.doFilter(request, response);
}
}
I won't go into details about the request wrapper. There are lots of examples of it on this site and elsewhere.
Your HttpServletRequestWrapper must implement the following methods:
public ServletInputStream getInputStream();
public BufferedReader getReader() throws IOException;
public Map<String, String[]> getParameterMap();
public String getParameter(String name);
After this you should no longer have problems verifying Slack signatures.
In my case, I didn't have issues with any of the encoded characters mentioned above (%20, %2A, etc). I only had problems verifying signatures of slash command requests. Message action request were verified correctly because they only had 1 request parameter in the body (payload).
I had this same issue using #RequestBody and not being able to validate the request. Here's how I fixed it in Kotlin. It should translate to Java fairly easily.
import org.springframework.web.bind.annotation.*
import org.apache.commons.io.IOUtils
import javax.crypto.Mac
import javax.crypto.spec.SecretKeySpec
import javax.servlet.http.HttpServletRequest
import org.apache.commons.codec.binary.Hex
fun validateSlackWebhook(
#RequestHeader("X-Slack-Request-Timestamp") slackRequestTimestamp: String,
#RequestHeader("X-Slack-Signature") slackSignature: String,
request: HttpServletRequest,
) {
val body = IOUtils.toString(request.reader)
val basestring = "v0:$slackRequestTimestamp:$body"
val sha256Hmac = Mac.getInstance("HmacSHA256")
val secretKey = SecretKeySpec(slackSigningSecret.toByteArray(), "HmacSHA256")
sha256Hmac.init(secretKey)
val finalHex = "v0=${Hex.encodeHexString(sha256Hmac.doFinal(basestring.toByteArray()))}"
val validated = finalHex.compareTo(slackSignature) == 0
if (!validated) {
// Code to run if request was not validated
return
}
// Code to run if request was validated
}
I finally solved it by looking at the official SDK source code of Slack!
https://github.com/slackapi/java-slack-sdk/blob/f283e45601157a0d2483ea3d3e8074e80b81a0e6/slack-app-backend/src/main/java/com/slack/api/app_backend/SlackSignature.java#L88-L122
public String generate(String slackRequestTimestamp, String requestBody) {
if (slackRequestTimestamp == null) {
return null;
}
// 1) Retrieve the X-Slack-Request-Timestamp header on the HTTP request, and the body of the request.
// "slackRequestTimestamp" here
// 2) Concatenate the version number, the timestamp, and the body of the request to form a basestring.
// Use a colon as the delimiter between the three elements.
// For example, v0:123456789:command=/weather&text=94070. The version number right now is always v0.
String baseString = "v0:" + slackRequestTimestamp + ":" + requestBody;
// 3) With the help of HMAC SHA256 implemented in your favorite programming, hash the above basestring,
// using the Slack Signing Secret as the key.
SecretKeySpec sk = new SecretKeySpec(slackSigningSecret.getBytes(), ALGORITHM);
try {
Mac mac = Mac.getInstance(ALGORITHM);
mac.init(sk);
byte[] macBytes = mac.doFinal(baseString.getBytes());
StringBuilder hashValue = new StringBuilder(2 * macBytes.length);
for (byte macByte : macBytes) {
hashValue.append(String.format("%02x", macByte & 0xff));
}
return "v0=" + hashValue.toString();
// 4) Compare this computed signature to the X-Slack-Signature header on the request.
} catch (NoSuchAlgorithmException | InvalidKeyException e) {
log.error("Failed to hash the base string value with HMAC-SHA256 because {}", e.getMessage(), e);
return null;
}
}
I want to create an employee information in the system by uploading an image along with employee data. I am able to do it with different rest calls using jersey. But I want to achieve in one rest call.
I provide below the structure. Please help me how to do in this regard.
#POST
#Path("/upload2")
#Consumes({MediaType.MULTIPART_FORM_DATA,MediaType.APPLICATION_XML, MediaType.APPLICATION_JSON})
public Response uploadFileWithData(
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition contentDispositionHeader,
Employee emp) {
//..... business login
}
Whenever I am trying to do, I get error in Chrome postman. The simple structure of my Employee json is given below.
{
"Name": "John",
"Age": 23,
"Email": "john#gmail.com",
"Adrs": {
"DoorNo": "12-A",
"Street": "Street-11",
"City": "Bangalore",
"Country": "Karnataka"
}
}
However I can do it by making two different call, but I want to achieve in one rest call so that I can receive the file as well as the actual data of the employee.
Request you to help in this regard.
You can't have two Content-Types (well technically that's what we're doing below, but they are separated with each part of the multipart, but the main type is multipart). That's basically what you are expecting with your method. You are expecting mutlipart and json together as the main media type. The Employee data needs to be part of the multipart. So you can add a #FormDataParam("emp") for the Employee.
#FormDataParam("emp") Employee emp) { ...
Here's the class I used for testing
#Path("/multipart")
public class MultipartResource {
#POST
#Path("/upload2")
#Consumes({MediaType.MULTIPART_FORM_DATA})
public Response uploadFileWithData(
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition cdh,
#FormDataParam("emp") Employee emp) throws Exception{
Image img = ImageIO.read(fileInputStream);
JOptionPane.showMessageDialog(null, new JLabel(new ImageIcon(img)));
System.out.println(cdh.getName());
System.out.println(emp);
return Response.ok("Cool Tools!").build();
}
}
First I just tested with the client API to make sure it works
#Test
public void testGetIt() throws Exception {
final Client client = ClientBuilder.newBuilder()
.register(MultiPartFeature.class)
.build();
WebTarget t = client.target(Main.BASE_URI).path("multipart").path("upload2");
FileDataBodyPart filePart = new FileDataBodyPart("file",
new File("stackoverflow.png"));
// UPDATE: just tested again, and the below code is not needed.
// It's redundant. Using the FileDataBodyPart already sets the
// Content-Disposition information
filePart.setContentDisposition(
FormDataContentDisposition.name("file")
.fileName("stackoverflow.png").build());
String empPartJson
= "{"
+ " \"id\": 1234,"
+ " \"name\": \"Peeskillet\""
+ "}";
MultiPart multipartEntity = new FormDataMultiPart()
.field("emp", empPartJson, MediaType.APPLICATION_JSON_TYPE)
.bodyPart(filePart);
Response response = t.request().post(
Entity.entity(multipartEntity, multipartEntity.getMediaType()));
System.out.println(response.getStatus());
System.out.println(response.readEntity(String.class));
response.close();
}
I just created a simple Employee class with an id and name field for testing. This works perfectly fine. It shows the image, prints the content disposition, and prints the Employee object.
I'm not too familiar with Postman, so I saved that testing for last :-)
It appears to work fine also, as you can see the response "Cool Tools". But if we look at the printed Employee data, we'll see that it's null. Which is weird because with the client API it worked fine.
If we look at the Preview window, we'll see the problem
There's no Content-Type header for the emp body part. You can see in the client API I explicitly set it
MultiPart multipartEntity = new FormDataMultiPart()
.field("emp", empPartJson, MediaType.APPLICATION_JSON_TYPE)
.bodyPart(filePart);
So I guess this is really only part of a full answer. Like I said, I am not familiar with Postman So I don't know how to set Content-Types for individual body parts. The image/png for the image was automatically set for me for the image part (I guess it was just determined by the file extension). If you can figure this out, then the problem should be solved. Please, if you find out how to do this, post it as an answer.
See UPDATE below for solution
And just for completeness...
See here for more about MultiPart with Jersey.
Basic configurations:
Dependency:
<dependency>
<groupId>org.glassfish.jersey.media</groupId>
<artifactId>jersey-media-multipart</artifactId>
<version>${jersey2.version}</version>
</dependency>
Client config:
final Client client = ClientBuilder.newBuilder()
.register(MultiPartFeature.class)
.build();
Server config:
// Create JAX-RS application.
final Application application = new ResourceConfig()
.packages("org.glassfish.jersey.examples.multipart")
.register(MultiPartFeature.class);
If you're having problems with the server configuration, one of the following posts might help
What exactly is the ResourceConfig class in Jersey 2?
152 MULTIPART_FORM_DATA: No injection source found for a parameter of type public javax.ws.rs.core.Response
UPDATE
So as you can see from the Postman client, some clients are unable to set individual parts' Content-Type, this includes the browser, in regards to it's default capabilities when using FormData (js).
We can't expect the client to find away around this, so what we can do, is when receiving the data, explicitly set the Content-Type before deserializing. For example
#POST
#Path("upload2")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFileAndJSON(#FormDataParam("emp") FormDataBodyPart jsonPart,
#FormDataParam("file") FormDataBodyPart bodyPart) {
jsonPart.setMediaType(MediaType.APPLICATION_JSON_TYPE);
Employee emp = jsonPart.getValueAs(Employee.class);
}
It's a little extra work to get the POJO, but it is a better solution than forcing the client to try and find it's own solution.
Another option is to use a String parameter and use whatever JSON library you use to deserialze the String to the POJO (like Jackson ObjectMapper). With the previous option, we just let Jersey handle the deserialization, and it will use the same JSON library it uses for all the other JSON endpoints (which might be preferred).
Asides
There is a conversation in these comments that you may be interested in if you are using a different Connector than the default HttpUrlConnection.
You can access the Image File and data from a form using MULTIPART FORM DATA By using the below code.
#POST
#Path("/UpdateProfile")
#Consumes(value={MediaType.APPLICATION_JSON,MediaType.MULTIPART_FORM_DATA})
#Produces(value={MediaType.APPLICATION_JSON,MediaType.APPLICATION_XML})
public Response updateProfile(
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition contentDispositionHeader,
#FormDataParam("ProfileInfo") String ProfileInfo,
#FormDataParam("registrationId") String registrationId) {
String filePath= "/filepath/"+contentDispositionHeader.getFileName();
OutputStream outputStream = null;
try {
int read = 0;
byte[] bytes = new byte[1024];
outputStream = new FileOutputStream(new File(filePath));
while ((read = fileInputStream.read(bytes)) != -1) {
outputStream.write(bytes, 0, read);
}
outputStream.flush();
outputStream.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} finally {
if (outputStream != null) {
try {
outputStream.close();
} catch(Exception ex) {}
}
}
}
When I tried #PaulSamsotha's solution with Jersey client 2.21.1, there was 400 error. It worked when I added following in my client code:
MediaType contentType = MediaType.MULTIPART_FORM_DATA_TYPE;
contentType = Boundary.addBoundary(contentType);
Response response = t.request()
.post(Entity.entity(multipartEntity, contentType));
instead of hardcoded MediaType.MULTIPART_FORM_DATA in POST request call.
The reason this is needed is because when you use a different Connector (like Apache) for the Jersey Client, it is unable to alter outbound headers, which is required to add a boundary to the Content-Type. This limitation is explained in the Jersey Client docs. So if you want to use a different Connector, then you need to manually create the boundary.
Your ApplicationConfig should register the MultiPartFeature.class from the glassfish.jersey.media.. so as to enable file upload
#javax.ws.rs.ApplicationPath(ResourcePath.API_ROOT)
public class ApplicationConfig extends ResourceConfig {
public ApplicationConfig() {
//register the necessary headers files needed from client
register(CORSConfigurationFilter.class);
//The jackson feature and provider is used for object serialization
//between client and server objects in to a json
register(JacksonFeature.class);
register(JacksonProvider.class);
//Glassfish multipart file uploader feature
register(MultiPartFeature.class);
//inject and registered all resources class using the package
//not to be tempered with
packages("com.flexisaf.safhrms.client.resources");
register(RESTRequestFilter.class);
}
I used file upload example from,
http://www.mkyong.com/webservices/jax-rs/file-upload-example-in-jersey/
in my resource class i have below method
#POST
#Path("/upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response attachupload(#FormDataParam("file") byte[] is,
#FormDataParam("file") FormDataContentDisposition fileDetail,
#FormDataParam("fileName") String flename){
attachService.saveAttachment(flename,is);
}
in my attachService.java i have below method
public void saveAttachment(String flename, byte[] is) {
// TODO Auto-generated method stub
attachmentDao.saveAttachment(flename,is);
}
in Dao i have
attach.setData(is);
attach.setFileName(flename);
in my HBM mapping is like
<property name="data" type="binary" >
<column name="data" />
</property>
This working for all type of files like .PDF,.TXT, .PNG etc.,
The request type is multipart/form-data and what you are sending is essentially form fields that go out as bytes with content boundaries separating different form fields.To send an object representation as form field (string), you can send a serialized form from the client that you can then deserialize on the server.
After all no programming environment object is actually ever traveling on the wire. The programming environment on both side are just doing automatic serialization and deserialization that you can also do. That is the cleanest and programming environment quirks free way to do it.
As an example, here is a javascript client posting to a Jersey example service,
submitFile(){
let data = new FormData();
let account = {
"name": "test account",
"location": "Bangalore"
}
data.append('file', this.file);
data.append("accountKey", "44c85e59-afed-4fb2-884d-b3d85b051c44");
data.append("device", "test001");
data.append("account", JSON.stringify(account));
let url = "http://localhost:9090/sensordb/test/file/multipart/upload";
let config = {
headers: {
'Content-Type': 'multipart/form-data'
}
}
axios.post(url, data, config).then(function(data){
console.log('SUCCESS!!');
console.log(data.data);
}).catch(function(){
console.log('FAILURE!!');
});
},
Here the client is sending a file, 2 form fields (strings) and an account object that has been stringified for transport. here is how the form fields look on the wire,
On the server, you can just deserialize the form fields the way you see fit. To finish this trivial example,
#POST
#Path("/file/multipart/upload")
#Consumes({MediaType.MULTIPART_FORM_DATA})
public Response uploadMultiPart(#Context ContainerRequestContext requestContext,
#FormDataParam("file") InputStream fileInputStream,
#FormDataParam("file") FormDataContentDisposition cdh,
#FormDataParam("accountKey") String accountKey,
#FormDataParam("account") String json) {
System.out.println(cdh.getFileName());
System.out.println(cdh.getName());
System.out.println(accountKey);
try {
Account account = Account.deserialize(json);
System.out.println(account.getLocation());
System.out.println(account.getName());
} catch (Exception e) {
e.printStackTrace();
}
return Response.ok().build();
}
I have a Jersey 2 application containing resources that consume and produce json. My requirement is to add a signature to an Authorization response header generated from a combination of various piece of response data (similar to the Amazon Webservices request signature). One of these pieces of data is the response body but I cant see that there are any filter or interception points that will allow me access to the json content. I imagine this is mainly because the response outputstream is for writing not reading.
Any ideas as to how I can read the response body - or alternative approaches ?
Thank you.
My understanding is that when your application is responding to a request, you want to modify the Authorization header by adding a signature to it's value.
If that's the case, you want to implement a ContainerResponseFilter:
public class MyContainerResponseFilter implements ContainerResponseFilter {
#Override
public void filter(ContainerRequestContext containerRequestContext, ContainerResponseContext containerResponseContext) throws IOException {
// You can get the body of the response from the ContainerResponseContext
Object entity = containerResponseContext.getEntity();
// You'll need to know what kind of Object the entity is in order to do something useful though
// You can get some data using these functions
Class<?> entityClass = containerResponseContext.getEntityClass();
Type entityType = containerResponseContext.getEntityType();
// And/or by looking at the ContainerRequestContext and knowing what the response entity will be
String method = containerRequestContext.getMethod();
UriInfo uriInfo = containerRequestContext.getUriInfo();
// Then you can modify your Authorization header in some way
String authorizationHeaderValue = containerResponseContext.getHeaderString(HttpHeaders.AUTHORIZATION);
authorizationHeaderValue = authorizationHeaderValue + " a signature you calculated";
containerResponseContext.getHeaders().putSingle(HttpHeaders.AUTHORIZATION, authorizationHeaderValue);
}
}
Be warned that the filter function will be called for all requests to your application, even when Jersey couldn't find a matching resource for the request path, so you may have to do some extra checking.
You can implement ContainerRequestFilter in order to access the content, and once you are finished with your interception logic, forward it to the request. E.g.
import java.io.*;
import com.sun.jersey.api.container.ContainerException;
import com.sun.jersey.core.util.ReaderWriter;
import com.sun.jersey.spi.container.ContainerRequest;
import com.sun.jersey.spi.container.ContainerRequestFilter;
public class ExampleFilter implements ContainerRequestFilter {
#Override
public ContainerRequest filter(ContainerRequest req) {
try(InputStream in = req.getEntityInputStream(); ByteArrayOutputStream out = new ByteArrayOutputStream();) {
if (in.available() > 0) {
StringBuilder content = new StringBuilder();
ReaderWriter.writeTo(in, out);
byte[] entity = out.toByteArray();
if (entity.length > 0) {
content.append(new String(entity)).append("\n");
System.out.println(content);
}
req.setEntityInputStream(new ByteArrayInputStream(entity));
}
} catch (IOException ex) {
//handle exception
}
return req;
}
}
My code is the consumer of an API (www.abc.com/public/news/apple.json). I get a json array in return which I then parse and populate in my own data structure. the code responsible for doing this is:
public Map<String,List<NewsItem>> populateNewsArray() throws Exception
{
url = domain + newsApiStr;
InputStream stream = getNews(url, true);
//jackson library object mapper
ObjectMapper mapper = new ObjectMapper();
//NewsApiObject class implements the structure of the json array returned.
List<NewsApiObject> mappedData = mapper.readValue(stream, NewsApiObject.class));
//populate the properties in a HashMap.
//return HashMap
}
public InputStream getNews(String request, boolean bulk) throws Exception
{
URL url = new URL(request);
connection = (HttpURLConnection) url.openConnection();
connection.setDoOutput(true);
connection.setInstanceFollowRedirects(false);
connection.setRequestMethod("GET");
connection.setRequestProperty("Content-Type", "text/plain");
connection.setRequestProperty("charset", "utf-8");
connection.connect();
return connection.getInputStream();
}
As you can see I am not the controller of the api, only the consumer. It is said that in unit tests, one is not suppose to make http requests. In this scenario, how can I unit test the populateNewsArray() function to see if the object mapping was correct (without any exceptions) and a valid hashmap was returned?
You should extract getNews() into a separate interface, e.g. NewsReader (although the word Reader has a specific meaning in the JDK, I like the name...)
public interface NewsReader {
InputStream getNews(String request, boolean bulk) throws Exception
}
Then implement that interface with using HttpURLConnection as per your code and update your code to allow injection of that particular interface. Then, if you need to test how your code handles an InputStream, you can create a mock of NewsReader which returns a InputStream with well-known content.
Remember to aim for high cohesion: your class shouldn't be an HTTP client and a stream parser.
I would create a subclass and overwrite the method getNews(...). In the subclass you then may return an InputStream for your test.
Since you should not depenend on some external file in a unit test and in order to get a better testable design I'd also change the getNews(...) method to return some kind of value which can be further processed by the mapper.
I would like to parse a local JSON file and marshal it into models using RestTemplate, but can't tell if this is possible.
I'm trying to pre-populate a database on an Android app that is using RestTemplate for syncing with the server. Rather than parsing the local JSON on my own, I thought, why not use RestTemplate? It's made exactly for parsing JSON into models.
But...I can't tell from the docs if there is any way to do this. There is the MappingJacksonHttpMessageConverter class which appears to convert the server's http response into a model...but is there any way to hack that to work with a local file? I tried, but kept getting deeper and deeper down the rabbit hole with no luck.
Figured this out. Instead of using RestTemplate, you can just use Jackson directly. There is no reason RestTemplate needs to be involved in this. It's very simple.
try {
ObjectMapper mapper = new ObjectMapper();
InputStream jsonFileStream = context.getAssets().open("categories.json");
Category[] categories = (Category[]) mapper.readValue(jsonFileStream, Category[].class);
Log.d(tag, "Found " + String.valueOf(categories.length) + " categories!!");
} catch (Exception e){
Log.e(tag, "Exception", e);
}
Yes, I think it is possible(with MappingJacksonHttpMessageConverter).
MappingJacksonHttpMessageConverter has method read() which takes two parameters: Class and HttpInputMessage
MappingJacksonHttpMessageConverter converter = new MappingJacksonHttpMessageConverter();
YourClazz obj = (YourClazz) converter.read(YourClazz.class, new MyHttpInputMessage(myJsonString));
With this method you can read single object from single json message, but YourClazz can be some collection.
Next, You have to create you own HttpInputMessage implementation, in this example it expected json as string but You probably can pass stream to your json file.
public class MyHttpInputMessage implements HttpInputMessage {
private String jsonString;
public MyHttpInputMessage(String jsonString) {
this.jsonString = jsonString;
}
public HttpHeaders getHeaders() {
// no headers needed
return null;
}
public InputStream getBody() throws IOException {
InputStream is = new ByteArrayInputStream(
jsonString.getBytes("UTF-8"));
return is;
}
}
PS. You can publish your app with database