We Keep Coding

How to deal with a web service that doesn't obey Accept: application/json using Jersey and Jackson

I am using Jersey and Jackson to access a REST web service, which is correctly returning well formed JSON data but has the response header:
Content-Type: text/html; charset=UTF-8
Even though I have specified Accept: application/json in the request header and as a result is causing Jersey to throw:
org.glassfish.jersey.message.internal.MessageBodyProviderNotFoundException: MessageBodyReader not found for media type=text/html;charset=UTF-8
I am consuming other web services fine with my code, but I am wondering if there is a way to create my own MessageBodyReader to deal with the mis-match, however I have yet to figure out how to implement it correctly. I plan to ask the owner of the web service to fix the mis-match but I don't hold out much hope.

Okay so I managed to figure it out by essentially following Stephen C's advice but thought I'd post a few more details in case anyone else is in the same boat. First I actually started from the Jersey guide a few sections back, specifically this one:
https://jersey.java.net/documentation/latest/user-guide.html#d0e6825
Obviously I am using Jersey for the javax.ws.rs.client and I am using Genson to do the JSON deserialisation. As a result I have implemented the following class to implement a MessageBodyReader:
public class BTCEURTradeMessageBodyReader
implements MessageBodyReader<BTCEURTrades> {
final org.slf4j.Logger logger =
LoggerFactory.getLogger(BTCEURTradeMessageBodyReader.class);
#Override
public boolean isReadable(Class<?> type, Type genericType,
Annotation[] annotations, MediaType mediaType) {
logger.info("isReadable being checked for: {} and media type: {}", type, mediaType);
return type == BTCEURTrades.class;
}
#Override
public BTCEURTrades readFrom(Class<BTCEURTrades> type, Type genericType,
Annotation[] annotations, MediaType mediaType,
MultivaluedMap<String, String> httpHeaders, InputStream entityStream)
throws IOException, WebApplicationException {
logger.info("readFrom being called for: {}", type);
BTCEURTrades btceurTrades;
try {
btceurTrades = new Genson().deserialize(entityStream, type);
} catch(Exception e) {
logger.error("Error processing JSON reponse.", e);
throw new ProcessingException("Error processing JSON reponse.");
}
return btceurTrades;
}
}
This then gets registered with the client after it is created as follows:
client = ClientBuilder.newClient();
client.register(BTCEURTradeMessageBodyReader.class);

I am wondering if there is a way to create my own MessageBodyReader to deal with the mis-match.
This page in the Jersey documentation explains how to create a custom MessageBodyReader:
https://jersey.java.net/documentation/latest/message-body-workers.html#d0e7151
In your case, you may be able to find the source code for the reader that is ordinarily used for your JSON, and "tweak" it. In theory.
However, after a bit more reading, I found this:
https://jersey.java.net/documentation/latest/media.html#json
which is telling me that Jersey already has extensive support for JSON. There is a good chance that you could fix your problem this by simply tweaking the configs so that Jersey knows what to do with the unusual content-type. But that will depend on which of the many possible ways that you are parsing JSON response bodies ... at the moment.
Someone commented thus:
I think however easier just to retrieve the data, ignore the header and just parse it into your json object.
That is a bad idea. The header is telling you that the JSON could contain multi-byte characters. If you simply ignored that and decoded the bytes to characters in the default character set, you would get "mojibake" if there were multibyte characters present.
If you are parsing the JSON yourself, then it should be a simple matter to configure the parser's input stream to use UTF-8, or whatever else the content-type header says the character encoding is.
Finally, there is the issue of "who is wrong".
I actually think it is your fault. If you send just an "Accept: application/json" header, you are telling the server that you don't care what the character set is. The server is then free to pick any charset for the response that it knows will correctly represent the response text. (In this case the text content of the JSON.)
If you specifically want (say) ASCII or Latin-1 then you should add an "Accept-charset:" header.
If >>THAT<< doesn't work, then maybe it is the server's fault. But bear in mind that if the response does / could contain characters that cannot be encoded on your preferred charset, then the server could / should send you a 406 error.

Spring REST tutorial [duplicate]

I'm building a REST API, but I've encountered a problem.
It seems that accepted practice in designing a REST API is that if the resource requested doesn't exist, a 404 is returned.
However, to me, this adds unnecessary ambiguity. HTTP 404 is more traditionally associated with a bad URI. So in effect we're saying "Either you got to the right place, but that specific record does not exist, or there's no such location on the Internets! I'm really not sure which one..."
Consider the following URI:
http://mywebsite/api/user/13
If I get a 404 back, is that because User 13 does not exist? Or is it because my URL should have been:
http://mywebsite/restapi/user/13
In the past, I've just returned a NULL result with an HTTP 200 OK response code if the record doesn't exist. It's simple, and in my opinion very clean, even if it's not necessarily accepted practice. But is there a better way to do this?

404 is just the HTTP response code. On top of that, you can provide a response body and/or other headers with a more meaningful error message that developers will see.

Use 404 if the resource does not exist. Don't return 200 with an empty body.
This is akin to undefined vs empty string (e.g. "") in programming. While very similar, there is definitely a difference.
404 means that nothing exists at that URI (like an undefined variable in programming). Returning 200 with an empty body means that something does exist there and that something is just empty right now (like an empty string in programming).
404 doesn't mean it was a "bad URI". There are special HTTP codes that are intended for URI errors (e.g. 414 Request-URI Too Long).

As with most things, "it depends". But to me, your practice is not bad and is not going against the HTTP spec per se. However, let's clear some things up.
First, URI's should be opaque. Even if they're not opaque to people, they are opaque to machines. In other words, the difference between http://mywebsite/api/user/13, http://mywebsite/restapi/user/13 is the same as the difference between http://mywebsite/api/user/13 and http://mywebsite/api/user/14 i.e. not the same is not the same period. So a 404 would be completely appropriate for http://mywebsite/api/user/14 (if there is no such user) but not necessarily the only appropriate response.
You could also return an empty 200 response or more explicitly a 204 (No Content) response. This would convey something else to the client. It would imply that the resource identified by http://mywebsite/api/user/14 has no content or is essentially nothing. It does mean that there is such a resource. However, it does not necessarily mean that you are claiming there is some user persisted in a data store with id 14. That's your private concern, not the concern of the client making the request. So, if it makes sense to model your resources that way, go ahead.
There are some security implications to giving your clients information that would make it easier for them to guess legitimate URI's. Returning a 200 on misses instead of a 404 may give the client a clue that at least the http://mywebsite/api/user part is correct. A malicious client could just keep trying different integers. But to me, a malicious client would be able to guess the http://mywebsite/api/user part anyway. A better remedy would be to use UUID's. i.e. http://mywebsite/api/user/3dd5b770-79ea-11e1-b0c4-0800200c9a66 is better than http://mywebsite/api/user/14. Doing that, you could use your technique of returning 200's without giving much away.

That is an very old post but I faced to a similar problem and I would like to share my experience with you guys.
I am building microservice architecture with rest APIs. I have some rest GET services, they collect data from back-end system based on the request parameters.
I followed the rest API design documents and I sent back HTTP 404 with a perfect JSON error message to client when there was no data which align to the query conditions (for example zero record was selected).
When there was no data to sent back to the client I prepared an perfect JSON message with internal error code, etc. to inform the client about the reason of the "Not Found" and it was sent back to the client with HTTP 404. That works fine.
Later I have created a rest API client class which is an easy helper to hide the HTTP communication related code and I used this helper all the time when I called my rest APIs from my code.
BUT I needed to write confusing extra code just because HTTP 404 had two different functions:
the real HTTP 404 when the rest API is not available in the given url, it is thrown by the application server or web-server where the rest API application runs
client get back HTTP 404 as well when there is no data in database based on the where condition of the query.
Important: My rest API error handler catches all the exceptions appears in the back-end service which means in case of any error my rest API always returns with a perfect JSON message with the message details.
This is the 1st version of my client helper method which handles the two different HTTP 404 response:
public static String getSomething(final String uuid) {
String serviceUrl = getServiceUrl();
String path = "user/" + , uuid);
String requestUrl = serviceUrl + path;
String httpMethod = "GET";
Response response = client
.target(serviceUrl)
.path(path)
.request(ExtendedMediaType.APPLICATION_UTF8)
.get();
if (response.getStatus() == Response.Status.OK.getStatusCode()) {
// HTTP 200
return response.readEntity(String.class);
} else {
// confusing code comes here just because
// I need to decide the type of HTTP 404...
// trying to parse response body
try {
String responseBody = response.readEntity(String.class);
ObjectMapper mapper = new ObjectMapper();
ErrorInfo errorInfo = mapper.readValue(responseBody, ErrorInfo.class);
// re-throw the original exception
throw new MyException(errorInfo);
} catch (IOException e) {
// this is a real HTTP 404
throw new ServiceUnavailableError(response, requestUrl, httpMethod);
}
// this exception will never be thrown
throw new Exception("UNEXPECTED ERRORS, BETTER IF YOU DO NOT SEE IT IN THE LOG");
}
BUT, because my Java or JavaScript client can receive two kind of HTTP 404 somehow I need to check the body of the response in case of HTTP 404. If I can parse the response body then I am sure I got back a response where there was no data to send back to the client.
If I am not able to parse the response that means I got back a real HTTP 404 from the web server (not from the rest API application).
It is so confusing and the client application always needs to do extra parsing to check the real reason of HTTP 404.
Honestly I do not like this solution. It is confusing, needs to add extra bullshit code to clients all the time.
So instead of using HTTP 404 in this two different scenarios I decided that I will do the following:
I am not using HTTP 404 as a response HTTP code in my rest application anymore.
I am going to use HTTP 204 (No Content) instead of HTTP 404.
In that case client code can be more elegant:
public static String getString(final String processId, final String key) {
String serviceUrl = getServiceUrl();
String path = String.format("key/%s", key);
String requestUrl = serviceUrl + path;
String httpMethod = "GET";
log(requestUrl);
Response response = client
.target(serviceUrl)
.path(path)
.request(ExtendedMediaType.APPLICATION_JSON_UTF8)
.header(CustomHttpHeader.PROCESS_ID, processId)
.get();
if (response.getStatus() == Response.Status.OK.getStatusCode()) {
return response.readEntity(String.class);
} else {
String body = response.readEntity(String.class);
ObjectMapper mapper = new ObjectMapper();
ErrorInfo errorInfo = mapper.readValue(body, ErrorInfo.class);
throw new MyException(errorInfo);
}
throw new AnyServerError(response, requestUrl, httpMethod);
}
I think this handles that issue better.
If you have any better solution please share it with us.

404 Not Found technically means that uri does not currently map to a resource. In your example, I interpret a request to http://mywebsite/api/user/13 that returns a 404 to imply that this url was never mapped to a resource. To the client, that should be the end of conversation.
To address concerns with ambiguity, you can enhance your API by providing other response codes. For example, suppose you want to allow clients to issue GET requests the url http://mywebsite/api/user/13, you want to communicate that clients should use the canonical url http://mywebsite/restapi/user/13. In that case, you may want to consider issuing a permanent redirect by returning a 301 Moved Permanently and supply the canonical url in the Location header of the response. This tells the client that for future requests they should use the canonical url.

So in essence, it sounds like the answer could depend on how the request is formed.
If the requested resource forms part of the URI as per a request to http://mywebsite/restapi/user/13 and user 13 does not exist, then a 404 is probably appropriate and intuitive because the URI is representative of a non-existent user/entity/document/etc. The same would hold for the more secure technique using a GUID http://mywebsite/api/user/3dd5b770-79ea-11e1-b0c4-0800200c9a66 and the api/restapi argument above.
However, if the requested resource ID was included in the request header [include your own example], or indeed, in the URI as a parameter, eg http://mywebsite/restapi/user/?UID=13 then the URI would still be correct (because the concept of a USER does exits at http://mywebsite/restapi/user/); and therefore the response could reasonable be expected to be a 200 (with an appropriately verbose message) because the specific user known as 13 does not exist but the URI does. This way we are saying the URI is good, but the request for data has no content.
Personally a 200 still doesn't feel right (though I have previously argued it does). A 200 response code (without a verbose response) could cause an issue not to be investigated when an incorrect ID is sent for example.
A better approach would be to send a 204 - No Contentresponse. This is compliant with w3c's description *The server has fulfilled the request but does not need to return an entity-body, and might want to return updated metainformation.*1 The confusion, in my opinion is caused by the Wikipedia entry stating 204 No Content - The server successfully processed the request, but is not returning any content. Usually used as a response to a successful delete request. The last sentence is highly debateable. Consider the situation without that sentence and the solution is easy - just send a 204 if the entity does not exist. There is even an argument for returning a 204 instead of a 404, the request has been processed and no content has been returned! Please be aware though, 204's do not allow content in the response body
Sources
http://en.wikipedia.org/wiki/List_of_HTTP_status_codes
1. http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html

This old but excellent article... http://www.infoq.com/articles/webber-rest-workflow says this about it...
404 Not Found - The service is far too lazy (or secure) to give us a real reason why our request failed, but whatever the reason, we need to deal with it.

This recently came up with our team.
We use both 404 Not found with a message body and 204 No Content based on the following rational.
If the request URI indicates the location of a single resource, we use 404 Not found. When the request queries a URI, we use 204 No Content
http://mywebsite/api/user/13 would return 404 when user 13 does not exist
http://mywebsite/api/users?id=13 would return 204 no content
http://mywebsite/api/users?firstname=test would return 204 no content
The idea here being, 'query routes' are expected to be able to return 1, many or no content.
Whatever pattern you choose, the most important things is to be consistent - so get buy in from your team.

The Uniform Resource Identifier is a unique pointer to the resource. A poorly form URI doesn't point to the resource and therefore performing a GET on it will not return a resource. 404 means The server has not found anything matching the Request-URI. If you put in the wrong URI or bad URI that is your problem and the reason you didn't get to a resource whether a HTML page or IMG.

Since this discussion seems to be able to survive the end of time I'll throw in the JSON:API Specifications
404 Not Found
A server MUST respond with 404 Not Found when processing a request to fetch a single resource that does not exist, except when the request warrants a 200 OK response with null as the primary data (as described above).
HTTP/1.1 200 OK
Content-Type: application/vnd.api+json
{
"links": {
"self": "http://example.com/articles/1/author"
},
"data": null
}
Also please see this Stackoverflow question

For this scenario HTTP 404 is response code for the response from the REST API
Like 400, 401, 404 , 422 unprocessable entity
use the Exception handling to check the full exception message.
try{
// call the rest api
} catch(RestClientException e) {
//process exception
if(e instanceof HttpStatusCodeException){
String responseText=((HttpStatusCodeException)e).getResponseBodyAsString();
//now you have the response, construct json from it, and extract the errors
System.out.println("Exception :" +responseText);
}
}
This exception block give you the proper message thrown by the REST API

Questions about implement server side and client side validation

I'm working with PlayFramework to build a registration user form that works via Ajax (it never refresh the page) and I obviously need to validate it. I'm considering two way to implement client and server side validation.
Send via Ajax the form data to server side validation, that save the
user in the db if the data is valid, or send an error message to the
frontend in case of invalid data. All of this happens via ajax
without refresh the page.
Pro: In this way I don't implement any
client side validation, but I get the same effect.
Contro: I'm not able to show specific
error messages for each type of invalid data (required, min lenght,
email format, ...)
Validate the form with a client side validation (jQuery Validation
plugin), then if it's valid it sends the data to the server side
validation. From the server side, if it's valid it registers the
user, otherwise nothing happens (?). Pro: Very simple way to show
specific error messages Contro: I need to implement 2 times the same
validation logic
These are my considerations, but I don't know if they're right or if there are other better ways. What is the best way in your opinion to do this?
The questions:
With the first way: Is it possible from server validation send specific error messages and show them on frontend, using PlayFramework (without writing from scratch all the validation rules)?
With the second way: To implement both server and side validations I have to first validate the data with client validation then pass to the server? And with disabled javascript on browser: server side validation has to show something? (for example: notable Mediafire doesn't do that, I have to assume that for correct)
Edit:
Here the code I'm using to receive the form data via ajax in json format:
Controller:
JsonNode json = request().body().asJson();
Form<User> userForm = User.form.bind(json);
if(userForm.hasErrors()) {
return badRequest("One or more inputs are invalid");
} else {
// register the user in the db
}
Model (User):
#Required
public String email;
#Required #MinLength(8) #MaxLength(16)
public String password;
public static Form<User> form = Form.form(User.class);
As it is, it returns a generic message for any invalid inputs. What I'm not able to do is: return (with the badRequest) the specific error messages and show them under each inputs of the form.

You don't need to tell somebody that he tried to login with too short username, in this particular case it's enough if you'll just display an error like: Login incorrect. It's generally adviced to DO NOT tell the user what is the exact reason of the login fail - it prevents (a little) against the tries of brute login attempts.
How do you handle your errors depends on you, you can easily return a JSON object containing list of all errors so you can display them where you want.
For other forms (like some registration, messeging, etc) using ready to use frontend validators is good approach, as you can avoid some reduntand requests and it's just works smoother, anyway main validation should be always be done at the backend - just use frontend one as a "interesting addition to your site".
And don't worry about disabled JS - in this case you will not be able to perform even simple AJAX request ;)
Edit
As mentioned above you can just return a JSON object (also with list of errors) and handle it with JavaScript, although there may be many different approaches, there's one of possibilities:
public static Result submit() {
Form<User> userForm = Form.form(User.class).bindFromRequest();
ObjectNode output = Json.newObject();
if (userForm.hasErrors()) {
output.put("status", 400);
output.put("errors", userForm.errorsAsJson());
return badRequest(output);
}
output.put("status", 200);
output.put("msg", "Your account was registered correctly!");
return ok(output);
}

What is the best way to implement kind of getNext on results using REST (restlet)

I'm writing a web service which returns results. This results are some data that is being created on the server all the time. So when the client asks for a resource i want to give him also a url for the next query he can preform to get the new data created on the server.
For example, client can start with the following url:
http://ip:port/server/{id}/resource
and next time he should use something like:
http://ip:port/server/{id}/resource/1234
where 1234 is some pointer to the server to know which result the client has already recieved.
So the question is, where do i return the url to the next set of results? should it be in the header or in the body?
I read some reference about usage of url parameters vs. query, and if i understood for chaching i better use uri rather then query.
Last thing, i need to pass info the body for the request and therefore the web service expects PUT and not GET.
Restlet example will be mostly appreciated.
One last thing, i must have {id} in the url, but there is no such uri as
http://ip:port/server
so what would be the right way for users to know there id? (the results are returned per user's id). the id ia allocated by completly different resource.

You might consider an alternate approach using conditional GETs. In this approach, the server responds to GET requests with an ETag and/or Last-Modified response header to indicate when the resource was last modified. This would be "1234" in your example above. The client then supplies that value on a subsequent GET to the same URI as earlier, providing it in an If-None-Match or If-Modified-Since header.
I have a similar service implemented using Restlet that is continuously receiving updates, and likewise, clients want to periodically fetch information newer than what they received previously. I use ETags.
In my ServerResource-derived class, I have code like this:
import org.restlet.data.Tag;
import org.restlet.representation.Representation;
import org.restlet.representation.Variant;
public class FooCollectionServerResource extends ServerResource implements FooCollectionResource {
private Tag mResponseTag;
#Override public FooCollection getFooCollection() {
List<Tag> tags = getRequest().getConditions().getNoneMatch();
Tag eTag = null;
if (!tags.isEmpty())
eTag = tags.get(0);
// might be an empty collection, if there are no new/modified Foos since eTag
FooCollection result = getFoosSince(eTag);
mResponseTag = new Tag("1234"); // hardcoded here, but you get the idea
return result;
}
#Override
public Representation toRepresentation(final Object source, final Variant target) {
Representation result = super.toRepresentation(source, target);
result.setTag(mResponseTag);
return result;
}
}
One trick here is that you can't set the ETag header value directly in the getFoo() method; you have to tuck it away in a field and use it in the toRepresentation() override. This is because of the timing of when the response Representation is created in Restlet.

AS2: Does xml.sendAndLoad use POST or GET?

All,
I'm trying to find out, unambiguously, what method (GET or POST) Flash/AS2 uses with XML.sendAndLoad.
Here's what the help/docs (http://livedocs.adobe.com/flash/9.0/main/wwhelp/wwhimpl/common/html/wwhelp.htm?context=LiveDocs_Parts&file=00002340.html) say about the function
Encodes the specified XML object into
an XML document, sends it to the
specified URL using the POST method,
downloads the server's response, and
loads it into the resultXMLobject
specified in the parameters.
However, I'm using this method to send XML data to a Java Servlet developed and maintained by another team of developers. And they're seeing log entries that look like this:
GET /portal/delegate/[someService]?svc=setPayCheckInfo&XMLStr=[an encoded version of the XML I send]
After a Google search to figure out why the POST shows up as a GET in their log, I found this Adobe technote (http://kb2.adobe.com/cps/159/tn_15908.html). Here's what it says:
When loadVariables or getURL actions are
used to send data to Java servlets it
can appear that the data is being sent
using a GET request, when the POST
method was specified in the Flash
movie.
This happens because Flash sends the
data in a GET/POST hybrid format. If
the data were being sent using a GET
request, the variables would appear in
a query string appended to the end of
the URL. Flash uses a GET server
request, but the Name/Value pairs
containing the variables are sent in a
second transmission using POST.
Although this causes the servlet to
trigger the doGet() method, the
variables are still available in the
server request.
I don't really understand that. What is a "GET/POST hybrid format"?
Why does the method Flash uses (POST or GET) depend on whether the data is sent to a Java servlet or elsewhere (e.g., a PHP page?)
Can anyone make sense of this? Many thanks in advance!
Cheers,
Matt

Have you try doing something like that :
var sendVar=new LoadVars();
var xml=new XML("<r>test</r>");
sendVar.xml=xml;
sendVar.svc="setPayCheckInfo";
var receiveXML=new XML();
function onLoad(success) {
if (success) {
trace("receive:"+receiveXML);
} else {
trace('error');
}
}
receiveXML.onLoad=onLoad;
sendVar.sendAndLoad("http://mywebserver", receiveXML, "POST");

The hybrid format is just a term Macromedia invented to paint over its misuse of HTTP.
HTTP is very vague on what you can do with GET and POST. But the convention is that no message body is used in GET. Adobe violates this convention by sending parameters in the message body.
Flash sends the same request regardless of the server. You have problem in Servlet because most implementation (like Tomcat) ignores message body for GET. PHP doesn't care the verb and it processes the message body for GET too.