We Keep Coding

When developing a RESTful web service, why do I need to write the method return value directly to the body of the response as JSON?

I am learning about REST apis and after some time of researching, I know the basics of how REST works and I can write an api that generates a JSON response in the web browser with appropriate url. However, I can't seem to understand what is the purpose of it. Surely just generating JSON response seems useless for production application, as you have to assume that most of your website visitors won't even know what JSON is, so there must be some other uses of rest api that I still don't know about. I have been searching a lot about why I should use a rest api, but besides websites praising it ("because it is scalable, portable, flexible" and so on and on) and saying that JSON is simpe yet powerful data exchange format, I have found no concrete answer.
As of this particular case, I am learning to develop RESTful api with Spring. I understand simple CRUD actions using #RestController, but It seems ridiculous to just sent JSON object as a response and especially to expect a client to understand JSON data.
So I am guessing that there must be some other uses for RESTful api that I am not aware of and can't seem to look up either.
So my questions are:
Why should I write RESTful controller if all it does is generate JSON data as a response.
1.1) I am assuming that it IS NOT all it does, so why is it necessary exactly?
Where and what should I use JSON data for?
I know that I am far from understanding this fully, so there probably are many misconceptions, but I want to understand the reasoning behind everything I do and not blindly follow whatever studying resources say, so I am asking it here. Thank you for your time

Its very important to understand that API (yeah, capital letters are quite important) is acronym for Application Programming Interface, it is not user interface, it is interface for other programs/applications to use.
So, API will be used solely by some other code to exchange data (interface with each other) and to do this you need some structure or protocol to follow by both sides of this exchange, otherwise applications will not understand each other.
So, you decided that your application (service) will provide some API for other applications (again, not users), you decided what kind of functions those applications will be able to consume (like get current weather, or create new user - basically methods in your controller).
Next step is to define protocol - how exactly information will be presented on wire (serialized to binary stream, because you can send only bytes via physical connection), JSON is quite popular choice because it provides quite easy format to parse for application (libraries for JSON exist for almost any programming language), but also is still readable enough for humans (there many offline and online formatters to help you).
But, JSON is not most efficient in terms of space, this is why you can pick many others - BSON, protobuff, kryo, java RMI and so on.
Now, lets actually answer you questions:
Why should I write RESTful controller if all it does is generate JSON data as a response.
Because you as developer decided to provide API for others, and you decided to use JSON as data format, and you decided so (probably) because it is quite famous and easy to work with, it is provided by default in many frameworks, etc, but there is no real objective reasons for that, several years ago SOAP/XML was holding the same niche, for the very same reasons.
Where and what should I use JSON data for?
Anywhere and for everything where you think it will fit: non performance critical inter-service communications, config files, personal notes, structured or non structured data, JSON is very flexible format (as almost any other generic purpose one, like XML or simple text files).

A ReST API will usually be called by another service or e.g. by a JavaScript based frontend application, not directly by users. JSON can be processed quite comfortably by other applications, that's why it's widely used (by now there are other formats as well, especially for high performance applications, but that's on an advanced level).
So to your questions specifically: If you are offering a service, that will be used by other applications, then use a ReST API with JSON responses, if you are offering an application that will be used by users directly, and don't want to have a JavaScript frontend application, you would use something like Thymeleaf to implement the frontend and have that returned in your controllers.

Auto-generating frontend and backend validation for Angular/Java

I hope my question is not too broad.
Our application has an Angular frontend and a Java backend and
we need both frontend and backened validation. At the moment, we manually code
validators in Typescript and Java that basically do the same thing.
Since that is not only tedious but also error-prone, we would like to define once (maybe in a JSON file)
which input fields have which constraints and ideally use a piece of software that auto-generates
Java and Typescript code based on that definition.
My question is: are there any open-source/commercial software solutions that take care of this?
It seems like a common enough problem, but I haven't come across
a proper solution yet. JSON schema seemed promising and since we're using swagger
anyway, I'd hoped that swagger-codegen would automatically take the supported JSON-schema keywords as constraints into account,
but I haven't figured out how it works. When I generate Angular code, constraints such as 'minimum' and 'maximum' are completely ignored,
and when I generate Java SpringBoot code, they only show up as annotations.

This is exactly what JSON Schema is designed for. Open API (swagger) modifies JSON Schema for it's own purposes, which doesn't include validation.
JSON Schema has validator implementations in most major languages allowing you to use the same schema to do the same validation on the front end and the back end. http://json-schema.org/implementations.html
Unfortunately, you likely won't be able to use the same schemas you use for your Open API service definition. I'm not aware of any validator implementations that support Open API's flavor of JSON Schema. I understand that Open API is trying to close the gap between their definition and JSON Schema, but a gap still exists.

Should web service be separate from web site? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I am building a website and also want to build a REST web service for accessing a lot of the same functionality (using google app engine and spring mvc3), and I'm not sure of the best practices for how integrated/separate the 2 parts should be.
For example if I want view a resource I can provide a url in the form:
{resourcetype}\{resourceid}
A GET request to this url can be redirected at a view which generates a webpage when the client is HTML/browser based. Spring has (from what I read - not tried it yet) the ability to use this same resource URL to serve up a view which returns HTML/Xml/JSON depending on the content type. This all seems great.
POST requests to a URL to create new resources in REST should return 201 CREATED (or so I read) along with the URL of the created resource, which seems fine for the Api but seems a little different from what would be expected from the norm in a web page (where you would likely be redirected to a page showing the resource you created or to a page saying it was created successfully or similar). Should I handle this by serving up a page at a different URL which contains the form for creating the resource, then submits to the Api URL via ajax and gets the response and redirects to the resource URL included in the response.
This pattern seems like it would work (and should work for DELETE too) but is this a good approach or am I better keeping the REST Api URLs and the web site URLs separate? this seems like it could introduce a fair bit of duplication and extra work. But having a single URL could mean that you are dependent on javascript being available on the client of a HTML 5 supporting browser.

I suggest you keep them separate. By doing this you gain several benefits.
First, you decouple your web urls from you API urls so they can each change independently. For example, you may need to release backwards incompatible change to your API, in which case you can create a /v2/ directory. Meanwhile, you might want an /about page on the website but don't need one for your API.
By using different URLs, you simplify your implementation. Now every method doesn't have to determine if it's fronting JSON/XML or HTML. This is true even if you have a framework like Spring doing the heavy lifting; you still have do extra things for the website with respect to the current user.
It also eliminates a whole class of bugs. For example, users won't get JSON or XML output back while browsing the site—even if they have custom browser anonymity settings.
You can easily separate the logic for authentication. With a website, you need a login page and cookies. With an API, these aren't required but extra authentication headers are (for example, an HMAC+sha256 signature).
Finally, by separating the site from the API, you allow for different scaling needs. If your API is getting hit hard but not the website, you can throw more hardware at the API while keeping the minimal needed for the website.
Update:
To clarify, I'm not suggesting you code up everything twice. There are two different ways to look at this to remove duplication.
First, in MVC parlance, you have one model and two different views on that model. This is the whole point of MVC so that the view and model are not tied together. The code to get a particular resource is the same in both clients, so it you could write your model such that only one line of code gets that resource from the database or wherever it comes from. In short, your model is an easy-to-use library with two clients.
Another way to look at it is that your website is your very first client of your public REST API; the web server actually calls your RESTful API to gets it's information. This is the whole Eat Your Own Dog Food principle.

I disagree with Michael's answer and use it as a basis for my own:
To "decouple your web urls from you API urls so they can each change indepentently." is to spit in the face of REST. Cool URIs don't change. Don't concern yourself with changing your URLs. Don't version your API using URIs. REST uses links to espouse the OCP - a client operating on the previous version of your API should happily follow the links that existed when it went live, unknowing of new links that you've added to enhance your API.
If you insist on versioning your API, I'd ask that you do so using the media type, not the URI.
Next, " you simplify your implementation. Now every method doesn't have to determine if it's fronting JSON/XML or HTML."
If you're doing it this way, you're doing it wrong. Coming from Jersey, I return the same damn object from every method whether or not I produce HTML, XML or JSON. That's a completely cross-cutting concern that a marshaller takes care of. I use a VelocityMessageBodyWriter to emit HTML templates surrounding my REST representations.
Every resource in my system follows the same basic behavior:
class FooResource extends Resource {
#GET
public FooRepresentation get() {
return new FooRepresentation();
}
#POST
#Consumes(MediaType.APPLICATION_FORM_URLENCODED)
public Response postForm(MulivaluedMap<String, String> form) {
return post(buildRepresentationFromForm(form));
}
#POST
#Consumes(MediaType.APPLICATION_XML)
public Resopnse post(FooRepresentation representation) {
// return ok, see other, whatever
}
}
The GET method may need to build Links to other resources. These are used by consumers of a REST API as well as the HTML template. A form may have to POST and I use some particular Link (defined by the Relation) for the "action" attribute).
The path through the system may be different between a user-browser and a user-machine but this is not a separation of implementation - it is REST! The state transfer happens how it needs to, how you define it. How users (in a browser) proceed.
"Finally, by separating the site from the API, you allow for different scaling needs. If your API is getting hit hard but not the website, you can throw more hardware at the API while keeping the minimal needed for the website." - your scaling should not depend on who uses what here. Odds are you're doing some work behind the scenes that is more intense than just serving up HTML. By using the same implementation, scaling is even easier. The API itself (the marshaling between XML and domain objects and back) is not going to exceed the business logic and processing, database, etc.
Lastly, by keeping them the same, it is much easier to think about your system as a whole. In fact, start with the HTML. Define the relationships. If you're having a hard time expressing a particular action or user story in HTML anchors and forms, you're probably straying from REST.
Remember, you're expressing things as links (of a particular relation) to other things. Those URIs can even be different depending on whether you're producing XML or HTML - the HTML page might POST to URI some/uri/a and the API might POST to some/uri/b - that's irrelevant and being concerned with what the actual URI contents are is the dark path to POX and RPC
Another nifty feature is that if you do it this way, you're not dependent on JavaScript. You've defined your system to work with basic HTML and you can 'flip on' JavaScript when it is available. Then you're really working with your "API" anyway (I cringe at referring to them as different things, but I'm also trying to bridge my response in to your wording)
** I will add one final comment, when producing HTML, I use 303 instead of 201 in order to facilitate POST-then-GET. If JS is enabled, you're actually talking XML (or JSON) and you're back to 201.

In support of Michael, and in contrast to Doug, you should keep them separate.
Turns out that the browser, in its basic form, is not a particularly good REST client. Through lack of full support of HTTP, to lousy authentication support, to weak support for media types, the browser is actually quite limited. If you're limited to simply consuming content, and that content happens to be HTML, then the browser is fine, but going beyond that and the API suffers do to poor support in the browser.
JavaScript can improve the capability of the browser, and make it a better REST citizen, but few things work better in the browser than a static HTML page. Portable, performant, scaleable to different devices with some CSS fun. Everyone loves the snap of a static page, especially one that isn't hosting a zillion images and what not from other slow providers. Click, BANG, a fast appearing, fast scrolling page.
Since the browser is a sad citizen, you shouldn't limit your API to its weak capabilities. By separating them, you can write a nice, rich, animated, bouncy, exciting interface in HTML + JS, or Flash, or Java, Obj-C for iOS, or Android, or whatever.
You could also write a nice front end in PHP, hosted on your server, and pushing the results out to browser clients. The PHP app doesn't have to be REST at all, it can be just a generic web app, working in the domain and constraints of a web app (stateful sessions, non-semantic markup, etc. etc.). Browser talks to PHP, PHP talks to your REST service. The PHP app lets you segregate the demands of the browser from the semantics of a REST service.
You can write more RESTful HTML apps, even with pure HTML. They just turn out to be pretty crummy apps that folks don't like to use.
Obviously there is a lot of possible overlap between a generic web app and a REST service, but overlap is not equality, and they are different. HTTP != REST, using HTTP does not mean you're using REST. HTTP is well suited to REST applications, but you can certainly use HTTP in non-RESTful ways. People do it all day long.
So, if you want to use REST as a service layer, then do that. Leverage REST for REST sake, and build up your service. Then, start working on clients and interfaces that leverage that service. Don't let your initial client choice color the REST service itself. Focus on use cases of functionality, then build your clients around that.
As the needs of each component change, they can grow in congruence with or separately from each other as necessary. You don't have to punish the mobile apps for a change that a browser requires, or vice a versa. Let each piece be their own master.
Addenda:
Sam -
There's nothing wrong with offering a hybrid approach where some of the requests are served directly by your REST service layer while others are handled through a server side proxy. As long as the semantics are the same, it doesn't really matter. Your REST service certainly doesn't care. But it does potentially become problematic if the REST service returns link rels that are specific to the "raw" REST service rather than the hybrid. Now you have an issue of translating the representation, etc. My basic point is to not let the browser limitations wag your REST API, you can use a separate facade and let the browser influence that.
And this is a logical separation, whether that's manifested in the URL patterns, I have no opinion. That's more a development/maintenance/deployment call. I find that logical separations that can be manifested physically has some benefits in terms of clarity and understanding, but that's me.
Doug -
The raw HTML user experience is a crummy one. If it weren't, there wouldn't be an entire industry surrounding making the browser user application experience un-crummy. Of course it can be functional, and using HTML is an excellent media type for REST applications BECAUSE of the tooling around browsers and such that make working with the interface, viewing artifacts, interacting with the service when possible easier to do. But you don't design your service API around your debugger, and the raw browser is an incomplete tool for fully exploiting HTTP. As a host for JS through XHR, it gets more capable, but now we're talking "rich client" rather than just straight HTML in a browser.
While you can make service POST facades for delete et al, as in your example, the only reason you ARE doing that is because of the limitations of the browser, not for the sake of the API itself. If anything it's cluttering and complicating the API. "More than one way to do it."
Now, obviously you can tunnel everything through POST, simply because you like to tunnel everything through POST. But by hiding things this way you bypass other aspects of the protocol. If you POST foo/1 to /deleteFoos, you won't be able to leverage things like caches and such. A DELETE on foo/1 would invalidate any caches that see the operation, but a POST would slip right through, leaving the old, now deleted, resource behind.
So, there's reasons why there are other verbs than POST and GET in the protocol, even if the native browser chooses not to use them.

I think it is reasonble of you to require javascript to allow you to utilize ajax techniques to do what you suggested in the third paragraph of your question.
Also, to clarify, no matter the client use the Location header of the 201 response to indicate the canonical URI of the newly created resource. These can be checked by your javascript, for clients that use it.
About 'dumber' clients (like: a browser with js disabled), a somewhat ugly way to have them do the redirect is to have a html meta refresh in the returned resource's html representation head section from the POST. The body of the response can then just briefly describe your new resource so long as you use the Location header.

Communication model: C++ and Java

Pals,
I have a requirement to establish a communication channel between C++ and Java layer of my application for the exchange of objects and their properties.
I have got the following options:
XML / SOAP
Postgre SQL
Can you please advice me the Pros & Cons on these. Please share your experiences on the implementation complexities.
Thanks,
Gtk

If the option is between those I would choose XML
Object <=> XML
Java side Simple, C++ side XML Objects
Reason, its simpler for what you want, i.e. pass language objects and not Data Base
Ah, could you specify the communication channel between the apps ?
UPDATE
If you can use JSON I would recommend it instead of XML, here is why.

Another option would be JMS. There are C++ clients out there.

Every time I see XML I think RESTful web service. Both platforms you mentioned have some form of tooling to marshal & unmarshal XML. There are plenty of working examples out in the wild, so a Google/Bing search is good. A nice side-effect is once you have those interfaces built, anything can connect to them.
If you really want to bother with generating a WSDL, then feel free to go the SOAP route. However, speaking with several years of web service integration experience, RESTful is so gosh darned simple compared to anything else.

I would like to suggest a third option : YAML
You have parsing library in YAML for both java and C++. In my experience, it's easier to debug exchange in YAML that in XML (especially if you got full text field or cyclic data structure).

I depends of the kind of message you transfer.
If your message are individual entity that have a short live, I would go for XML, YAML or something similar.
If your message contains information that is going to be used later on and refer to information in previous messages, I would use a database.

How to extract data from a SOAP response in Java?

I have a client set up to send a request to the National Weather Service SOAP server. I am receiving the response that I expect, but I am unsure as to the best way to extract the data from it that I need.
For example, there is a lot of extra data in the XML (in the SOAPBody), but I only want to grab the data for the parameters that I set (such as temperature) to my POJO.
What's the best way to extract this data?

I started out trying to consume SOAP Web Services by hand like you describe - there are better ways.
There are libraries out there that will do all the work for you - no need to parse anything by hand.
Check out JAX-WS. Most Modern IDEs (Certainly Netbeans and Eclipse) also provide point and click support for building web service clients given a WSDL.
The biggest potential problem down this route is if there's no WSDL, or the WSDL is wrong, in which case the tooling I've linked might struggle.
The next safest thing would be to use an XML Parser like JAXP's SAX & DOM etc (they're right there in your JRE) to parse the response and then walk the data structures involved.
Finally you could go the string hacking route using splits or regexes but down that path lies a great deal of potential pain - there's more to the XML spec then nested tags.

It's strongly receommended that you not try and decode SOAP by hand :)
Just to expand on what #Brabster said,
Netbeans has extensive Web Service support, especially using the JAX-WS library.
http://www.netbeans.org/kb/60/websvc/jax-ws.html#Exercise_3_1