We Keep Coding

how to consume a Restful Web Service (Restful API) in Java

I just want to know the high level steps of the process. Here's my thought on the process:
Assumption: the API returns JSON format
Check the API document to see the structure of the returned JSON
Create a corresponding Java class (ex: Employee)
Make Http call to the endpoint to get the JSON response
Using some JSON library (such as GSON, Jackson) to unmarshall the JSON string to Employee object.
Manipulate the Employee object
However, what if the API returned JSON is changed? it's really tedious task to exam the JSON string every now and then to adjust the corresponding Java class.
Can anyone help me out with this understanding. Thanks

You describe how to consume a json over http API, which is fine since most of the APIs out there are just that. If you are interested in consuming Restful HTTP resources however, one way would be:
Check the API documentation, aka. the media-types that your client will need to support in order to communicate with its resources. Some RESTafarians argue that all media-types should be standardized, so all clients could potentially support them, but I think that goes a bit far.
Watch out for link representations, and processing logic. media-types do not only describe the format of the data, but also how to process them. How to display it if its an image, how to run code that might be part of the message, how to layout onto the screen, how to use embedded controls like forms, etc.
Create corresponding Java classes. If the resources "only" describe data (which they usually do in API context), then simple Java classes will do, otherwise more might be needed. For example: can the representation contain JavaScript to run on the client? You need to embed a JavaScript engine, and prepare your class to do just that.
Make call to a bookmarked URI if you have it. There should be no hardcoded SOAP-like "endpoint" you call. You start with bookmarks and work your way to the state your client need to be in.
Usually your first call goes to the "start" resource. This is the only bookmark you have in the beginning. You specify the media-types you support for this resource in the Accept header.
You then check whether the returned Content-Type matches one of your accepted media-types (remember, the server is free to ignore your preferences), and then you process the returned representation according to its rules.
For example you want to get all the accounts for customer 123456 for which you don't yet have a bookmark to. You might first GET the start resource for account management. The processing logic there might describe a link to go to for account listings. You follow the link. The representation there might give you a "form" in which you have to fill out the customer number and POST. Finally, you get your representation of the account list. You may at this point bookmark the page, so you don't have to go through the whole chain the next time.
Process representation. This might involve displaying, running, or just handing over the data to some other class.
Sorry for the long post, slow day at work :) Just for completeness, some other points the client needs to know about: caching, handling bookmarks (reacting to 3xx codes), following links in representations.
Versioning is another topic you mention. This is a whole discussion onto itself, but in short: some people (myself included) advocate versioning the media-type. Non-backwards compatible changes simply change the media type's name (for example from application/vnd.company.customer-v1+json, to application/vnd.company.customer-v2+json), and then everything (bookmarks for example) continues to work because of content negotiation.

There are many ways to consume RESTful APIs.
Typically, you need to know what version of the API you are going to use. When the API changes (i.e. a different version is exposed) you need to decide if the new functionality is worth migrating your application(s) to the latest and greatest or not...
In my experience, migrating to a new API always requires some effort and it really depends on the value of doing so (vs. not doing it) and/or whether the old API is going to be deprecated and/or not supported by the publisher.

Read HL7 message without prescribing message type or event type or version

I need to read an HL7, pipe-delimited message into a custom Java structure that represents the message structure. I need to do all this without using any message type, event type, or version specific methods or classes, because I need this code to work no matter what kind of HL7 message is read. I don't want to write a reader for every different message and event type. I need to get all the segments and subfields. I have looked at HAPI a lot to see if it has a prebuilt solution for this problem, but I have found nothing.
Is there anyway to extend a HAPI class or iterate over the underlying message structure that HAPI creates from a parser to solve this problem?
Also, any good resources on HAPI would be appreciated.

Have a look at the HL7X library for Java. This library offers you to transform any HL7 to an XML. It does not depend on MessageType, EventType nor version (big difference to HAPI).
Example:
MSH|^~\&|||||20121116122025||ADT^A01|5730224|P|2.5||||||UNICODE UTF-8
EVN|A01|20130120151827
PID||0|123||Name^Firstname^^^^||193106170000|w
PV1||E|
Gets transformed to
<?xml version="1.0" encoding="UTF-8"?>
<HL7X>
<HL7X>
<MSH>
<MSH.1>^~\&</MSH.1>
<MSH.6>20121116122025</MSH.6>
<MSH.8>
<MSH.8.1>ADT</MSH.8.1>
<MSH.8.2>A01</MSH.8.2>
</MSH.8>
<MSH.9>5730224</MSH.9>
<MSH.10>P</MSH.10>
<MSH.11>2.5</MSH.11>
<MSH.17>UNICODE UTF-8</MSH.17>
</MSH>
<EVN>
<EVN.1>A01</EVN.1>
<EVN.2>20130120151827</EVN.2>
</EVN>
<PID>
<PID.2>0</PID.2>
<PID.3>123</PID.3>
<PID.5>
<PID.5.1>Name</PID.5.1>
<PID.5.2>Firstname</PID.5.2>
</PID.5>
<PID.7>193106170000</PID.7>
<PID.8>F</PID.8>
</PID>
<PV1>
<PV1.2>E</PV1.2>
</PV1>
</HL7X>
Then you can get your Information from the String based XML or you parse it with any XMLDocument builder.

It is possible to parse messages in a generic way with HAPI though you may find yourself coding some extra and casting some of it's output as it was not designed to be used that way.
By this I mean that it is in fact supported, just that you will not have the benefits of many of HAPI's more advanced features.
With the use of the Terser, Varies, GenericComposite and GenericPrimitives classes I was able to get a simple generic parser up and running in no time at all. It's pretty basic mind you but it does what I need it to.
Next I want to have a look at the MessageVisitor see if I can't get some of the work done in a less hackish way (to read by not relying on casting as much).
Good luck, hope this helps

Metro stack java web services write own serializer/deserializer

I am facing some problem while returning data from web services.
I am using Metro stack.
Problem:
The problem is my data contain some control characters. I don't want to encrypt my data, as this will involve client side code changes as well.
Better way is to remove control characters, but character by characters scanning and removing/replaceAll will have some performance impact.
Probable solution:
What I want to do is intercept SOAP message before metro send it to client and check control characters. And then replace all control characters of XML before returning.
Do anyone here have any idea how can I intercept response before returning? Or
how I can write my own serializer/deserializer where I can replace control char?
Thanks.

I suppose you could use the Provider interface to do this. Your implementation of the provider would be called instead of the actual web service. Then you would parse the XML and forward the call to you service implementation.
I find it somewhat cumbersome to work with, but if it works..

I found XMLAdapter very useful in my case.
Adding info as in case any buddy can find it useful.
Reference: http://weblogs.java.net/blog/kohsuke/archive/2005/09/using_jaxb_20s.html
XMLAdapter contain abstract method marshal and unmarshal. So I used appropriate encoder in marshal/unmarshal method.

Efficiently implementing an existing Protocol

I have an existing Protocol I'd like to write a java Client for. The Protocol consists of messages that have a header containing message type and message length, and then the announced number of bytes which is the payload.
I'm having some trouble modeling it, since creating a Class for each message type seems a bit excessive to me (that would turn out to be 20+ classes just to represent the messages that go over the wire) I was thinking about alternative models. But I can't come up with one that works.
I don't want anything fancy to work on the messages aside from notifying via publish subscribe when a message comes in and in some instances reply back.
Any pointers as to where to look?

A class for each message type is the natural OO way to model this. The fact that there are 20 classes should not put you off. (Depending on the relationship between the messages, you can probably implement common featues in superclasses.)
My advice is to not worry too much about efficiency to start with. Just focus on getting clean APIs that provide the required functionality. Once you've got things working, profile the code and see if the protocol classes are a significant bottleneck. If they are ... then think about how to make the code more efficient.

Would using a WSDL to generate REST clients be the wrong direction?

I'm out to create a fairly simple web service for intranet use. The service will ultimately be the interface to a DB that will allow me to keep track of what the various internal tools within the company are doing. I'm thinking I want a web service so that various tools (and thus different languages) within the organization can easily update the DB without needing to know the schema.
I've read many of the related REST vs SOAP questions already that come up from searching https://stackoverflow.com/search?q=soap+rest but I'm not sure I've found my answer.
My dilemma seems to be that I want the simplicity of REST while also having the code generation abilities of a WSDL, which seems to imply SOAP. It is of utmost importance to me that various internal tools (JAVA, Perl, Python, PHP, C++) be able to talk to this service and it would seem silly to have to rewrite/maintain the interface layer for each of these languages manually when the WSDL route would do that for me. From what I can tell, if the WS needs to change, you would update the WSDL, regenerate the client stubs, and make any changes necessary to the code that uses the stubs (which would need to be done anyway).
For example - say I have a tool written in JAVA that utilizes a RESTful web service. I imagine that the tool will have specific code for dealing with certain URLs, launching the request, doing something with the response, translating that into some data structure if I'd like, etc. Now lets say I also have a Perl tool doing the same thing. Now I'll need Perl code to do the same, make requests on specific URLs get the responses, do something with them, etc. In each case, and thus in C++ and Python and C#, where code cannot be shared, eventually I'll end up with a wrapper classes/methods that hide a lot of that ugliness from me. I'd much rather call a function on a class that returns my data encapsulated in an object instead of having to worry about the URL, the arguments, the response, etc. Sure, maybe its not a lot of code in any particular place but it starts to add up over time. Multiply that out over each of the tools and now when I make a change to the service I must go update the URLs in each CRUD operation and all that goes along with that. I guess I imagine that with a WSDL that is the aspect that is done for you. Your code simply interacts with the stubs. What the stubs do, who cares? Urls, arguments, responses - if anything changes just regenerate the stubs from the WSDL. If that process causes your code to break, so be it, but at least I didn't also have to update all the code that deals with the specifics of making the request and dealing with the response. Is this really not a problem? Perhaps what I need to do is just create a service and a few clients and see what I'm really up against.
While I'm a fairly seasoned programmer with experience in JAVA, Perl, Python, C++, etc, this is the first time I've considered authoring a WS and don't have prior experience with other WSs, so I'm looking for some guidance. Do I just go with WSDL/SOAP and forget about what everybody is saying about how popular, simple, and useful REST is?

I don't get the code generation issue here.
REST rarely requires any code generation of any kind. It's just HTTP requests with simple JSON (or XML) payloads.
You use the existing HTTP libraries (e.g. Apache Commons, or Python urrlib2). You use existing JSON (or XML) libraries. (the Jersey project has a nice JAXB-JSON conversion, for example).
What's "generated"? Our RESTful library in Java and Python are nearly identical and simply make REST requests through the HTTP library.
class OurService {
def getAResource( String argValue ) {
path = { "fixed", argValue };
URI uri= build_path( path );
return connection.get( uri )
[I've left out the exception handling.]
Are you trying to layer in the complex SOAP interface/implementation separation?
A client "written in JAVA that utilizes a RESTful web service"... A "Perl tool doing the same thing" ... "in C++ and Python and C#".
Correct.
"where code cannot be shared"
Correct. The code cannot be shared. You have to write each client in the appropriate language. Writing some kind of "generator" to create this code from WSDL is (1) a huge amount of work and (2) needless. Each language has unique syntax and unique libraries for making REST requests. But it's so simple and generic that there's hardly anything there.
The canonical example in Python is
class Some_REST_Stub( object ):
def get_some_resource( self, arg ): # This name is from the WSDL
uri = "http://host:port/path/to/resource/%s/" % arg # This path is from the WSDL
data= urllib2.open( uri )
return json.load( data )
This block of code is shorter than the WSDL required to describe it. And it seems easier to change because the name is the method name and the URI is simply a string.
In most languages the client is approximately that simple. I just wrote a bunch of REST client code in Java. It's wordier, but it's generic stuff. Build a request, parse the JSON that comes back. That's it.
A RESTful WSDL declaration buries two pieces of trivial information in a lot of XML.
It provides an "interface name" for the URI.
It documents the meaning of GET, PUT, POST and DELETE by providing Stub class method names.
It doesn't help you write much code, since there isn't much code. Note that all REST requests have the same generic HttpRequest and HttpResponse structure. The request contains generic an Entities. The response, also, contains a generic Entity that must be parsed. There's very little to REST -- the point is to be as simple as possible.
It largely eliminates the need for WSDL since everything is a generic JSONObject or XML URLEncoded and sent as a string.
"I'd much rather call a function on a class that returns my data encapsulated in an object instead of having to worry about the URL, the arguments, the response, etc."
Correct, you'll have to write a "stub" class. It has almost no code, and it will be shorter than the WSDL required to describe it.
"Multiply that out over each of the tools and now when I make a change to the service I must go update the URLs in each CRUD operation and all that goes along with that."
You can either update the stub class in each language for each client. Or you can update the WSDL and then regenerate each client. It seems like the same amount of work, since the URI is trivially encapsulated in the client code.
"I guess I imagine that with a WSDL that is the aspect that is done for you."
I'm unclear on what's "done". Translating the wordy and complex WSDL into a simple stub class? I suppose this could be helpful. Except the WSDL is bigger than the stub class. I'm suggesting that you'll probably find it easier to simply write the stub class. It's shorter than the WSDL.
"Your code simply interacts with the stubs."
Correct.
"What the stubs do, who cares? Urls, arguments, responses - if anything changes just regenerate the stubs from the WSDL."
Sadly, almost none of this requires any real programming. Generating it from WSDL is more work than simply writing it. URI's a strings. Arguments are generic JSONObjects. Responses are generic HttpResponses including a JSONArray. That's it.
"I didn't also have to update all the code that deals with the specifics of making the request and dealing with the response."
Except, there aren't any interesting specifics of making the request. HTTP is simple, generic stuff. GET, POST, PUT and DELETE are all nearly identical.

Fossill,
I recommend that you do not bother to learn SOAP for this. Ws-* has a very high learning curve and the (unnecessary) complexity will likely eat you alive.
Looking at your skill set (Java, Perl, Python, C++) you should be very satisfied with a REST (or at least HTTP-based) approach. And: you'll get results very fast.
As S.Lott said, do not worry about the code generation. You'll not need it.
For questions, I suggest you join rest-discuss on Yahoo groups:
http://tech.groups.yahoo.com/group/rest-discuss/
You usually get immediate help with all things REST there.
Personally, I have yet to see any use case that could benefit from using WS-*.
Jan

The code-generation aspect that you value is a maintenance item, but I question its real worth.
Be it thru a WSDL document or your own grammar documentation for a REST-style implementations, clients have to comply to the published interface. The WS/SOAP (code-generation) development model might have more tools, but I think that's because it's clunky and needs them.
There's no impact in the 'integrateability' of your web service. That's an issue of publishing the formal interface (in whatever form that takes), in either case. And the speed with which you move from an interface change to an implementation change is arguably faster with REST-style services. Firing up (and fighting with) WS-* code generation tools takes time.

FYI - REST does have a WSDL-like auto-generation schema definition called WADL. But almost no one uses it.

Apache CXF has Java support for REST clients that gives you the same sorts of 'code generation' advantages, in many cases, as full SOAP.