I am implementing web services for a PHP application and am trying to understand what both standard web services and RESTful web services have to offer.
My intent is to write wrapper code to abstract away the web service details so that developers can just "instantiate remote objects" and use them.
Here are my thoughts, perhaps some of you could add your experience and expand this:
RESTful Web Servcies
are basically just "XML feeds on demand", so e.g. you could write wrapper code for a client application so it could query the server application in this way:
$users = Users::getUsers("state = 'CO'");
this would in turn get an XML feed form a remote URL
$users could be made into a collection of full User objects, or
left as XML, or
turned into an array, etc.
the query script ("state = 'CO'") would be translated into SQL on the server side
RESTful Web Services are in general read-only from the view of the client, although you could write code which could use POST or GET to make changes on the server, e.g. passing an encrypted token for security, e.g.:
$users = Users::addUser($encryptedTrustToken, 'jim',$encryptedPassword, 'James', 'Taylor');
and this would create a new user on the server application.
Standard Web Services
Standard Web Servcies in the end basically do the same thing. The one advantage they have is that client can discover their details via WSDL. But other than that, if I want to write wrapper code which allows a developer to instantiate, edit and save objects remotely, I still need to implement the wrapper code. SOAP doesn't do any of that for me, it can do this:
$soap = new nusoap_client('http://localhost/test/webservice_user.php?wsdl', true);
$user = $soap->getProxy();
$lastName = $user->lastName();
but if I want to edit and save:
$user->setLastName('Jones');
$user->save();
then I need to e.g. handle all of the state on the server side, SOAP doesn't seem to hold that object on the server side for each client.
Perhaps there are limitations in the PHP SOAP implementation I am using (nusoap). Perhaps Java and .NET implementations do much more.
Will enjoy hearing your feedback to clear up some of these clouds.
They are different models... REST is data-centric, where-as SOAP is operation-centric. i.e. with SOAP you tend to have discrete operations "SubmitOrder", etc; but with REST you are typically a lot more fluid about how you are querying the data.
SOAP also tends to be associated with a lot more complexity (which is sometimes necessary) - REST is back to POX etc, and YAGNI.
In terms of .NET, tools like "wsdl.exe" will give you a full client-side proxy library to represent a SOAP service (or "svcutil.exe" for a WCF service):
var someResult = proxy.SubmitOrder(...);
For REST with .NET, I guess ADO.NET Data Services is the most obvious player. Here, the tooling (datasvcutil.exe) will give you a full client-side data-context with LINQ support. LINQ is .NET's relatively new way of forming complex queries. So something like (with strong/static type checking and intellisense):
var qry = from user in ctx.Users
where user.State == 'CO'
select user;
(this will be translated to/from the appropriate REST syntax for ADO.NET Data Services)
I'm echoing what Marc Gravell mentioned. When people ask me about REST (and they usually have an idea about SOAP and SOA), I will tell them REST = ROA as it is resource/data oriented, it's a different paradigm and therefore has different design concerns.
For your case, if I'm reading you correctly, you want to avoid writing the wrapper code and need a solution that can store objects and their attributes remotely (and having them completely hidden from the developers). I can't really suggest a better solution.. Umm, let me know if either of these ever come close to meet your requirements:
EJB3 / JPA
CouchDB (REST/JSON)
Let me know too if I've interpreted your question wrongly.
If we compare XML-RPC and SOAP, the latter will give you better data types handling, for the former although you will be dealing with simpler data types but you will have to write a layer or adapter to convert them to your domain objects.
yc
Soap is just a set of agreed upon XML schemas blessed by a standards group. It's strength is that it was designed for interoperability and it supports many enterprise-class features. Soap on any platform will not provide the operations you are looking for. You need to design & implement the service contract to get those features.
Sounds like you want remote objects which neither Soap or REST are particularly good for. Maybe you'd be better off looking at XML-RPC.
The key differences are basically tooling.
Many of the high end SOAP stacks shroud the vast bulk of the SOAP infrastructure from the developer, to where you are working pretty much solely with DTO's/Documents and RPC.
REST over HTTP puts more of that burden upon you the developer (negotiating formats [XML, JSON, HTTP POST], parsing results [DOM, maps, DTO marshalling, etc.]).
Obviously, you can write a layer of logic to make dealing with this detail easier. And some REST frameworks have arrived to make this easier, but at the moment, it's still a task on your TODO list when you wish to use or consume such services.
My feedback is that if you want remote state, you are not talking about web services. I don't know about any contemporary model that deal with remote state. I think that if you need remote state you are on your own ( without religion to follow ). Just throw xml from here to there and don't mind the theory.
I have to add that remote state is evil. Avoid it if you can.
Related
I just want to know the high level steps of the process. Here's my thought on the process:
Assumption: the API returns JSON format
Check the API document to see the structure of the returned JSON
Create a corresponding Java class (ex: Employee)
Make Http call to the endpoint to get the JSON response
Using some JSON library (such as GSON, Jackson) to unmarshall the JSON string to Employee object.
Manipulate the Employee object
However, what if the API returned JSON is changed? it's really tedious task to exam the JSON string every now and then to adjust the corresponding Java class.
Can anyone help me out with this understanding. Thanks
You describe how to consume a json over http API, which is fine since most of the APIs out there are just that. If you are interested in consuming Restful HTTP resources however, one way would be:
Check the API documentation, aka. the media-types that your client will need to support in order to communicate with its resources. Some RESTafarians argue that all media-types should be standardized, so all clients could potentially support them, but I think that goes a bit far.
Watch out for link representations, and processing logic. media-types do not only describe the format of the data, but also how to process them. How to display it if its an image, how to run code that might be part of the message, how to layout onto the screen, how to use embedded controls like forms, etc.
Create corresponding Java classes. If the resources "only" describe data (which they usually do in API context), then simple Java classes will do, otherwise more might be needed. For example: can the representation contain JavaScript to run on the client? You need to embed a JavaScript engine, and prepare your class to do just that.
Make call to a bookmarked URI if you have it. There should be no hardcoded SOAP-like "endpoint" you call. You start with bookmarks and work your way to the state your client need to be in.
Usually your first call goes to the "start" resource. This is the only bookmark you have in the beginning. You specify the media-types you support for this resource in the Accept header.
You then check whether the returned Content-Type matches one of your accepted media-types (remember, the server is free to ignore your preferences), and then you process the returned representation according to its rules.
For example you want to get all the accounts for customer 123456 for which you don't yet have a bookmark to. You might first GET the start resource for account management. The processing logic there might describe a link to go to for account listings. You follow the link. The representation there might give you a "form" in which you have to fill out the customer number and POST. Finally, you get your representation of the account list. You may at this point bookmark the page, so you don't have to go through the whole chain the next time.
Process representation. This might involve displaying, running, or just handing over the data to some other class.
Sorry for the long post, slow day at work :) Just for completeness, some other points the client needs to know about: caching, handling bookmarks (reacting to 3xx codes), following links in representations.
Versioning is another topic you mention. This is a whole discussion onto itself, but in short: some people (myself included) advocate versioning the media-type. Non-backwards compatible changes simply change the media type's name (for example from application/vnd.company.customer-v1+json, to application/vnd.company.customer-v2+json), and then everything (bookmarks for example) continues to work because of content negotiation.
There are many ways to consume RESTful APIs.
Typically, you need to know what version of the API you are going to use. When the API changes (i.e. a different version is exposed) you need to decide if the new functionality is worth migrating your application(s) to the latest and greatest or not...
In my experience, migrating to a new API always requires some effort and it really depends on the value of doing so (vs. not doing it) and/or whether the old API is going to be deprecated and/or not supported by the publisher.
I am building a service that depends on another service. A typical Service oriented architecture. The service i am dependent on exposes some API and data types. I am confused should i be converting the object types exposed by that service into specific objects which my service understands. I do expect their service to change with time as these are two different services. I have two options:
Directly use those data types in my service and pass those in methods.
Transform those into specific data types which only my service understands. ( objects will look exactly same if i do this with 0 changes ).
I tried to answer these questions but still could not make the final call. I need help in making this decision.
Why should I have encapsulated/transformed types ?
To prevent building every time they build changes in the service.
To prevent widespread changes ( adapter pattern ) : Changes to the wire
format will lead me to change only the encapsulating classes.
Why should I not have the changes for the types encapsulated ?
The classes will look exactly same as the wire format classes. ( Useless effort to maintain extra classes )
As i understand the impact will be same if i go with either approach. Help ?
I am no architect or SOA specialist, so excuse me if I am saying anything stupid :-)
But I really think the way here is to keep your services simple.
In your shoes, I'd just directly use the existent API. I would not spent any time wrapping or adapting the methods into another API. Your second service (that uses the existent first service) business logic should take care of this convertion, IMO, except if you're being forced to do something that is really expensive with the existent API.
Remember that services are mutable. They're software. They have bugs, business logic changes as time goes and you'll have to change the API and sometimes you'll have to keep older methods compatible for other service consumers. You probably don't want to maintain two APIs that provide the same information without any good practical reason. Not for twice the maintenance work.
Creating another API just to adapt the data format sounds to me a little like that old "DTOs are evil" flame war. And I think a very few people write about the advantages of using DTO nowadays :-)
This is sort of opinion based question, so my opinion is, you should make your own data-types to let your piece of code understand what should be contained in which variable.
I think of services as a data provider, which accepts certain request and fulfill our needs and in return may give us some data. I think role of service is just providing services to client.
It should be responsiblity of client to accept the data returned by service and store them in certain data-structure as there can be n different clients for single service and they can have n different requirements which may lead them to design client specific data-structure to contain data.
Also, as you said client service is subject to change over the period of time, then if you make your own data-structure, then you will need to make change in one single place, and rest of your code will be safe.
I have a desktop client that will be using Netty to connect to a server (your basic client/server setup). The client has many objects (POJO's) to not only CRUD on, but also to do some complex business logic which is why I can't just have the clients talk directly to the database. As well, I don't want to place all the SQL logic in the clients.
So for example I have:
boolean RaceTrackDAO.saveTrack(Track track);
int RaceTrackDAO.getFastestLapTime(Track track);
Driver[] RaceTrackDAO.getTopDriversForTrack(int trackID)
...
boolean DriverDAO.saveDriver(Driver driver);
bollean DriverDAO.updateDriver(Driver driver);
Driver[] DriverDAO.getTopTenDrivers(Driver driver);
...
and so on...
In the client, I have:
someUICode()
{
...
Driver[] topTenDrivers = DriverDAO.getTopTenDrivers();
...
}
The question is how do I architecture/implement the DriverDAO layer to use Netty in the client/server system?
I understand you can send POJO's through the Channel, but the documentation only really talks about sending one type of POJO. I've done some research and find there's all kinds of options. There's the command and state patterns which look interesting.
The key for me is avoiding as much boilerplate coding as possible. I know Netty has built-in tools to marhsall and unmarshall POJO's through serialization. That's the easier part. I just don't want to be stuck with a million if statements to deal with all the possible pojo's (and to know which backend calls to make).
The key for me is to keep the whole very maintainable. Performance is nice to have, but the system is never going to need to handle thousands of connections at once, so there's a bit of leeway here.
With all that in mind, what's the best architecture to design such a system using Netty at the DAO layer?
If you want to keep things simple you can use RMI(?) or serialize your object over Netty. That way you don't need to be able to handle all the possible objects. (You should still try to keep them to a manageable level)
Magento has a "Webservices API" that's accesible via SOAP and RPC.
The terminology gets a screwy here, as the SOAP (or RPC) API itself only exposes 8 methods. To access a method in the Magento API you call the SOAP api method call, and then pass in a string that indicates the method you really want to call, (something like catalog_product.info)
Over the years the Magento team introduced two additional API backends, SOAP V2, and SOAP WSI. These backends were introduced "To increase compatibility with .NET and Java"
What does this mean, exactly? From a code point of view these APIs change the WSDL that's exposed, and when using PHP's SOAP client the whole call thing goes away. You're calling actual methods on the client itself goes away
$client->call($session, 'catalog_product.info',...);
vs.
$client->catalogProductInfo($sessionId, ...
It's never been clear to me how this improves .NET or Java compatibility (because I'm not familiar with those tool-chains).
Is this just a code generation convenience/culture thing, or is it literally impossible for .NET and Java users to use the original SOAP API?
What was the improvement in general?
The building blocks of a SOAP service described by WSDL are roughly:
Contract: Specified by wsdl:portType, wsdl:message and wsdl:type, that describes operations as well as the messages and types these operate on. So this tells: What can the service do?
Binding: A wsdl:binding binds a wsdl:portType to a concrete transport, a document style, policies, etc. This tells: How to talk to the service?
Address: wsdl:service section links a wsdl:binding to an concrete service endpoint address. This answers: Where is the service?
The above mentioned points are more or less the ABC-tenet of the Windows Communication Foundation (WCF).
So the first RPC-approach of magento was simply an abuse of the SOAP protocol. Every request going through the key-hole of the few entry-functions has another, wrapped RPC protocol as message payload. So SOAP was just used as an message transport, just like SOAP uses eg. HTTP-POST. To keep the contract valid, it has to be either very complex (an enormous amount of possible messages) or very generic (simply a string). The contract becomes much easier if every method/operation has it's own declaration. That is what the V2 API does.
But why is this also a big improvement for static typed languages like Java or C#?
Like macki mentioned, there is tooling (proxy generators, etc) and frameworks (like WCF or JAX-WS) for these languages. They generate depending on the above mentioned contract data transfer objects (DTO)s for the message-types and proxies for the method invocation (they do much more but that's not important here). As you see in the Magento Wiki on C# magento soap v2 api an filters-object is instantiated and the salesOrderList proxy method. A good part of the contract is based on static typing of the method and its parameter. If you would use the V1 keyhole-methods you had to build the messages to satisfy the contract by yourself in an more arbitrary data structure like a string, array of strings or dictionary of strings.
So of course it is possible to use the old API from static typed languages it doesn't feel right. It is much better to have on type per message-type and a statically typed signature of operation proxies. Beside that V2 are also better SOAP service regardless the language you call them with.
From what I know at least in .NET with the original ones you have to do everything manually (as there is no WSDL specification of available methods) and with the V2 you can use various "wizards" and "WSDL importers" that will create proxy classes with correct data types and names. Makes it much easier to code. Also there is a problem with complex data types with the original methods - they are hard to work with directly.
There are 105 Magento SOAP v2 function APIs, that you should be aware of their use and application.
First go to Magento SOAP on how to create a SOAP client and login. The check out the rather brief description of the SOAP API v2. Then run the following code from your PHP CLI (via command line) or PHP CGI (via Web browser) respective script engine.
$client = new SoapClient('http://mymagentohost/soap/api/?wsdl');
// If some SOAP APIs require API authentication, then get a session token
$session = $client->login('apiUser', 'apiKey');
foreach($client->__getFunctions() as $function)
echo "function: $function".PHP_EOL; // or .<br> if running the PHP from your Web browser
Cut and paste the SOAP v2 API function list to a text file and save it. You'll find the API list will be invaluable, so you will want to keep handy.
I'm out to create a fairly simple web service for intranet use. The service will ultimately be the interface to a DB that will allow me to keep track of what the various internal tools within the company are doing. I'm thinking I want a web service so that various tools (and thus different languages) within the organization can easily update the DB without needing to know the schema.
I've read many of the related REST vs SOAP questions already that come up from searching https://stackoverflow.com/search?q=soap+rest but I'm not sure I've found my answer.
My dilemma seems to be that I want the simplicity of REST while also having the code generation abilities of a WSDL, which seems to imply SOAP. It is of utmost importance to me that various internal tools (JAVA, Perl, Python, PHP, C++) be able to talk to this service and it would seem silly to have to rewrite/maintain the interface layer for each of these languages manually when the WSDL route would do that for me. From what I can tell, if the WS needs to change, you would update the WSDL, regenerate the client stubs, and make any changes necessary to the code that uses the stubs (which would need to be done anyway).
For example - say I have a tool written in JAVA that utilizes a RESTful web service. I imagine that the tool will have specific code for dealing with certain URLs, launching the request, doing something with the response, translating that into some data structure if I'd like, etc. Now lets say I also have a Perl tool doing the same thing. Now I'll need Perl code to do the same, make requests on specific URLs get the responses, do something with them, etc. In each case, and thus in C++ and Python and C#, where code cannot be shared, eventually I'll end up with a wrapper classes/methods that hide a lot of that ugliness from me. I'd much rather call a function on a class that returns my data encapsulated in an object instead of having to worry about the URL, the arguments, the response, etc. Sure, maybe its not a lot of code in any particular place but it starts to add up over time. Multiply that out over each of the tools and now when I make a change to the service I must go update the URLs in each CRUD operation and all that goes along with that. I guess I imagine that with a WSDL that is the aspect that is done for you. Your code simply interacts with the stubs. What the stubs do, who cares? Urls, arguments, responses - if anything changes just regenerate the stubs from the WSDL. If that process causes your code to break, so be it, but at least I didn't also have to update all the code that deals with the specifics of making the request and dealing with the response. Is this really not a problem? Perhaps what I need to do is just create a service and a few clients and see what I'm really up against.
While I'm a fairly seasoned programmer with experience in JAVA, Perl, Python, C++, etc, this is the first time I've considered authoring a WS and don't have prior experience with other WSs, so I'm looking for some guidance. Do I just go with WSDL/SOAP and forget about what everybody is saying about how popular, simple, and useful REST is?
I don't get the code generation issue here.
REST rarely requires any code generation of any kind. It's just HTTP requests with simple JSON (or XML) payloads.
You use the existing HTTP libraries (e.g. Apache Commons, or Python urrlib2). You use existing JSON (or XML) libraries. (the Jersey project has a nice JAXB-JSON conversion, for example).
What's "generated"? Our RESTful library in Java and Python are nearly identical and simply make REST requests through the HTTP library.
class OurService {
def getAResource( String argValue ) {
path = { "fixed", argValue };
URI uri= build_path( path );
return connection.get( uri )
[I've left out the exception handling.]
Are you trying to layer in the complex SOAP interface/implementation separation?
A client "written in JAVA that utilizes a RESTful web service"... A "Perl tool doing the same thing" ... "in C++ and Python and C#".
Correct.
"where code cannot be shared"
Correct. The code cannot be shared. You have to write each client in the appropriate language. Writing some kind of "generator" to create this code from WSDL is (1) a huge amount of work and (2) needless. Each language has unique syntax and unique libraries for making REST requests. But it's so simple and generic that there's hardly anything there.
The canonical example in Python is
class Some_REST_Stub( object ):
def get_some_resource( self, arg ): # This name is from the WSDL
uri = "http://host:port/path/to/resource/%s/" % arg # This path is from the WSDL
data= urllib2.open( uri )
return json.load( data )
This block of code is shorter than the WSDL required to describe it. And it seems easier to change because the name is the method name and the URI is simply a string.
In most languages the client is approximately that simple. I just wrote a bunch of REST client code in Java. It's wordier, but it's generic stuff. Build a request, parse the JSON that comes back. That's it.
A RESTful WSDL declaration buries two pieces of trivial information in a lot of XML.
It provides an "interface name" for the URI.
It documents the meaning of GET, PUT, POST and DELETE by providing Stub class method names.
It doesn't help you write much code, since there isn't much code. Note that all REST requests have the same generic HttpRequest and HttpResponse structure. The request contains generic an Entities. The response, also, contains a generic Entity that must be parsed. There's very little to REST -- the point is to be as simple as possible.
It largely eliminates the need for WSDL since everything is a generic JSONObject or XML URLEncoded and sent as a string.
"I'd much rather call a function on a class that returns my data encapsulated in an object instead of having to worry about the URL, the arguments, the response, etc."
Correct, you'll have to write a "stub" class. It has almost no code, and it will be shorter than the WSDL required to describe it.
"Multiply that out over each of the tools and now when I make a change to the service I must go update the URLs in each CRUD operation and all that goes along with that."
You can either update the stub class in each language for each client. Or you can update the WSDL and then regenerate each client. It seems like the same amount of work, since the URI is trivially encapsulated in the client code.
"I guess I imagine that with a WSDL that is the aspect that is done for you."
I'm unclear on what's "done". Translating the wordy and complex WSDL into a simple stub class? I suppose this could be helpful. Except the WSDL is bigger than the stub class. I'm suggesting that you'll probably find it easier to simply write the stub class. It's shorter than the WSDL.
"Your code simply interacts with the stubs."
Correct.
"What the stubs do, who cares? Urls, arguments, responses - if anything changes just regenerate the stubs from the WSDL."
Sadly, almost none of this requires any real programming. Generating it from WSDL is more work than simply writing it. URI's a strings. Arguments are generic JSONObjects. Responses are generic HttpResponses including a JSONArray. That's it.
"I didn't also have to update all the code that deals with the specifics of making the request and dealing with the response."
Except, there aren't any interesting specifics of making the request. HTTP is simple, generic stuff. GET, POST, PUT and DELETE are all nearly identical.
Fossill,
I recommend that you do not bother to learn SOAP for this. Ws-* has a very high learning curve and the (unnecessary) complexity will likely eat you alive.
Looking at your skill set (Java, Perl, Python, C++) you should be very satisfied with a REST (or at least HTTP-based) approach. And: you'll get results very fast.
As S.Lott said, do not worry about the code generation. You'll not need it.
For questions, I suggest you join rest-discuss on Yahoo groups:
http://tech.groups.yahoo.com/group/rest-discuss/
You usually get immediate help with all things REST there.
Personally, I have yet to see any use case that could benefit from using WS-*.
Jan
The code-generation aspect that you value is a maintenance item, but I question its real worth.
Be it thru a WSDL document or your own grammar documentation for a REST-style implementations, clients have to comply to the published interface. The WS/SOAP (code-generation) development model might have more tools, but I think that's because it's clunky and needs them.
There's no impact in the 'integrateability' of your web service. That's an issue of publishing the formal interface (in whatever form that takes), in either case. And the speed with which you move from an interface change to an implementation change is arguably faster with REST-style services. Firing up (and fighting with) WS-* code generation tools takes time.
FYI - REST does have a WSDL-like auto-generation schema definition called WADL. But almost no one uses it.
Apache CXF has Java support for REST clients that gives you the same sorts of 'code generation' advantages, in many cases, as full SOAP.