We Keep Coding

CQRS + Event Sourcing handling update of entity using requests

I'm developing this JavaEE REST microservice oriented CQRS + EventSourcing app, I have this entity (Artwork) with many fields and I have to record each update to this entity according to EventSourcing pattern (basically each update creates a new event, the artwork is then rebuild using these events).
My approach basically works, but I'm stuck with a "compliance" towards HTTP standards, basically I want to avoid a "generic" update in which you update the whole entity because it will be a mess to handle each single field update (and consequent event generation).
So this is what I did.
Let's say that I have this entity:
public entity{
int id;
String field1;
String field2;
...
Then I created as many requests as many fields I have to update (not all fields can be updated, such as the ID)
public field1UpdateRequest{
field1 newvalue;
}
and the same for field 2.
These updated are handled using a PUT request, when such a request arrives, it is handled by something like this:
HTTP → Controller→ Service → (DAOS etc.)
So in the controller class I have a PUT http://...//updatefield1 method that accepts field1UpdateRequest objects.
My question is:
Is this right to do? How can I explain that this is right (if it is)? should these requests be PATCH more than PUT? Should a generic PUT request also be included? (Even if I'm scared that this will make the event sourcing part more difficult)?

In a CQRS spproach, it's important to remember that the C stands for Command. Every request to your "write-side" is thus a command. A generic "here is the new value for this resource" request (which is what REST tends to lead to) can be interpreted as a "use this value henceforth" command, but it is a bit of an impedance mismatch with CQRS, because it's a fairly anemic command. There are definitely cases where having that in an API can be called for (and if it's an exceptionally rare request, you may even be able to get away with modeling it as a single "new beginning" event rather than teasing out finer-grained events; this has the cost of shifting some complexity out to consumers of the events).
With that in mind, an alternative approach that updates parts of an object is a little more of a fit with CQRS (though in this case, you are shifting some complexity to requestor, at least if that requestor wants to do wholesale updates). HTTP PUTsounds proper to me: the command is "use this value for this field/component of the entity".
That said, an even more CQRSy API would instead focus on the higher-level activities which motivate a change to the entity. For instance if you're tracking the current owner of the artwork as of when it was sold, you might have a currentOwner and a currentOwnerAcquired field in your artwork entity. When recording a sale, you would want to update both, so a POST /artworks/{artworkId}/transferOwnership endpoint taking something like
{
"transferor": "Joe Bloggs",
"transferee": "Jack Schmoe",
"date": "2021-12-24T00:00:01Z"
}
would allow the update to be a single transaction and allow you to encode the "why" as well as the "what" in your events (which is an advantage of event sourcing).

So in the controller class i have a PUT http://...//updatefield1 method that accepts field1UpdateRequest objects.
Is this right to do?
It might be, but it probably isn't.
Here's the key idea: your REST API is a facade; it supports the illusion that your server is stores and produces documents. In other words, your providing an interface to your data that makes it look like every other site on the web.
The good news: when you do that, you get (for free!) the benefits of a bunch of general purpose work that has already been done for you.
But the cost (of these things that you get for free) is that - for everything to "just work", you need to handle the impedance mismatch between HTTP (which is based on an idiom of documents) and your domain model.
So I send to you messages to "edit your documents", and you in turn figure out how to translate those messages into commands for your domain model.
In HTTP, both PUT and PATCH have remote authoring semantics. Both of those messages mean "make your copy of the document look like my copy". They are the flavor of HTTP messages you would use to (for example) edit the title of an HTML document on your web server.
The semantics are fundamentally anemic. My message tells you how I want your copy of the document to look, your server is responsible for figuring out how to achieve that.
And that's fine when you are working with documents, or using documents as a facade in front of a data model. But matching remote authoring requests with a domain model are a lot harder.
(Recommended reading: Greg Young 2010 on task based user interfaces).
In the case of a domain model, you normally want to send to the server a representation of a command message. HTTP really wants you to deal with command messages in one of two ways:
treat the command message as a document/resource of its own, to be stored on the server (the changes to the domain model are a side effect of storing a new command message)
POST the command message to the resource most directly impacted by the change.
(See Fielding, 2009; it is okay to use POST).
In both cases, the HTTP application itself knows nothing about what's going on at the domain level, it is only concerned with the transfer of documents over the network.
HTTP doesn't really place any constraints on your resource model - if you want to publish all of your information in one document, that's fine. If you want to distribute your information across many documents, that's also fine.
So taking a single entity in your domain, and distributing its information across many resources is fine.
BUT: remember caching. HTTP has simple rules for automatically invalidating previously cached responses; separating the resource you use for reading information from the resource that you use for editing information makes caching harder (caution: caching is already one of the two hard problems).
In other words: trade offs.

Is a REST request normally JSON output or something else?

A full REST api like with Java's jax-rs contains definitions for defining a path for a resource, uses the full GET, POST, PUT requests.
But typically when I encounter a REST API, it is typically a standard HTTP GET request and the response is a JSON output. It seems like the meat of a real-world REST request is using JSON output but the true definition of REST allows for XML, JSON or other output types.
For example, the twitter API has "JSON" output, they use a GET request and here are some of the URL's:
https://dev.twitter.com/docs/api/1.1/get/search/tweets
And you can still use the "GET" parameters to modify the request. It seems like the twitter 'search/tweets' function is just a simple http request with a well defined URI path that happens to return a JSON response. Is that really REST?
What is a REST api to you?
On Jax-rs
http://en.wikipedia.org/wiki/Java_API_for_RESTful_Web_Services
(Sorry if this is slightly subjective or anecdotal but I believe developers were wondering about this)

REST (Representational State Transfer) is a vague architectural design pattern which was first written about in a paper by Roy Fieldings (aka the guy who created HTTP).
Most of the time (99% of the time) when somebody wants a REST API they mean they want a web API where they send a Request containing an HTTP verb and a URL which describes the location of a Resource that will be acted upon by the HTTP verb. The web server then performs the requested verb on the Resource and sends back a Response back to the user. The Response will usually (depending on the HTTP verb used) contain a Representation of the resulting Resource. Resources can be represented as HTML, JSON, XML or a number of other different mime types.
Which representation used in the response doesn't really indicate whether an API is RESTful or not; it's how well the interface's URLs are structured and how the web server's behaviors are defined using the HTTP Verbs. A properly compliant REST API should use a GET verb to only read a resource, a POST verb to modify a resource, a PUT to add/replace a resource, and a DELETE to remove a resource. A more formal definition of the expected verb behaviors are listed in the HTTP Specification.

REST is (in a nutshell) a paradigm that resources should be accessible through a URI and that HTTP-like verbs can be used to manipulate them (that is to say, HTTP was designed with REST principles in mind). This is, say, in contrast to having one URI for your app and POSTing a data payload to tell the server what you want to achieve.
With a rough analogy, a filesystem is usually RESTful. Different resources live at different addresses (directories) that are easy to access and write to, despite not being necessarily stored on disk in a fashion that reflects the path. Alternatively, most databases are not RESTful - you connect to the database and access the data through a declarative API, rather than finding the data by a location.
As far as what the resource is - HTML, JSON, a video of a waterskiing squirrel - that is a different level of abstraction than adhering to RESTful principles.

REST is a pretty "loose" standard, but JSON is a reasonable format to standardize around. My only major concern with JSON overall is that it doesn't have a method for explicitly defining its character encoding the way XML does. As for the verbs, sticking to meaningful usages of the verbs is nice, but they don't necessarily map one for one in every situation, so as usual, use common sense :)

It can be JSON, it can be XML. JSON is not exactly industry "standard," but many developers like it (including me) because it's lightweight and easy.

That's pretty much it. REST is designed to be simple. The gist of it is that each url corresponds to a unique resource. The format of the resource is commonly json, but can be anything, and is typically determined by the "extension" or "format" part of the url. For example, "/posts/1.json" and "/posts/1.xml" are just two different representations of the same logical resource, formatted in json and xml, respectively. Another common trait of a RESTful interface is the use of HTTP verbs such as GET, PUT, and POST for retrieving, modifying and creating new resources. That's pretty much all there is to it.

HATEOAS and dynamic discovery of API

The HATEOAS principle "Clients make state transitions only through actions that are dynamically identified within hypermedia by the server"
Now I have a problem with the word dynamically, though I guess it's the single most important word there.
If I change one of my parameters from say optional to mandatory in the API, I HAVE to fix my client else the request would fail.
In short, all HATEOAS does is give the server side developer extreme liberty to change the API at will, at the cost of all clients using his/her API.
Am I right in saying this, or am I missing something like versioning or maybe some other media-type than JSON which the server has to adopt?

Any time you change a parameter from optional to mandatory in an API, you will break consumers of that API. That it is a REST API that follows HATEOAS principles does not change this in any way. Instead, if you wish to maintain compatibility you should avoid making such changes; ensure that any call made or message sent by a client written against the old API will continue to function as expected.
On the other hand, it is also a good idea to not write clients to expect the set of returned elements to always be identical. They should be able to ignore additional information given by a server if the server chooses to provide it. Again, this is just good API design.
HATEOAS isn't the problem. Excessively rigid API expectations are the problem. HATEOAS is just part of the solution to the problem (as it potentially relieves clients from having to know vast amounts about the state model of the service, even if it doesn't necessarily make it straight-forward).

Donal Fellows has a good answer, but there's another side to the same coin. The HATEOAS principle doesn't have anything to say itself about the format of your messages (other parts of REST do); instead, it means essentially that the client should not try to know which URI's to act upon out of band. Instead, the server should tell the client which URI's are of interest via hyperlinks (or forms/templates which construct hyperlinks). How it works:
The client starts at state 0.
The client requests a well-known resource.
The server's response moves the client to a new state N. There may be multiple states achievable at this point depending on the response code and payload.
The response includes links (or forms/templates) which tell the client, in band, the set of potential next states.
The client selects one of the potential next states by issuing a method on a URI.
Repeat 3 through 5 to states N+1 and beyond until the client's application needs are met.
In this fashion, the server is free to change the URI that moves the client from state N to state N+1 without breaking the client.

It seems to me that you misunderstood the quoted principle. Your question suggests that you think about the resources and that they could be "dynamically" defined. Like a mandatory property added to certain resource type at the application runtime. This is not what the principle says and this was correctly pointed out in other answers. The quoted principle says that the actions within the hypermedia should be dynamically identified.
The actions available for a given resource may change in time (e.g. because someone added/removed a relationship in the meantime) and there may be different actions available for the same resource but for different users (e.g. because users have different authorization levels). The idea of HATEOAS is that clients should not have any assumptions about actions available for certain resource at any given time. The client should identify available actions each time it reads that resource.
Edit: The below paragraph may be wrong. See the comments for discussion about it.
On the other hand clients may have expectation for the data available in the resource. Like that a book resource must have a title and that it there may be links to the book's author or authors. There is no way of avoiding the coupling introduced by these assumptions but both service providers and clients should use backward-compatibility and versioning techniques to deal with it.

How to determine JAX-RS resource paths programatically?

Suppose I have a set of JAX-RS locators and sublocators, like the following:
#Path("/users")
public class UserListResource {
#Path("/{id}")
public UserResource getCustomer(#PathParam("id") int id) {
// Find and return user object
}
}
public class UserResource {
#GET
public String get() {...}
}
For example, a UserResource object with the ID 5 would have the path "/users/5". In my system, I have several different resources.
Now the question is: How can the server figure out the path of a given resource? Can I do this programmatically via some JAX-RS API or do I have to implement code that uses reflection? (I know how to do the latter, but would prefer the other approach.)
At the point when I need to know the path, I do not have a request object at all. For example, I have a timer which does some background processing, then changes some entities in the domain model, then informs all clients about the changed entities (including their paths).
I know that within the scope of a request, I can inject a UriInfo object that provides this, but I need to know the path in advance (to inform clients of a change that did not necessarily happen through the JAX-RS resource).
I don't want to repeat the path information in another place, and I also don't want to have a set of path fragment constants for each resource type (in this case "/users" and "/{id}").

As I read your question, you need to build a URI knowing only the resource class and the id parameter.
It can be done using the UriBuilder class as in:
UriBuilder builder=UriBuilder.fromResource(UserListResource.class);
URI uri=builder.path(UserListResource.class,"getCustomer").build(5);
It uses reflection under the hood, so it is not so easy to refactor, but it is all it is available at the moment.

Overall, be aware that something sounds rather strange with the architecture of your application. It's hard to put a finger on, but the pattern of questions you are asking is raising a number of red flags about how you're going about this. Be aware that if you are seeking to create a RESTful API to your application that you may need to stop, take a few steps back, and rethink what you are trying to do.
To your explicit questions…
Now the question is: How can the server figure out the path of a given resource? Can I do this programmatically via some JAX-RS API or do I have to implement code that uses reflection? (I know how to do the latter, but would prefer the other approach.)
The server knows the path, as that's always supplied by the user and is used to navigate through the collection of resource classes that make up your application. If you need a UriInfo instance for a particular call, you should inject it as part of that specific call:
#GET
public String get(#Context UriInfo info) {...}
Any information required from the outer context (e.g., what the resource's ID is) is best passed in during construction. You can reparse it out of the URL (obtainable from the UriInfo) again, but that's probably the wrong approach.
Otherwise, if you're doing something much more complex then you need to be more specific in your question.
At the point when I need to know the path, I do not have a request object at all. For example, I have a timer which does some background processing, then changes some entities in the domain model, then informs all clients about the changed entities (including their paths).
I know that within the scope of a request, I can inject a UriInfo object that provides this, but I need to know the path in advance (to inform clients of a change that did not necessarily happen through the JAX-RS resource).
How are you going to have the clients be informed? There's normally no mechanism to push messages from the server to the clients, and clients are typically firewalled so that they can't directly host a service.
Theoretically, you could associate (explicitly, by URL) each resource with its own RSS feed to which a client could listen to if they chose. You wouldn't be able to force clients to listen, but you could give them the option to do so. If you go this route, you don't need to know the UriInfo “ahead of time” as the location information will be present at key times (i.e., at resource creation) and afterwards you're just referring to something that you have control over.
But that's just one way to do it and it adds a lot of complexity; you'd only do it if it was critical to your application. It's often simpler to just have clients poll from time to time. (Note that some sorts of modifications are inherently very destructive; particularly altering the ID or deleting the resource. Don't expect things to cope smoothly with those.)
I don't want to repeat the path information in another place, and I also don't want to have a set of path fragment constants for each resource type (in this case "/users" and "/{id}").
Tough. Repeating information in multiple places, provided you draw it consistently from a single source, is a common practice. There's nothing actually wrong with it.

As I understand your question, you want to know the path as the request is coming in but before it hits your resource; are you open to using Servlet Filters?
JAX-RS specific filters are only supported in 2.0

For the record: after I had posted the question, I thought about our architecture a bit more and came to the conclusion that sending URLS is not as useful as I thought. The application has to know some details about the application structure anyway:
Continuing the example above: even if the client did not know the URL pattern for individual users, it must assume that there is a list of users and know its URL; it also has hard-coded knowledge what dialog to display for editing a user etc.
So all in all, attempting to tell the client (most) URLs it needs is not worth the effort. Instead, we decided to go with a custom API definition file which includes data about the resource contents and their URL scheme. This file is used to generate the following:
the server-side resource classes with the correct JAX-RS annotations
a URL scheme specification document for other developers to code against
classes for our own client (including the URL know how, e.g. user with ID 5 has the URL ...), so we don't have to worry about inconsistencies between our client and server.
This approach has the following advantages:
The need for the server to figure out the URLs from the annotations vanishes, as the client can now do that on its own once it receives a notification that includes the object ID.
We don't have to worry about inconsistencies between our client and the server, as all information is drawn from a single source.
We have one source for the API definition under version control which can be used to verify backwards compatibility with older releases.
Note:
I would probably not claim that the resulting API stays "faithful" to the idea of RESTful webservices, but it works for us and the elements that it borrows from "actual" REST architectural style should make the API clearer and easier to learn than a traditional contract-first webservice.

Would using a WSDL to generate REST clients be the wrong direction?

I'm out to create a fairly simple web service for intranet use. The service will ultimately be the interface to a DB that will allow me to keep track of what the various internal tools within the company are doing. I'm thinking I want a web service so that various tools (and thus different languages) within the organization can easily update the DB without needing to know the schema.
I've read many of the related REST vs SOAP questions already that come up from searching https://stackoverflow.com/search?q=soap+rest but I'm not sure I've found my answer.
My dilemma seems to be that I want the simplicity of REST while also having the code generation abilities of a WSDL, which seems to imply SOAP. It is of utmost importance to me that various internal tools (JAVA, Perl, Python, PHP, C++) be able to talk to this service and it would seem silly to have to rewrite/maintain the interface layer for each of these languages manually when the WSDL route would do that for me. From what I can tell, if the WS needs to change, you would update the WSDL, regenerate the client stubs, and make any changes necessary to the code that uses the stubs (which would need to be done anyway).
For example - say I have a tool written in JAVA that utilizes a RESTful web service. I imagine that the tool will have specific code for dealing with certain URLs, launching the request, doing something with the response, translating that into some data structure if I'd like, etc. Now lets say I also have a Perl tool doing the same thing. Now I'll need Perl code to do the same, make requests on specific URLs get the responses, do something with them, etc. In each case, and thus in C++ and Python and C#, where code cannot be shared, eventually I'll end up with a wrapper classes/methods that hide a lot of that ugliness from me. I'd much rather call a function on a class that returns my data encapsulated in an object instead of having to worry about the URL, the arguments, the response, etc. Sure, maybe its not a lot of code in any particular place but it starts to add up over time. Multiply that out over each of the tools and now when I make a change to the service I must go update the URLs in each CRUD operation and all that goes along with that. I guess I imagine that with a WSDL that is the aspect that is done for you. Your code simply interacts with the stubs. What the stubs do, who cares? Urls, arguments, responses - if anything changes just regenerate the stubs from the WSDL. If that process causes your code to break, so be it, but at least I didn't also have to update all the code that deals with the specifics of making the request and dealing with the response. Is this really not a problem? Perhaps what I need to do is just create a service and a few clients and see what I'm really up against.
While I'm a fairly seasoned programmer with experience in JAVA, Perl, Python, C++, etc, this is the first time I've considered authoring a WS and don't have prior experience with other WSs, so I'm looking for some guidance. Do I just go with WSDL/SOAP and forget about what everybody is saying about how popular, simple, and useful REST is?

I don't get the code generation issue here.
REST rarely requires any code generation of any kind. It's just HTTP requests with simple JSON (or XML) payloads.
You use the existing HTTP libraries (e.g. Apache Commons, or Python urrlib2). You use existing JSON (or XML) libraries. (the Jersey project has a nice JAXB-JSON conversion, for example).
What's "generated"? Our RESTful library in Java and Python are nearly identical and simply make REST requests through the HTTP library.
class OurService {
def getAResource( String argValue ) {
path = { "fixed", argValue };
URI uri= build_path( path );
return connection.get( uri )
[I've left out the exception handling.]
Are you trying to layer in the complex SOAP interface/implementation separation?
A client "written in JAVA that utilizes a RESTful web service"... A "Perl tool doing the same thing" ... "in C++ and Python and C#".
Correct.
"where code cannot be shared"
Correct. The code cannot be shared. You have to write each client in the appropriate language. Writing some kind of "generator" to create this code from WSDL is (1) a huge amount of work and (2) needless. Each language has unique syntax and unique libraries for making REST requests. But it's so simple and generic that there's hardly anything there.
The canonical example in Python is
class Some_REST_Stub( object ):
def get_some_resource( self, arg ): # This name is from the WSDL
uri = "http://host:port/path/to/resource/%s/" % arg # This path is from the WSDL
data= urllib2.open( uri )
return json.load( data )
This block of code is shorter than the WSDL required to describe it. And it seems easier to change because the name is the method name and the URI is simply a string.
In most languages the client is approximately that simple. I just wrote a bunch of REST client code in Java. It's wordier, but it's generic stuff. Build a request, parse the JSON that comes back. That's it.
A RESTful WSDL declaration buries two pieces of trivial information in a lot of XML.
It provides an "interface name" for the URI.
It documents the meaning of GET, PUT, POST and DELETE by providing Stub class method names.
It doesn't help you write much code, since there isn't much code. Note that all REST requests have the same generic HttpRequest and HttpResponse structure. The request contains generic an Entities. The response, also, contains a generic Entity that must be parsed. There's very little to REST -- the point is to be as simple as possible.
It largely eliminates the need for WSDL since everything is a generic JSONObject or XML URLEncoded and sent as a string.
"I'd much rather call a function on a class that returns my data encapsulated in an object instead of having to worry about the URL, the arguments, the response, etc."
Correct, you'll have to write a "stub" class. It has almost no code, and it will be shorter than the WSDL required to describe it.
"Multiply that out over each of the tools and now when I make a change to the service I must go update the URLs in each CRUD operation and all that goes along with that."
You can either update the stub class in each language for each client. Or you can update the WSDL and then regenerate each client. It seems like the same amount of work, since the URI is trivially encapsulated in the client code.
"I guess I imagine that with a WSDL that is the aspect that is done for you."
I'm unclear on what's "done". Translating the wordy and complex WSDL into a simple stub class? I suppose this could be helpful. Except the WSDL is bigger than the stub class. I'm suggesting that you'll probably find it easier to simply write the stub class. It's shorter than the WSDL.
"Your code simply interacts with the stubs."
Correct.
"What the stubs do, who cares? Urls, arguments, responses - if anything changes just regenerate the stubs from the WSDL."
Sadly, almost none of this requires any real programming. Generating it from WSDL is more work than simply writing it. URI's a strings. Arguments are generic JSONObjects. Responses are generic HttpResponses including a JSONArray. That's it.
"I didn't also have to update all the code that deals with the specifics of making the request and dealing with the response."
Except, there aren't any interesting specifics of making the request. HTTP is simple, generic stuff. GET, POST, PUT and DELETE are all nearly identical.

Fossill,
I recommend that you do not bother to learn SOAP for this. Ws-* has a very high learning curve and the (unnecessary) complexity will likely eat you alive.
Looking at your skill set (Java, Perl, Python, C++) you should be very satisfied with a REST (or at least HTTP-based) approach. And: you'll get results very fast.
As S.Lott said, do not worry about the code generation. You'll not need it.
For questions, I suggest you join rest-discuss on Yahoo groups:
http://tech.groups.yahoo.com/group/rest-discuss/
You usually get immediate help with all things REST there.
Personally, I have yet to see any use case that could benefit from using WS-*.
Jan

The code-generation aspect that you value is a maintenance item, but I question its real worth.
Be it thru a WSDL document or your own grammar documentation for a REST-style implementations, clients have to comply to the published interface. The WS/SOAP (code-generation) development model might have more tools, but I think that's because it's clunky and needs them.
There's no impact in the 'integrateability' of your web service. That's an issue of publishing the formal interface (in whatever form that takes), in either case. And the speed with which you move from an interface change to an implementation change is arguably faster with REST-style services. Firing up (and fighting with) WS-* code generation tools takes time.

FYI - REST does have a WSDL-like auto-generation schema definition called WADL. But almost no one uses it.

Apache CXF has Java support for REST clients that gives you the same sorts of 'code generation' advantages, in many cases, as full SOAP.