I'm still quite new to microservices and have a few basic architectural questions that I can't get solved right now.
I'm using the Quarkus framework with the standard extensions like quarkus-resteasy and quarkus-rest-client for the realization.
The scenario:
I have an example of a "Persistence" service that I want to externally populate with data via a REST call in a dedicated Maven project.
#Path("/api/persistence")
#Products(MediaType.APPLICATION_JSON)
public class Persistence{
#Inject
EntityManager entityManager;
#POST
#Transactional
public Response create(PostDto postDto) {
Post post = toPostMapper.toResource(postDto);
entityManager.persist(post);
return Response.ok(postDto).status(201).build();
}
}
At the same time I would like to have a microservice DataGenerator which generates the corresponding data and passes it to the Persistence Service.
My problem : API sharing
Both services were created as Maven projects.
According to the tutorials I found the correct way would be to declare an interface (here called PersistenceApi) in the DataGenerator project like this
#Path("/api/persistence")
#Products(MediaType.APPLICATION_JSON)
#RegisterRestClient
public interface PersistenceApi {
#POST
#Transactional
public Response create(PostDto post) ;
}
This interface is then integrated into the DataGenerator service via #Inject, which leads to the following exemplary service.
#RequestScoped
#Path("/api/datagenerator")
#Products("application/json")
#Consumes("application/json")
public class DataGenerator{
#Inject
#RestClient
PersistenceApi persistenceApi
#POST
public void getPostExamplePostToPersistence() {
PostDto post = new PostDto();
post.setTitle("Find me in db in persistence-service")
persistenceApi.create(post);
}
}
I have the PersistenceService running locally on port 8181 and have added the following entry in the application.properties of the DataGenerator project so that the service can be found.
furnace.collection.item.service.PersistenceApi/mp-rest/url=http://localhost:8181
furnace.collection.item.service.PersistenceApi/mp-rest/scope=javax.inject.Singleton
I find it "wrong" to declare the interface in my DataGenerator, because at this point I don't notice when the api provided by the Persistence service changes. Accordingly one could come up with the idea to position the interface in the Persistence service, which is then implemented by my concrete Persistence implementation and leads to the following code.
#Path("/api/persistence")
#Products(MediaType.APPLICATION_JSON)
#RegisterRestClient
public class PersistenceApiImpl implements PersistenceApi {
#Inject
EntityManager entityManager;
#POST
#Transactional
public Response create(PostDto fruit) {
Post post = toPostMapper.toResource(fruit);
entityManager.persist(post);
return Response.ok(fruit).status(201).build();
}
}
In order to use them in my DataGenerator project, I would have to include the Persistence project as a dependency in my DataGenerator project, which sounds like a "monolith with extra steps" to me and therefore feels wrong in terms of "separation of concerns".
I have tried the following approach:
I created another Maven project called PersistenceApi which only contains the corresponding PersistenceApi. This PersistenceApi project was then included as a dependency in both the "Persistence" and "DataGenerator" projects. In the "Persistence"-Project I implement the service from the example above and try to address the corresponding interface in the "DataGenerator"-Project via #Inject.
Unfortunately this does not work. When I'm building the service, I get the message that the required dependency PersistenceApi, which I want to include via #Inject in the DataGenerator service, cannot be injected in the form of an UnsatisfiedResolutionException.
Now my questions:
I don't see what I'm missing here. Could you help me?
Is this kind of API-sharing with dedicated Api projects a viable way or is the "monolith with extra steps" approach really the way to go?
Thank you in advance.
Thats a common problem with microservices. Like in the book "Microservices: Grundlagen flexibler Softwarearchitekturen" by Eberhard Wolff (I saw that you are German too) i follow the idea that microservices should have the same coupling like the teams developing them and like the organization your developing it for(have a look at Conway's law). Therefore services of mostly independent teams should be developed independly and the api changes of one service should not affect another at the time of the update.
If you develop both services in your team then i think you can couple them the way you are doing it because you dont have to work together with other teams and there will be no huge overhead. Note that you will be forced to release both services together. If that is always ok for you then save your time and do it your way, if not have a look at API-Versioning:
I use api versioning so the old api is still reachable under "v1/" and the new one under "v2/". This way the team behind the other microservice has enough time to update their service.
Have a look at Domain-driven Design for different ways of integrating bounded contexts (=services) and the coupling consequences. Without API-Versioning you are forced to a partnership and you need to release together. Maybe you prefer Customer-Supplier or even conformist.
To test compatibility between both services have a look at consumer driven contracts and Pact. You can also generate open api files and track their changes but that will only help to notify people about changes.
We have several legacy java-services with RMI-api, implemented by the old JRMP approach requiring 'rmic' pre-compilation.
As part of migrating everything to latest JDK,
I am also trying to rewrite the RMI stuff to the more current approach, where the implementation-classes extend from UnicastRemoteObject, thus getting rid of the rmic pre-compilation step.
Following a simple example, like here:
https://www.mkyong.com/java/java-rmi-hello-world-example/
but I have not been able to find such example with commit/rollback transaction-logic.
In the current legacy-code, all transaction-logic is handled in a single, common method invokeObject() in the JRMP container-code that will intercept all RMI api-calls in one place,
and will simply commit if the RMI-call is successful, or rollback if an Exception was thrown.
I haven't been able to figure out how to do this in the new approach with no JRMP container present.
Obviously, I do not want to code commit/rollback-logic into every single api-method (there are many dozens of them),
but still keep that uniform logic in one place.
Any advice, hints, references, etc, how to intercept all RMI-calls in a single point to implement the transaction-logic?
To begin with, I agree with #df778899 the solution he provides is a robust solution. Although,I will give you an alternative choice if you don't want to work with spring framework and you want to dig it further.
Interceptors provide a powerful and flexible design include invocation monitoring, logging, and message routing. In some sense, the key value you are looking is a kind of RMI-level AOP (aspect-oriented programming) support
Generally speaking:
it is not fair to ask RMI to directly support such capabilities as it
is only a basic remote method invocation primitive, while CORBA ORB is
at a layer close to what the J2EE EJB (Enterprise JavaBean) container
offers. In the CORBA specification, service context is directly
supported at the IIOP level (GIOP, or General Inter-Orb Protocol) and
integrated with the ORB runtime. However, for RMI/IIOP, it is not easy
for applications to utilize the underlying IIOP service-context
support, even when the protocol layer does have such support. At the
same time, such support is not available when RMI/JRMP (Java Remote
Method Protocol) is used. As a result, for RMI-based distributed
applications that do not use, or do not have to use, an ORB or EJB
container environment, the lack of such capabilities limits the
available design choices, especially when existing applications must
be extended to support new infrastructure-level functions. Modifying
existing RMI interfaces often proves undesirable due to the
dependencies between components and the huge impact to client-side
applications. The observation of this RMI limitation leads to the
generic solution that I describe in this article
Despite all the above
The solution is based on Java reflection techniques and some common
methods for implementing interceptors. More importantly, it defines an
architecture that can be easily integrated into any RMI-based
distributed application design.
The solution below is demonstrated through an example implementation that supports the transparent passing of transaction-context data, such as a transaction ID (xid), with RMI. The solution contains the following three important components:
RMI remote interface naming-function encapsulation and interceptor plug-in
Service-context propagation mechanism and server-side interface support
Service-context data structure and transaction-context propagation support
The implementation assumes that RMI/IIOP is used. However, it by no
means implies that this solution is only for RMI/IIOP. In fact, either
RMI/JRMP or RMI/IIOP can be used as the underlying RMI environments,
or even a hybrid of the two environments if the naming service
supports both
Naming-function encapsulation
First we encapsulate the naming function that provides the RMI remote
interface lookup, allowing interceptors to be transparently plugged
in. Such an encapsulation is always desirable and can always be found
in most RMI-based applications. The underlying naming resolution
mechanism is not a concern here; it can be anything that supports JNDI
(Java Naming and Directory Interface). In this example, to make the
code more illustrative, we assume all server-side remote RMI
interfaces inherit from a mark remote interface ServiceInterface,
which itself inherits from the Java RMI Remote interface. Figure
shows the class diagram, which is followed by code snippets that I
will describe further
RMI invocation interceptor
To enable the invocation interceptor, the original RMI stub reference
acquired from the RMI naming service must be wrapped by a local proxy.
To provide a generic implementation, such a proxy is realized using a
Java dynamic proxy API. In the runtime, a proxy instance is created;
it implements the same ServiceInterface RMI interface as the wrapped
stub reference. Any invocation will be delegated to the stub
eventually after first being processed by the interceptor. A simple
implementation of an RMI interceptor factory follows the class diagram
shown in Figure
package rmicontext.interceptor;
public interface ServiceInterfaceInterceptorFactoryInterface {
ServiceInterface newInterceptor(ServiceInterface serviceStub, Class serviceInterfaceClass) throws Exception;
}
package rmicontext.interceptor;
public class ServiceInterfaceInterceptorFactory
implements ServiceInterfaceInterceptorFactoryInterface {
public ServiceInterface newInterceptor(ServiceInterface serviceStub, Class serviceInterfaceClass)
throws Exception {
ServiceInterface interceptor = (ServiceInterface)
Proxy.newProxyInstance(serviceInterfaceClass.getClassLoader(),
new Class[]{serviceInterfaceClass},
new ServiceContextPropagationInterceptor(serviceStub)); // ClassCastException
return interceptor;
}
}
package rmicontext.interceptor;
public class ServiceContextPropagationInterceptor
implements InvocationHandler {
/**
* The delegation stub reference of the original service interface.
*/
private ServiceInterface serviceStub;
/**
* The delegation stub reference of the service interceptor remote interface.
*/
private ServiceInterceptorRemoteInterface interceptorRemote;
/**
* Constructor.
*
* #param serviceStub The delegation target RMI reference
* #throws ClassCastException as a specified uncaught exception
*/
public ServiceContextPropagationInterceptor(ServiceInterface serviceStub)
throws ClassCastException {
this.serviceStub = serviceStub;
interceptorRemote = (ServiceInterceptorRemoteInterface)
PortableRemoteObject.narrow(serviceStub, ServiceInterceptorRemoteInterface.class);
}
public Object invoke(Object proxy, Method m, Object[] args)
throws Throwable {
// Skip it for now ...
}
}
To complete the ServiceManager class, a simple interface proxy cache is implemented:
package rmicontext.service;
public class ServiceManager
implements ServiceManagerInterface {
/**
* The interceptor stub reference cache.
* <br><br>
* The key is the specific serviceInterface sub-class and the value is the interceptor stub reference.
*/
private transient HashMap serviceInterfaceInterceptorMap = new HashMap();
/**
* Gets a reference to a service interface.
*
* #param serviceInterfaceClassName The full class name of the requested interface
* #return selected service interface
*/
public ServiceInterface getServiceInterface(String serviceInterfaceClassName) {
// The actual naming lookup is skipped here.
ServiceInterface serviceInterface = ...;
synchronized (serviceInterfaceInterceptorMap) {
if (serviceInterfaceInterceptorMap.containsKey(serviceInterfaceClassName)) {
WeakReference ref = (WeakReference) serviceInterfaceInterceptorMap.get(serviceInterfaceClassName);
if (ref.get() != null) {
return (ServiceInterface) ref.get();
}
}
try {
Class serviceInterfaceClass = Class.forName(serviceInterfaceClassName);
ServiceInterface serviceStub =
(ServiceInterface) PortableRemoteObject.narrow(serviceInterface, serviceInterfaceClass);
ServiceInterfaceInterceptorFactoryInterface factory = ServiceInterfaceInterceptorFactory.getInstance();
ServiceInterface serviceInterceptor =
factory.newInterceptor(serviceStub, serviceInterfaceClass);
WeakReference ref = new WeakReference(serviceInterceptor);
serviceInterfaceInterceptorMap.put(serviceInterfaceClassName, ref);
return serviceInterceptor;
} catch (Exception ex) {
return serviceInterface; // no interceptor
}
}
}
}
You can find more details in the following guideline here. It is important to understand all this above if you desire to make it from scratch, in contrast with Spring-AOP Framework robust solution.In addition,I think you will find very interesting the Spring Framework plain source code for implementing an RmiClientInterceptor. Take another look here also.
You might consider using Spring with RMI. It offers a simplified way to use transactions. You just set some persistence settings from the #EnableTransactionManagement annotation and it manages transactions for you at a point defined by #Transactional. It can be a class or methods of a class.
example code here
The explanation is in the project README.
I don't know if you've already considered this, one possibility that would fit well with the dual requirement of RMI and transactional boundaries is clearly Spring. An example of Spring Remoting is here.
The #Transactional annotation is very widely used for declarative transaction management - Spring will automatically wrap your bean in an AOP transaction advisor.
These two would then fit together well, for example with a single transactional bean that could be exported as a remote service.
The Spring Remoting link above is based around Spring Boot, which is an easy way to get started. By default Boot would want to bring in an embedded server to the environment, though this can be disabled. Equally there are other options like a standalone AnnotationConfigApplicationContext, or a WebXmlApplicationContext within an existing web application.
Naturally with a question asking for recommendations, there will an element of opinion in any reply - I would be disappointed not to see some other suggestions shortly.
I've been using Spring annotations such as #RestController and #RequestMapping to generate simple services in a Spring Boot Web application.
So I have this trivial example working correctly:
#RestController
public class HelloController {
#RequestMapping("/")
public String sayIt() {
return "Hello!";
}
}
Now, I would like to separate out an API library (jar) with only the REST interface and the DTOs. One or more separate libraries would provide the actual implementations of this interface. I can then use the (lightweight) API library on the client-side to generate REST client proxies to talk to any of the implementations.
So... are there any annotations or configuration to mark REST interfaces vs. implementations separately? If not, what is the Spring-y way to achieve this instead of using JAX-RS annotations?
#Something1
public class HelloServiceApi {
#RequestMapping("/")
public String sayIt();
}
#Something2
public class HelloServiceImpl implements HelloServiceApi {
public String sayIt() {
return "Hello!";
}
}
I would advise to have a jar that contains the DTO objects only, without any logic. It then can be used by both the REST server and the client to transfer objects.
The client should not be dependent on the REST war/jar or the logic.
Furthermore, I would try to make sure my controller doesn't hold any logic aside from, maybe, transferring the DTOs into domain model objects which will then be passed to the business logic layer.
In my opinion, the REST layer should be only responsible for the external API, argument handling, sending to a layer bellow (service layer) and preparing a response.
That being said, you should have your different implementations at the service layer. This allows having the API/REST layer untouched and constant.
The service layer (being providing in different implementations) should respect some common interface that is later injected in the above rest layer.
Have I responded to your question?
I have a collection of stateless scala/Java APIs, which might look like:
class UserService {
def get(id: UserIDType): User { ... }
def update( user:User): User { ... }
...
}
where User has a set of inspectable bean properties. I'd like to make these same APIs not only available over HTTP (which I know how to do), but also other, more performant non-HTTP protocols (ideally also running in the same process as the HTTP server). And more importantly, automate as much as possible including generation of client APIs that match the original Java API, and the dispatching of method calls following network requests.
I've found Spring's guide on remoting
However, this looks limited to HTTP (text not binary). What I'd really love is a library/other method that:
lets me scan for registered/annotated services and describes methods and parameters as data
lets me easily dispatch method calls (so that I'm in control of the sockets, communication protocols and whatnot and can chose a more performant one than HTTP).
i.e. the kinds of things that Spring's DispatcherServlet does internally, but without the HTTP limitations.
Ideas?
Background
I'd like to make a set of stateless service API calls available as network services with the following goals:
Some of the APIs should be available as REST calls from web pages/mobile clients
A superset of these APIs should be available internally within our company (e.g. from C++ libraries, python libraries) in a way that is as high-performance (read low-latency) as possible, particularly when the service and client are:
co-located in the same process, or
co-located on the same machine.
automate wrapper code for the client and server. If I add a method a service API, or and an attribute to a class, no-one should have to write additional code in client or server. (e.g. I hit a button and a new client API that matches the original Java/scala service is auto-generated).
(1) is easily achievable with Spring MVC. Without duplication of classes I can simply markup the service: (i.e. service is also a controller)
#Controller
#Service
#RequestMapping(...)
class UserService {
#RequestMapping(...)
def get(#PathVariable("id") id: UserIDType): User { ... }
#RequestMapping(...)
def update( #RequestBody user:User): User { ... }
...
}
With tools like Swagger (3) is easily achievable (I can trivially read the Swagger-generated JSON spec and can auto-generate C++ or Python code that mimics the original service class, it's methods, parameter names and the POJO parameter types).
However, HTTP is not particularly performant, so I'd like to use a different (binary) protocol here. Trivially, I can use the this same spec to generate a thrift .idl or even directly generate client code to talk Thrift/ProtocolBuffers (or any other binary protocol) while also providing to the client an API that looks just like the original Java/scala one.
Really -- the only tricky part is getting a description of the service method and then actually dispatching a method calls. So -- are there libraries to help me do that?
Notes:
I'm aware that mixing service and controller annotations disgusts some Spring MVC purists. However, my goal really is to have a logical API, and automate all of the service embodiments of it (i.e. the controller part is redundant boilerplate that should be auto-generated). I'd REALLY rather not spend my life writing boilerplate code.
I am trying to create a dynamic proxy that would wrap an EJB around a web service because the application server does not support creating an EJB based web service without a proprietary router project generation.
My thought was to create a dynamic proxy, and some how just start it using an InitServlet. Right now I am kind of stuck on figuring out how to set the annotations dynamically so that I won't get the following error.
class $Proxy0 has neither #WebSerivce nor #WebServiceProvider annotation
at com.sun.xml.internal.ws.server.EndpointFactory.verifyImplementorClass(EndpointFactory.java:277)
at com.sun.xml.internal.ws.transport.http.server.EndpointImpl.getPrimaryWsdl(EndpointImpl.java:273)
at com.sun.xml.internal.ws.transport.http.server.EndpointImpl.createEndpoint(EndpointImpl.java:213)
at com.sun.xml.internal.ws.transport.http.server.EndpointImpl.publish(EndpointImpl.java:143)
Recently I have had the same problem. It seems that most people say is not possible. See http://softwarecarnival.blogspot.be/2009/02/java-annotations-and-proxies.html
If the interface that you have is:
interface XXXInterface{
Result doStuff1(String param1)
}
then a workaround is to create a delegator to the proxy that will also implement the web service.
#WebService
public class WebServiceDelegateToXXXServer implements XXXInterface{
public WebServiceDelegateToXXXServer(XXXInterface actualImplementor){
this.actualImplementor = actualImplementor;
}
public Result doStuff1(String param1){
return actualImplementor.doStuff1(param1);
}
}
Then you will publish
XXXInterface proxy = createProxyAsXXXInterface();
Endpoint.publish(url, new WebServiceDelegateToXXXServer(proxy));