Sometimes when I post some data if connection is turned off or there is a problem I lose the data and that's not good ,I'm looking for a way to save the data in the background or something like this and it will send automatically when there is a connection and in the same time waiting I must be able to use the application, any ideas ?
Post PostContact = new Post();
json = PostContact.ContactJson(id_device1,id_device2);
response = PostContact.post("url", json);
HTTP request functions as below
Client make requests
Server reply responses
They don't really care about what happens in between. This means that once the client has done the request after that is like a black box. If the server's connection is interrupted while your request is coming the data will be corrupted, and since there is no both-ways communication the client won't know about it to re-send the request.
the best option is to create a transaction:
Client creates a unique transaction ID for the request and attach to the Rest call
Server receives the request, check the transaction log for previous calls with the same id.
if it is a new transaction, server stores this transaction ID in a log with a status 'pending'. Server then processes the request. Server updates the transaction log status to 'processed'. Server then sends the response to the client and stores response in the transaction log
If it is an old transaction, server resend response to client
Client receives response, client sends transaction completed signal to the server.
Server removes/updates transaction log to 'finished'
Failure in the 5th step, the client can resend the request with the same transaction id, without the risk of duplicate records
Related
Problem:
In my api-gateway I call other microservices using a message broker and wait for response using method, annotated using Spring #MessageHandler. When response arrives how am I suppose to connect it with the waiting http request?
My solution is to hang request to CompletableFuture, generate a unique key and save the CompletableFuture with the key to the Spring Data Key-Value (or other in-memory) database and than call other services via Message Broker and send my request key along with the data. When async response arrives to the MessageHandler, check the in-memory database for my request key that is also part of the response and complete the stage.
If there are more nodes in cluster, I'll use a topic and check every key-value instance and complete stage only on the correct node, since I don't want to use any distributed cache for that.
Is it a good solution? Or how do you handle this particular problem?
I am currently migrating our existing Spring asynchronous REST architecture to Spring's new WebFlux library and have a question around joining multiple requests so that they can listen for the same published response.
Use Case is as follows:
Client A connects to our web server and requests data
We hit our cache to check if we have the data there
We don't, so we go and retrieve this data (Client A has subscribed and waits for a response)
Client B connects to our web server and requests the same data (hits the same endpoint)
We check the cache, data is still not there
As we are already fetching this data for Client A we don't want to make another request, however, we also do not want to turn Client B away. Client B should be able to listen for the same information
How can Client B subscribe to the same response stream that Client A is waiting for?
"Client A has subscribed and waits for a response"
I suppose the request is coded as a Mono and client A sibscribes to it literally:
Subscriber<Response> clientA = ...
Mono<Response> request = makeRequest(...);
request.subscribe(clientA);
then clientB should subscribe the same way:
Subscriber<Response> clientB = ...
request.subscribe(clientB);
Moreover, the cache should contain not the previously saved response data, but the requests themselves, of type Mono<Response>. Then, if such a request is found in the cache, new clients simply subscribe to it, regardless of was that request already completed or not.
A client sends a request and catches a timeout exception. However the server is still processing the request and saving it to the database. Before that happening, the client already sent a second request which doubles the record on the database. How do I prevent that from happening? Im using java servlets and javascript.
A few suggestions:-
1) Increase the client timeout.
2) Make the server more efficient so it can respond faster.
3) Get the server to respond with an intermediate "I'm working on it" response before returning with the main response.
4) Does the server need to do all the work before it responds to the client, or can some be offloaded to a seperate process for running later?
A client sends a request and catches a timeout exception. However the server is still processing the request
Make the servlet generate some output (can be just blank spaces) and flush the stream every so often (every 15 seconds for example).
If the connection has been closed on the client side, the write will fail with a socket exception.
Before that happening, the client already sent a second request which doubles the record on the database
Use the atomicity of the database, for example, a unique key. Start the process by creating a unique record (maybe in some "unfinished" status), it will fail if the record already exists.
i am trying to create a http client based on netty. I have written the code based on the HttpSnoopClient example given in Netty site. But the problem is HttpResponse are handled by HttpSnoopClientHandler & HttpRequests are sent in HttpSnoopClient & i want to sync it. as in if i send a request i want to make sure that i will send the next request once i know the response to the previous. But since both are handled in different class, It is becoming difficult to do the same.
One thing i did was to create a setResponse() method in HttpTarget & HttpSnoopClientHandler will be setting the HttpResponse when it receives the Response from the sever. But i don't think it is a good approach since i won't be able to know the reposne was for which request.
So basically i want to do it synchronously i.e. send a request(channel.writeandFlush(req)) in HttpSnoopClient then wait till the response is received by the HttpSnoopCLientHandler & once it recieves a HTTP 1.1 200 OK then send the next request.
Can anyone tell me a good approach for doing it. Thanks in advance!
I had a similar use case where I had to block concurrent requests till one completes for a resource. I implemented a ConcurrentHashMap<RequestKey, ArrayList<ChannelHandlerContext>>> which will hold all the concurrent requests ChannelHandlerContext (ctx) and on completion of the first request raise an event which would trigger all other ctx to consume the cached response. In all this I had to make sure the AUTO_READ was set to false for fine grain control over the reads on each channel.
channelRead ->
if(map.contains(reqKey)){
map.add(reqKey, list.add(ctx))
//do nothing with AUTO_READ = false and do not initiate any ctx.* methods
}else{
//firstRequest
map.add(reqKey, new ArrayList<CTX>(){{ add(ctx);}})
//continue with request execution
//cache response and raise event on completion
}
userEventTriggered ->
onCompletionEvent {
ctxList = map.get(reqKey).clone();
map.remove(reqKey);
for(blockedCtx : ctxList){
//respond back with cached response to each blockedCtx
}
}
#norman-maurer would you give your take on this !!!
As you're creating a new HttpSnoopClientHandler for each connection, I would consider turning HttpSnoopClientHandler into a ChannelDuplexHandler. In the write method you can store a reference to the outgoing http request. When the response is received you can call your setResponse method with (channel, request, response). This should provide enough context so you can process the response correctly.
If your client is pure request/response, does not issue unrelated requests separately, and you want your application thread to process responses sequentially, then you could use a SynchronousQueue to coordinate responses with allowing the main thread to continue. Alternatively your callback can process the responses internally.
You can also extends this technique to use HTTP pipelining. HTTP pipelining guarantees that responses are returned in the order that requests are issued. In HttpSnoopClientHandler you maintain a queue of requests. As each response is returned you match it to the request at the front of the queue.
I'm having trouble establishing AsyncContexts for users and using them to push notifications to them. On page load I have some jQuery code to send the request:
$.post("TestServlet",{
action: "registerAsynchronousContext"
},function(data, textStatus, jqXHR){
alert("Server received async request"); //Placed here for debugging
}, "json");
And in "TestServlet" I have this code in the doPost method:
HttpSession userSession = request.getSession();
String userIDString = userSession.getAttribute("id").toString();
String paramAction = request.getParameter("action");
if(paramAction.equals("registerAsynchronousContext"))
{
AsyncContext userAsyncContext = request.startAsync();
HashMap<String, AsyncContext> userAsynchronousContextHashMap = (HashMap<String, AsyncContext>)getServletContext().getAttribute("userAsynchronousContextHashMap");
userAsynchronousContextHashMap.put(userIDString, userAsyncContext);
getServletContext().setAttribute("userAsynchronousContextHashMap", userAsynchronousContextHashMap);
System.out.println("Put asynchronous request in global map");
}
//userAsynchronousContextHashMap is created by a ContextListener on the start of the web-app
However, according to Opera Dragonfly (a debugging tool like Firebug), it appears that the server sends an HTTP 500 response about 30000ms after the request is sent.
Any responses created with userAsyncContext.getResponse().getWriter().print(SOME_JSON) and sent before the HTTP 500 response is not received by the browser, and I don't know why. Using the regular response object to send a response (response.print(SOME_JSON)) is received by the browser ONLY if all the code in the "if" statement dealing with AsyncContext is not present.
Can someone help me out? I have a feeling this is due to my misunderstanding of how the asynchronous API works. I thought that I would be able to store these AsyncContexts in a global map, then retrieve them and use their response objects to push things to the clients. However, it doesn't seem as if the AsyncContexts can write back to the clients.
Any help would be appreaciated.
I solved the issue. It seems as though there were several problems wrong with my approach:
In Glassfish, AsyncContext objects all have a default timeout period of 30,000 milliseconds (.5 minutes). Once this period expires, the entire response is committed back to the client, meaning you won't be able to use it again.
If you're implementing long-polling this might not be much of an issue (since you'll end up sending another request after the response anyway), but if you wish to implement streaming (sending data to back to the client without committing the response) you'll want to either increase the timeout, or get rid of it all together.
This can be accomplished with an AsyncContext's .setTimeout() method. Do note that while the spec states: "A timeout value of zero or less indicates no timeout.", Glassfish (at this time) seems to interpret 0 as being "immediate response required", and any negative number as "no timeout".
If you're implementing streaming , you must use the printwriter's .flush() method to push the data to the client after you're done using its .print() .println() or .write() methods to write the data.
On the client side, if you've streamed the data, it will trigger a readyState of 3 ("interactive", which means that the browser is in the process of receiving a response). If you are using jQuery, there is no easy way to handle readyStates of 3, so you're going to have to revert to regular Javascript in order to both send the request and handle the response if you're implementing streaming.
I have noticed that in Glassfish if you use AsyncContext and use .setTimeOut() to a negative number the connection is broken anyway, to fix this I had to go to my Glassfish admin web configurator : asadmin set
configs.config.server-config.network-config.protocols.protocol.http-listener-1.http. And set timeout to -1. All this to avoid glassfish finish the connections after 30 sec.