We Keep Coding

Error in backend of REST API: "INFO: The connection was broken. It was probably closed by the client. Reason: Closed"

So I am trying out a simple full stack project of my own that involves a java backend implementation of a REST API, for which I am using the org.restlet.com framework/package and jetty as the server.
Whilst I was testing my API using Postman I noticed something wierd: Every time I started the server only the first POST/PUT/DELETE HTTP Request would get an answer, while the next ones would not receive one and on the console this error message would appear:
/* Timestamp-not-important */ org.restlet.engine.adapter.ServerAdapter commit
INFO: The connection was broken. It was probably closed by the client.
Reason: Closed
The GET HTTP Requests however do not share that problem.
I said "Fair enough, probably it's postman's fault".. after all the request made it to the server and their effects were applied. However, now that I am building the front-end this problem blocks the server's response: instead of a JSON object I get an undefined (edit: actually I get 204 No Content) on the front-end and the same "INFO" on the back-end for every POST/PUT/DELETE after the first one.
I have no idea what it is or what I am doing wrong. It has to be the backend's problem, right? But what should I look for?

Nevermind, it was the stupidest thing ever. I tried to be "smart" about returning the same Representation object (with only a 'success' JSON field) on multiple occasions by making one instance on a static final field of a class. Turns out a new instance must be returned each time.

How to continue on client when heavy server computation is done

This might be a simple problem, but I can't seem to find a good solution right now.
I've got:
OldApp - a Java application started from the command line (no web front here)
NewApp - a Java application with a REST api behind Apache
I want OldApp to call NewApp through its REST api and when NewApp is done, OldApp should continue.
My problem is that NewApp is doing a lot of stuff that might take a lot of time which in some cases causes a timeout in Apache, and then sends a 502 error to OldApp. The computations continue in NewApp, but OldApp does not know when NewApp is done.
One solution I thought of is fork a thread in NewApp and store some kind of ID for the API request, and return it to OldApp. Then OldApp could poll NewApp to see if the thread is done, and if so - continue. Otherwise - keep polling.
Are there any good design patterns for something like this? Am I complicating things? Any tips on how to think?

If NewApp is taking a long time, it should immediately return a 202 Accepted. The response should contain a Location header indicating where the user can go to look up the result when it's done, and an estimate of when the request will be done.
OldApp should wait until the estimate time is reached, then submit a new GET call to the location. The response from that GET will either be the expected data, or an entity with a new estimated time. OldApp can then try again at the later time, repeating until the expected data is available.
So The conversation might look like:
POST /widgets
response:
202 Accepted
Location: "http://server/v1/widgets/12345"
{
"estimatedAvailableAt": "<whenever>"
}
.
GET /widgets/12345
response:
200 OK
Location: "http://server/v1/widgets/12345"
{
"estimatedAvailableAt": "<wheneverElse>"
}
.
GET /widgets/12345
response:
200 OK
Location: "http://server/v1/widgets/12345"
{
"myProperty": "myValue",
...
}

Yes, that's exactly what people are doing with REST now. Because there no way to connect from server to client, client just polls very often. There also some improved method called "long polling", when connection between client and server has big timeout, and server send information back to connected client when it becomes available.

The question is on java and servlets ... So I would suggest looking at Servlet 3.0 asynchronous support.
Talking from a design perspective, you would need to return a 202 accepted with an Id and an URL to the job. The oldApp needs to check for the result of the operation using the URL.
The thread that you fork on the server needs to implement the Callable interface. I would also recommend using a thread pool for this. The GET url for the Job that was forked can check the Future object status and return it to the user.

What causes "java.io.IOException: stream was reset: CANCEL" with okhttp and spdy?

I'm experimenting with OKHttp (version 2.0.0-RC2) and SPDY and seeing IOException: stream was reset: CANCEL quite a lot, maybe 10% or more of all requests in some preliminary testing. When using Apache HttpClient and regular https we were not seeing any equivalent issue as far as I'm aware. I'm pretty sure we also don't see anything equivalent with OkHttp when SPDY is disabled (client.setProtocols(ImmutableList.of(Protocol.HTTP_1_1))) but I haven't done enough testing to be 100% confident.
This previous question sees these exceptions among others and the advice there is to ignore them, but this seems crazy: we get an exception while reading data from the server, so we abort the data processing code (which using Jackson). We need to do something in such cases. We could retry the request, of course, but sometimes it's a POST request which is not retry-able, and if we've already started receiving data from the server then it's a good bet that the server as already taken the requested action.
Ideally there is some configuration of the client and/or the server that we can do in order to reduce the incidence of these exceptions, but I don't understand SPDY well enough to know even where to start looking or to advise our server-admin team to start looking.
Stack trace, in case it's helpful:
java.io.IOException: stream was reset: CANCEL
at com.squareup.okhttp.internal.spdy.SpdyStream$SpdyDataSource.checkNotClosed(SpdyStream.java:442)
at com.squareup.okhttp.internal.spdy.SpdyStream$SpdyDataSource.read(SpdyStream.java:344)
at com.squareup.okhttp.internal.http.SpdyTransport$SpdySource.read(SpdyTransport.java:273)
at okio.RealBufferedSource.exhausted(RealBufferedSource.java:60)
at okio.InflaterSource.refill(InflaterSource.java:96)
at okio.InflaterSource.read(InflaterSource.java:62)
at okio.GzipSource.read(GzipSource.java:80)
at okio.RealBufferedSource$1.read(RealBufferedSource.java:227)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.loadMore(UTF8StreamJsonParser.java:174)
at com.fasterxml.jackson.core.base.ParserBase.loadMoreGuaranteed(ParserBase.java:431)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString2(UTF8StreamJsonParser.java:2111)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser._finishString(UTF8StreamJsonParser.java:2092)
at com.fasterxml.jackson.core.json.UTF8StreamJsonParser.getText(UTF8StreamJsonParser.java:275)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:205)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeArray(JsonNodeDeserializer.java:230)
at com.fasterxml.jackson.databind.deser.std.BaseNodeDeserializer.deserializeObject(JsonNodeDeserializer.java:202)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:58)
at com.fasterxml.jackson.databind.deser.std.JsonNodeDeserializer.deserialize(JsonNodeDeserializer.java:15)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:2765)
at com.fasterxml.jackson.databind.ObjectMapper.readTree(ObjectMapper.java:1546)
at com.fasterxml.jackson.core.JsonParser.readValueAsTree(JsonParser.java:1363)
at (application-level code...)

Your best bet is to set a breakpoint in the two places where the CANCEL error code is assigned: that's SpdyStream#closeInternal (line 246) and SpdyStream#receiveRstStream (line 304). If you can put a breakpoint here, you can capture who is canceling your stream and that'll shed light on the problem.
If for whatever reason you cannot attach a debugger, you can instrument the code to print a stacktrace when those lines are reached:
new Exception("SETTING ERROR CODE TO " + errorCode).printStackTrace();
In either case, I'm the author of that code and I'd love to help you resolve this problem.

Had the same problem and this was a result of network connection timeout, this was a result of downloading a large file from the web service
i had my timeout set to 2-min so i changed it to 5-min and it solved my problem
val okkHttpclient = OkHttpClient.Builder()
.connectTimeout(5, TimeUnit.MINUTES)
.writeTimeout(5, TimeUnit.MINUTES) // write timeout
.readTimeout(5, TimeUnit.MINUTES) // read timeout
.addInterceptor(networkConnectionInterceptor)
.build()

We had this issue because of broken http headers. The android Base64 encoder by default adds newlines which broke our Authorization headers.

Tomcat request logging - how to?

Env: Tomcat 7.
Would like to log http requests and their headers. Actually I could do without the headers as long as I can log the IP address of the caller, the resource he's requesting (the URL) and the type of request (GET, POST, etc)
This may seem like a trivial question, but it really isn't.
The standard way would be to use the AccessLogValve, but as far as I understand that one is actually not request logging, it is request/response logging, meaning that it will not log anything before at the end of the response cycle. It will only log those requests where a response has successfully been delivered to the http client. If something goes wrong before that AccessLogValve will not log the request.
Question 1: Is this correctly understood?
Question 2: Are there other options?
UPDATE 1:
I've done a test with Tomcat7 using a dummy-servlet that does blocking for x seconds based on an URL parameter. My findings are that indeed that request gets logged by the AccessLogValve ... although as expected this does not happen until the end of the response, i.e. after the x seconds. There will be a log entry regardless if the client has aborted before the request finishes and regardless if the servlet throws an exception during processing.
Therefore the answer to question 1 is : "No".
Conclusion
AcccesLogValve will eventually produce a log entry. At least I haven't been able to produce a scenario where this is not the case.

All the access logs that I have seen are written after the request/response has been processed because it is useful to log info like the size of the response or the total processing time.
"If something goes wrong before that AccessLogValve will not log the request.
Question 1: Is this correctly understood?"
No, not based on my experience. The request/response is always logged, even if there is an error processing it. In that case the HTTP status code field (%s in the log pattern) will contain an error code, like 500.

Salesforce/PHP - Bulk Outbound message (SOAP), Time out issue - See update #2

Salesforce can send up to 100 requests inside 1 SOAP message. While sending this type of Bulk Ooutbound message request my PHP script finishes executing but SF fails to accept the ACK used to clear the message queue on the Salesforce side of things. Looking at the Outbound message log (monitoring) I see all the messages in a pending state with the Delivery Failure Reason "java.net.SocketTimeoutException: Read timed out". If my script has finished execution, why do I get this error?
I have tried these methods to increase the execution time on my server as I have no access on the Salesforce side:
set_time_limit(0); // in the script
max_execution_time = 360 ; Maximum execution time of each script, in seconds
max_input_time = 360 ; Maximum amount of time each script may spend parsing request data
memory_limit = 32M ; Maximum amount of memory a script may consume
I used the high settings just for testing.
Any thoughts as to why this is failing the ACK delivery back to Salesforce?
Here is some of the code:
This is how I accept and send the ACK file for the imcoming SOAP request
$data = 'php://input';
$content = file_get_contents($data);
if($content) {
respond('true');
} else {
respond('false');
}
The respond function
function respond($tf) {
$ACK = <<<ACK
<?xml version = "1.0" encoding = "utf-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<notifications xmlns="http://soap.sforce.com/2005/09/outbound">
<Ack>$tf</Ack>
</notifications>
</soapenv:Body>
</soapenv:Envelope>
ACK;
print trim($ACK);
}
These are in a generic script that I include into the script that uses the data for a specific workflow. I can process about 25 requests (That are in 1 SOAP response) but once I go over that I get the timeout error in the Salesforce queue. for 50 requests is usually takes my PHP script 86.77 seconds.
Could it be Apache? PHP?
I have also tested just accepting the 100 request SOAP response and just accepting and sending the ACK the queue clears out, so I know it's on my side of things.
I show no errors in the apache log, the script runs fine.
I did find some info on the Salesforce site but still no luck. Here is the link.
Also I'm using the PHP Toolkit 11 (From Salesforce).
Other forum with good SF help
Thanks for any insight into this,
--Phill
UPDATE:
If I receive the incoming message and print the response, should this happen first regardless if I do anything else after? Or does it wait for my process to finish and then print the response?
UPDATE #2:
okay I think I have the problem:
PHP uses the single thread processing approach and will not send back the ACK file until the thread has completed it's processing. Is there a way to make this a mutli thread process?
Thread #1 - accept the incoming SOAP request and send back the ACK
Thread #2 - Process the SOAP request
I know I could break it up into like a DB table or flat file, but is there a way to accomplish this without doing that?
I'm going to try to close the socket after the ACK submission and continue the processing, cross my fingers it will work.

Sounds like the outbound message is hitting the timeout. Other users have reported timeouts as low as 10 seconds (see forum link below). The sandbox instance that I use (cs1) is timing out after about 1 minute, from my testing. It's possible that the timeout is an organization or instance level setting that Salesforce controls.
Two things you could try:
Open a support ticket with
Salesforce to see if they can
increase the timeout value for
outbound messages. From my
experience, there are lot of
settings that they can modify on the
organization level - this might be
one of them.
Offload processing of your data, so
that the ACK is sent immediately
back to Salesforce. Then the actual
processing of your data will take
place asynchronously. ie. Message
queue, separate thread, etc.
Some other resources that might be helpful:
related Salesforce forum discussion
Outbound messaging documentation

I think they timeout the thing waiting for Your script to end.
There is a way You could try to fix this.
Output the envelope with ack message at the beginning and then flush the thing so that their server gets it before You end processing. No threading, just plain priorities rethinking :)
read this for best info on flushing content

Are you 100% sure that Salesforce will wait the amount of time your scripts need too run? 80 seconds seem like a loong time too me.
If all requests failed I would guess that Salesforce expects you to set the Content-Type header appropriately, but this does not seem to be the case.

I don't know about Salesforce, but if you want to make some multithreading with PHP you should take a look at this code example and more precisely to pcntl_fork().
N.B: pcntl is not enabled by default and won't work on Windows platforms.

So what I've done is:
Accept all incoming OBM's, parse them into a DB
When this is done kick of a process that runs in the background (Actually I send it to the background so the script can end)
Send ACK file back
By just accepting the raw data, parsing into fields and inserting it into a DB is fairly quick. Then I issue a Linux Command Line command that also send the processing script to run in the background. Then I send the ACK file to SF and the script ends within the allotted time. It is cumbersome to split the script process into two separate stages but it works.