I'm connecting from an android device using java to a server running PHP using DefaultHttpClient.
One test I am doing is to check that the java code gracefully handles itself if the server takes to long in sending data back. If it does take to long it disconnects and retries.
Currently I have setup the connection timeout to 3 seconds by:
HttpConnectionParams.setSoTimeout(httpParameters, 3000);
On the server the PHP script is sleeping for 10 seconds:
sleep(10);
The java code works, if the script takes longer than 3 seconds then it throws a java.net.SocketTimeoutException and then retries again after a small amount of time.
The PHP script continues to run which is not what I want. I've tried testing using connection_aborted straight after the sleep function but it does not catch the client disconnect which has already happened.
ignore_user_abort(true);
sleep(10);
print "black hole";
flush();
if(connection_aborted()!=0){
// You would think this works but it does not.
}
Whats the recommended way to handle this?
I actually wrote an article on this very subject not too long ago, I will give a brief answer here, and assume it's ok to also post a related link?
Essentially, PHP will only figure out that a remote client has disconnected when it tries to use the socket that is connected to that remote socket, until you ask it to do anything with that socket it will assume that everything is fine. Here is the code I use to check for remote disconnections:
public function isAlive(){
$res = #socket_recv($this->sockHandle, $data, 1024, MSG_PEEK);
if($res === 0){
return false;
}else{
return true;
}
}
The important part here is the MSG_PEEK stops any pending messages from being cleared, and the "#" mutes errors if the socket is ok, but no messages are pending.
For the full article, it's available here:
http://www.bracketbrotherhood.com/remote-disconnections-php-non-blocking-server-sockets/programming-and-development/
Regards,
Phil,
Not sure if it's the same now, but a long time ago I ran into php not realizing the connection was aborted until it actually tried to write or flush its output buffers. See php's flush() and ob_flush(); I didn't need to actually write anything to output, just flushing empty buffers was enough to make it check.
I would imagine the webserver(s) and other stuff in front of php could affect this behavior.
You probably need to have already called ignore_user_abort(true); otherwise php will probably stop execution on the flush() call(default behavior is to stop when it realizes the connection is gone), so your if statement will never get executed.
print
"black hole";
flush();
if(connection_aborted()!=0){
// You would think this works but it does not.
}
Actually, you wouldn't think that that would work, because it would require the php script to get to that point in the script. If you wrote it at the end, then the script must run its course.
I would suggest instead:
for($i=0;$i<10;$i++){
echo ' '; ## echo "\0"; might also work
flush();
if(connection_aborted()!=0){
// You would think this works but maybe it will now.
die();
}
sleep(1); # the sleep should come after the check
}
Give it a try. Clearly you are trying to sleep for 10 seconds and then kill the script... in all cases that script will run for 10 seconds.
Related
I've created server (10.32.240.50) with SslHandler. Client (10.32.240.5) connects to server and everything works fine. After some time client disconects with no reason. I've took tcp dump and saw there ncrypted alert right before disconect:
I have no idea what client send me in this alert - it's encrypted. What could be the cause of this alert and why it leads to disconect? Is there any way to trace this events with netty?
At this stage it is difficult to see if your question is really related to programming, and hence ontopic here or not.
A TLS 1.2 alert can be many things, see https://www.rfc-editor.org/rfc/rfc5246#section-7.2 which gives you the whole list:
enum { warning(1), fatal(2), (255) } AlertLevel;
enum {
close_notify(0),
unexpected_message(10),
bad_record_mac(20),
decryption_failed_RESERVED(21),
record_overflow(22),
decompression_failure(30),
handshake_failure(40),
no_certificate_RESERVED(41),
bad_certificate(42),
unsupported_certificate(43),
certificate_revoked(44),
certificate_expired(45),
certificate_unknown(46),
illegal_parameter(47),
unknown_ca(48),
access_denied(49),
decode_error(50),
decrypt_error(51),
export_restriction_RESERVED(60),
protocol_version(70),
insufficient_security(71),
internal_error(80),
user_canceled(90),
no_renegotiation(100),
unsupported_extension(110),
(255)
} AlertDescription;
struct {
AlertLevel level;
AlertDescription description;
} Alert;
Of course it is encrypted so if you really wanted to see it, you need to:
change the client so that it outputs the master secret and client random when doing the connection that triggers this error
record the relevant connection with wireshark
and then you will be able, inside wireshark, with the items in first point, to decrypt it (you can find numerous tutorials on how to do that)
From experience, if the alert happens after some application data the most probable case is "close_notify". It is a "normal" case it just means that the server decides to shutdown the TLS socket (but not necessarily the TCP one) and hence warns (alerts) the other party about it.
If it is this case, then it is expected for the other party to send the same alert, and then the connection is shut down at the TCP level with FIN. So the chain of observations you have is expected. The only reason remaining is about the initial alert.
After clarification, since the first alert comes from .5 which is the client, and not the server, it means the client that you do not control has decided to shutdown the TLS stream, for reasons only known by it
(if we still guess correctly that the alert is "close_notify" which is still only a guess that can be tested only if you decrypt the exchange per the instructions above, or maybe increase server verbosity, like this idea given by #dave_thompson_085 in comment: "If you set sysprop javax.net.debug=ssl it will trace all JSSE (SSL/TLS) operations, which includes the received alert. ")
Other than that, except asking the client operator/developer I see no way to understand why the client decided not to talk to you anymore. It also depends on the underlying application data exchanged, maybe it was indeed the end of the transmission and the client does not need the TLS stream anymore?
I have a bot which analyses chat command and provides a small database. When I tell the bot it should write the content of the database to the irc, it gets kicked because of excess flooding. How can I prevent this?
(I'm using freenode.net)
PS: Here is my code:
for (User u : users.values()) {
ps.println("PRIVMSG " + CHANNEL + " :" + u.toString());
}
ps is a PrintStream which writes to Socket.getOutputStream().
CHANNEL is the name of my channel. users.size() is ~ 60.
Obviously a client can't change a servers thresholds for detecting flooding and disconnecting clients which do so. The whole idea of flood protection would be pointless, otherwise.
So, you must limit your output to such a level, that it will not trigger flooding kick. Add a (configurable) sleep between ps.println calls, and set the sleep duration so that you don't get kicked.
Work-around would be to use a pastebin-like service, and just send URL and summary/title text to IRC in one line. Using these services is a standard procedure for any IRC paste longer than 3 lines these days anyway. Various "pastebin" services have REST APIs (example), so this should be rather easy way to go.
This might be a simple problem, but I can't seem to find a good solution right now.
I've got:
OldApp - a Java application started from the command line (no web front here)
NewApp - a Java application with a REST api behind Apache
I want OldApp to call NewApp through its REST api and when NewApp is done, OldApp should continue.
My problem is that NewApp is doing a lot of stuff that might take a lot of time which in some cases causes a timeout in Apache, and then sends a 502 error to OldApp. The computations continue in NewApp, but OldApp does not know when NewApp is done.
One solution I thought of is fork a thread in NewApp and store some kind of ID for the API request, and return it to OldApp. Then OldApp could poll NewApp to see if the thread is done, and if so - continue. Otherwise - keep polling.
Are there any good design patterns for something like this? Am I complicating things? Any tips on how to think?
If NewApp is taking a long time, it should immediately return a 202 Accepted. The response should contain a Location header indicating where the user can go to look up the result when it's done, and an estimate of when the request will be done.
OldApp should wait until the estimate time is reached, then submit a new GET call to the location. The response from that GET will either be the expected data, or an entity with a new estimated time. OldApp can then try again at the later time, repeating until the expected data is available.
So The conversation might look like:
POST /widgets
response:
202 Accepted
Location: "http://server/v1/widgets/12345"
{
"estimatedAvailableAt": "<whenever>"
}
.
GET /widgets/12345
response:
200 OK
Location: "http://server/v1/widgets/12345"
{
"estimatedAvailableAt": "<wheneverElse>"
}
.
GET /widgets/12345
response:
200 OK
Location: "http://server/v1/widgets/12345"
{
"myProperty": "myValue",
...
}
Yes, that's exactly what people are doing with REST now. Because there no way to connect from server to client, client just polls very often. There also some improved method called "long polling", when connection between client and server has big timeout, and server send information back to connected client when it becomes available.
The question is on java and servlets ... So I would suggest looking at Servlet 3.0 asynchronous support.
Talking from a design perspective, you would need to return a 202 accepted with an Id and an URL to the job. The oldApp needs to check for the result of the operation using the URL.
The thread that you fork on the server needs to implement the Callable interface. I would also recommend using a thread pool for this. The GET url for the Job that was forked can check the Future object status and return it to the user.
I´ve been looking for a good book or article about this topic but didnt find much. I didnt find a good example - piece of code - for a specific scenario. Like clients/server conversation.
In my application´s protocol they have to send/recieve messages. Like:
Server want to send a file to a client
Client can accpet or no,
if he accepts, server will send bytes over the same connection/socket.
The rest of my application all uses blocking methods, server has a method
Heres what I did:
Server method:
public synchronized void sendFile(File file)
{
//send messsage asking if I can send a file
//block on read, waiting for client responde
//if client answers yes, start sending the bytes
//else return
}
Client methods:
public void reciveCommand()
{
//read/listen for a command from socket
//if is a send file command handleSendFileCommand();
//after the return of handleSendFileCommand() listen for another command
}
public void handleSendFileCommand()
{
//get the file server want to send
//check if it already has the file
//if it already has, then send a command to the socket saying it already has and return
//else send a command saying server can send the file
//create a FileInputStream, recive bytes and then return method
}
I am 100% sure this is wrong because, there is no way server and clients would talk bidirecional, I mean, when server wants to send a command to a server, they have to follow an order of commands until that conversation is finished, only then, they can send/recive another sequence of commands. Thats why I made all methods that send requests synchronized
It didnt took me a lot of time to realize I need to study about design patterns for that kind of application...
I read about Chain of Responsibility design pattern but I dont get it how can I use it or another good design pattern in that situation.
I hope someone can help me with some code example-like.
Thanks in advance
synchronized keyword in Java means something completely different - it marks a method or a code block as a critical section that only single thread can execute at a time. You don't need it here.
Then, a TCP connection is bi-directional on the byte-stream level. The synchronization between the server and a client is driven by the messages exchanged. Think of a client (same pretty much applies to the server) as a state machine. Some types of messages are acceptable in the current state, some are not, some switch the node into different state.
Since you are looking into design patterns, the State pattern is very applicable here.
Salesforce can send up to 100 requests inside 1 SOAP message. While sending this type of Bulk Ooutbound message request my PHP script finishes executing but SF fails to accept the ACK used to clear the message queue on the Salesforce side of things. Looking at the Outbound message log (monitoring) I see all the messages in a pending state with the Delivery Failure Reason "java.net.SocketTimeoutException: Read timed out". If my script has finished execution, why do I get this error?
I have tried these methods to increase the execution time on my server as I have no access on the Salesforce side:
set_time_limit(0); // in the script
max_execution_time = 360 ; Maximum execution time of each script, in seconds
max_input_time = 360 ; Maximum amount of time each script may spend parsing request data
memory_limit = 32M ; Maximum amount of memory a script may consume
I used the high settings just for testing.
Any thoughts as to why this is failing the ACK delivery back to Salesforce?
Here is some of the code:
This is how I accept and send the ACK file for the imcoming SOAP request
$data = 'php://input';
$content = file_get_contents($data);
if($content) {
respond('true');
} else {
respond('false');
}
The respond function
function respond($tf) {
$ACK = <<<ACK
<?xml version = "1.0" encoding = "utf-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<notifications xmlns="http://soap.sforce.com/2005/09/outbound">
<Ack>$tf</Ack>
</notifications>
</soapenv:Body>
</soapenv:Envelope>
ACK;
print trim($ACK);
}
These are in a generic script that I include into the script that uses the data for a specific workflow. I can process about 25 requests (That are in 1 SOAP response) but once I go over that I get the timeout error in the Salesforce queue. for 50 requests is usually takes my PHP script 86.77 seconds.
Could it be Apache? PHP?
I have also tested just accepting the 100 request SOAP response and just accepting and sending the ACK the queue clears out, so I know it's on my side of things.
I show no errors in the apache log, the script runs fine.
I did find some info on the Salesforce site but still no luck. Here is the link.
Also I'm using the PHP Toolkit 11 (From Salesforce).
Other forum with good SF help
Thanks for any insight into this,
--Phill
UPDATE:
If I receive the incoming message and print the response, should this happen first regardless if I do anything else after? Or does it wait for my process to finish and then print the response?
UPDATE #2:
okay I think I have the problem:
PHP uses the single thread processing approach and will not send back the ACK file until the thread has completed it's processing. Is there a way to make this a mutli thread process?
Thread #1 - accept the incoming SOAP request and send back the ACK
Thread #2 - Process the SOAP request
I know I could break it up into like a DB table or flat file, but is there a way to accomplish this without doing that?
I'm going to try to close the socket after the ACK submission and continue the processing, cross my fingers it will work.
Sounds like the outbound message is hitting the timeout. Other users have reported timeouts as low as 10 seconds (see forum link below). The sandbox instance that I use (cs1) is timing out after about 1 minute, from my testing. It's possible that the timeout is an organization or instance level setting that Salesforce controls.
Two things you could try:
Open a support ticket with
Salesforce to see if they can
increase the timeout value for
outbound messages. From my
experience, there are lot of
settings that they can modify on the
organization level - this might be
one of them.
Offload processing of your data, so
that the ACK is sent immediately
back to Salesforce. Then the actual
processing of your data will take
place asynchronously. ie. Message
queue, separate thread, etc.
Some other resources that might be helpful:
related Salesforce forum discussion
Outbound messaging documentation
I think they timeout the thing waiting for Your script to end.
There is a way You could try to fix this.
Output the envelope with ack message at the beginning and then flush the thing so that their server gets it before You end processing. No threading, just plain priorities rethinking :)
read this for best info on flushing content
Are you 100% sure that Salesforce will wait the amount of time your scripts need too run? 80 seconds seem like a loong time too me.
If all requests failed I would guess that Salesforce expects you to set the Content-Type header appropriately, but this does not seem to be the case.
I don't know about Salesforce, but if you want to make some multithreading with PHP you should take a look at this code example and more precisely to pcntl_fork().
N.B: pcntl is not enabled by default and won't work on Windows platforms.
So what I've done is:
Accept all incoming OBM's, parse them into a DB
When this is done kick of a process that runs in the background (Actually I send it to the background so the script can end)
Send ACK file back
By just accepting the raw data, parsing into fields and inserting it into a DB is fairly quick. Then I issue a Linux Command Line command that also send the processing script to run in the background. Then I send the ACK file to SF and the script ends within the allotted time. It is cumbersome to split the script process into two separate stages but it works.