We Keep Coding

Tomcat max connections on a per context basis

I have multiple web applications running under a single Tomcat container. Since they all run under a single Tomcat connector (as defined in the server.xml file), attributes such as maxConnections and maxThreads govern the container as a whole. As a result it is possible for a single application to consume all available Tomcat threads, starving the other applications of threads and making them unresponsive. I would like to be able to define the maximum http threads on a per context basis so that this is no longer possible.
Here's what I've tried so far:
Create a custom filter in the application that keeps track of the current thread count and limits additional connections. (Got the filter here: How to set limit to the number of concurrent request in servlet?). I'm not sure I like this solution, as it isn't as full-featured (support for attributes such as acceptCount, maxConnections, maxThreads, and minSpareThreads) as Tomcat provides by default to the container; and adding in the features feels like I am attempting to build what already exists in Tomcat.
Create a separate Tomcat connector in the server.xml file for each context. This has a few issues. For one, each connector requires a separate port; this means I'll have to account for this in my apache config. Secondly, I plan to add more webapps regularly; this means a config change followed by a tomcat restart, which is disruptive to clients.
Has anyone else encountered something like this? I feel like there should be a "Tomcat supported" workflow to accomplish what I'm after.

I'm going to post an answer that was provided to me from the Tomcat user group: http://tomcat.apache.org/tomcat-9.0-doc/config/valve.html#Semaphore_Valve (The Semaphore Valve is not Tomcat 9 specific, but was actually introduced in Tomcat 6). I experimented with this concept, and I found the following practical applications:
(Untested) The Semaphore Valve should be able to be nested within the Host element in the server.xml file.
(Tested) A [context-name].xml file can be placed inside [tomcat-home]/conf/Catalina/localhost with the valve nested within the Context element.
This is not necessarily the solution that I am going with, as more testing will need to be performed. However, I thought I'd add this as it is a potential answer to the problem.
Update:
As a recap, the SemaphoreValve was an option that was recommended to me through the Tomcat user mailing list as a solution to the issue that I described above. It turns out it was easier to implement than I anticipated. Adding the following to context.xml in the Tomcat/conf directory did the trick:
<Valve className="org.apache.catalina.valves.SemaphoreValve"
concurrency="10"
fairness="true" />
Thanks to Mark Thomas from the Apache group for supplying the solution.

Elastic Beanstalk host specific application configuration

I have a java web application I'm trying to re-factor to work with the elastic beanstalk way of doing things. The application will be load balanced and have (for the moment) 2 hosts without taking any advantage of auto-scaling. The issue is that there are slight configuration differences between the nodes, in particular authenticating to certain web-services is done with different credentials to effectively double throughput as there are per account throttling restrictions.
Currently my application treats configuration separately from the archive so its relatively simple on fixed hosts where the configuration remains in a relatively static file path and deployment of the war files is all that is required.
Going down the elastic beanstalk path I think I'll have to include all the configuration options inside the deployable artifact and some how get the application to load up the relevant host specific configuration. The problem I have is deciding which configuration to load inside the application. I could use a physical aspect about the host, i.e. an IP address or Instance ID that would effectively load the relevant config;
/config-<InstanceID-1>.properties
/config-<InstanceID-2>.properties
This approach is totally flawed given that if I create an entirely new environment in beanstalk, it would require me to update all the configuration files in the project to reflect the new Instance-id's created.
Has anyone come up with a good way of doing this in beanstalk?

If you have to have two different types of nodes, then you should consider SOA architecture for your application.
Create two environments, environment-a and environment-b. Either set all properties for the environments through AWS web console, or can reuse your existing configuration files and just set the specific configuration file name for each environment.
#environment-a
PARAM1 = config-environment-a.properties
#environment-b
PARAM1 = config-environment-b.properties
You share the same code base and push to either environment with -e modifier.
#push to environment-a
$ git aws.push -e environment-a
#push to environment-b
$ git aws.push -e environment-b
You can also create git alias to push to both environments at the same time :-)
Now, the major benefit of SOA approach is that you can scale and manage those environments separately. It is simple and elegant.
If you want more complex and less elegant, use simple token distribution service. On every environment initialization, send two messages to Amazon SQS. Each message should contain configuration name. Then pull those messages from SQS, each instance will get exactly one from the queue. Whichever configuration name the message contains, configure your node with that configuration. :-)
Hope it helps.
Update after #vcetinick comment:
All still seems rather complex for what should be pretty simple.
That's why I suggested separate environments. You can make your own registration service, when the node comes up, it registers with the service and in return gets configuration params. You keep available configurations in persistent DB. If the node dies and the service gets another registration request, the registration service can quickly check registered all nodes (because they all left their info during the registration), and if any of the nodes is not responding, its configuration data is reassigned to the new node. And now you have single point of failure on your hands :-)
Again, there might be other ways to approach that problem.

Propagating configuration within the WAS cluster by means of MOM

I am developing application which is embedded within the cluster environment in Websphere AS. I am using several nodes and sometimes I would like to change configuration settings on the fly and propagate it to all nodes within the cluster. I don't want to hold the config in the db or at least I would like to cache it on the node level and trigger config refresh action which forces each node to refresh the config from some common ground (i.e. db or net drive)
to avoid constant round-trips to the config storage.
More over some configuration can't be stored in db i.e. log level needs to be applied on the logger object in each node separately.
I was thinking about using JMS Topics and publish/subscribe approach to achive that goal.
The idea is that each node could subscribe to each Topic and no matter which nodes initate the config change modification would be propagated to all nodes within the cluster.
Has anyone ever tried to do that in WAS and whether there are any obstacles with this approach. If there are or if you have any other suggestion on how to solve that problem I would be very greatfull for your help.
Tx in advance,
Marcin

Here are a few options to consider as alternatives to JMS -
Use Java EE environment entries. These are scoped to the application, and WAS will automatically propagate any changes to all servers against which the application is deployed. This is a good approach since it is the standard Java EE approach to application configuration, if it is robust enough to meet your use case.
Use a WebSphere Shared Library. This allows you to link your applications to static files external to your application (i.e. on the filesystem), such that they are available on your classpath. Although these files are located on the node file systems, there is a way that you can place these files in WebSphere's centralized configuration repository such that they are automatically propagated to all WAS nodes. For more details on this, see this answer.
Both of these options are optimized for static configuration; in other words, configuration settings that are intended to be set at assembly-time, deployment-time, or to be changed by system administrators, but they are not typically used for values that change frequently, nor are they generally changed programmatically at runtime. WAS does allow your applications to pick these configuration settings in a rolling fashion, such that no application downtime is required though.

Currently we solved the problem with maybe not the most pretty approach but with the most simple one. Since we are using only 2 nodes we have possibility to enter web interface of specific node where we modify settings per each node. Maybe it is not very pretty but for now it is the easiest way. The config is stored in DB and we are planning to trigger config reload in each node and change the log level per node as well.

Memory Leak in WebSphere Portal Relating to Portal URIs

I've got an application leaking out java heap at a decent rate (400 users leaves 25% free after 2hours...after logoff all memory is restored) and we've identified the items causing the memory leak as Strings placed in session that appear to be generated by Portal itself. The values are the encoded Portal URIs (very long endcoded strings ... usually sized around 19kb), and the keys seem to be seven (7) randomly generated characters prefixed by RES# (for example, RES#NhhEY37).
We've stepped through the application using session tracing and snapping off heapdumps which has resulted in determining that there is one of these objects created and added to session on almost every page ... in fact, it seems like it is on each page that submits data (which is most pages). So, it's either 1:1 with pages in general, or 1:1 with forms.
Has anyone encountered a similar problem as this? We are opening a ticket with IBM, but wanted to ask this community as well. Thanks in advance!

Can it be the portlet cache? You could have servlet caching activated and declare a long portlet expiry time. Quoting from techjournal:
Portlets can advertise their ability to be cached in the fragment cache by setting their expiry time in their portlet.xml descriptor (see Portlet descriptor example)
<!-Expiration value is in seconds, -1 = no time limit, 0 = deactivated-->
<expiration-cache>3600</expiration-cache> <!- 1 Hour cache -->
To use the fragment caching functions, servlet caching needs to be activated in the Web Container section of WebSphere Application Server administrative console (see Portlet descriptor example). WebSphere Application Server also provides also a cache monitor enterprise application (CacheMonitor.ear), which is very useful for visualizing the contents of the fragment cache.
Update
Do you have portlets that set EXPIRATION_CACHE? Quote:
Modifying the local cache at runtime
For standard portlets, the portlet window can modify the expiration time at runtime by setting the EXPIRATION_CACHE property in the RenderResponse, as follows:
RenderResponse.setProperty(
PortletResponse.EXPIRATION_CACHE,
(new Integer(3000)).toString() );
Note that for me the value is a bit counter-intuitive, -1 means never expire, 0 means don't cache.

The actual issue turned out to be a working feature within Portal. Specifically, Portal's action protection which prevents the same action from being submitted twice, while keeping the navigational ability of the portal. There is a cache that retains the actions results for every successful action and uses them to compare and reject duplicates.
The issue for us was the fact that we required "longer than normal" user sessions (60+ minutes) and with 1,000+ concurrent users, we leaked out on this protection mechanism after just a couple hours.
IBM recommended that we just shut off the cache entirely using the following portlet.xml configuration entry:
wps.multiple.action.execution = true
This allows double submits, which may or may not harm business functionality. However, our internal Portal framework already contained a mechanism to prevent double submits, so this was not an issue for us.
At our request, IBM did come back with a patch for this issue which makes the cache customizeable, that is, let's you configure the number of action results that you store in cache for each user and thus you can leverage Portal's mechanism again, at a reduced session overhead. Those portal configuration settings were:
wps.multiple.action.cache.bound.enabled = true
wps.multiple.action.cache.key.maxsize = 40
wps.multiple.action.cache.value.maxsize = 10
You'll need to contact IBM about this patch as it is not currently in a released fixpack.

Is your Websphere Portal Server having latest fix pack installed?
http://www-01.ibm.com/support/docview.wss?uid=swg24024380&rs=0&cs=utf-8&context=SSHRKX&dc=D420&loc=en_US&lang=en&cc=US
Also you may be interested in following discussion
http://www.ibm.com/developerworks/forums/thread.jspa?messageID=14427700&tstart=0
Update:
Just throwing some blind folded darts.
"RES#" to me sounds like resource.
From the forum stack trace,
"DefaultActionResultManager.storeDocument"
indicates it is storing the document.
Hence looks like your resources(generated portal pages) are being cached. Check if there is some paramater that can lmit cache size of resource.
Also in another test set cache expiration to 5 minutes instead of an hour.

Resolving Session Fixation in JBoss

I need to prevent Session Fixation, a particular type of session hijacking, in a Java web application running in JBoss. However, it appears that the standard idiom doesn't work in JBoss. Can this be worked around?

This defect (found here) points the way to the solution. The Tomcat instance that runs in JBoss is configured with emptySessionPath="true", rather than "false", which is the default. This can be modified in .../deploy/jboss-web.deployer/server.xml; both the HTTP and AJP connectors have this option.
The feature itself is used to eliminate the context path (eg. "foo" in http://example.com/foo) from being included in the JSESSIONID cookie. Setting it to false will break applications that rely on cross-application authentication, which includes stuff built using some portal frameworks. It didn't negatively affect the application in question, however.

This problem and the specific case in which it occurs is a problem in Tomcat as well as JBoss. Tomcat shares the emptySessionPath="true" effect (and actually JBoss inherits it from Tomcat).
This really seems like a bug in Tomcat and JBoss when you are trying to prevent session fixation attacks but the servlet spec (at least version 2.3) does not actually require the JSESSIONID to be defined or redefined according to any specific logic. Perhaps this has been cleaned up in later versions.

One workaround is to store the client address in the session. A response wrapper should validate the client address set in the session is same as the one accessing the session.

I came to know below code setting snippet from one of the forum. And I added below lines. But when I print the session ID after and before log in into the application it is same. How would I test session Fixation.
D:\jboss-5.1.0.GA\bin\run.cof file and add the below line.
set "JAVA_OPTS=%JAVA_OPTS% -Dorg.apache.catalina.connector.Request.SESSION_ID_CHECK=false"
in each context.xml of the jboss applications.
D:\jboss-5.1.0.GA\server\default\deploy\jbossweb.sar\context.xml