I'm using Play to write a webapp which is deployed in Tomcat. Because the app won't be processing very much data I'm using the default H2 database with Hibernate. When I want to deploy a new version of the app, I shut down tomcat, wipe the old webapp and WAR, add my new WAR, and start back up.
This worked until a few days ago, when I added the database component. Now, I am often unable to redeploy the app. When I delete the old directory, it is automatically regenerated with this structure:
$ ls -laR myapp/
myapp/:
total 24
drwxr-xr-x 3 root root 4096 Aug 24 17:20 .
drwxr-xr-x 13 root root 4096 Aug 24 17:20 ..
drwxr-xr-x 3 root root 4096 Aug 24 17:20 WEB-INF
myapp/WEB-INF:
total 24
drwxr-xr-x 3 root root 4096 Aug 24 17:20 .
drwxr-xr-x 3 root root 4096 Aug 24 17:20 ..
drwxr-xr-x 3 root root 4096 Aug 24 17:20 application
myapp/WEB-INF/application:
total 24
drwxr-xr-x 3 root root 4096 Aug 24 17:20 .
drwxr-xr-x 3 root root 4096 Aug 24 17:20 ..
drwxr-xr-x 3 root root 4096 Aug 24 17:20 db
myapp/WEB-INF/application/db:
total 24
drwxr-xr-x 3 root root 4096 Aug 24 17:20 .
drwxr-xr-x 3 root root 4096 Aug 24 17:20 ..
drwxr-xr-x 2 root root 4096 Aug 24 17:20 h2
myapp/WEB-INF/application/db/h2:
total 24
drwxr-xr-x 2 root root 4096 Aug 24 17:20 .
drwxr-xr-x 3 root root 4096 Aug 24 17:20 ..
-rw-r--r-- 1 root root 100 Aug 24 17:20 play.lock.db
The same happens when the WAR unzips.
I recently noticed a message whiz by in the catalina.out log complaining about my app not shutting down a process called something like "H2 File Lock Watchdog". Based on a brief search of the H2 docs, I think that process is what's interfering with my app.
EDIT
Here's the complaining line in the log file:
SEVERE: The web application [/myapp] appears to have started a thread named [H2 File Lock Watchdog /var/lib/apache-tomcat-6.0.32/webapps/myapp/WEB-INF/application/db/h2/play.lock.db] but has failed to stop it. This is very likely to create a memory leak.
So, how do I kill this process? I can't restart the machine because it's not mine, and I can't find the watchdog with top or ps. I'd prefer a way for Play to shut it down automagically, but I'm not above building it into my deployment script.
Thanks a million if you've read this far!
I shut down tomcat
Are you sure you have shut down tomcat completely? Because the H2 database is sill running. If you shut down the tomcat process, the database is also stopped (because H2 is running within the tomcat process). Except if you run the database in a different process.
Or did you just shut down the web application within tomcat? If that is the case, then at least one database connection was not closed, so that the database keeps running (and creates this .lock.db file).
Now, I don't know the play framework, and can't say how to ensure all database connections are closed.
One way to force the database to close is to run the SQL statement SHUTDOWN.
I can't find the watchdog with top or ps
top and ps only display processes. The H2 watchdog is a thread within a java process. To see the thread, use:
jps -l (to get the list of Java processes)
jstack -l <pid> (to get a full thread dump)
Related
I am trying to change the client_max_body_size property of my Elastic Beanstalk NGINX reverse-proxy in order to allow uploads for larger JPEG files. Therefore, I added the folder ".ebextensions" to the root directory of my WAR file (the WAR file is also including a Spring Boot application) and added a file ".ebextensions/01_files.config" with the following content:
files:
"/etc/nginx/conf.d/proxy.conf" :
mode: "000755"
owner: root
group: root
content: |
client_max_body_size 20M;
I deploy the WAR file via Travis-CI to Elastic Beanstalk. However, it seems that the file is beeing ignored by Elastic Beanstalk since uploads with a filesize e.g. 2MB do not work and when connecting with SSH to the instance and looking for "/etc/nginx/conf.d/proxy.conf" the file does not exist.
I already successfully validated above content with an YAML validator. I know, there exists plenty of related questions but non of those seem to fix my problem. I also checked if ".ebextensions/01_files.config" is included in the WAR file in root directory. And when I check "/tmp/eb_extracted_jar", the file ".ebextensions/01_files.config" also exists with the correct content. I can't even find any errors in the "/var/log/cfn-init.log". I noticed that, just for some seconds, the file "proxy.conf" appeared in "/etc/nginx/conf.d/" during deployment but then it has been removed.
Can this problem occure because of the deployment to Elastic Beanstalk via Travis-CI? Or did I miss something else that is important?
EDIT:
I just recognized that the "proxy.conf" file is created every time for a few seconds when the application is deployed but after a few seconds it disappears (checked with ls -lsa in "/etc/nginx/conf.d/", see the timestamps with 13:34 for "elasticbeanstalk" directory and "healthd_http.conf" and 13:43 for "proxy.conf")
4 drwxr-xr-x 3 root root 4096 6. Dec 13:43 .
4 drwxr-xr-x 4 root root 4096 6. Dec 13:34 ..
4 drwxr-xr-x 2 root root 4096 6. Dec 13:34 elasticbeanstalk
4 -rw-r--r-- 1 root root 148 6. Dec 13:34 healthd_http.conf
4 -rwxr-xr-x 1 root root 26 6. Dec 13:43 proxy.conf
And after a few seconds ls -lsa "/etc/nginx/conf.d/":
4 drwxr-xr-x 3 root root 4096 6. Dec 13:44 .
4 drwxr-xr-x 4 root root 4096 6. Dec 13:44 ..
4 drwxr-xr-x 2 root root 4096 6. Dec 13:44 elasticbeanstalk
4 -rw-r--r-- 1 root root 148 6. Dec 13:44 healthd_http.conf
After hours of reading docs, I found out that I missed some important parts of the official AWS docs for the Elastic Beanstalk Java SE Platform (see http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-se-platform.html).
Definitely my mistake: I used the wrong file extension, the file extension in the folder ".ebextensions" has to be ".conf" but not ".config".
At least for the Java SE Platform: one can directly add NGINX config files within the ".ebextensions" directory without using the "files: ..." syntax to generate a file with a specific content, i.e. to create the proxy file in "/etc/nginx/conf.d/proxy.conf" just add ".ebextension/nginx/conf.d/proxy.conf" with the content client_max_body_size 20M; directly. Subsequently, "proxy.conf" will then be deployed to "/etc/nginx/conf.d/proxy.conf" and automatically included by the default NGINX config.
Hope this answer saves someone else the time it took me to figure that out.
I'm deploying a war file (Thingworx Iot Software) on tomcat with elastic beanstalk. The software is trying to create some folders/files under Several folders (at server root) I've given tomcat permissions to write to.
drwxrwxr-x 3 tomcat tomcat 4096 Jan 12 22:41 ThingworxBackupStorage
drwxr-xr-x 2 tomcat tomcat 4096 Jan 13 20:47 ThingworxPlatform
drwxrwxr-x 9 tomcat tomcat 4096 Jan 12 22:41 ThingworxStorage
Heres the error I'm getting:
Caused by: java.io.IOException: THINGWORX WARNING: could not create directory: /ThingworxStorage/database
at com.thingworx.common.utils.PathUtilities.createPath(PathUtilities.java:39)
at com.thingworx.system.configuration.PlatformSettings.prepareLocation(PlatformSettings.java:302)
... 20 more
I'm running Java 8 and Tomcat 8 on an AWS EB setup.
Seams like it should just be a simple permissions thing, but from what I can tell I've already done all that properly. What am I missing?
A CentOS 7 server has Java 7 and tomcat 8 installed. I need to set the permgen space, but typing printenv on the command line does not include JAVA_OPTS in the list of environmental variables. I typed ps to get the list of running processes, and then I typed jmap -heap <pid> with the pid from the running java process, which listed values for MaxPermSize and PermSize among other values.
How do I set new values for these properties in a way that will persist when the system is rebooted, etc.? I hesitate to just create a JAVA_OPTS variable if the server in question has another place where it stores these persistent values.
EDIT:
As per #ChrisRobak's suggestion, I went searching for tomcat conf files. There is no /etc/tomcat directory, but /etc has directories called /etc/java, /etc/.java, /etc/jvm, and /etc/jvm-common. Also, the tomcat conf is as follows:
[root#myserver tomcat]# cd /opt/tomcat/conf
[root#myserver conf]# ls -al
total 280
drwxrwx---. 3 root root 4096 Oct 30 20:02 .
drwxr-xr-x. 14 root root 4096 Oct 30 20:58 ..
drwxrwx---. 7 root root 4096 Oct 30 21:00 Catalina
-rwxrwx---. 1 root root 12624 Nov 2 2014 catalina.policy
-rwxrwx---. 1 root root 6560 Nov 2 2014 catalina.properties
-rwxrwx---. 1 root root 1852 Jan 2 2015 context.xml
-rwxrwx---. 1 root root 3451 Nov 2 2014 logging.properties
-rwxr-x--- 1 root root 5018 Oct 30 21:52 server.xml
-rwxrwx---. 1 root root 1783 Nov 2 2014 tomcat-users.xml
-rwxrwx---. 1 root root 1888 Nov 2 2014 tomcat-users.xsd
-rwxrwx---. 1 root root 168082 Nov 2 2014 web.xml
[root#myserver conf]#
Which file should I look in?
Alternatively, is there a command line way of setting the Java options which will not only be persistent, but also not cause side effects due to conflicts with config files? I would just create a JAVA_OPTS variable if I thought the rest of the server would go to the JAVA_OPTS as the default.
ANSWER?
When none of the config files and scripts seemed to have JAVA_OPTS set, I finally just decided to type the following in the CENTOS 7 terminal:
export JAVA_OPTS="-Xms256m -Xmx1024m -XX:PermSize=256m -XX:MaxPermSize=512m"
The result is that the MaxPermSize has been reset. If anyone else has a specific answer to CentOS 7 that is better than this, please alert me.
I think you can persist those in
/etc/tomcat8/tomcat8.conf
or a similar variant, the server i had access to was /etc/tomcat5/tomcat5.conf but i don't know about your version in particular
I have a war file deployed on tomcat ( /var/lib/tomcat7/webapps folder ), say, rest-api-webapp-0.0.1.war
To access the rest endpoint check, exposed in this war, I use curl in the format
curl -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "remarks=Tester" https://localhost:8080/rest-api-webapp-0.0.1/check The problem I face is, whenever I up the patch/major/minor versions of my webapp, I need to change the curl appropriately (say the version is now 0.1.4, then the curl must change as curl -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "remarks=Tester" https://localhost:8080/rest-api-webapp-0.1.4/check. I donot wish to change the way the client calls the endpoint (because it requires the client to upgrade their app, which they resent and see as high maintenance) Can this be avoided by doing something like this
Create a symbolic link as below ln -s rest-api-webapp-0.0.1.war rest-api inside the /var/log/tomcat7/webapps folder so that whennever I up the version, I just change the symlink to point to the new version and the client need not do anything to use the new version of the api. In effect, I need the api endpoint to be fixed and not change as and when I up the versions on the server. For Ex: I need the endpoint to be fixed as https:gva.atr.in/colouring-api/check and whenever I have a major change in the controllers and all that I need to do is just update the symlink and not change the endpoint. If you find that this approach is flawed, please show me the right direction as I have been trying to read about this from the past 2 days, but found very less articles that address my problem.
I read the tomcat documentation and realised that I need to create a symbolic link to the war file and give the symbolic file the war extension. So If this is what you have in your tomcat webapps directory
/var/lib/tomcat7/webapps user1$ ls
drwxr-xr-x 11 user1 wheel 374 Oct 19 21:52 .
drwxr-xr-x 9 user1 admin 306 Oct 19 19:36 ..
drwxr-xr-x 19 user1 wheel 646 Aug 29 20:19 ROOT
drwxr-xr-x 55 user1 wheel 1870 Aug 29 20:19 docs
drwxr-xr-x 4 user1 wheel 136 Oct 19 19:46 rest-api-webapp-0.2.1
-rw-r--r-- 1 user1 wheel 48258097 Oct 19 19:46 rest-api-webapp-0.2.1.war
drwxr-xr-x 7 user1 wheel 238 Aug 29 20:19 host-manager
drwxr-xr-x 8 user1 wheel 272 Aug 29 20:19 manager
Do this,
ln -s rest-api-webapp-0.2.1.war rest-api.war So that the directory looks like this (Wait for sometime for the tomcat engine to deploy the new war)
drwxr-xr-x 11 user1 wheel 374 Oct 19 21:52 .
drwxr-xr-x 9 user1 admin 306 Oct 19 19:36 ..
drwxr-xr-x 19 user1 wheel 646 Aug 29 20:19 ROOT
drwxr-xr-x 4 user1 wheel 136 Oct 19 21:52 rest-api
drwxr-xr-x 4 user1 wheel 136 Oct 19 19:46 rest-api-webapp-0.2.1
-rw-r--r-- 1 user1 wheel 48258097 Oct 19 19:46 rest-api-webapp-0.2.1.war
lrwxr-xr-x 1 user1 wheel 25 Oct 19 21:51 rest-api.war -> rest-api-webapp-0.2.1.war
drwxr-xr-x 7 user1 wheel 238 Aug 29 20:19 host-manager
drwxr-xr-x 8 user1 wheel 272 Aug 29 20:19 manager If needed restart your tomcat and you can use the curl command like this curl -X POST -H "Content-Type:application/x-www-form-urlencoded" -d "remarks=Tester" https://localhost:8080/rest-api/check totally not bothering about the major, minor, patch versions. All that you need to do once you have a new version is unlink rest-apiln -s rest-api-webapp-X.Y.Z.war rest-api.war
In eclipse it does it anyway when exporting jars, and will only give a warning unless you tell it not to. However in IntelliJ, it refuses to delete the file when building a new version, and I have to go through and manually delete the jar myself for IntelliJ to export properly. Is there a way I force IntelliJ to overwrite jars when it exports?
IntelliJ always overrides artifacts. Try it again. Make sure no other process is using the file with:
$ lsof |grep [file name]
I just tried to make a project with artifact generation, no issues whatsoever.
bender:queues_jar demo$ ls -ltra
total 1408
-rw-r--r-- 1 demo staff 718737 Feb 11 21:26 queues.jar
drwxr-xr-x 3 demo staff 102 Feb 11 21:26 ..
drwxr-xr-x 3 demo staff 102 Feb 11 21:26 .
bender:queues_jar demo$ ls -ltra
total 1408
drwxr-xr-x 3 demo staff 102 Feb 11 21:26 ..
-rw-r--r-- 1 demo staff 718737 Feb 11 21:27 queues.jar
drwxr-xr-x 3 demo staff 102 Feb 11 21:27 .