We Keep Coding

Configuring Jetty 10/11 Request Logs

I am going through the post jetty logging and trying to figure out what is meaning of every attribute that prints
123.4.5.6 - - [27/Aug/2004:10:16:17 +0000]
"GET /jetty/tut/XmlConfiguration.html HTTP/1.1"
200 76793 "http://localhost:8080/jetty/tut/logging.html"
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8"
I am got the answers for some but still not able to figure out for some as mentioned below.
123.4.5.6 : request.getRemoteAddr()
dash(-) : not able to figure out
dash(-) : not able to figure out
[27/Aug/2004:10:16:17 +0000] : timestamp
GET : request.getMethod()
jetty/tut/XmlConfiguration.html : request.getRequestURI()
HTTP/1.1 : request.getProtocol()
200 : response.getStatus()
76793 : response.getHttpChannel().getBytesWritten()
http://localhost:8080/jetty/tut/logging.html : not able to figure out
Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8 : request.getHeader("User-Agent")
Please correct me if I am wrong for other attributes as well.

Recent versions of Jetty log requests with the CustomRequestLog format.
Which has 2 default "NCSA" formats for output (along with a few others and the ability to customize the format)
NCSA_FORMAT - syntax declaration of "%{client}a - %u %t \"%r\" %s %O"
EXTENDED_NCSA_FORMAT - syntax of NCSA_FORMAT + " \"%{Referer}i\" \"%{User-Agent}i\""
So, according to the documentation on CustomRequestLog, that means that the following output ...
123.4.5.6 - - [27/Aug/2004:10:16:17 +0000] "GET /jetty/tut/XmlConfiguration.html HTTP/1.1" 200 76793 "http://localhost:8080/jetty/tut/logging.html" "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8"
is actually one of the extended NCSA formats.
Broken down like this ...
123.4.5.6 : the %{client}a which is the request.getRemoteHost()
- : the - a hardcoded string of - in Jetty, that entry would have been the "remote log name user", but since Jetty doesn't support identd, we don't have a means to fill this value out (identd is old school tech that never saw a big adoption for http)
- : the %u which is the authenticated user name, which comes from Jetty internal APIs (eg: Request.getAuthentication() or Authentication.getUserIdentity() - this only works if your webapp is using Servlet security / authentication / authorization - custom authentication techniques will not have this entry filled out.
[27/Aug/2004:10:16:17 +0000] : the %t which is the Jetty API Request.getTimeStamp() which is set in stone when the request was finished being parsed, but before its dispatched to a handler or webapp for processing.
"GET /jetty/tut/XmlConfiguration.html HTTP/1.1" : the \"%r\" which is the raw "Request Line" used in HTTP. Which is the first line of the HTTP request. (or request.getMethod() + request.getOriginalURI() + request.getProtocol())
200 : the %s which is the status as committed on the response obtained from the Jetty internal API response.getCommittedMetadata().getStatus() (this API exists because the HttpServletResponse is mutable, and many webapps tend to modify it after it's been sent which would mean we log a value that wasn't actually sent if we used the standard servlet API here)
76793 : the %O which is the bytes sent on the network as part of the response which comes from the Jetty internal APIs response.getHttpChannel().getBytesWritten()
"http://localhost:8080/jetty/tut/logging.html" : the \"%{Referer}i\" which is request Referer line (yes, it's spelled incorrectly, but there's ancient HTTP history here for why that's the case). It comes from request.getHeader("Referer"), which can be empty in many cases.
"Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8" : the \"%{User-Agent}i\" is the same as above, but for the User-Agent request header.
You can customize this output in countless ways, just read the CustomRequestLog apidoc and create a format of your own that has what you want and/or what you are looking for.
https://javadoc.io/doc/org.eclipse.jetty/jetty-server/latest/org.eclipse.jetty.server/org/eclipse/jetty/server/CustomRequestLog.html

Jetty logs in the NCSA format by default. According to that,
(2) would be the RFC 931 "remote log name of the user"
(3) would be the "The username as which the user has authenticated himself"
Not sure, but I don't think that any of these will have a value anytime.
(10) this is the referrer (the page which led to this request via a hyperlink), the data is in the header - see Stack Overflow question 2648984

How to resolve an exception related to rest api POST request?

I have a call from angular 5 app:
const httpOptions = {
headers: new HttpHeaders({ 'Accept': 'application/json', 'Content-Type': 'application/json' })
};
...
addTrip(trip: Trip): Observable<any> {
return this.http.post(`${this.baseUrl}/Trips/Trip`, JSON.stringify(trip), httpOptions);
}
The java code is seam component which use rest api:
#Name("tripFacadeREST")
#Scope(ScopeType.EVENT)
#Path("Trips")
public class TripFacadeREST {
#In private TripDaoHibernateImpl tripDao;
...
#POST
#Path("Trip")
#Consumes("application/json")
public Response addNewTrip(Trip newTrip) {
tripDao.addTrip(newTrip);
return Response.ok().entity("trip added successfully").build();
}
I get an exception when trying to send POST request to the server (tomcat - java 7):
java.lang.AbstractMethodError:
javax.ws.rs.core.Response.getStatusInfo()Ljavax/ws/rs/core/Response$StatusType
tomcat log:
0:0:0:0:0:0:0:1 - - [23/May/2018:16:07:12 +1000] "OPTIONS
/jboss-seam/seam/resource/rest/Trips/Trip HTTP/1.1" 200 -
0:0:0:0:0:0:0:1 - - [23/May/2018:16:07:12 +1000] "POST
/jboss-seam/seam/resource/rest/Trips/Trip HTTP/1.1" 500 1386
I can't get this call to work, I have tried also PUT but I understand that put is for update and because the id is created in the server side I've used post. Any idea how to solve this problem? (I was able to use GET annotation in the same service with no issues).

This problem was solved by update the resteasy jars to version 2.1.0.
For anyone encountered similar problem.
Seam 2.1.2 with resteasy 2.1.

Tomcat Jersey REST - Spaces in #PathParam

Should spaces be possible in #PathParams when creating REST-Services via Jersey 1.19 in Tomcat 2.0.27?
Jersey returns 400 Bad Request whenever I try to call my REST-Service with a space in the filename path param. It does not even reach my code. If there is no space in filename, everything is working as expected. The annotations of my REST method are as follows:
#GET
#Path("/trennblatt/{objId}/{filename}")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getTrennblatt(
#PathParam("objId") final long objId,
#PathParam("filename") final String filename,
#Context final HttpServletResponse response)
Tomcat's access log, the encoding seems fine:
[18/Apr/2016:11:09:17 +0200] "GET /.../trennblatt/228342/01%20Aktendecke.pdf HTTP/1.1" 400 1004
[18/Apr/2016:11:09:29 +0200] "GET /.../trennblatt/228342/inspectionsheet.html HTTP/1.1" 200 44870
[18/Apr/2016:11:13:29 +0200] "GET /.../trennblatt/228342/inspectionsheet.html/ HTTP/1.1" 200 44870
[18/Apr/2016:11:13:33 +0200] "GET /.../trennblatt/228342/01%20Aktendecke.pdf/ HTTP/1.1" 400 1004
As the access log shows, I even tried appending another / to my URI template without success.
#Path("/trennblatt/{objId}/{filename}/")
I also tried specifying the character set for filename (everything):
#Path("/trennblatt/{objId}/{filename:.+}")
I tried to to activate Jersey logging by adding:
<init-param>
<param-name>com.sun.jersey.spi.container.ContainerRequestFilters</param-name>
<param-value>com.sun.jersey.api.container.filter.LoggingFilter</param-value>
</init-param>
but could not find any additional output. I even procmon'd for accessed files containing "log" or "out".
Googling for things like "jersey rest spaces #PathParam" always returns fun things like "How do I enabled slashes and dots in #PathParam". I can see how this would be an issue. But I could not find any clear statement about whether or not simple (and properly encoded) spaces in #PathParam should work.
When I'm using #QueryParam everything is fine.
#GET
#Path("/trennblatt")
#Produces(MediaType.APPLICATION_OCTET_STREAM)
public Response getTrennblatt(
#QueryParam("objId") final long objId,
#QueryParam("filename") final String filename,
#Context final HttpServletResponse response)
[18/Apr/2016:11:17:47 +0200] "GET /.../trennblatt?objId=228342&filename=inspectionsheet.html HTTP/1.1" 200 44870
[18/Apr/2016:11:18:04 +0200] "GET /.../trennblatt?objId=228342&filename=01%20Aktendecke.pdf HTTP/1.1" 200 45081
This is my current workaround but there is a point to using #PathParam (keeping it as similar to our WebDAV service as possible). So I'd really like to get your input on this. Thanks!

A simple HTTPS proxy server with Apache Camel

I am trying to implement a simple HTTP proxy service with Apache Camel.
My code looks like this:
from("jetty:http://localhost:80?matchOnUriPrefix=true")
.recipientList(simple("jetty:${in.header.CamelHttpUrl}?bridgeEndpoint=true&throwExceptionOnFailure=false&disableStreamCache=true"));
It is essentially this with a dynamic recipient list to support multiple destinations. I also had to add the disableStreamCache=true bit, otherwise I would get weird exceptions with path duplication (like /index.html would become /index.html/index.html).
Nevertheless, it seems to work. But only with HTTP requests. When I try accessing an HTTPS site, I always get a 404.
According to the logs, the jetty component just doesn't seem to find the remote server. I have no idea why.
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG org.eclipse.jetty.server.Server - REQUEST www.google.cz:443 on AsyncHttpConnection#6964b063,g=HttpGenerator{s=0,h=-1,b=-1,c=-1},p=HttpParser{s=-5,l=17,c=0},r=1
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG o.e.j.server.handler.ContextHandler - scope null||www.google.cz:443 # o.e.j.s.ServletContextHandler{/,null}
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG o.e.j.server.handler.ContextHandler - context=null||www.google.cz:443 # o.e.j.s.ServletContextHandler{/,null}
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG o.e.jetty.servlet.ServletHandler - servlet null||www.google.cz:443 -> null
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG o.e.jetty.servlet.ServletHandler - chain=null
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG o.e.jetty.servlet.ServletHandler - Not Found www.google.cz:443
01:36:37.495 [qtp85415531-22 - www.google.cz:443] DEBUG org.eclipse.jetty.server.Server - RESPONSE www.google.cz:443 200 handled=false
What should I do to enable this HTTPS support? Is it even possible with standard Camel components?
Edit:
I managed to update my route definition to not use a recipient list. I don't know whether this improves anything performance-wise (does it?) but it feels better. I was also able to remove the path duplication problem when not using disableStreamCache=true by this.
from("jetty:http://localhost:80?matchOnUriPrefix=true")
.to("http4:dummy?bridgeEndpoint=true&urlRewrite=#urlRewrite&throwExceptionOnFailure=false");
And the URL rewriter implementation:
UrlRewrite urlRewrite = new HttpServletUrlRewrite() {
#Override
public String rewrite(String url, String relativeUrl, Producer producer, HttpServletRequest request) throws Exception {
return request.getRequestURL().toString();
}
#Override
public String rewrite(String url, String relativeUrl, Producer producer) throws Exception {
return null;
}
};
Edit 2:
I should probably mention that I would like to intercept those requests and read/alter content (actually only HTTP headers). In effect I would like to implement an MITM proxy.
Edit 3:
I tried replacing the target component with log to see whether the request gets through:
from("jetty:http://localhost:80?matchOnUriPrefix=true")
.to("log:test")
The message gets logged when used as a proxy with HTTP. It also gets logged when I replace the URI with jetty:https://localhost?matchOnUriPrefix=true and try opening https://localhost directly in the browser. However, when trying to use this as a proxy with HTTPS, I cannot get it to log. It seems like the Jetty component doesn't support this behavior. Is it correct?
I also tried using the Netty-http component with similar results (route tracer logged the CONNECT request but the message doesn't get passed to the Log component)

The key was to define a handler for jetty that would take care of the CONNECT method:
from("jetty:http://localhost:80?matchOnUriPrefix=true&handlers=#connectHandler")
The connectHandler in this case can be either a Jetty ConnectHandler (for normal HTTPS proxies) or a custom one (for MITM proxies):
class CustomConnectHandler extends ConnectHandler {
#Override
protected SocketChannel connect(HttpServletRequest request, String host, int port) throws IOException {
SocketChannel channel = SocketChannel.open();
channel.socket().setTcpNoDelay(true);
channel.socket().connect(new InetSocketAddress("localhost", 443), getConnectTimeout());
return channel;
}
}
This one connects to port 443 on the localhost and routes SSL data there. So we need to define another route from there:
from("jetty:https://localhost:443?matchOnUriPrefix=true")
(we can keep two separate routes or join them later by using a SEDA component)
To define custom SSL context parameters (key store, trust store, etc.), we can use the sslContextParametersRef parameter on the endpoint.
Next, we need to alter the URL. We do this by populating the URI header with the full URL (it's only the relative path by default):
.setHeader(Exchange.HTTP_URI, simple("${header.CamelHttpUrl}"))
We also need to remove the PATH header because the HTTP4 component builds the final URL by concatenating the URI and PATH headers:
.removeHeader(Exchange.HTTP_PATH)
Finally, we can forward the message to the HTTP4 component. We should also set throwExceptionOnFailure=false to forward errors to the client and httpClient.redirectsEnabled=false to forward redirects (this can otherwise mess with some sites):
.to("http4:dummy?throwExceptionOnFailure=false&httpClient.redirectsEnabled=false");
I hope this helps anyone who is looking to implement an HTTPS proxy using Camel and was struggling like me.

On most computer systems, localhost resolves to the IP address 127.0.0.1
Note:- This is a valid implementation in case of using Fiddler.***
<camelContext id="context" xmlns="http://camel.apache.org/schema/spring">
<properties>
<property key="http.proxyHost" value="127.0.0.1"/>
<property key="http.proxyPort" value="8888"/>
</properties>
</camelContext>

Shiro and Tomcat AccessLogValve

I'm using Shiro (v1.2.1) as security framework, it works perfectly, but in access log for Tomcat (v7.0.32) you cannot get remote user in logging when authenticated user access any resource. for my sample web application I used default configuration provided in Shiro SVN example repo.
127.0.0.1 - - [13/Nov/2012:08:22:55 +0200] "POST /pacs/login.jsp HTTP/1.1" 302 - // User here not logged
127.0.0.1 - - [13/Nov/2012:08:22:55 +0200] "GET /pacs/ HTTP/1.1" 200 821 // Here user is accessing protected page, so we got permission.
Is there any way this could be solved?

I'm not sure if this is THE official solution, but it is a solution which works for me:
for the AccessLogValve pattern, use
%u - Remote user that was authenticated (if any), else '-'
this will print the "guessed user name". If you ask google how tomcat will guess the user name, you'll find something like this:
For the user name, the manager uses the Session.getPrincipal() if available;
if not, it tries the following Session attribute names:
Login
User
userName
UserName
Utilisateur
using the given case, lower case, and upper case. Failing that, it searches for
attributes that are instances of java.security.Principal or
javax.security.auth.Subject.
In my case, I just modified the login code to write the user object to the session. Tomcat will invoke a .toString() on this object in order to print the "guessed user name".
If the result from .toString() is not what you want, you can also try to use
%{xxx}s xxx is an attribute in the HttpSession
in your AccessLogValve configuration in order to print another attribute from your session.
hope that helps.

If you're using Shiro's native session management, session.getPrincipal() will work as expected. If you'd like to see this call work in default session management, please open a Jira issue and we'll see if we can fix it asap.