I know a similar problem has been posted several times on StackOverflow, still all the answers did not help in my case.
Target: I need to log the SOAP-Request XML into a database before processing it further.
For this discussion, I'm happy if I can get it as a String and log it to console.
Problem: Request XML is always empty.
Environment: IBM WebSphere Application Server v8.5, EJB 3.1 web service (session bean)
I actually produced a working solution using javax.xml.soap.SOAPBody and SOAPMessage, but on production there seems to be another component which causes the following JAX-WS conflict:
ERROR org.apache.axis2.engine.AxisEngine receive - com.sun.xml.messaging.saaj.soap.ver1_1.Body1_1Impl incompatible with com.ibm.ws.webservices.engine.xmlsoap.SOAPBody
Yes, there are multiple workarounds on StackOverflow and IBM about changing the ClassLoader policy, like "local class loader first (parent last)", but we currently can't go this way.
So, my current approach (which was working on a different servlet) is to get the HTTPServletRequest, get it's InputStream and convert it to a String using IOUtils.toString().
I am aware that a request can only be consumed once and I found several approaches to avoid it (like using a HTTPServletRequestWrapper) but even using these workarounds the request XML is always empty.
Eventually I want to do the logging in an adapter, but for testing reasons I also put my attempts into the service itself (it did not have any effect).
The strange thing is: I can read all Headers and Attributes from the request (using request.getHeader() and request.getAttribute()! Only the body itself is blank!
I'm testing the application using SoapUI.
Request-XML:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:ping="http://services.axonactive.com/wsdl/cet/pingservice-v1.0">
<soapenv:Header/>
<soapenv:Body>
<ping:PingPingIn/>
</soapenv:Body>
</soapenv:Envelope>
Response is actually irrelevant, but correctly working.
Console output:
[ch.zek.ecls.EclsPingServiceImpl]: Initialization successful.
EclsPingServiceImpl ping - start ping()...
EclsPingServiceImpl ping - Body:
EclsPingServiceImpl ping - body2:
EclsUtil printRequestData - [H] Accept-Encoding: gzip,deflate
EclsUtil printRequestData - [H] Content-Type: text/xml;charset=UTF-8
EclsUtil printRequestData - [H] SOAPAction: ""
EclsUtil printRequestData - [H] Content-Length: 254
EclsUtil printRequestData - [H] Host: localhost:9443
EclsUtil printRequestData - [H] Connection: Keep-Alive
EclsUtil printRequestData - [H] User-Agent: Apache-HttpClient/4.1.1 (java 1.5)
EclsUtil printRequestData - [A] javax.servlet.request.key_size: 128
EclsUtil printRequestData - [A] javax.servlet.request.cipher_suite: SSL_RSA_WITH_AES_128_CBC_SHA
EclsUtil printRequestData - [A] com.ibm.websphere.servlet.uri_non_decoded: /NewZekEclsHTTPRouter/PingService
EclsPingServiceImpl ping - end ping()
Edit: Note the headers [H] for Content-Type and Content-Length:
They "know" the content - if I put some more characters into the request XML then Content-Length is updated accordingly.
So I conclude that the content is not lost, but somehow not accessible... .
Webservice:
public class EclsPingServiceImpl{
#javax.annotation.Resource
WebServiceContext wsContext;
#javax.annotation.Resource
SessionContext sessionContext;
private Log log = LogFactory.getLog(EclsPingServiceImpl.class);
public PingOut ping(PingIn parameters) throws PingEntityNotFoundException, PingPermissionException, PingSystemException {
MessageContext messageContext = wsContext.getMessageContext();
HttpServletRequest request = (HttpServletRequest) messageContext.get(MessageContext.SERVLET_REQUEST);
// start Try 1
MultiReadHttpServletRequest multiReadHttpServletRequest = new MultiReadHttpServletRequest(request);
try {
InputStream bodyInputStream = multiReadHttpServletRequest.getInputStream();
String body = IOUtils.toString(bodyInputStream);
if (log.isDebugEnabled()) {
log.debug("Body: " + body);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// end Try 1
// start Try 2
try {
InputStream body2 = request.getInputStream();
String xml = IOUtils.toString(body2, "UTF-8");
if (log.isDebugEnabled()) {
log.debug("body2: " + xml);
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// end Try 2
// Get Header data:
Enumeration<String> headerNames = request.getHeaderNames();
while(headerNames.hasMoreElements()) {
String headerName = headerNames.nextElement();
if (log.isDebugEnabled()) {
log.debug("[H] " + headerName + ": " + request.getHeader(headerName));
}
}
// Get Attribute data:
Enumeration<String> attributeNames = request.getAttributeNames();
while(attributeNames.hasMoreElements()) {
String attributeName = attributeNames.nextElement();
if (log.isDebugEnabled()) {
log.debug("[A] " + attributeName + ": " + request.getAttribute(attributeName));
}
}
PingOut pingOut = new PingOut();
// some irrelevant stuff...
return pingOut;
}
}
MultiReadHttpServletRequestWrapper:
Copied from StackOverflow: Http Servlet request lose params from POST body after read it once
public class MultiReadHttpServletRequest extends HttpServletRequestWrapper {
private ByteArrayOutputStream cachedBytes;
public MultiReadHttpServletRequest(HttpServletRequest request) {
super(request);
}
#Override
public ServletInputStream getInputStream() throws IOException {
if (cachedBytes == null)
cacheInputStream();
return new CachedServletInputStream();
}
#Override
public BufferedReader getReader() throws IOException{
return new BufferedReader(new InputStreamReader(getInputStream()));
}
private void cacheInputStream() throws IOException {
/* Cache the inputstream in order to read it multiple times. For
* convenience, I use apache.commons IOUtils
*/
cachedBytes = new ByteArrayOutputStream();
IOUtils.copy(super.getInputStream(), cachedBytes);
}
/* An inputstream which reads the cached request body */
public class CachedServletInputStream extends ServletInputStream {
private ByteArrayInputStream input;
public CachedServletInputStream() {
/* create a new input stream from the cached request body */
input = new ByteArrayInputStream(cachedBytes.toByteArray());
}
#Override
public int read() throws IOException {
return input.read();
}
}
}
LogHandler - just to show what I tried, too...:
public class EclsSimpleLogHandler implements javax.xml.ws.handler.soap.SOAPHandler
{
private Log log = LogFactory.getLog(EclsSimpleLogHandler.class);
#Override
public boolean handleMessage(MessageContext context) {
boolean success = false;
if (log.isDebugEnabled()) {
log.debug("handleMessage()...");
}
Boolean outboundProperty = (Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
// check if handler is called for inbound (request) or outbound (response) message
if (outboundProperty.booleanValue()) {
success = handleResponse(context);
} else {
//success = handleRequest(context);
success = true;
}
return success;
}
private boolean handleRequest(MessageContext messageContext) {
if(log.isDebugEnabled()) {
log.debug("handling request (inbound)");
}
// Initially logging planned here, moved out for testing reasons
boolean success = false;
return success;
}
private boolean handleResponse(MessageContext messageContext) {
if(log.isDebugEnabled()) {
log.debug("handling response (outbound)");
}
boolean success = false;
ByteArrayOutputStream outputStream = null;
SOAPMessageContext context = (SOAPMessageContext) messageContext;
SOAPMessage soapMessage = (SOAPMessage) context.getMessage();
try {
/*
Initial solution, but sometimes causing:
ERROR org.apache.axis2.engine.AxisEngine receive - com.sun.xml.messaging.saaj.soap.ver1_1.Body1_1Impl
incompatible with com.ibm.ws.webservices.engine.xmlsoap.SOAPBody
// SOAPBody soapBody = soapMessage.getSOAPBody();
// String soapBodyXml = soapBody.toString();
*/
// This is working - but I want to avoid using SOAPMessage:
outputStream = new ByteArrayOutputStream();
soapMessage.writeTo(outputStream);
String soapBodyXml = new String(outputStream.toByteArray(), "UTF-8");
if (log.isDebugEnabled()) {
log.debug("responseXml:\n" + soapBodyXml);
}
success = true;
} catch (SOAPException e) {
if (log.isErrorEnabled()) {
log.error("Error while accessing SOAPMessage: " + e.getMessage());
}
} catch (IOException e) {
if (log.isErrorEnabled()) {
log.error("IOException for soapMessage.writeTo(): " + e.getMessage());
}
e.printStackTrace();
}
return success;
}
// Another method, but also resulting in an empty body:
static String extractPostRequestBody(HttpServletRequest request) {
if ("POST".equalsIgnoreCase(request.getMethod())) {
Scanner s = null;
try {
s = new Scanner(request.getInputStream(), "UTF-8").useDelimiter("\\A");
} catch (IOException e) {
e.printStackTrace();
}
return s.hasNext() ? s.next() : "";
}
return "";
}
// Another method, but also resulting in an empty body:
private String getRequestBody(HttpServletRequest request) {
HttpServletRequestWrapper requestWrapper = new HttpServletRequestWrapper(request);
StringBuilder stringBuilder = new StringBuilder();
BufferedReader bufferedReader = null;
try {
InputStream inputStream = requestWrapper.getInputStream();
if (inputStream != null) {
bufferedReader = new BufferedReader(new InputStreamReader(inputStream));
char[] charBuffer = new char[128];
int bytesRead = -1;
while ((bytesRead = bufferedReader.read(charBuffer)) != -1) {
stringBuilder.append(charBuffer, 0, bytesRead);
}
}
} catch (IOException ex) {
log.error("Error reading the request payload", ex);
} finally {
if (bufferedReader != null) {
try {
bufferedReader.close();
} catch (IOException iox) {
// ignore
}
}
}
return stringBuilder.toString();
}
}
Related
I am trying to log all the outgoing Http requests in my spring based web application. Is there is interceptor for this purpose? I want to log all outgoing the contents and headers before it leaves the application. I am using spring-ws to send SOAP requests. So basically, I want to log not only the SOAP request xml (as mentioned here How can I make Spring WebServices log all SOAP requests?) but the http request as a whole.
Intercept the request/response using a ClientInterceptor on the WebServiceGatewaySupport:
// soapClient extends WebServiceGatewaySupport
soapClient.setInterceptors(new ClientInterceptor[]{new ClientInterceptor() {
#Override
public boolean handleRequest(MessageContext messageContext) throws WebServiceClientException {
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
messageContext.getRequest().writeTo(os);
} catch (IOException e) {
throw new WebServiceIOException(e.getMessage(), e);
}
String request = new String(os.toByteArray());
logger.trace("Request Envelope: " + request);
return true;
}
#Override
public boolean handleResponse(MessageContext messageContext) throws WebServiceClientException {
ByteArrayOutputStream os = new ByteArrayOutputStream();
try {
messageContext.getResponse().writeTo(os);
} catch (IOException e) {
throw new WebServiceIOException(e.getMessage(), e);
}
String response = new String(os.toByteArray());
logger.trace("Response Envelope: " + response);
return true;
}
...
To get the headers as well you need an instance of TransportOutputStream.
Unfortunately the class is abstract, so you need to subclass is. Here's how it might look:
class ByteArrayTransportOutputStream extends TransportOutputStream {
private ByteArrayOutputStream outputStream;
#Override
public void addHeader(String name, String value) throws IOException {
createOutputStream();
String header = name + ": " + value + "\n";
outputStream.write(header.getBytes());
}
public byte[] toByteArray() {
return outputStream.toByteArray();
}
#Override
protected OutputStream createOutputStream() throws IOException {
if (outputStream == null) {
outputStream = new ByteArrayOutputStream();
}
return outputStream;
}
}
I've got org.apache.http.HttpResponse object, which I'm using at different places in the code. One of those places is for logging.
The problem is that when I run following log code:
HttpEntity entity = response.getEntity();
try {
String content = Base64.encodeToString(
EntityUtils.toByteArray(entity), Base64.DEFAULT);
sb.append(content + "\r\n");
} catch (Exception e) {
sb.append("\r\n\r\n====EXCEPTION=====\r\n" + e.toString()
+ "\r\n");
}
and than I try to read entry content in the actual processing code, that causes the code to throw following exception:
java.lang.IllegalStateException: Content has been consumed
My question is: how do I read the entity without consuming it in the log code?
UPDATE
here's the full code of the function I use to transform httpresponse to string:
static String toString(org.apache.http.HttpResponse response) {
try {
if (response == null) {
return "null";
}
StringBuilder sb = new StringBuilder();
sb.append("==============BEGIN HttpResponse================\r\n");
StatusLine sl = response.getStatusLine();
if (sl == null) {
sb.append("status line is null\r\n");
} else {
sb.append(String.format("%s %s\r\n", sl.getStatusCode(),
sl.getReasonPhrase()));
}
for (Header h : response.getAllHeaders()) {
if (h == null) {
sb.append("header is null\r\n");
continue;
}
sb.append(String.format("%s: %s\r\n", h.getName(), h.getValue()));
}
sb.append("\r\r\r\n");
HttpEntity entity = response.getEntity();
if (entity == null) {
sb.append("content is null");
} else {
try {
String content = Base64.encodeToString(
EntityUtils.toByteArray(entity), Base64.DEFAULT);
sb.append(content + "\r\n");
} catch (Exception e) {
sb.append("\r\n\r\n====EXCEPTION=====\r\n" + e.toString()
+ "\r\n");
}
}
sb.append("\r\n==============END HttpResponse================\r\n");
return sb.toString();
} catch (Exception e) {
return e.toString();
}
}
Ok. So what I ended up doing is implementing my own HttpEntity class, and than just using response.setEntity(...) to replace the previous entity. That class stores the result as binary array and returns it as many times as necessary.
It might give you some performance issues, but will work:
Example of my HttpClient with logging.
private CloseableHttpResponse invoke(HttpRequestBase http) {
try {
CloseableHttpResponse response = client.execute(http);
if (http instanceof HttpPost) {
InputStream inputStream = ((HttpPost) http).getEntity().getContent();
String body = IOUtils.toString(inputStream, Charset.defaultCharset());
HttpEntity respBody = response.getEntity();
String responseBody = StreamUtils.copyToString(respBody.getContent(), Charset.defaultCharset());
response.setEntity(new StringEntity(responseBody));
LOG.info(String.format("Sending request: [%s] %s => [%s] \nPayload:\n%s \nResponse:\n%s", http.getMethod(), http.getURI(), response.getStatusLine(), body, responseBody));
} else {
LOG.info(String.format("Sending request: [%s] %s => [%s]", http.getMethod(), http.getURI(), response.getStatusLine()));
}
return response;
} catch (IOException e) {
throw new RuntimeException("HTTP request failed: " + http.toString(), e);
}
}
Main idea is following:
1. make http call
2. copy to string your response body:
HttpEntity respBody = response.getEntity();
String responseBody = StreamUtils.copyToString(respBody.getContent(), Charset.defaultCharset());
log it
set new response entity like response.setEntity(new StringEntity(responseBody));
This example work good for small test framework, not sure it's good code for production application
I have a piece of software which establishes a communication between a PC and a machine, which used to work in Java 1.6, but doesn't anymore in 1.7. The IOException --> "Invalid Http response" is what I get as soon as the getInputStream() method is called. It seems like the method had been improved an is much more sensitive, meaning that responseCode=
-1 result is not specifically checked in Java 1.6.
Assited with Wireshark I checked if both versions (1.6 & 1.7) sent and received same Ethernet frames, and so did both.
I can only resolve this from the PC (client) side...that means no posibility to edit or change the Server code.
I would apreciate any help on how to modify or implement something new to make the code compatible for ver. 1.7 as Im not a former programmer... Thanks
Sequence:
get is called
return readResponse(..) is called
getInputStream() --> IOException
catch (Exception e) {System.out.println("error sending get request " + getURL() + " string: " + requestString); return Error.ethernetException; //TODO
here the console output:
-1
error sending get request ... string: *APPLETCOMMINFO_
strResponse: Ethernet Exception
and the Code:
private String get(String requestString)/* throws ComunicationException*/ {
HttpURLConnection httpURLConnection = null;
OutputStream out = null;
try {
String encodedRequestString = URLEncoder.encode(requestString, "iso-8859-1");
String path = getURL().getPath();
if (!path.endsWith("/")) path = path + "/";
path = path + encodedRequestString;
URL fullURL = new URL(getURL().getProtocol(), getURL().getHost(), getURL().getPort(), path);
httpURLConnection = (HttpURLConnection)fullURL.openConnection();
httpURLConnection.setRequestMethod("GET");
httpURLConnection.setDoOutput(false);
httpURLConnection.setDoInput(true);
httpURLConnection.setUseCaches(false);
// Set timeout at 5s
httpURLConnection.setConnectTimeout(m_nTimeOut);
httpURLConnection.setReadTimeout(m_nTimeOut);
return readResponse(httpURLConnection);
} catch (Exception e) {
System.out.println("error sending get request " + getURL() + " string: " + requestString);
return Error.ethernetException; //TODO
} finally {
if (out != null) {
try {
out.close();
} catch (Throwable t) {
System.out.println("GET: out.close(), Class: Client");
}
}
if (httpURLConnection != null) {
try {
httpURLConnection.disconnect();
} catch (Throwable t) {
System.out.println("GET: httpURLConnection.disconnect(), Class: Client");
}
}
}
}
/**
* Reads the response from the server into a string.
* The content length header must be set!
* #param connection
* #return
* #throws IOException
*/
private String readResponse(HttpURLConnection connection) throws IOException {
if (connection == null) throw new IllegalStateException("connection must not be null");
connection.connect();
int status = connection.getResponseCode();
System.out.println(status);
// InputStream aaa = connection.getInputStream();
Reader reader = null;
try {
reader = new InputStreamReader(connection.getInputStream(), "iso-8859-1");
int readBufferSize = connection.getContentLength();
if (readBufferSize < 0) {
// use default buffer size
readBufferSize = DEFAULT_READ_BUFFER_SIZE;
}
// if content length was set, read will be done in a single
// iteration because buffer fits...
StringBuffer response = new StringBuffer();
char[] readBuffer = new char[readBufferSize];
int len;
while ((len = reader.read(readBuffer)) > 0) {
response.append(new String(readBuffer, 0, len));
}
return response.toString();
} catch (IOException ioe) {
throw ioe;
} finally {
if (reader != null) {
try {
reader.close();
} catch (Throwable t) {
System.out.println("readResponse: reader.close(), Class: Client");
//log
}
}
}
}
/**
*
* #return the url
*/
public URL getURL() {
return url;
}
}
I am attempting to send a user update to valance and I am looking for an example of how to do a put, specifically, a put to update a user.
I have looked around but do not see an example of how to use the UserContext to send a json block using Java.
Any pointers to documentation would be appreciated.
After tinkering with this, and with a great number of suggestions from my colleague (Who I am not sure wants to be identified so i will just call him Bill). We came up with the following Java method(should really be split into separate methods but it is understandable)
private static String getValanceResult(ID2LUserContext userContext,
URI uri, String query, String sPost, String sMethod, int attempts) {
String sError = "Error: An Unknown Error has occurred";
if (sMethod == null) {
sMethod = "GET";
}
URLConnection connection;
try {
URL f = new URL(uri.toString() + query);
//connection = uri.toURL().openConnection();
connection = f.openConnection();
} catch (NullPointerException e) {
return "Error: Must Authenticate";
} catch (MalformedURLException e) {
return "Error: " + e.getMessage();
} catch (IOException e) {
return "Error: " + e.getMessage();
}
StringBuilder sb = new StringBuilder();
try {
// cast the connection to a HttpURLConnection so we can examin the
// status code
HttpURLConnection httpConnection = (HttpURLConnection) connection;
httpConnection.setRequestMethod(sMethod);
httpConnection.setConnectTimeout(20000);
httpConnection.setReadTimeout(20000);
httpConnection.setUseCaches(false);
httpConnection.setDefaultUseCaches(false);
httpConnection.setDoOutput(true);
if (!"".equals(sPost)) {
//setup connection
httpConnection.setDoInput(true);
httpConnection.setRequestProperty("Content-Type", "application/json");
//execute connection and send xml to server
OutputStreamWriter writer = new OutputStreamWriter(httpConnection.getOutputStream());
writer.write(sPost);
writer.flush();
writer.close();
}
BufferedReader in;
// if the status code is success then the body is read from the
// input stream
if (httpConnection.getResponseCode() == 200) {
in = new BufferedReader(new InputStreamReader(
httpConnection.getInputStream()));
// otherwise the body is read from the output stream
} else {
in = new BufferedReader(new InputStreamReader(
httpConnection.getErrorStream()));
}
String inputLine;
while ((inputLine = in.readLine()) != null) {
sb.append(inputLine);
}
in.close();
// Determine the result of the rest call and automatically adjusts
// the user context in case the timestamp was invalid
int result = userContext.interpretResult(
httpConnection.getResponseCode(), sb.toString());
if (result == ID2LUserContext.RESULT_OKAY) {
return sb.toString();
// if the timestamp is invalid and we haven't exceeded the retry
// limit then the call is made again with the adjusted timestamp
} else if (result == userContext.RESULT_INVALID_TIMESTAMP
&& attempts > 0) {
return getValanceResult(userContext, uri, query, sPost, sMethod, attempts - 1);
} else {
sError = sb + " " + result;
}
} catch (IllegalStateException e) {
return "Error: Exception while parsing";
} catch (FileNotFoundException e) {
// 404
return "Error: URI Incorrect";
} catch (IOException e) {
}
return sError;
}
There is a php code snippet I can share from a project that uses the api (same rough logic with java). The User context just prepares the url and the framework of the specific environment (java runtime, or php library) is used to do the post and retrieve the results (in this case it is using php CURL).
$apiPath = "/d2l/api/le/" . VERSION. "/" . $courseid . "/content/isbn/";
$uri = $opContext->createAuthenticatedUri ($apiPath, 'POST');
$uri = str_replace ("https", "http", $uri);
curl_setopt ($ch, CURLOPT_URL, $uri);
curl_setopt ($ch, CURLOPT_POST, true);
$response = curl_exec ($ch);
$httpCode = curl_getinfo ($ch, CURLINFO_HTTP_CODE);
$contentType = curl_getinfo ($ch, CURLINFO_CONTENT_TYPE);
$responseCode = $opContext->handleResult ($response, $httpCode, $contentType);
$ret = json_decode($response, true);
if ($responseCode == D2LUserContext::RESULT_OKAY)
{
$ret = "$response";
$tryAgain = false;
}
elseif ($responseCode == D2LUserContext::RESULT_INVALID_TIMESTAMP)
{
$tryAgain = true;
}
elseif (isset ($ret['Errors'][0]['Message']))
{
if ($ret['Errors'][0]['Message'] == "Invalid ISBN")
{
$allowedOrgId[] = $c;
}
$tryAgain = false;
}
A sample of a trace of a post message is:
PUT https://valence.desire2learn.com/d2l/api/lp/1.0/users/3691?x_b=TwULqrltMXvTE8utuLCN5O&x_a=L2Hd9WvDTcyiyu5n2AEgpg&x_d=OKuPjV-a0ZoSBuZvJkQLpFva2D59gNjTMiP8km6bdjk&x_c=UjCMpy1VNHsPCJOjKAE_92g1YqSxmebLHnQ0cbhoSPI&x_t=1336498251 HTTP/1.1
Accept-Encoding: gzip,deflate
Accept: application/json
Content-Type: application/json
{
"OrgDefinedId": "85033380",
"FirstName": "First",
"MiddleName": "Middle",
"LastName": "Last",
"ExternalEmail": "me#somehostname.com",
"UserName": "Username",
"Activation": {
"IsActive": true
}
}
I have an assignment to write a proxy server. Simple tests work, but when I configure firefox to use the proxy, the response input stream is never ready. Can you help?
ProxyServer (the important method)
public void start() {
while (true) {
Socket serverSocket;
Socket clientSocket;
BufferedWriter toClient;
BufferedWriter toServer;
try {
//The client is meant to put data on the port, read the socket.
clientSocket = listeningSocket.accept();
Request request = new Request(clientSocket.getInputStream());
System.out.println("Accepted a request!\n" + request);
while(request.busy);
//Make a connection to a real proxy.
//Host & Port - should be read from the request
URL url = null;
try {
url = new URL(request.getRequestURL());
} catch (MalformedURLException e){
url = new URL("http:\\"+request.getRequestHost()+request.getRequestURL());
}
//remove entry from cache if needed
if (!request.getCacheControl().equals(CacheControl.CACHE) && cache.containsRequest(request)) {
cache.remove(request);
}
Response response = null;
if (request.getRequestType() == RequestType.GET && request.getCacheControl().equals(CacheControl.CACHE) && cache.containsRequest(request)) {
response = cache.get(request);
} else {
//Get the response from the destination
int remotePort = (url.getPort() == -1) ? 80 : url.getPort();
System.out.println("I am going to try to connect to: " + url.getHost() + " at port " + remotePort);
serverSocket = new Socket(url.getHost(), remotePort);
System.out.println("Connected.");
//write to the server - keep it open.
System.out.println("Writing to the server's buffer...");
toServer = new BufferedWriter(new OutputStreamWriter(serverSocket.getOutputStream()));
toServer.write(request.getFullRequest());
toServer.flush();
System.out.println("flushed.");
System.out.println("Getting a response...");
response = new Response(serverSocket.getInputStream());
System.out.println("Got a response!\n" + response);
//wait for the response
while(response.isBusy());
}
if (request.getRequestType() == RequestType.GET && request.getCacheControl().equals(CacheControl.CACHE)) {
cache.put(request, response);
}
response = filter.filter(response);
// Return the response to the client
toClient = new BufferedWriter(new OutputStreamWriter(clientSocket.getOutputStream()));
toClient.write(response.getFullResponse());
toClient.flush();
toClient.close();
} catch (IOException e) {
e.printStackTrace();
continue;
}
}
}
The Response and Request classes are just very simple parsers for HTTP requests/responses. When I try to load a website through the proxy, I get:
EDIT 2
Here's another attempt. I added a debug print just before the line toClient.write(response.getFullResponse());
Accepted a request!
Request
==============================
GET http://t2.technion.ac.il/~srachum/ HTTP/1.1
Host: t2.technion.ac.il
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0) Gecko/20100101 Firefox/4.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-us,en;q=0.5
Accept-Encoding: gzip, deflate
Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7
Keep-Alive: 115
Proxy-Connection: keep-alive
I am going to try to connect to: t2.technion.ac.il at port 80
Connected.
Writing to the server's buffer...
flushed.
Getting a response...
Got a response!
Response
==============================
HTTP/1.1 200 OK
Date: Sat, 23 Apr 2011 15:54:08 GMT
Server: Apache/2.0.52 (Red Hat)
Last-Modified: Fri, 18 Mar 2011 23:45:24 GMT
ETag: "14928fc-877-49eca5f29cd00"
Accept-Ranges: bytes
Content-Length: 2167
Connection: close
Content-Type: text/html
X-Pad: avoid browser bug
<html>
...
</html>
I am going to write the following response:
HTTP/1.1 200 OK
Date: Sat, 23 Apr 2011 15:54:08 GMT
Server: Apache/2.0.52 (Red Hat)
Last-Modified: Fri, 18 Mar 2011 23:45:24 GMT
ETag: "14928fc-877-49eca5f29cd00"
Accept-Ranges: bytes
Content-Length: 2167
Connection: close
Content-Type: text/html
X-Pad: avoid browser bug
<html>
...
</html>
EDIT 3:
Request
package cs236369.proxy;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import cs236369.proxy.types.CacheControl;
import cs236369.proxy.types.HttpPatterns;
import cs236369.proxy.types.RequestHeader;
import cs236369.proxy.types.RequestType;
public class Request {
private String fullRequest = "";
private BufferedReader reader;
private RequestHeader requestHeader;
private String requestHost;
boolean busy = true;
private CacheControl cacheControl = CacheControl.CACHE;
public CacheControl getCacheControl() {
return cacheControl;
}
Request(String request) {
this(new ByteArrayInputStream(request.getBytes()));
}
Request(InputStream input){
reader = new BufferedReader(new InputStreamReader(input));
try {
while(!reader.ready()); //wait for initialization.
String line;
while ((line = reader.readLine()) != null) {
fullRequest += "\r\n" + line;
if (HttpPatterns.CACHE_CONTROL.matches(line)) {
cacheControl = (CacheControl) HttpPatterns.RESPONSE_CODE.process(line);
} else if (HttpPatterns.REQUEST_HEADER.matches(line)) {
requestHeader = (RequestHeader) HttpPatterns.REQUEST_HEADER.process(line);
} else if (HttpPatterns.HOST.matches(line)) {
requestHost = (String) HttpPatterns.HOST.process(line);
}
}
fullRequest = "\r\n" + fullRequest.trim() + "\r\n\r\n";
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
busy = false;
}
public String getFullRequest() {
return fullRequest;
}
public RequestType getRequestType() {
return requestHeader.type;
}
public String getRequestURL() {
return requestHeader.url;
}
public String getRequestProtocol() {
return requestHeader.protocol;
}
public String getRequestHost() {
return requestHost;
}
public boolean isBusy() {
return busy;
}
#Override
public String toString() {
return "Request\n==============================\n" + fullRequest;
}
}
Response
package cs236369.proxy;
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import cs236369.proxy.types.CacheControl;
import cs236369.proxy.types.HttpPatterns;
public class Response {
private String fullResponse = "";
private BufferedReader reader;
private boolean busy = true;
private int responseCode;
private CacheControl cacheControl;
public Response(String input) {
this(new ByteArrayInputStream(input.getBytes()));
}
public Response(InputStream input) {
reader = new BufferedReader(new InputStreamReader(input));
try {
while (!reader.ready());//wait for initialization.
String line;
while ((line = reader.readLine()) != null) {
fullResponse += "\r\n" + line;
if (HttpPatterns.RESPONSE_CODE.matches(line)) {
responseCode = (Integer) HttpPatterns.RESPONSE_CODE.process(line);
} else if (HttpPatterns.CACHE_CONTROL.matches(line)) {
cacheControl = (CacheControl) HttpPatterns.CACHE_CONTROL.process(line);
}
}
reader.close();
fullResponse = "\r\n" + fullResponse.trim() + "\r\n\r\n";
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
busy = false;
}
public CacheControl getCacheControl() {
return cacheControl;
}
public String getFullResponse() {
return fullResponse;
}
public boolean isBusy() {
return busy;
}
public int getResponseCode() {
return responseCode;
}
#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result
+ ((fullResponse == null) ? 0 : fullResponse.hashCode());
return result;
}
#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (!(obj instanceof Response))
return false;
Response other = (Response) obj;
if (fullResponse == null) {
if (other.fullResponse != null)
return false;
} else if (!fullResponse.equals(other.fullResponse))
return false;
return true;
}
#Override
public String toString() {
return "Response\n==============================\n" + fullResponse;
}
}
Once I was writing a program to send HTTP requests... My code was like this:
String host="www.google.com";
String request="GET / HTTP/1.0\r\nHost: "+host+"\r\nAccept-Encoding: gzip\r\n\r\n";
System.out.println(request);
Socket sock=new Socket(host,80);
InputStream inp=sock.getInputStream();
OutputStream outp=sock.getOutputStream();
outp.write(request.getBytes());
byte[] buff=new byte[999];
while(true){
int n=inp.read(buff);
if(n<0) break;
System.out.println(new String(buff,0,n));
}
inp.close();
outp.close();
sock.close();
This code works. At first sight, it looks like yours. You can try combining the two and watch when problems start to occur. Maybe there is something wrong in your Response parser?
P.S. Are you sure that the original request you got from the browser ends with two line breaks? In the log you posted it seems that there is only one blank line...
EDIT: I compiled your code with minor modifications, and it works fine. The things I did:
Commented out everything about CacheControl, filtering and RequestHeader because I don't have sources for these classes;
Added simple parsing of URL and host because it won't work without them;
Added a check to remove Accept-Encoding because many servers use gzip which gets corrupted in this program;
Added a check to stop parsing the request after two line breaks.
This program serves as a proxy server for Firefox and works fine with HTML code. Please compile my version and try whether it works for you. Probably there is something to do with Firefox settings?
Please note that this proxy server is corrupting binary data such as images and gzipped HTML. This must be caused by the usage of InputStreamReader and OutputStreamWriter; they convert bytes to characters and vice versa, this is good for text but for binary data you'd better use InputStream and OutputStream "as is".
public class AmirRachum {
public static void main(String[] args) {
try{
int port=38824;
ServerSocket listeningSocket=new ServerSocket(port);
System.out.println("Socket created");
while (true) {
Socket serverSocket;
Socket clientSocket;
BufferedWriter toClient;
BufferedWriter toServer;
try {
//The client is meant to put data on the port, read the socket.
clientSocket = listeningSocket.accept();
Request request = new Request(clientSocket.getInputStream());
System.out.println("Accepted a request!\n" + request);
while(request.busy);
//Make a connection to a real proxy.
//Host & Port - should be read from the request
URL url = null;
try {
url = new URL(request.getRequestURL());
} catch (MalformedURLException e){
url = new URL("http:\\"+request.getRequestHost()+request.getRequestURL());
}
//remove entry from cache if needed
/* if (!request.getCacheControl().equals(CacheControl.CACHE) && cache.containsRequest(request)) {
cache.remove(request);
}*/
Response response = null;
/* if (request.getRequestType() == RequestType.GET && request.getCacheControl().equals(CacheControl.CACHE) && cache.containsRequest(request)) {
response = cache.get(request);
} else*/ {
//Get the response from the destination
int remotePort = (url.getPort() == -1) ? 80 : url.getPort();
System.out.println("I am going to try to connect to: " + url.getHost() + " at port " + remotePort);
serverSocket = new Socket(url.getHost(), remotePort);
System.out.println("Connected.");
//write to the server - keep it open.
System.out.println("Writing to the server's buffer...");
toServer = new BufferedWriter(new OutputStreamWriter(serverSocket.getOutputStream()));
toServer.write(request.getFullRequest());
toServer.flush();
System.out.println("flushed.");
System.out.println("Getting a response...");
response = new Response(serverSocket.getInputStream());
System.out.println("Got a response!\n" + response);
//wait for the response
while(response.isBusy());
}
/* if (request.getRequestType() == RequestType.GET && request.getCacheControl().equals(CacheControl.CACHE)) {
cache.put(request, response);
}
response = filter.filter(response);*/
// Return the response to the client
toClient = new BufferedWriter(new OutputStreamWriter(clientSocket.getOutputStream()));
toClient.write(response.getFullResponse());
toClient.flush();
toClient.close();
} catch (IOException e) {
e.printStackTrace();
continue;
}
}
} catch (Exception e) {
e.printStackTrace();
}
}
public static class Request {
private String fullRequest = "";
private BufferedReader reader;
// private RequestHeader requestHeader;
private String requestHost;
private String requestURL;
boolean busy = true;
// private CacheControl cacheControl = CacheControl.CACHE;
/* public CacheControl getCacheControl() {
return cacheControl;
}*/
Request(String request) {
this(new ByteArrayInputStream(request.getBytes()));
}
Request(InputStream input){
reader = new BufferedReader(new InputStreamReader(input));
try {
while(!reader.ready()); //wait for initialization.
String line;
while ((line = reader.readLine()) != null) {
if(!line.startsWith("Accept-Encoding:")) fullRequest += "\r\n" + line;
/* if (HttpPatterns.CACHE_CONTROL.matches(line)) {
cacheControl = (CacheControl) HttpPatterns.RESPONSE_CODE.process(line);
} else if (HttpPatterns.REQUEST_HEADER.matches(line)) {
requestHeader = (RequestHeader) HttpPatterns.REQUEST_HEADER.process(line);
} else if (HttpPatterns.HOST.matches(line)) {
requestHost = (String) HttpPatterns.HOST.process(line);
}*/
if(line.startsWith("GET ")){requestURL=line.split(" ")[1];System.out.println("url \""+requestURL+"\"");}
if(line.startsWith("Host:")){requestHost=line.substring(6);System.out.println("Host \""+requestHost+"\"");}
if(line.length()==0){System.out.println("empty line");break;}
}
fullRequest = "\r\n" + fullRequest.trim() + "\r\n\r\n";
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
busy = false;
}
public String getFullRequest() {
return fullRequest;
}
/* public RequestType getRequestType() {
return requestHeader.type;
}*/
public String getRequestURL() {
return requestURL;
}
/* public String getRequestProtocol() {
return requestHeader.protocol;
}*/
public String getRequestHost() {
return requestHost;
}
public boolean isBusy() {
return busy;
}
//#Override
public String toString() {
return "Request\n==============================\n" + fullRequest;
}
}
public static class Response {
private String fullResponse = "";
private BufferedReader reader;
private boolean busy = true;
// private int responseCode;
// private CacheControl cacheControl;
public Response(String input) {
this(new ByteArrayInputStream(input.getBytes()));
}
public Response(InputStream input) {
reader = new BufferedReader(new InputStreamReader(input));
try {
while (!reader.ready());//wait for initialization.
String line;
while ((line = reader.readLine()) != null) {
fullResponse += "\r\n" + line;
/* if (HttpPatterns.RESPONSE_CODE.matches(line)) {
responseCode = (Integer) HttpPatterns.RESPONSE_CODE.process(line);
}/* else if (HttpPatterns.CACHE_CONTROL.matches(line)) {
cacheControl = (CacheControl) HttpPatterns.CACHE_CONTROL.process(line);
}*/
}
reader.close();
fullResponse = "\r\n" + fullResponse.trim() + "\r\n\r\n";
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
busy = false;
}
/* public CacheControl getCacheControl() {
return cacheControl;
}*/
public String getFullResponse() {
return fullResponse;
}
public boolean isBusy() {
return busy;
}
/* public int getResponseCode() {
return responseCode;
}*/
//#Override
public int hashCode() {
final int prime = 31;
int result = 1;
result = prime * result
+ ((fullResponse == null) ? 0 : fullResponse.hashCode());
return result;
}
//#Override
public boolean equals(Object obj) {
if (this == obj)
return true;
if (obj == null)
return false;
if (!(obj instanceof Response))
return false;
Response other = (Response) obj;
if (fullResponse == null) {
if (other.fullResponse != null)
return false;
} else if (!fullResponse.equals(other.fullResponse))
return false;
return true;
}
//#Override
public String toString() {
return "Response\n==============================\n" + fullResponse;
}
}
}