Uploading large files with user metadata to Amazon S3 using java sdk - java

I'm trying to use the Java API to upload large files, but I need to also add user metadata. If I just use
TransferManager tm = new TransferManager(new BasicAWSCredentials(accessKey, secretKey));
Upload upload = tm.upload(AmazonS3Manager.bucketName, imageKey, file);
upload.waitForCompletion();
Then everything works fine, but if I use:
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(file.length());
metadata.addUserMetadata("My key", "My value");
FileInputStream input = new FileInputStream(file);
Upload upload = tm.upload(AmazonS3Manager.bucketName, imageKey, input, metadata);
Then it doesn't work, and I get the following output in the console:
Jul 5, 2011 4:33:15 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: POST https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploads: null, ) Headers: (Content-Type: application/x-www-form-urlencoded; charset=utf-8, x-amz-meta-length: 312612077, )
Jul 5, 2011 4:33:16 PM com.amazonaws.http.AmazonHttpClient handleResponse
INFO: Received successful response: 200, AWS Request ID: 2A5B3538795CE730
Jul 5, 2011 4:33:16 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: PUT https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploadId: rwUlbXqtRyMUWiVYKzGqRQH90fVLi9_|Secret Key Removed|_w--, partNumber: 1, ) Headers: (Content-Length: 5242880, Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient handleResponse
INFO: Received successful response: 200, AWS Request ID: 5E5AF291FBDBDD36
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: PUT https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploadId: rwUlbXqtRyMUWiVYKzGqRQH90fVLi9_|Secret Key Removed|_w--, partNumber: 2, ) Headers: (Content-Length: 5242880, Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
Jul 5, 2011 4:34:00 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: I/O exception (java.io.IOException) caught when processing request: Read error
Jul 5, 2011 4:34:00 PM org.apache.http.impl.client.DefaultRequestDirector tryExecute
INFO: Retrying request
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient executeHelper
WARNING: Unable to execute HTTP request: null
Jul 5, 2011 4:34:00 PM com.amazonaws.http.AmazonHttpClient executeHelper
INFO: Sending Request: DELETE https://mybucket.s3.amazonaws.com /test.jpg Parameters: (uploadId: rwUlbXqtRyMUWiVYKzGqRQH90fVLi9_|Secret Key Removed|_w--, ) Headers: (Content-Type: application/x-www-form-urlencoded; charset=utf-8, )
Jul 5, 2011 4:34:01 PM com.amazonaws.http.AmazonHttpClient handleResponse
INFO: Received successful response: 204, AWS Request ID: 0EFC3F8D0FA6097E
Any help greatly appreciated!

You may also need to set content type and, possibly, checksum as it can't be determined from input stream. This has some information for similar situation http://docs.amazonwebservices.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/AmazonS3.html#putObject(com.amazonaws.services.s3.model.PutObjectRequest)

If you are trying to upload more than 5 GB file with updating metadata then amazon not allowing to do so. Because amazon s3 not providing copy operation for more then 5Gb file.
Amazon S3 recently started to support copy operation wtih multipart operation so for that i can not say you something yet.
https://forums.aws.amazon.com/thread.jspa?messageID=256605&#256605
Thanks

You can use following code for your file upload:
ObjectMetadata metadata = new ObjectMetadata();
metadata.setContentLength(file.length());
metadata.addUserMetadata("My key", "My value");
FileInputStream input = new FileInputStream(file);
PutObjectRequest putObjectRequest = new PutObjectRequest(AmazonS3Manager.bucketName, imageKey, input, metadata);
s3client.putObject(putObjectRequest.withCannedAcl(CannedAccessControlList.PublicRead));
Thanks

Related

HttpMethodDirectory executeWithRetry and SSLProtocolException in Java

I am using httpclient-3.0 library to parse data to cloud. When I run the application on my local machine (Windows 10), it works fine and the data gets parsed to the server and I receive success response, but when I deployed it on our server which runs on Windows server 2012 R2, it throws below error. I have used the same JDK as well I tried many ways like adding -Djdk.tls.client.protocols="TLSv1,TLSv1.1,TLSv1.2" in my java.security jdk file, but still the issue is not resolved.
Here is my code
PostMethod post = new PostMethod(apiUrl);
post.setParameter("authtoken", authToken);
post.setParameter("dateFormat", dateTimeFormat);
post.setParameter("data", emloyeesAttendanceJsonArr.toString());
HttpClient httpclient = new HttpClient();
// Configuring proxy
httpclient.getHostConfiguration().setProxy("**.**.**.**", ****);
try {
long timeTrace = System.currentTimeMillis();
int result = httpclient.executeMethod(post);
System.out.println(">> HTTP Response status code: "+result);
System.out.println(">> Response Time: "+(System.currentTimeMillis() - timeTrace));
.......
.......
.......
}
I appreciate any quick help and guidelines.
Here is the error I get
Mar 11, 2020 4:23:08 PM org.apache.commons.httpclient.HttpMethodDirector execute
WithRetry
INFO: I/O exception (javax.net.ssl.SSLProtocolException) caught when processing
request: Connection reset
Mar 11, 2020 4:23:08 PM org.apache.commons.httpclient.HttpMethodDirector execute
WithRetry
INFO: Retrying request
Mar 11, 2020 4:23:23 PM org.apache.commons.httpclient.HttpMethodDirector execute
WithRetry
INFO: I/O exception (javax.net.ssl.SSLProtocolException) caught when processing
request: Connection reset
Mar 11, 2020 4:23:23 PM org.apache.commons.httpclient.HttpMethodDirector execute
WithRetry
INFO: Retrying request
Mar 11, 2020 4:23:38 PM org.apache.commons.httpclient.HttpMethodDirector execute
WithRetry
INFO: I/O exception (javax.net.ssl.SSLProtocolException) caught when processing
request: Connection reset
Mar 11, 2020 4:23:38 PM org.apache.commons.httpclient.HttpMethodDirector execute
WithRetry
INFO: Retrying request
javax.net.ssl.SSLProtocolException: Connection reset
at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:126)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.ja
va:321)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.ja
va:264)
at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.ja
va:259)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:137)
at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:11
52)
at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocke
tImpl.java:1063)
at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl
.java:402)
at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketIm
pl.java:716)
at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSoc
ketImpl.java:970)
at java.base/java.io.BufferedOutputStream.flushBuffer(BufferedOutputStre
am.java:81)
at java.base/java.io.BufferedOutputStream.flush(BufferedOutputStream.jav
a:142)
at java.base/java.io.FilterOutputStream.flush(FilterOutputStream.java:15
3)
at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequ
estBody(EntityEnclosingMethod.java:502)
at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodB
ase.java:1973)
at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.j
ava:993)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(Htt
pMethodDirector.java:397)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMe
thodDirector.java:170)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
a:396)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.jav
a:324)
at af.aib.etl.AttendanceETL.fetchAndParseAttendanceRecord(AttendanceETL.
java:99)
at af.aib.attendance.ApplicationStartPoint.main(ApplicationStartPoint.ja
va:28)
Caused by: java.net.SocketException: Connection reset
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRe
cord.java:448)
at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInput
Record.java:165)
at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:108)
The local proxy and the server proxy were different that is why it was running fine on my local machine but not on the server, once I changed the proxy to the specific proxy which server was using then the application was working fine.

getting 500 internal server error on rest call

getting this error after hibernate outputs the data, any idea why this would be happening. please help!
Sep 07, 2016 12:07:00 PM org.hibernate.engine.jdbc.connections.internal.DriverManagerConnectionProviderImpl stop
INFO: HHH000030: Cleaning up connection pool [jdbc:postgresql://localhost:5432/bendb]
Sep 07, 2016 12:07:00 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 3 * Server responded with a response on thread http-nio-8080-exec-5
3 < 200
3 < Access-Control-Allow-Methods: GET, POST, DELETE, PUT
3 < Access-Control-Allow-Origin: *
3 < Allow: OPTIONS
3 < Content-Type: application/json
Sep 07, 2016 12:07:00 PM org.glassfish.jersey.filter.LoggingFilter log
INFO: 4 * Server responded with a response on thread http-nio-8080-exec-5
4 < 500
Sorry found a bug in my code so apparently the code change i made was trying to map all the junction tables (collections) in that get user rest call hence jersey just breaks out while attempting to do that. un Commenting that code and just passing the normal data sets solved the issue.

Google app engine mail sends without error, but cannot be found in inbox

I am using google app engine's mail to send email. Bellow is my code
Properties props = new Properties();
Session session = Session.getDefaultInstance(props, null);
Message msg = new MimeMessage(session);
try {
msg.setFrom(new InternetAddress(from));
msg.addRecipient(Message.RecipientType.TO, new InternetAddress(to));
msg.setSubject(subject);
msg.setText(message);
Transport.send(msg);
} catch (AddressException e) {
throw new InvalidMailInputException("Invalid email.");
} catch (MessagingException e) {
throw new InvalidMailInputException("Unable to send eamil at this time.");
}
Log:
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: MailService.send
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: From: myaddress#gmail.com
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: To: myaddress#gmail.com
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Reply-to: myaddress#gmail.com
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Subject: Appoint has been cancelled successfully (5)
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Body:
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Content-type: text/plain
Apr 13, 2013 5:26:29 PM com.google.appengine.api.mail.dev.LocalMailService log
INFO: Data length: 147
In the log message I have not recived any errors. However, I am not receiving email as well.
Appriceate your help.
It looks like you're using the local development server. In such case no mail is actually sent.
From:
https://developers.google.com/appengine/docs/java/mail/overview#Development_Server
When an application running in the development server calls the Mail service to send an email message, the message is printed to the log. The Java development server does not send the email message.

Problems when uploading large files to Amazon S3

I tried to use Amazon-SDK(Java) sample code S3TransferProgressSample.java to upload large files to Amazon-S3 storage (also posted here on AWS docs).
But when I am trying to upload 11 GB files, the upload is getting stuck at different points with the error message:
Unable to upload file to Amazon S3: Unable to upload part: Unable toexecute HTTP request: Unbuffered entity enclosing request can not be repeated " (attached screenshot).
It looks like that after IOException occurs SDK is not able to retry the request (see below).
Does anyone encounter this? What is the best-practice to resolve this? Any code is appreciated.
INFO: Received successful response: 200, AWS Request ID:
2B66E7669E24DA75<br> Jan 15, 2011 6:44:46 AM
com.amazonaws.http.HttpClient execute<br> INFO: Sending Request: PUT
s3.amazonaws.com /test_file_upload/autogenerated.txt Parameters:
(uploadId:
m9MqxzD484Ys1nifnX._IzJBGbCFIoT_zBg0xdd6kkZ4TAtmcG0lXQOE.LeiSEuqn6NjcosIQLXJeKzSnKllmw--, partNumber: 1494, )<br> Jan 15, 2011 6:45:10 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
**INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error**<br>
Jan 15, 2011 6:45:10 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
INFO: Retrying request<br> Jan 15, 2011 6:45:12 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:12 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
**INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error**<br>
Jan 15, 2011 6:45:12 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
INFO: Retrying request<br> Jan 15, 2011 6:45:13 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
**INFO: I/O exception (java.net.SocketException) caught when processing request: Connection reset by peer: socket write error**<br>
Jan 15, 2011 6:45:13 AM
org.apache.commons.httpclient.HttpMethodDirector executeWithRetry<br>
INFO: Retrying request<br> Jan 15, 2011 6:45:13 AM
com.amazonaws.http.HttpClient execute<br>
**WARNING: Unable to execute HTTP request: Unbuffered entity enclosing request can not be repeated.**<br> Jan 15, 2011 6:45:14 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:14 AM com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:14 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:15 AM com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:16 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:16 AM
com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:17 AM
com.amazonaws.http.HttpClient execute<br> WARNING: Unable to execute
HTTP request: Unbuffered entity enclosing request can not be
repeated.<br> Jan 15, 2011 6:45:19 AM com.amazonaws.http.HttpClient
execute<br> WARNING: Unable to execute HTTP request: Unbuffered entity
enclosing request can not be repeated.<br> Jan 15, 2011 6:45:19 AM
com.amazonaws.http.HttpClient execute<br> ....<br> Jan 15, 2011
6:45:21 AM com.amazonaws.http.HttpClient handleResponse<br>
**INFO: Received successful response: 204, AWS Request ID: E794B8FCA4C3D007**<br> Jan 15, 2011 6:45:21 AM
com.amazonaws.http.HttpClient execute<br> ...<br> Jan 15, 2011 6:45:19
AM com.amazonaws.http.HttpClient execute<br> INFO: Sending Request:
DELETE s3.amazonaws.com /test_file_upload/autogenerated.txt
Parameters:<br> ...<br> Jan 15, 2011 6:47:01 AM
com.amazonaws.http.HttpClient handleErrorResponse<br> INFO: Received
error response: Status Code: 404, AWS Request ID: 0CE25DFE767CC595,
AWS Error Code: NoSuchUpload, AWS Error Message: The specified upload
does not exist. The upload ID may be invalid, or the upload may have
been aborted or completed.<br>
Try using the low level API.
This will give you far more control when things go wrong, as they are likely to do with an 11GB file.
Requests to and from S3 do fail from time to time. With the low level API, you'll be able to retry a part of the upload if it fails.
Refactoring the example in the Amazon docs a bit:
// Step 2: Upload parts.
long filePosition = 0;
for (int i = 1; filePosition < contentLength; i++) {
// Last part can be less than 5 MB. Adjust part size.
partSize = Math.min(partSize, (contentLength - filePosition));
// Create request to upload a part.
UploadPartRequest uploadRequest = new UploadPartRequest()
.withBucketName(existingBucketName).withKey(keyName)
.withUploadId(initResponse.getUploadId()).withPartNumber(i)
.withFileOffset(filePosition)
.withFile(file)
.withPartSize(partSize);
// repeat the upload until it succeeds.
boolean anotherPass;
do {
anotherPass = false; // assume everythings ok
try {
// Upload part and add response to our list.
partETags.add(s3Client.uploadPart(uploadRequest).getPartETag());
} catch (Exception e) {
anotherPass = true; // repeat
}
} while (anotherPass);
filePosition += partSize;
}
// Step 3: complete.
CompleteMultipartUploadRequest compRequest = new
CompleteMultipartUploadRequest(
existingBucketName,
keyName,
initResponse.getUploadId(),
partETags);
s3Client.completeMultipartUpload(compRequest);
Note: I am not a java developer so I could have messed things up syntactically, but hopefully this gets you going in the right direction. Also, you'll want to add in a 'retry counter' to prevent an endless loop if the upload repeatedly fails.
As a side note, 404 errors can be thrown if you try to do a multipart upload to a key that is already under a multipart upload.
I think you should try Multipart API supported by AWS.
Check this out : http://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/s3/transfer/TransferManager.html
The answer of Geoff Appleford works for me.
However, I would add a && retryCount < MAX_RETRIES to the while loop control statement and increment of the retryCount on every exception caught inside the while.
Aviad
I wanted to add a comment to Geoff Appleford's answer but SO wouldn't allow me to. In general his answer to use low level API works fine but even if we do now have a do-while loop the way for loop is designed there is in-built retry logic. In his code snippet the file position increases only when there is a success otherwise you are uploading the same part again.

HTTP Builder/Groovy - lost 302 (redirect) handling?

I am reading here
http://groovy.codehaus.org/modules/http-builder/doc/handlers.html
"In cases where a response sends a redirect status code, this is handled internally by Apache HttpClient, which by default will simply follow the redirect by re-sending the request to the new URL. You do not need to do anything special in order to follow 302 responses."
This seems to work fine when I simply use the get() or post() methods without a closure.
However, when I use a closure, I seem to lose 302 handling. Is there some way I can handle this myself? Thank you
p.s. Here is my log output showing it is a 302 response
[java] FINER: resp.statusLine: "HTTP/1.1 302 Found"
Here is the relevant code:
// Copyright (C) 2010 Misha Koshelev. All Rights Reserved.
package com.mksoft.fbbday.main
import groovyx.net.http.ContentType
import java.util.logging.Level
import java.util.logging.Logger
class HTTPBuilder {
def dataDirectory
HTTPBuilder(dataDirectory) {
this.dataDirectory=dataDirectory
}
// Main logic
def logger=Logger.getLogger(this.class.name)
def closure={resp,reader->
logger.finer("resp.statusLine: \"${resp.statusLine}\"")
if (logger.isLoggable(Level.FINEST)) {
def respHeadersString='Headers:';
resp.headers.each() { header->respHeadersString+="\n\t${header.name}=\"${header.value}\"" }
logger.finest(respHeadersString)
}
def text=reader.text
def lastHtml=new File("${dataDirectory}${File.separator}last.html")
if (lastHtml.exists()) {
lastHtml.delete()
}
lastHtml<<text
new XmlSlurper(new org.cyberneko.html.parsers.SAXParser()).parseText(text)
}
def processArgs(args) {
if (logger.isLoggable(Level.FINER)) {
def argsString='Args:';
args.each() { arg->argsString+="\n\t${arg.key}=\"${arg.value}\"" }
logger.finer(argsString)
}
args.contentType=groovyx.net.http.ContentType.TEXT
args
}
// HTTPBuilder methods
def httpBuilder=new groovyx.net.http.HTTPBuilder ()
def get(args) {
httpBuilder.get(processArgs(args),closure)
}
def post(args) {
args.contentType=groovyx.net.http.ContentType.TEXT
httpBuilder.post(processArgs(args),closure)
}
}
Here is a specific tester:
#!/usr/bin/env groovy
import groovyx.net.http.HTTPBuilder
import groovyx.net.http.Method
import static groovyx.net.http.ContentType.URLENC
import java.util.logging.ConsoleHandler
import java.util.logging.Level
import java.util.logging.Logger
// MUST ENTER VALID FACEBOOK EMAIL AND PASSWORD BELOW !!!
def email=''
def pass=''
// Remove default loggers
def logger=Logger.getLogger('')
def handlers=logger.handlers
handlers.each() { handler->logger.removeHandler(handler) }
// Log ALL to Console
logger.setLevel Level.ALL
def consoleHandler=new ConsoleHandler()
consoleHandler.setLevel Level.ALL
logger.addHandler(consoleHandler)
// Facebook - need to get main page to capture cookies
def http = new HTTPBuilder()
http.get(uri:'http://www.facebook.com')
// Login
def html=http.post(uri:'https://login.facebook.com/login.php?login_attempt=1',body:[email:email,pass:pass])
assert html==null
// Why null?
html=http.post(uri:'https://login.facebook.com/login.php?login_attempt=1',body:[email:email,pass:pass]) { resp,reader->
assert resp.statusLine.statusCode==302
// Shouldn't we be redirected???
// http://groovy.codehaus.org/modules/http-builder/doc/handlers.html
// "In cases where a response sends a redirect status code, this is handled internally by Apache HttpClient, which by default will simply follow the redirect by re-sending the request to the new URL. You do not need to do anything special in order to follow 302 responses. "
}
Here are relevant logs:
FINE: Receiving response: HTTP/1.1 302 Found
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << HTTP/1.1 302 Found
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Cache-Control: private, no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Expires: Sat, 01 Jan 2000 00:00:00 GMT
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Location: http://www.facebook.com/home.php?
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << P3P: CP="DSP LAW"
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Pragma: no-cache
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Set-Cookie: datr=1275687438-9ff6ae60a89d444d0fd9917abf56e085d370277a6e9ed50c1ba79; expires=Sun, 03-Jun-2012 21:37:24 GMT; path=/; domain=.facebook.com
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Set-Cookie: lxe=koshelev%40post.harvard.edu; expires=Tue, 28-Sep-2010 15:24:04 GMT; path=/; domain=.facebook.com; httponly
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Set-Cookie: lxr=deleted; expires=Thu, 04-Jun-2009 21:37:23 GMT; path=/; domain=.facebook.com; httponly
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Set-Cookie: pk=183883c0a9afab1608e95d59164cc7dd; path=/; domain=.facebook.com; httponly
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Content-Type: text/html; charset=utf-8
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << X-Cnection: close
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Date: Fri, 04 Jun 2010 21:37:24 GMT
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.DefaultClientConnection receiveResponseHeader
FINE: << Content-Length: 0
Jun 4, 2010 4:37:22 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
FINE: Cookie accepted: "[version: 0][name: datr][value: 1275687438-9ff6ae60a89d444d0fd9917abf56e085d370277a6e9ed50c1ba79][domain: .facebook.com][path: /][expiry: Sun Jun 03 16:37:24 CDT 2012]".
Jun 4, 2010 4:37:22 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
FINE: Cookie accepted: "[version: 0][name: lxe][value: koshelev%40post.harvard.edu][domain: .facebook.com][path: /][expiry: Tue Sep 28 10:24:04 CDT 2010]".
Jun 4, 2010 4:37:22 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
FINE: Cookie accepted: "[version: 0][name: lxr][value: deleted][domain: .facebook.com][path: /][expiry: Thu Jun 04 16:37:23 CDT 2009]".
Jun 4, 2010 4:37:22 PM org.apache.http.client.protocol.ResponseProcessCookies processCookies
FINE: Cookie accepted: "[version: 0][name: pk][value: 183883c0a9afab1608e95d59164cc7dd][domain: .facebook.com][path: /][expiry: null]".
Jun 4, 2010 4:37:22 PM org.apache.http.impl.client.DefaultRequestDirector execute
FINE: Connection can be kept alive indefinitely
Jun 4, 2010 4:37:22 PM groovyx.net.http.HTTPBuilder doRequest
FINE: Response code: 302; found handler: post302$_run_closure2#7023d08b
Jun 4, 2010 4:37:22 PM groovyx.net.http.HTTPBuilder doRequest
FINEST: response handler result: null
Jun 4, 2010 4:37:22 PM org.apache.http.impl.conn.SingleClientConnManager releaseConnection
FINE: Releasing connection org.apache.http.impl.conn.SingleClientConnManager$ConnAdapter#605b28c9
You can see there is clearly a location argument.
Thank you
Misha
I've had the same problem with HTTPBuilder until I realized that the HTTP/1.1 spec states:
Redirection 3xx
[..]
This class of status code indicates
that further action needs to be
taken by the user agent in order to
fulfill the request. The action
required MAY be carried out by the
user agent without interaction with
the user if and only if the method
used in the second request is GET
or HEAD.
302 Found
[..]
If the 302 status code is received in response to a request other
than GET or HEAD, the user agent MUST NOT automatically redirect the
request unless it can be confirmed by the user, since this might
change the conditions under which the request was issued.
Essentially this means that the request following a POST and 302 redirect won't work automatically and will require user intervention if the HTTP/1.1 spec is followed by the letter. Not all Http clients follow this practice, in fact most browsers do not. However the Apache Http Client (which is the underlying Http client for HttpBuilder) is spec compliant. There is an issue in the Apache Http Client bugtracker that contains more information and a possible solution for the problem.
void test_myPage_shouldRedirectToLogin() {
def baseURI = "http://servername"
def httpBuilder = new HTTPBuilder(baseURI)
// Make sure that HttpClient doesn't perform a redirect
def dontHandleRedirectStrategy = [
getRedirect : { request, response, context -> null},
isRedirected : { request, response, context -> false}
]
httpBuilder.client.setRedirectStrategy(dontHandleRedirectStrategy as RedirectStrategy)
// Execute a GET request and expect a redirect
httpBuilder.request(Method.GET, ContentType.TEXT) {
req ->
uri.path = '/webapp/de/de/myPage'
response.success = { response, reader ->
assertThat response.statusLine.statusCode, is(302)
assertThat response.headers['Location'].value, startsWith("${baseURI}/webapp/login")
}
response.failure = { response, reader ->
fail("Expected redirect but received ${response.statusLine} \n ${reader}")
}
}
}
302 status coming because, after action on any link redirected url not follow by HttpBuilder so we need to add "RedirectStrategy" explicitly.
What other headers do you see when you process the 302 response? If you were to turn on http client logging you'd expect to see HttpClient process the 302 response and automatically request URL in the Location header. What do you see when you process that URL? Does it work for any URL?
Try http://www.sun.com (it redirects to Oracle now.) I'm just wondering if the server you're working with is doing something wonky like sending a 302 with no Location header.

Categories