I am trying to do a PHP GET request to a website:
The problem is that this website will only process my request if I attach Cookie information to the header of the request.
Or in picture terms, if I disable cookies in my browser, I get this:
Which means the website recognises that it's my first time 'visiting' the site.
Problem is, that if I now use the search bar on the top right, it will not process this request:
it will just show the same (general) screen.
E.g.: if I have cookies disabled and I search for "AAPL", it will not show any results.
Now if I have cookies enabled, the request is handled just fine:
And so the "AAPL" results are shown.
You can try this yourself as well:
With cookies enabled, visit http://www.pennystocktweets.com/user_posts/feeds?cat=search&lptyp=prep&usrstk=AAPL
With cookies disabled, visit the link again: http://www.pennystocktweets.com/user_posts/feeds?cat=search&lptyp=prep&usrstk=AAPL
Now compare the responses, only the first one is correct.
This means that the website only works after the client has downloaded a cookie, and then has made another (new) GET request to the server with this Cookie information attached.
(Does this imply that the website needs a session-cookie to function correctly?)
Now what I'm trying to do is imitate the request with Apache HttpClient like so:
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.Date;
import java.util.List;
import java.util.StringTokenizer;
import org.apache.http.HttpResponse;
import org.apache.http.NameValuePair;
import org.apache.http.client.HttpClient;
import org.apache.http.client.entity.UrlEncodedFormEntity;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.impl.client.DefaultHttpClient;
import org.apache.http.message.BasicNameValuePair;
public class downloadTweets {
private String cookies;
private HttpClient client = new DefaultHttpClient();
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws Exception {
String ticker = "AAPL";
String lptyp = "prep";
int opid = 0;
int lpid = 0;
downloadTweets test = new downloadTweets();
String url = test.constructURL(ticker, lptyp, opid, lpid);
// make sure cookies is turn on
CookieHandler.setDefault(new CookieManager());
downloadTweets http = new downloadTweets();
String page = http.GetPageContent(url, ticker);
System.out.println(page);
}
public String constructURL(String ticker, String lptyp, int opid, int lpid)
{
String link = "http://www.pennystocktweets.com/user_posts/feeds?cat=search" +
"&lptyp=" + lptyp +
"&usrstk=" + ticker;
if (opid != 0)
{
link = link +
"&opid=" + opid +
"&lpid=" + lpid;
}
return link;
}
private String GetPageContent(String url, String ticker) throws Exception {
HttpGet request = new HttpGet(url);
String RefererLink = "http://www.pennystocktweets.com/search/post/" + ticker.toUpperCase();
request.setHeader("Host", "www.pennystocktweets.com");
request.setHeader("Connection", "Keep-alive");
request.setHeader("Accept", "*/*");
request.setHeader("X-Requested-With", "XMLHttpRequest");
request.setHeader("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/31.0.1650.57 Safari/537.36");
request.setHeader("Referer", RefererLink);
request.setHeader("Accept-Language", "nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4,fr;q=0.2");
HttpResponse response = client.execute(request);
int responseCode = response.getStatusLine().getStatusCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader rd = new BufferedReader(
new InputStreamReader(response.getEntity().getContent()));
StringBuffer result = new StringBuffer();
String line = "";
while ((line = rd.readLine()) != null) {
result.append(line);
}
// set cookies
setCookies(response.getFirstHeader("Set-Cookie") == null ? "" :
response.getFirstHeader("Set-Cookie").toString());
return result.toString();
}
public String getCookies() {
return cookies;
}
public void setCookies(String cookies) {
this.cookies = cookies;
}
}
Now, the same thing holds: if I attach (my) cookie information, the response works just fine, and if I don't the response doesn't work.
But I don't know how to get the cookie information and then use it in a new GET request.
So my question is:
How can I make 2 requests to a website such that:
On the first GET request, I get cookie information from the website and store this in my Java program
On the second GET request, I use the stored cookie information (as a Header) to make a new request.
Note: I don't know if the cookie is a normal cookie or a session cookie but I suspect it's a session-cookie!
All help is greatly appreciated!
As the documents of Apache commons httpclient states in the HttpClient Cookie handling part:
HttpClient supports automatic management of cookies, including allowing the server to set cookies and automatically return them to the server when required. It is also possible to manually set cookies to be sent to the server.
Whenever the http client receives cookies they are persisted into HttpState and added automatically to the new request. This is the default behavior.
In the following example code, we can see the cookies returned by two GET requests. We can't see directly the cookies sent to the server, but we can use a tool such as a protocol/net sniffer or ngrep to see the data transmitted over the network:
import java.io.IOException;
import org.apache.commons.httpclient.Cookie;
import org.apache.commons.httpclient.HttpClient;
import org.apache.commons.httpclient.HttpException;
import org.apache.commons.httpclient.HttpMethod;
import org.apache.commons.httpclient.HttpState;
import org.apache.commons.httpclient.cookie.CookiePolicy;
import org.apache.commons.httpclient.methods.GetMethod;
public class HttpTest {
public static void main(String[] args) throws HttpException, IOException {
String url = "http://www.whatarecookies.com/cookietest.asp";
HttpClient client = new HttpClient();
client.getParams().setCookiePolicy(CookiePolicy.BROWSER_COMPATIBILITY);
HttpMethod method = new GetMethod(url);
int res = client.executeMethod(method);
System.out.println("Result: " + res);
printCookies(client.getState());
method = new GetMethod(url);
res = client.executeMethod(method);
System.out.println("Result: " + res);
printCookies(client.getState());
}
public static void printCookies(HttpState state){
System.out.println("Cookies:");
Cookie[] cookies = state.getCookies();
for (Cookie cookie : cookies){
System.out.println(" " + cookie.getName() + ": " + cookie.getValue());
}
}
}
This is the output:
Result: 200
Cookies:
active_template::468: %2Fresponsive%2Fthree_column_inner_ad3b74de5a1c2f311bee7bca5c368aaa4e:b326b5062b2f0e69046810717534cb09
Result: 200
Cookies:
active_template::468: %2Fresponsive%2Fthree_column_inner_ad%2C+3b74de5a1c2f311bee7bca5c368aaa4e%3Db326b5062b2f0e69046810717534cb09
3b74de5a1c2f311bee7bca5c368aaa4e: b326b5062b2f0e69046810717534cb09
Here is an excerpt of ngrep:
MacBook$ sudo ngrep -W byline -d en0 "" host www.whatarecookies.com
interface: en0 (192.168.11.0/255.255.255.0)
filter: (ip) and ( dst host www.whatarecookies.com )
#####
T 192.168.11.70:56267 -> 54.228.218.117:80 [AP]
GET /cookietest.asp HTTP/1.1.
User-Agent: Jakarta Commons-HttpClient/3.1.
Host: www.whatarecookies.com.
.
####
T 54.228.218.117:80 -> 192.168.11.70:56267 [A]
HTTP/1.1 200 OK.
Server: nginx/1.4.0.
Date: Wed, 27 Nov 2013 10:22:14 GMT.
Content-Type: text/html; charset=iso-8859-1.
Content-Length: 36397.
Connection: keep-alive.
Vary: Accept-Encoding.
Vary: Cookie,Host,Accept-Encoding.
Set-Cookie: active_template::468=%2Fresponsive%2Fthree_column_inner_ad; expires=Fri, 29-Nov-2013 10:22:01 GMT; path=/; domain=whatarecookies.com; httponly.
Set-Cookie: 3b74de5a1c2f311bee7bca5c368aaa4e=b326b5062b2f0e69046810717534cb09; expires=Thu, 27-Nov-2014 10:22:01 GMT.
X-Middleton-Response: 200.
Cache-Control: max-age=0, no-cache.
X-Mod-Pagespeed: 1.7.30.1-3609.
.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/1998/REC-html40-19980424/loose.dtd">
...
##
T 192.168.11.70:56267 -> 54.228.218.117:80 [AP]
GET /cookietest.asp HTTP/1.1.
User-Agent: Jakarta Commons-HttpClient/3.1.
Host: www.whatarecookies.com.
Cookie: active_template::468=%2Fresponsive%2Fthree_column_inner_ad.
Cookie: 3b74de5a1c2f311bee7bca5c368aaa4e=b326b5062b2f0e69046810717534cb09.
.
##
T 54.228.218.117:80 -> 192.168.11.70:56267 [A]
HTTP/1.1 200 OK.
Server: nginx/1.4.0.
Date: Wed, 27 Nov 2013 10:22:18 GMT.
Content-Type: text/html; charset=iso-8859-1.
Content-Length: 54474.
Connection: keep-alive.
Vary: Accept-Encoding.
Vary: Cookie,Host,Accept-Encoding.
Set-Cookie: active_template::468=%2Fresponsive%2Fthree_column_inner_ad%2C+3b74de5a1c2f311bee7bca5c368aaa4e%3Db326b5062b2f0e69046810717534cb09; expires=Fri, 29-Nov-2013 10:22:05 GMT; path=/; domain=whatarecookies.com; httponly.
Set-Cookie: 3b74de5a1c2f311bee7bca5c368aaa4e=b326b5062b2f0e69046810717534cb09; expires=Thu, 27-Nov-2014 10:22:05 GMT.
X-Middleton-Response: 200.
Cache-Control: max-age=0, no-cache.
X-Mod-Pagespeed: 1.7.30.1-3609.
.
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/1998/REC-html40-19980424/loose.dtd">
...
Related
Hi im building a REST API to upload files.
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import javax.ws.rs.Consumes;
import javax.ws.rs.GET;
import javax.ws.rs.POST;
import javax.ws.rs.Path;
import javax.ws.rs.core.MediaType;
import javax.ws.rs.core.Response;
import org.apache.http.HttpEntity;
#Path("/api")
public class RestAPI {
private final String UPLOADED_FILE_PATH = "C:/ProgramData/XXXX/";
#GET
public String getFile() {
return "Loading File...";
}
#POST
#Path("/image-upload")
#Consumes(MediaType.MULTIPART_FORM_DATA)
public Response uploadFile(HttpEntity input) throws IOException {
// Do stuff
return Response.status(200).entity("Uploaded file name : " + "").build();
}
Uploader Class:
import java.io.File;
import org.apache.http.HttpEntity;
import org.apache.http.client.methods.CloseableHttpResponse;
import org.apache.http.client.methods.HttpPost;
import org.apache.http.entity.ContentType;
import org.apache.http.entity.mime.MultipartEntityBuilder;
import org.apache.http.entity.mime.content.FileBody;
import org.apache.http.entity.mime.content.StringBody;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClientBuilder;
public class DemoFileUploader {
public static void main(String args[]) throws Exception {
DemoFileUploader fileUpload = new DemoFileUploader();
File file = new File("C:/Users/tdr/Desktop/TestFile.txt");
// Upload the file
fileUpload.executeMultiPartRequest("http://localhost:8080/MediaHandler/mediahandler/api/image-upload",
file, file.getName(), "File Uploaded :: TestFile.txt");
}
public void executeMultiPartRequest(String urlString, File file, String fileName, String fileDescription)
throws Exception {
// default client builder
CloseableHttpClient httpClient = HttpClientBuilder.create().build();
HttpPost postRequest = new HttpPost(urlString);
try {
FileBody fileBody = new FileBody(file, ContentType.DEFAULT_BINARY);
// Set various attributes
HttpEntity multiPartEntity = MultipartEntityBuilder.create()
.addPart("fileDescription",
new StringBody(fileDescription != null ? fileDescription : "",
ContentType.MULTIPART_FORM_DATA))
.addPart("fileName", new StringBody(fileName != null ? fileName : file.getName(),
ContentType.MULTIPART_FORM_DATA))
.addPart("attachment", fileBody).build();
// Set to request body
postRequest.setEntity(multiPartEntity);
System.out.println("Sending Request....");
System.out.println("Request: " + postRequest);
System.out.println("Request Entity: " + postRequest.getEntity().getContentType());
// Send request
CloseableHttpResponse response = httpClient.execute(postRequest);
System.out.println("Request executed.");
// Verify response if any
if (response != null) {
System.out.println("Response Status Code: " + response.getStatusLine().getStatusCode());
System.out.println("Response: " + response);
System.out.println("Response Entity: " + response);
}
} catch (Exception ex) {
ex.printStackTrace();
}
}
}
and i get the following output:
Sending Request....
Request: POST http://localhost:8080/MediaHandler/mediahandler/api/image-upload HTTP/1.1
Request Entity: Content-Type: multipart/form-data; boundary=eINJSk3iptTJP7wf-cXlS-uznnnGMl99FyFmlet
Request executed.
Response Status Code: 415
Response: HTTP/1.1 415 [Content-Type: text/html;charset=utf-8, Content-Language: de, Content-Length: 785, Date: Wed, 31 Mar 2021 12:19:35 GMT, Keep-Alive: timeout=20, Connection: keep-alive]
Response Entity: HTTP/1.1 415 [Content-Type: text/html;charset=utf-8, Content-Language: de, Content-Length: 785, Date: Wed, 31 Mar 2021 12:19:35 GMT, Keep-Alive: timeout=20, Connection: keep-alive]
I tried to follow all examples i found, but all of them are similar to my code. Do u guys can tell me where the bug is?
im sending a multipart/form-data and my restapi is expecting multipart/form-data...
Can you remove this annotation:
#Consumes(MediaType.MULTIPART_FORM_DATA)
or request header must contains Content-Type: MediaType.MULTIPART_FORM_DATA
I spent a fair bit of time looking for the solution but couldnt find any. I am sending HTTP request and need to retrieve certain information from the http header. I'm doing manually with socket library. How do I omit other information and only retrieve information that i want to display from the http header? Is there any way or what library to use so I can format the http header?
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.Socket;
import java.util.*;
import java.text.SimpleDateFormat;
public class socketv1 {
public static void main(String[] args) throws Exception {
InetAddress addr = InetAddress.getByName("www.google.com");
Socket socket = new Socket(addr, 80);
//socket.bind (new InetSocketAddress (socket.getLocalAddress().getHostAddress(), 0));
//socket.connect (new InetSocketAddress (socket.getInetAddress().getHostAddress(), 80), 1000);
boolean autoflush = true;
System.out.println("URL requested: " + socket.getInetAddress().getHostName());
System.out.println("Client: " + socket.getLocalAddress().getHostAddress() + " " + socket.getLocalPort());
System.out.println("Server: " + socket.getInetAddress().getHostAddress() + " " + socket.getPort());
System.out.println("");
PrintWriter out = new PrintWriter(socket.getOutputStream(), autoflush);
BufferedReader in = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// send an HTTP request to the web server
out.println("GET / HTTP/1.1");
out.println("Host: www.google.com:80");
out.print ("Date Accessed: date" + "\r\n");
out.println("Connection: Close");
out.println();
// read the response
boolean loop = true;
StringBuilder sb = new StringBuilder(8096);
while (loop) {
if (in.ready()) {
int i = 0;
while (i != -1) {
i = in.read();
sb.append((char) i);
}
loop = false;
}
}
System.out.println(sb.toString());
socket.close();
}
}
Expected:
URL requested: www.google.com
Client: 192.168.1.110 53954
Server: 172.217.167.68 80
Date Accessed: 24/03/2019 14:53:59 AEST
Actual:
URL requested: www.google.com
Client: 192.168.1.110 53954
Server: 172.217.167.68 80
HTTP/1.1 200 OK
Date: Sun, 24 Mar 2019 12:52:16 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Server: gws
X-XSS-Protection: 1; mode=block
X-Frame-Options: SAMEORIGIN
Set-Cookie: 1P_JAR=2019-03-24-12; expires=Tue, 23-Apr-2019 12:52:16 GMT; path=/; domain=.google.com
Set-Cookie: NID=179=E-IxZRjdPtqBWrSM-bdqfdYDdzPlEaC7gkdFKxYoGRJpBIdD__1ZQiVFPrSuoEqme-yBucdcczqMw_EOJaUpfuXYy1auuQWd1-AZQ6WKmQR_pz8kFZqemdm4Bc-yH0P1Zc7ODKWEmtHKpE3nT2kqIhwfp7pLZrYd3YGMrZFUwZs; expires=Mon, 23-Sep-2019 12:52:16 GMT; path=/; domain=.google.com; HttpOnly
Accept-Ranges: none
Vary: Accept-Encoding
Connection: close
Do you have to work with sockets, can you not do something like:
import java.net.URL;
import java.net.URLConnection;
public class socketv1 {
public static void main(String[] args) throws Exception {
URL url = new URL("http://www.google.com");
URLConnection c = url.openConnection();
System.out.println(c.getHeaderField("Content-Type"));
}
}
so work with the connection from the start?
When I'm using Apache HttpClient and loading a webpage via GET request after the page is loaded in the response I have the headers that are different from ones I have when loading the same page in browser. Here is the example of the page: http://empoweredfoundation.org/wp-login.php?action=register
In browser I have the following headers:
Status code: 302
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Location: http://empoweredfoundation.org/register/
Content-Encoding: gzip
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:04:35 GMT
Age: 0
Vary: User-Agent
X-Cache: uncached
X-Cache-Hit: MISS
X-Backend: all_requests
When I use HttpClient in my application I have these headers in response:
Status code: 200
Content-Type: text/html; charset=UTF-8
X-Port: port_10210
X-Cacheable: YES:Forced
Transfer-Encoding: chunked
Date: Thu, 22 Feb 2018 04:44:58 GMT
Age: 28224
Vary: Accept-Encoding, User-Agent
X-Cache: cached
X-Cache-Hit: HIT
X-Backend: all_requests
Server: nginx/1.12.1
Date: Thu, 22 Feb 2018 04:45:24 GMT
Content-Type: text/html; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
X-Powered-By: PHP/5.4.45
So, I should have a 302 status code but I have 200. And also I see other headers are different than the ones from the browser. I can't figure out why and what should I do to fix this.
Here is the code:
HttpClient httpclient = null;
HttpClientBuilder builder = HttpClients.custom();
Builder requestConfigBuilder = RequestConfig.custom();
// here goes the cookie store creation, ssl configuration etc
builder.setDefaultRequestConfig(requestConfigBuilder.build());
httpclient = builder.build();
HttpResponse response = null;
HttpGet httpget = null;
Escaper escaper = UrlEscapers.urlFragmentEscaper();
httpget = new HttpGet(escaper.escape(url));
httpget.getParams().setParameter("http.socket.timeout", new Integer(socketTimeout));
httpget.getParams().setParameter("http.connection.timeout", new Integer(connectTimeout));
httpget.addHeader("Accept", "text/html, application/xml;q=0.9, application/xhtml+xml, image/png, image/jpeg, image/gif, image/x-xbitmap, */*;q=0.1");
httpget.addHeader("Accept-Language", "en-US,en;q=0.9");
httpget.addHeader("Accept-Encoding", "identity, *;q=0");
response = httpclient.execute(httpget);
I also tried CloseableHttpClient, had the same result.
I resolved this issue, this solution works: https://memorynotfound.com/apache-httpclient-redirect-handling-example/
I still have 200 status code, not 302. But now I can handle 302 redirects (even when response.getStatusLine() shows 200).
Here is the code from the article:
package com.memorynotfound.httpclient;
import org.apache.http.HttpHost;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.client.protocol.HttpClientContext;
import org.apache.http.client.utils.URIUtils;
import org.apache.http.impl.client.CloseableHttpClient;
import org.apache.http.impl.client.HttpClients;
import org.apache.http.impl.client.LaxRedirectStrategy;
import java.io.IOException;
import java.net.URI;
import java.net.URISyntaxException;
import java.util.List;
/**
* This example demonstrates the use of {#link HttpGet} request method.
* and handling redirect strategy with {#link LaxRedirectStrategy}
*/
public class HttpClientRedirectHandlingExample {
public static void main(String... args) throws IOException, URISyntaxException {
CloseableHttpClient httpclient = HttpClients.custom()
.setRedirectStrategy(new LaxRedirectStrategy())
.build();
try {
HttpClientContext context = HttpClientContext.create();
HttpGet httpGet = new HttpGet("http://httpbin.org/redirect/3");
System.out.println("Executing request " + httpGet.getRequestLine());
System.out.println("----------------------------------------");
httpclient.execute(httpGet, context);
HttpHost target = context.getTargetHost();
List<URI> redirectLocations = context.getRedirectLocations();
URI location = URIUtils.resolve(httpGet.getURI(), target, redirectLocations);
System.out.println("Final HTTP location: " + location.toASCIIString());
} finally {
httpclient.close();
}
}
}
And also I added builder.setRedirectStrategy(new LaxRedirectStrategy()); when created the HttpClient class object.
If you know any solution to get the correct status code (which should be 302), please tell me.
I am trying to send a HTTP/1.1 POST to http://uploaded.net/io/login, using HttpURLConnection, in order to login to the website with my username and password.
For some reason it appears like byte[] postData is not being sent to the server.
I receive the response code and message 200 OK, but the server replies with the JSON-formatted error message {"err":"Please type in id and password"} as if I never actually sent the data.
I've tried different methods (sending a String, byte[] etc., using different output writers) of sending the data but so far none of them have worked and I can't figure out where exactly it fails.
Here is the code snippet that I am working with:
package post;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLEncoder;
import java.nio.charset.Charset;
public class PostTest {
public static void login() throws MalformedURLException, IOException {
String urlParameters = String.format("id=%s&password=%s",
URLEncoder.encode(Config.getProperty("account.1.username"), "UTF-8"),
URLEncoder.encode(Config.getProperty("account.1.password"), "UTF-8")
);
byte[] postData = urlParameters.getBytes(Charset.forName("UTF-8"));
URL url = new URL(Config.getProperty("login.url"));
HttpURLConnection httpConn = (HttpURLConnection)url.openConnection();
httpConn.setDoInput(true);
httpConn.setDoOutput(true);
httpConn.setUseCaches(false);
httpConn.setRequestMethod("POST");
httpConn.setRequestProperty("Content-Length",
Integer.toString(postData.length));
//prints: 26
System.out.println(postData.length);
//prints: id=test&password=123456789
System.out.println(urlParameters);
httpConn.getOutputStream().write(postData);
httpConn.getOutputStream().flush();
try (BufferedReader reader = new BufferedReader(
new InputStreamReader(httpConn.getInputStream()))) {
for (String line; (line = reader.readLine()) != null;)
System.out.println(line);
}
}
}
Live HTTP Headers (Firefox plugin) output:
POST /io/login HTTP/1.1
Host: uploaded.net
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101 Firefox/45.0
Accept: text/javascript, text/html, application/xml, text/xml, */*
Accept-Language: de,en-US;q=0.7,en;q=0.3
Accept-Encoding: gzip, deflate
X-Requested-With: XMLHttpRequest
X-Prototype-Version: 1.6.1
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Referer: http://uploaded.net/
Content-Length: 20
Cookie: __utma=9[censored...]
Connection: keep-alive
id=test&pw=123456789
HTTP/1.1 200 OK
[...]
Edit: Added the Firefox' Live HTTP Headers output which I used to see what data is being sent through POST.
Is my assumption that byte[] postData is not sent correctly, if at all, correct?
What changes do I have to make to get the login to work properly?
Uhh, I've finally noticed my mistake. Hence I tried the first logins using my localhost/test.php where the parameter was password=<pw> I completly overlooked that the actual website I send the data to is using pw=<pw> instead.
After changing String.format("id=%s&password=%s", [...] to String.format("id=%s&pw=%s", [...] it works like a charm.
Basically i just want to get cookies from server. Here's simple code, but i guess it's not enough to get every cookie like e.g sessionID etc.
import java.net.*;
import java.util.*;
public class Cookie {
public static void main(String[] args) {
try {
URL url = new URL("http://www.google.com/");
URLConnection conn = url.openConnection();
System.out.println(con.getHeaderField("Set-Cookie"));
} catch (Exception e) { }
}
}
Result shows that i get only one cookie:
NID=61=nNSsAl4g7DfxqHE7t__ghfoCc_J-RmY7PaTNoPOd_khLdvvuqI-fnteHgnQXMzmxj_C5HdMozcGfOTt8PvePLKpbDdUlzdVZiRKwNpQIej6_T69jt9C7Oi6E4F-gWPHD; expires=Mon, 14-Jan-2013 17:16:40 GMT; path=/; domain=.google.pl; HttpOnly
Firefox gets NID and PREF, so it's not working.
Is there any way to get PREF cookie?
Can anyone tell me is this proper way(code below) to send POST.
import java.io.*;
import java.net.URL;
import java.util.Iterator;
import java.util.List;
import java.util.Map;
import javax.net.ssl.HttpsURLConnection;
public class Cookie_Example {
public static void main(String[] args){
try{
String https_url = "https://ssl.filmweb.pl/j_login";
URL url = new URL(https_url);
String post = "_login_redirect_url=http%253A%252F%252Fwww.filmweb.pl%252F&j_username=test.filmweb%40gmail.com&j_password=test.filmweb&_rememberMe=on&pass=zaloguj";
HttpsURLConnection con = (HttpsURLConnection)url.openConnection();
con.setInstanceFollowRedirects(false);
con.setDoInput(true);
con.setDoOutput(true);
con.setRequestMethod("GET");
con.setRequestProperty("Host", "ssl.filmweb.pl");
con.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:13.0) Gecko/20100101 Firefox/13.0.1");
con.setRequestProperty("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
con.setRequestProperty("Accept-Language","pl,en-us;q=0.7,en;q=0.3");
con.setRequestProperty("Accept-Encoding","gzip, deflate");
con.setRequestProperty("Connection","keep-alive");
con.setRequestProperty("Referer","https://ssl.filmweb.pl/login");
con.setRequestProperty("Content-Type","application/x-www-form-urlencoded");
con.setRequestProperty("Content-Length",post.length()+"");
DataOutput output = new DataOutputStream(new BufferedOutputStream(con.getOutputStream()));
output.write(post.getBytes());
Map<String, List<String>> hf = con.getHeaderFields();
Iterator it = hf.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pairs = (Map.Entry)it.next();
System.out.println(pairs.getKey() + " = " + pairs.getValue());
}
} catch (Exception e) {
}
}
}
Result is empty cookie,
Set-Cookie = [_artuser_rememberMe=;Path=/;Domain=filmweb.pl;Expires=Thu, 01-Jan-1970 00:00:00 GMT]
http://img72.imageshack.us/img72/1134/testgof.jpg
Screen from Tamper Data
It means this code also doesn't work. How Can I get __utma, __utmz etc. cookie?
Can anyone help me? Which books will be helpful?
I think that with your statement, con.getHeaderField("Set-Cookie")" you only retrieve one header. But maybe the server sends you two cookies, in two different headers.
I use another statement for retrieve all headers of cookies:
List<String> cookies = connection.getHeaderFields().get("Set-Cookie");
To see what you send to server, and what the server sends to you, I recommend use a proxy between your program (or web navigator) and the server. Maybe Charles proxy can help you.