I have this browser class:
public class browser{
private List<String> cookies;
private HttpURLConnection conn;
<....>
public String GetPageContent(String url) throws Exception {
URL obj = new URL(url);
conn = (HttpURLConnection) obj.openConnection();
// default is GET
conn.setRequestMethod("GET");
conn.setUseCaches(false);
// act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language",
"en-GB,en;q=0.8,en-US;q=0.6,ro;q=0.4,fr;q=0.2");
if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
System.out.println(cookies);
}
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(
conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// Get the response cookies
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
<...>
public List<String> getCookies() {
return cookies;
}
public void setCookies(List<String> cookies) {
this.cookies = cookies;
}
Everything is fine if all my request comes from MainApp:
public class main_program {
public static void main(String[] args) {
Browser br = new Browser();
Check_List users = new Check_List ();
br.GetPageContent(//some url) >> see the cookies and retrives the url fine
but when I do something like this in my main program:
users.getList()
which is a another class in the same package that calls GetPageContent method from Browser class the things gets strange: most of the request don't have cookies attached.
The class looks like this:
public class Check_List {
private HashMap<String, String> dictionar = new HashMap<String, String>();
public void getList(HashMap dictionar) {
http_handler webpage = new http_handler();
try {
Iterator it = dictionar.entrySet().iterator();
while (it.hasNext()) {
Map.Entry pairs = (Map.Entry) it.next();
System.out.println(pairs.getKey() + " = " + pairs.getValue());
webpage.GetPageContent("http://example.com/profile.php?id="+pairs.getKey()+"&actionid=15");
System.out.println("I'am on user: " + pairs.getValue());
Related
I am trying to make it so I can login to facebook using proxies and threads. I had this working single thread with jsoup.connect but it will not allow me to do multithreading with multiple proxies. I am trying to login to Facebook using HttpsURLConnection.
public class HttpUrlConnectionExample {
private List<String> cookies;
private HttpsURLConnection conn;
private final String USER_AGENT = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.108 Safari/537.36";
public static void main(String[] args) throws Exception {
String url = "https://facebook.com/login.php?login_attempt=1&lwv=110";
String gmail = "https://facebook.com";
HttpUrlConnectionExample http = new HttpUrlConnectionExample();
// make sure cookies is turn on
CookieHandler.setDefault(new CookieManager());
// 1. Send a "GET" request, so that you can extract the form's data.
String page = http.GetPageContent(url);
String postParams = http.getFormParams(page, "email#live.com", "password");
// 2. Construct above post's content and then send a POST request for
// authentication
http.sendPost(url, postParams);
// 3. success then go to gmail.
String result = http.GetPageContent(gmail);
System.out.println(result);
}
private void sendPost(String url, String postParams) throws Exception {
URL obj = new URL(url);
conn = (HttpsURLConnection) obj.openConnection();
// Acts like a browser
conn.setUseCaches(true);
conn.setRequestMethod("POST");
conn.setRequestProperty("Host", "facebook.com");
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
/*
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
*/
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Referer", "https://facebook.com/login.php?login_attempt=1&lwv=110");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));
conn.setDoOutput(true);
conn.setDoInput(true);
// Send post request
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(postParams);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + postParams);
System.out.println("Response Code : " + responseCode);
System.out.println(conn.toString());
BufferedReader in =
new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// System.out.println(response.toString());
}
private String GetPageContent(String url) throws Exception {
URL obj = new URL(url);
conn = (HttpsURLConnection) obj.openConnection();
// default is GET
conn.setRequestMethod("GET");
conn.setUseCaches(false);
// act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept",
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in =
new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// Get the response cookies
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
public String getFormParams(String html, String username, String password)
throws UnsupportedEncodingException {
System.out.println("Extracting form's data...");
Document doc = Jsoup.parse(html);
System.out.println(doc.body());
// Facebook form id
Element loginform = doc.getElementById("login_form");
Elements inputElements = loginform.getElementsByTag("input");
List<String> paramList = new ArrayList<String>();
for (Element inputElement : inputElements) {
String key = inputElement.attr("name");
System.out.println("Key: " + key);
String value = inputElement.attr("value");
System.out.println("Value: " + value);
if (key.equals("email"))
{
value = username;
System.out.println("Email: "+value);
}
else if (key.equals("pass"))
{
value = password;
System.out.println("Password: "+value);
//paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
// build parameters list
StringBuilder result = new StringBuilder();
for (String param : paramList) {
if (result.length() == 0) {
result.append(param);
} else {
result.append("&" + param);
}
}
System.out.println("result: " + result.toString());
return result.toString();
/*
// Google form id
Element loginform = doc.getElementById("gaia_loginform");
Elements inputElements = loginform.getElementsByTag("input");
List<String> paramList = new ArrayList<String>();
for (Element inputElement : inputElements) {
String key = inputElement.attr("name");
String value = inputElement.attr("value");
if (key.equals("Email"))
value = username;
else if (key.equals("Passwd"))
value = password;
paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
// build parameters list
StringBuilder result = new StringBuilder();
for (String param : paramList) {
if (result.length() == 0) {
result.append(param);
} else {
result.append("&" + param);
}
}
*/
//return result.toString();
//return "";
}
public List<String> getCookies() {
return cookies;
}
public void setCookies(List<String> cookies) {
this.cookies = cookies;
}
}
Originally I was able to login, but now it it is forcing me to change my password, and giving a message saying an unusual login attempt has been detected. Is there anyone who would be able to give me some guidance of where I should look to achieve what I am looking to do. I have also tried logging into m.facebook utilizing fiddler with no avail. I would like to do this without the API to just see how it works.
I am trying to adapt this to learn from - https://www.mkyong.com/java/how-to-automate-login-a-website-java-example/
UPDATE:
So not only it showing the unusual login attempt, but it is giving a diffrent user agent and ip then the one I am attempting with. The login is working, but it's not passing the string that says invalid user or email when I enter a wrong password. Can anyone shed some light?
This line conn.setRequestProperty("Cookie",cookieHeader); wont reed cookieHeader if i manualy pass in the string like this conn.setRequestProperty("Cookie", "ci_session=5dsjqh39nkj7nm98nokrukoesr6iir67"); then it works fine, anny idea why this dos not work?
package javaapplication5;
import java.io.*;
import java.net.*;
import java.util.*;
public class JavaApplication5 {
public static void main(String[] args) throws Exception {
String urlString = "http://example.se/";
String params ="";
int timeout =1000;
String cookieHeader = getcookieHeader(urlString,params);
System.out.println(post(urlString,params,cookieHeader,timeout));
}
public static String getcookieHeader(String urlString, String params) throws MalformedURLException, IOException{
// your first request that does the authentication
URL authUrl = new URL(urlString);
HttpURLConnection authCon = (HttpURLConnection) authUrl.openConnection();
authCon.connect();
// temporary to build request cookie header
StringBuilder sb = new StringBuilder();
// find the cookies in the response header from the first request
List<String> cookies = authCon.getHeaderFields().get("Set-Cookie");
if (cookies != null) {
for (String cookie : cookies) {
if (sb.length() > 0) {
sb.append("; ");
}
// only want the first part of the cookie header that has the value
String value = cookie.split(";")[0];
sb.append(value);
}
}
// build request cookie header to send on all subsequent requests
String cookieHeader = sb.toString();
return cookieHeader;
}
public static String post(String urlString, String params, String cookieHeader, int timeout) throws Exception {
String response = null;
HttpURLConnection conn = null;
DataOutputStream dos = null;
InputStream is = null;
try {
URL url = new URL(urlString);
conn = (HttpURLConnection)url.openConnection();
conn.setDoInput(true);
conn.setDoOutput(true);
conn.setConnectTimeout(timeout);
conn.setRequestMethod("POST");
conn.setRequestProperty("Cookie", cookieHeader);// does not work
//conn.setRequestProperty("Cookie", "ci_session=5dsjqh39nkj7nm98nokrukoesr6iir67"); works fine..
System.out.println(cookieHeader);// prints out "ci_session=5dsjqh39nkj7nm98nokrukoesr6iir67"
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("charset", "utf-8");
conn.setRequestProperty("Content-Length", Integer.toString(params.getBytes().length));
conn.connect();
dos = new DataOutputStream(conn.getOutputStream());
dos.writeBytes(params);
dos.flush();
is = conn.getInputStream();
Scanner s = new Scanner(is).useDelimiter("\\A");
response = s.hasNext() ? s.next() : null;
} finally {
if (dos != null) {
dos.close();
}
if (is != null) {
is.close();
}
if (conn != null) {
conn.disconnect();
}
}
return response;
}
}
I found the solution.. Just added this line: CookieHandler.setDefault(new CookieManager()); to getcookieHeader() under authCon.connect(); and removed
conn.setRequestProperty("Cookie", cookieHeader);
With CookieHandler.setDefault(new CookieManager()); HttpURLConnection will automatically save any cookies it receives and send them back with the next request to the same host.
I want to make java program to download a file from website, but i'm stuck in following situation. When I run the following code logging to page gets done successfully, but after it is done, when I send request for the next page, it shows response code 500.
public class Ss2mydb
{
private List<String> cookies;
private HttpURLConnection conn;
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws Exception
{
String indexPage = "http://10.100.100.142/index.asp";
String validatePage = "http://10.100.100.142/validate.asp";
String ccmenuPage = "http://10.100.100.142/callcentre/ccmenu.asp";
String reportPage = "http://10.100.100.142/topmgmt/reports/PROJECTVIJAY/get_download_cafs.asp";
String reportPageDownload = "http://10.100.100.142/topmgmt/reports/PROJECTVIJAY/get_download_cafs.asp?view=N";
Ss2mydb http = new Ss2mydb();
// make sure cookies is turn on
CookieHandler.setDefault(new CookieManager());
// 1. Send a "GET" request, so that you can extract the form's data.
String page = http.GetPageContent(indexPage);
String postParams = http.getFormParams(page, "username", "password");
// 2. Construct above post's content and then send a POST request for
// authentication
http.sendPost(validatePage, postParams);
// System.exit(0);
// 3. success then go to gmail.
http.GetPageContent2(ccmenuPage);
String result = http.GetPageContent2(reportPage);
System.out.println(result);
}
private void sendPost(String url, String postParams) throws Exception {
URL obj = new URL(url);
conn = (HttpURLConnection) obj.openConnection();
// Acts like a browser
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("Host", "10.100.100.142");
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
for (String cookie : this.cookies) {
System.out.println(" sendPost : "+cookie.split(";", 1)[0]);
conn.addRequestProperty("Cookie",cookie.split(";", 1)[0]);
}
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Referer", "http://10.100.100.142/index.asp");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));
conn.setDoOutput(true);
conn.setDoInput(true);
// Send post request
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(postParams);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + postParams);
System.out.println("Response Code : " + responseCode);
// setCookies(conn.getHeaderFields().get("Set-Cookie"));
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// System.out.println(response.toString());
}
private String GetPageContent2(String url) throws Exception {
URL obj = new URL(url);
conn = (HttpURLConnection) obj.openConnection();
// default is GET
conn.setRequestMethod("GET");
conn.setUseCaches(false);
// act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
System.out.println(" L : "+cookies);
if (cookies != null) {
for (String cookie : this.cookies) {
System.out.println(" GetPageContent : "+cookie.split(";", 1)[0]);
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// Get the response cookies
// System.out.println(" Get : "+conn.getHeaderFields().get("Set-Cookie"));
// setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
private String GetPageContent(String url) throws Exception {
URL obj = new URL(url);
conn = (HttpURLConnection) obj.openConnection();
// default is GET
conn.setRequestMethod("GET");
conn.setUseCaches(false);
// act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept","text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
System.out.println(" L : "+cookies);
if (cookies != null) {
for (String cookie : this.cookies) {
System.out.println(" GetPageContent : "+cookie.split(";", 1)[0]);
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// Get the response cookies
System.out.println(" Get : "+conn.getHeaderFields().get("Set-Cookie"));
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
public String getFormParams(String html, String username, String password)
throws UnsupportedEncodingException {
System.out.println("Extracting form's data...");
Document doc = Jsoup.parse(html);
// Google form id
Element loginform = doc.getElementById("right");
Elements inputElements = loginform.getElementsByTag("input");
List<String> paramList = new ArrayList<String>();
for (Element inputElement : inputElements) {
String key = inputElement.attr("name");
String value = inputElement.attr("value");
if (key.equals("USERname"))
value = username;
else if (key.equals("password"))
value = password;
if(!key.equals(""))
paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
// build parameters list
StringBuilder result = new StringBuilder();
for (String param : paramList) {
if (result.length() == 0) {
result.append(param);
} else {
result.append("&" + param);
}
}
return result.toString();
}
public List<String> getCookies() {
return cookies;
}
public void setCookies(List<String> cookies) {
this.cookies = cookies;
}
}
500 means "Internal server error" (see HTTP codes and their meaning).
It does mean that the request you sent was understood and was valid HTTP, but either there is an internal, unrelated issue, or the request you send is missing some parameter or is sending some incorrect value, and that omission/mistake causes the server logic to crash (either because it fails a validation of the request content or in a deeper, uncontrolled way).
The only way to get more specific info would be seeing the server logs (and that if the developer was careful checking parameters and logging info).
Your best option is to analize the traffic between the web browser and the server using some tool like wireshark, and then analize the traffic between your client and the server and try to spot the differences.
I am looking for a way to login to Facebook without a browser.
I want to obtain the access token without a browser.
Login is required in order to obtain the access token.
-Login Java examples
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.CookieHandler;
import java.net.CookieManager;
import java.net.URL;
import java.net.URLEncoder;
import java.util.ArrayList;
import java.util.List;
import javax.net.ssl.HttpsURLConnection;
import org.jsoup.Jsoup;
import org.jsoup.nodes.Document;
import org.jsoup.nodes.Element;
import org.jsoup.select.Elements;
public class HttpUrlConnectionExample {
private List<String> cookies;
private HttpsURLConnection conn;
private final String USER_AGENT = "Mozilla/5.0";
public static void main(String[] args) throws Exception {
String url = "https://www.facebook.com/login.php?login_attempt=1";
String facebook = "https://www.facebook.com/";
HttpUrlConnectionExample http = new HttpUrlConnectionExample();
// make sure cookies is turn on
CookieHandler.setDefault(new CookieManager());
// 1. Send a "GET" request, so that you can extract the form's data.
String page = http.GetPageContent(url);
String postParams = http.getFormParams(page, "sainstia#gmail.com", "1111");
// 2. Construct above post's content and then send a POST request for
// authentication
http.sendPost(url, postParams);
// 3. success then go to facebook.
String result = http.GetPageContent(facebook);
try {
String content = result;
File file = new File(".\\facebook_page.html");
// if file doesnt exists, then create it
if (!file.exists()) {
file.createNewFile();
}
FileWriter fw = new FileWriter(file.getAbsoluteFile());
BufferedWriter bw = new BufferedWriter(fw);
bw.write(content);
bw.close();
System.out.println("Done");
} catch (IOException e) {
e.printStackTrace();
}
}
private void sendPost(String url, String postParams) throws Exception {
URL obj = new URL(url);
conn = (HttpsURLConnection) obj.openConnection();
// Acts like a browser
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.setRequestProperty("Host", "www.facebook.com");
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
conn.setRequestProperty("Connection", "keep-alive");
conn.setRequestProperty("Referer", "https://www.facebook.com/login.php?login_attempt=1");
conn.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));
conn.setDoOutput(true);
conn.setDoInput(true);
// Send post request
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(postParams);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + url);
System.out.println("Post parameters : " + postParams);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// System.out.println(response.toString());
}
private String GetPageContent(String url) throws Exception {
URL obj = new URL(url);
conn = (HttpsURLConnection) obj.openConnection();
// default is GET
conn.setRequestMethod("GET");
conn.setUseCaches(false);
// act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "en-US,en;q=0.5");
if (cookies != null) {
for (String cookie : this.cookies) {
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + url);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
// Get the response cookies
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
public String getFormParams(String html, String username, String password) throws UnsupportedEncodingException {
System.out.println("Extracting form's data...");
Document doc = Jsoup.parse(html);
// Google form id
Element loginform = doc.getElementById("login_form");
Elements inputElements = loginform.getElementsByTag("input");
List<String> paramList = new ArrayList<String>();
for (Element inputElement : inputElements) {
String key = inputElement.attr("name");
String value = inputElement.attr("value");
if (key.equals("email"))
value = username;
else if (key.equals("pass"))
value = password;
paramList.add(key + "=" + URLEncoder.encode(value, "UTF-8"));
}
// build parameters list
StringBuilder result = new StringBuilder();
for (String param : paramList) {
if (result.length() == 0) {
result.append(param);
} else {
result.append("&" + param);
}
}
return result.toString();
}
public List<String> getCookies() {
return cookies;
}
public void setCookies(List<String> cookies) {
this.cookies = cookies;
}
}
GetPageContent (String url) {
.....
.....
setCookies (. conn.getHeaderFields () get ("Set-Cookie"));
......
}
I could not have a "Set-Cookie" value.
I can not find the "Set-Cookie".
What should I do to solve this problem?
You should use Jsoup i just finished making a program that did exactly this it logged onto a website and then i could manipulate anything on the website as i needed as long as you match the GETrequests you can log on to the wesbsite like this.
Connection.Response response = Jsoup.connect(url).method(Connection.Method.GET).execute();
response = Jsoup.connect(url)
.cookies(response.cookies())
.data("Action", "Login")
.data("User", "My_UserName")
.data("Password", "My_Password")
.method(Connection.Method.POST)
.followRedirects(true)
.timeout(50000)
.execute();
I have used this tutorial to try to login to a website from the company I am doing my internship at: http://www.mkyong.com/java/how-to-automate-login-a-website-java-example/
However, It is not logging in, since the URL when you're logged in to the site is the same as when you're not logged in. So all that happens is me adding the data to the fields, and then the page refreshes, with nothing happening. Can anyone tell me how I am supposed to continue?
Thanks
(Edit): I only have a redirect link
public class ProfileLogin extends AsyncTask<Void, Void, Void>{
private List<String> cookies;
private HttpsURLConnection conn;
private final String USER_AGENT = "Mozilla/5.0";
private String page;
private String userName;
private String passWord;
private String postParams;
URL obj;
//Setting up out connection
public ProfileLogin(String user, String pass){
CookieManager cManager = new CookieManager();
CookieHandler.setDefault(cManager);
page = null;
try {
obj = new URL(LOGIN_URL);
} catch (MalformedURLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//Setting the username and password
userName = user;
passWord = pass;
try {
conn = (HttpsURLConnection) obj.openConnection();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
//Starting our Asynctask to do all of the networking
execute();
}
public VpnProfile getProfiles(){
VpnProfile profile = new VpnProfile(null);
return profile;
}
public String getPageContent() throws Exception{
//Set Get method
conn.setRequestMethod("GET");
conn.setUseCaches(false);
//The properties of our site, we need to act like a browser
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4");
if(cookies != null){
for(String cookie : this.cookies){
conn.addRequestProperty("Cookie", cookie.split(";", 1)[0]);
}
}
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'GET' request to URL : " + LOGIN_URL);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while((inputLine = in.readLine()) != null){
response.append(inputLine);
}
in.close();
setCookies(conn.getHeaderFields().get("Set-Cookie"));
return response.toString();
}
private String getFormParams(String userName2, String passWord2) throws Exception{
System.out.println("Extract form's data...");
Document doc = Jsoup.parse(page);
//Elements of the login page
Element userNameElement = doc.getElementById("username");
Element passWElement = doc.getElementById("password");
List<String> paramList = new ArrayList<String>();
paramList.add(userNameElement.attr("name") + "=" + URLEncoder.encode(userName2, "UTF-8"));
paramList.add(passWElement.attr("name") + "=" + URLEncoder.encode(passWord2, "UTF-8"));
StringBuilder result = new StringBuilder();
for(String param : paramList){
if(result.length() == 0){
result.append(param);
} else {
result.append("&" + param);
}
}
return result.toString();
}
private void sendPost(String postParams) throws Exception {
conn = (HttpsURLConnection) obj.openConnection();
//Act like a browser
conn.setUseCaches(false);
conn.setRequestMethod("POST");
//This line is deleted, I can't show the url, I set the host here
conn.setRequestProperty("User-Agent", USER_AGENT);
conn.setRequestProperty("Accept", "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8");
conn.setRequestProperty("Accept-Language", "nl-NL,nl;q=0.8,en-US;q=0.6,en;q=0.4");
if(cookies != null){
for(String cookie : this.cookies){
System.out.println("This is cookies: " + cookie);
conn.addRequestProperty("cookie", cookie.split(";", 1)[0]);
}
}
conn.setRequestProperty("Connection", "keep-alive");
//Deleted line setting referer URL
conn.setRequestProperty("Content-Type", "text/html; charset=UTF-8");
conn.setRequestProperty("Content-Length", Integer.toString(postParams.length()));
conn.setDoOutput(true);
conn.setDoInput(true);
DataOutputStream wr = new DataOutputStream(conn.getOutputStream());
wr.writeBytes(postParams);
wr.flush();
wr.close();
int responseCode = conn.getResponseCode();
System.out.println("\nSending 'POST' request to URL : " + LOGIN_URL);
System.out.println("Post parameters : " + postParams);
System.out.println("Response Code : " + responseCode);
BufferedReader in = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while((inputLine = in.readLine()) != null){
response.append(inputLine);
}
in.close();
}
#Override
protected Void doInBackground(Void... arg0) {
try {
page = getPageContent();
postParams = getFormParams(userName, passWord);
sendPost(postParams);
page = getPageContent();
Document doc = Jsoup.parse(page);
Element userNameElement = doc.getElementById("username");
if(userNameElement.toString() != null){
System.out.println("Not logged in");
}else{
System.out.println("Logged in!");
}
}catch(Exception e){
e.printStackTrace();
}
return null;
}
private void setCookies(List<String> cookies) {
this.cookies = cookies;
}
}
I kind of found a way around: How to log into Facebook programmatically using Java?
This works really well for me and gives me really clean and short code :)