Yahoo Finance URL not working (again since 05/2018) - java

Thanks to this valuable site, I found useful tips since 08/2017 to retrieve cookies and crumbs for Yahoo Finance site in order to solve my bulk quote download problem.
Nevertheless my program (written in Java) doesn't work anymore since end of May 2018.
I get the following error message :
CookieHandler retrieved cookie:
GUCS="AX62rEgH";$Path="/";$Domain=".yahoo.com" Added cookie using
cookie handler getContent on quote failed: java.io.IOException: Server
returned HTTP response code: 401 for URL:
https://query1.finance.yahoo.com/v7/finance/download/AC.PA?period1=1526594400&period2=1527631200&interval=1d&events=history&crumb=null
I think that the crumb search is failing..
FYI : I am a Java programmer "amateur" since 2003
Please advise if anybody knows how to solve this problem

Thanks to Maxzoom and Dave for their prompt answer. I apologize for the lack of details in my question.For that reason I am adding the complete java method I was using successfully until last month, thanks on one hand to the code of Serge dated Aug 27 2017
Since the method is too long in the comment page I will paste in a new answer Here is the method below:
public static String getQuote3(String quoteString, String stock) throws IOException {
int curByte;
char curChar;
String curQuote,z;
boolean priceFlag;
z="rr";
//////////////Search for cookies
try {
CookieManager manager = new CookieManager();
manager.setCookiePolicy(CookiePolicy.ACCEPT_ALL);
CookieHandler.setDefault(manager);
URL quoteURL = new URL("https://fr.finance.yahoo.com/quote"+stock+"/history?p="+stock);
URLConnection con = quoteURL.openConnection();
con.getContent();
// get cookies from underlying CookieStore
CookieStore cookieJar = manager.getCookieStore();
java.util.List <HttpCookie> cookies = cookieJar.getCookies();
for (HttpCookie cookie: cookies) {
System.out.println("CookieHandler retrieved cookie: " + cookie);
}
//now you can search for the crumb in the yahoo site:
String crumb = null;
InputStream inStream = con.getInputStream();
InputStreamReader irdr = new InputStreamReader(inStream);
BufferedReader rsv = new BufferedReader(irdr);
Pattern crumbPattern = Pattern.compile(".*\"CrumbStore\":\\{\"crum\":\"([^\"]+)\"\\}.*");
String line = null;
while (crumb == null && (line = rsv.readLine()) != null) {
Matcher matcher = crumbPattern.matcher(line);
if (matcher.matches() && matcher.group(1).length()< 12)
crumb = matcher.group(1);
if(crumb!= null)
{
System.out.println ("crumb= " + crumb) ;
}
}
rsv.close();
String quoteUrls = quoteString + crumb;
// create cookie
HttpCookie cookie = new HttpCookie("UserName", "John Doe");
// add cookie to CookieStore for a particular URL quoteURL = new URL(quoteUrls);
try {
cookieJar.add(quoteURL.toURI(), cookie);
System.out.println("Added cookie using cookie handler");
} catch(Exception e) {
System.out.println("Unable to set cookie using CookieHandler");
e.printStackTrace();
}
con.connect();
try {
DataInputStream quoteStream = new DataInputStream(quoteURL.openStream());
priceFlag = false;
curQuote = "";
while( (curByte = quoteStream.read()) != -1) {
curChar = (char) curByte;
curQuote += curChar;
}
System.out.println(curQuote);
priceFlagn = true;
return curQuote;
} catch (IOException e) {
System.err.println("getContent on quote failed: " + e);
priceFlagn = false;
}
} catch (MalformedURLException e) {
System.err.println("Yikes. URL exception");
}
return z;
}

Related

How to implement HTTP POST method in web proxy through Java

I am trying to make a web proxy for HTTP communication between server and client. The GET method is working fine but I am POST method part is not working. I am sure I have missed out something. I want to know what have I missed or not implemented.
// request from client is handle from here
while ((inputLine = in.readLine()) != null) {
try {
StringTokenizer tok = new StringTokenizer(inputLine);
tok.nextToken();
} catch (Exception e) {
break;
}
if (cnt == 0) {
System.out.println("inputLine "+inputLine);
String[] tokens = inputLine.split(" ");
urlToCall = tokens[1];
//hum inputline sy URL nikaal rahay hai
if(tokens[0]=="POST")
{
f=1;
}
System.out.println("Request for : " + urlToCall);
}
cnt++;
}
BufferedReader rd = null;
try {
//yaha sy hum ab server ko request send karay gy
URL url = new URL(urlToCall);
URLConnection conn = url.openConnection();
HttpURLConnection huc = (HttpURLConnection) conn;
conn.setDoInput(true);
conn.setDoOutput(false);
// now we will get the response from the server
if (f == 1) {
huc.setDoOutput(true);
huc.setInstanceFollowRedirects(false);
huc.setRequestMethod("POST");
huc.setRequestProperty("Content-Type", "application/x-www-form-urlencoded");
huc.setRequestProperty("charset", "utf-8");
}
InputStream is = null;
if (conn.getContentLength() > 0)
{
try {
is = conn.getInputStream();
rd = new BufferedReader(new InputStreamReader(is));
} catch (IOException ioe) {
System.out.println(
"********* IO EXCEPTION **********: " + ioe);
}
}
What is the error you are getting on the post? And what is a sample GET request that is being passed into your code?
When I use a simple GET request, the code fails because the urlToCall does not have the host or protocol. The below code worked for me, but I would highly suggest you change your code to not hide the exceptions that are being thrown because they will have important information about what is going wrong with your code.
if (cnt == 1) {
System.out.println("host: " + inputLine);
String[] tokens = inputLine.split(" ");
urlToCall = "HTTP://" + tokens[1] + urlToCall;
}

Unable to get the subscription information from Google Play Android Developer API

I'trying to use Google APIs Client Library for Java to get information about user's subscriptions purchased in my android app. Here is how I'm doing for now:
HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
JsonFactory JSON_FACTORY = new JacksonFactory();
GoogleCredential credential = new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setServiceAccountId(GOOGLE_CLIENT_MAIL)
.setServiceAccountScopes("https://www.googleapis.com/auth/androidpublisher")
.setServiceAccountPrivateKeyFromP12File(new File(GOOGLE_KEY_FILE_PATH))
.build();
Androidpublisher publisher = new Androidpublisher.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).
setApplicationName(GOOGLE_PRODUCT_NAME).
build();
Androidpublisher.Purchases purchases = publisher.purchases();
Get get = purchases.get("XXXXX", subscriptionId, token);
SubscriptionPurchase subscripcion = get.execute(); //Exception returned here
GOOGLE_CLIENT_MAIL is the email address from API Access from the Google Console.
GOOGLE_KEY_FILE_PATH is the p12 file downloaded from the API Access.
GOOGLE_PRODUCT_NAME is the product name from the branding information.
In Google APIS Console the Service "Google Play Android Developer API" is enabled.
What I'm getting is:
{
"code" : 401,
"errors" : [ {
"domain" : "androidpublisher",
"message" : "This developer account does not own the application.",
"reason" : "developerDoesNotOwnApplication"
} ],
"message" : "This developer account does not own the application."
}
I really appreciate your help for this issue...
I got it working! The steps I followed:
Prerequisite
Before start, we need to generate a refresh token. To do this first we have to create an APIs console project:
Go to the APIs Console and log in with your Android developer
account (the same account used in Android Developer Console to upload the APK).
Select Create project.
Go to Services in the left-hand navigation panel.
Turn the Google Play Android Developer API on.
Accept the Terms of Service.
Go to API Access in the left-hand navigation panel.
Select Create an OAuth 2.0 client ID:
On the first page, you will need to fill in the product name, but a
logo is not required.
On the second page, select web application and set the redirect URI
and Javascript origins. We will use it later the redirect URI.
Select Create client ID. Keep in mind the Client ID and the Client secret, we will use them later.
So, now we can generate the refresh token:
Go to the following URI (note that the redirect URI must match the value entered in the client ID exactly, including any trailing backslashes):
https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/androidpublisher&response_type=code&access_type=offline&redirect_uri=REDIRECT_URI&client_id=CLIENT_ID
Select Allow access when prompted.
The browser will be redirected to your redirect URI with a code parameter, which will look similar to 4/eWdxD7b-YSQ5CNNb-c2iI83KQx19.wp6198ti5Zc7dJ3UXOl0T3aRLxQmbwI. Copy this value.
Create a main class with:
public static String getRefreshToken(String code)
{
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://accounts.google.com/o/oauth2/token");
try
{
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(5);
nameValuePairs.add(new BasicNameValuePair("grant_type", "authorization_code"));
nameValuePairs.add(new BasicNameValuePair("client_id", GOOGLE_CLIENT_ID));
nameValuePairs.add(new BasicNameValuePair("client_secret", GOOGLE_CLIENT_SECRET));
nameValuePairs.add(new BasicNameValuePair("code", code));
nameValuePairs.add(new BasicNameValuePair("redirect_uri", GOOGLE_REDIRECT_URI));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
org.apache.http.HttpResponse response = client.execute(post);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer buffer = new StringBuffer();
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
String refreshToken = json.getString("refresh_token");
return refreshToken;
}
catch (Exception e) { e.printStackTrace(); }
return null;
}
GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET and GOOGLE_REDIRECT_URI are the previously values.
Finally, we have our refresh token! This value does not expire, so we can store in some site, like a property file.
Accessing to Google Play Android Developer API
Getting the access token. We will need our previosly refresh token:
private static String getAccessToken(String refreshToken){
HttpClient client = new DefaultHttpClient();
HttpPost post = new HttpPost("https://accounts.google.com/o/oauth2/token");
try
{
List<NameValuePair> nameValuePairs = new ArrayList<NameValuePair>(4);
nameValuePairs.add(new BasicNameValuePair("grant_type", "refresh_token"));
nameValuePairs.add(new BasicNameValuePair("client_id", GOOGLE_CLIENT_ID));
nameValuePairs.add(new BasicNameValuePair("client_secret", GOOGLE_CLIENT_SECRET));
nameValuePairs.add(new BasicNameValuePair("refresh_token", refreshToken));
post.setEntity(new UrlEncodedFormEntity(nameValuePairs));
org.apache.http.HttpResponse response = client.execute(post);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent()));
StringBuffer buffer = new StringBuffer();
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
String accessToken = json.getString("access_token");
return accessToken;
}
catch (IOException e) { e.printStackTrace(); }
return null;
}
Now, we can access to the Android API. I'm interesting in the expiration time of a subscription, so:
private static HttpTransport HTTP_TRANSPORT = new NetHttpTransport();
private static JsonFactory JSON_FACTORY = new com.google.api.client.json.jackson2.JacksonFactory();
private static Long getSubscriptionExpire(String accessToken, String refreshToken, String subscriptionId, String purchaseToken){
try{
TokenResponse tokenResponse = new TokenResponse();
tokenResponse.setAccessToken(accessToken);
tokenResponse.setRefreshToken(refreshToken);
tokenResponse.setExpiresInSeconds(3600L);
tokenResponse.setScope("https://www.googleapis.com/auth/androidpublisher");
tokenResponse.setTokenType("Bearer");
HttpRequestInitializer credential = new GoogleCredential.Builder().setTransport(HTTP_TRANSPORT)
.setJsonFactory(JSON_FACTORY)
.setClientSecrets(GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET)
.build()
.setFromTokenResponse(tokenResponse);
Androidpublisher publisher = new Androidpublisher.Builder(HTTP_TRANSPORT, JSON_FACTORY, credential).
setApplicationName(GOOGLE_PRODUCT_NAME).
build();
Androidpublisher.Purchases purchases = publisher.purchases();
Get get = purchases.get(GOOGLE_PACKAGE_NAME, subscriptionId, purchaseToken);
SubscriptionPurchase subscripcion = get.execute();
return subscripcion.getValidUntilTimestampMsec();
}
catch (IOException e) { e.printStackTrace(); }
return null;
}
And that's all!
Some steps are from https://developers.google.com/android-publisher/authorization.
You can use com.google.api-client and google-api-services-androidpublisher libraries.
First go to the project on google developer console (https://console.developers.google.com)
APIs & Auth -> APIs
Enable "Google Play Android Developer API"
Go to Credentials -> Create new Client ID
Select service account
Create client ID
Save the p12 file somewhere safe
Then add the just generated email address for the service account to your google play developer console (https://play.google.com/apps/publish/)
Settings -> Users and permissions -> Invite new users
Paste the #developer.gserviceaccount.com email account
Select "View financial reports"
Send invitation
Now to the code. Add the following dependencies to your pom.xml file:
<dependency>
<groupId>com.google.api-client</groupId>
<artifactId>google-api-client</artifactId>
<version>1.18.0-rc</version>
</dependency>
<dependency>
<groupId>com.google.http-client</groupId>
<artifactId>google-http-client-jackson2</artifactId>
<version>1.18.0-rc</version>
</dependency>
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-androidpublisher</artifactId>
<version>v1.1-rev25-1.18.0-rc</version>
</dependency>
Then first validate the signature:
byte[] decoded = BASE64DecoderStream.decode(KEY.getBytes());
KeyFactory keyFactory = KeyFactory.getInstance("RSA");
PublicKey publicKey = keyFactory.generatePublic(new X509EncodedKeySpec(decoded));
Signature sig = Signature.getInstance("SHA1withRSA");
sig.initVerify(publicKey);
sig.update(signedData.getBytes());
if (sig.verify(BASE64DecoderStream.decode(signature.getBytes())))
{
// Valid
}
If the signature verifies fetch subscription details:
// fetch signature details from google
HttpTransport httpTransport = GoogleNetHttpTransport.newTrustedTransport();
JsonFactory jsonFactory = JacksonFactory.getDefaultInstance();
GoogleCredential credential = new GoogleCredential.Builder()
.setTransport(httpTransport)
.setJsonFactory(jsonFactory)
.setServiceAccountId(ACCOUNT_ID)
.setServiceAccountScopes(Collections.singleton("https://www.googleapis.com/auth/androidpublisher"))
.setServiceAccountPrivateKeyFromP12File(new File("key.p12"))
.build();
AndroidPublisher pub = new AndroidPublisher.Builder(httpTransport, jsonFactory, credential)
.setApplicationName(APPLICATION_NAME)
.build();
AndroidPublisher.Purchases.Get get = pub.purchases().get(
APPLICATION_NAME,
PRODUCT_ID,
token);
SubscriptionPurchase subscription = get.execute();
System.out.println(subscription.toPrettyString());
This will take care of all the token issues by generating a JWT token so you don't have to handle it yourself.
For those who want to check subscription status on Google's AppEngine with Java, here is my working example based on many codes found on SO. I spent couple of days to solve many mistakes caused by lack of experience. I see lot of suggestions to check subscription status on server but it was not easy for me to do on AppEngine. Without answers found on SO, I could not come up with this.
Step 1
First we need to go through "Prerequisite" section found on Jonathan Naguin's answer, until you get code from web browser. Now you have;
Client ID
Client secret
Redirect URI
code
ready.
Note we run all codes shown below on AppEngine. And I used logger like this.
static final Logger log = Logger.getLogger(MyClassName.class.getName());
Step 2
We need to get refresh token. Run code shown below after replacing [YOUR CLIENT ID], [YOUR CLIENT SECRET], [YOUR CODE], [YOUR REDIRECT URI] with your string.
private String getRefreshToken()
{
try
{
Map<String,Object> params = new LinkedHashMap<>();
params.put("grant_type","authorization_code");
params.put("client_id",[YOUR CLIENT ID]);
params.put("client_secret",[YOUR CLIENT SECRET]);
params.put("code",[YOUR CODE]);
params.put("redirect_uri",[YOUR REDIRECT URI]);
StringBuilder postData = new StringBuilder();
for(Map.Entry<String,Object> param : params.entrySet())
{
if(postData.length() != 0)
{
postData.append('&');
}
postData.append(URLEncoder.encode(param.getKey(),"UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(param.getValue()),"UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
URL url = new URL("https://accounts.google.com/o/oauth2/token");
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.getOutputStream().write(postDataBytes);
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuffer buffer = new StringBuffer();
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
String refreshToken = json.getString("refresh_token");
return refreshToken;
}
catch (Exception ex)
{
log.severe("oops! " + ex.getMessage());
}
return null;
}
Since refresh token won't expire, we can save it somewhere or simply hard-code in our code. (We only need to run above code once to get refresh token.)
Step 3
We need to get access token. Run code shown below after replacing [YOUR CLIENT ID], [YOUR CLIENT SECRET], [YOUR REFRESH TOKEN] with your string.
private String getAccessToken()
{
try
{
Map<String,Object> params = new LinkedHashMap<>();
params.put("grant_type","refresh_token");
params.put("client_id",[YOUR CLIENT ID]);
params.put("client_secret",[YOUR CLIENT SECRET]);
params.put("refresh_token",[YOUR REFRESH TOKEN]);
StringBuilder postData = new StringBuilder();
for(Map.Entry<String,Object> param : params.entrySet())
{
if(postData.length() != 0)
{
postData.append('&');
}
postData.append(URLEncoder.encode(param.getKey(),"UTF-8"));
postData.append('=');
postData.append(URLEncoder.encode(String.valueOf(param.getValue()),"UTF-8"));
}
byte[] postDataBytes = postData.toString().getBytes("UTF-8");
URL url = new URL("https://accounts.google.com/o/oauth2/token");
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
conn.setDoOutput(true);
conn.setUseCaches(false);
conn.setRequestMethod("POST");
conn.getOutputStream().write(postDataBytes);
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
StringBuffer buffer = new StringBuffer();
for (String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
String accessToken = json.getString("access_token");
return accessToken;
}
catch (Exception ex)
{
log.severe("oops! " + ex.getMessage());
}
return null;
}
Step 4
What I wanted to know is just expire UTC of the subscription. Code shown below returns expire UTC, 0 when found error. You need to provide your package name, product id (=subscription id), access token you got on Step 3, and purchase token found in your purchase data.
private long getExpireDate(String packageName,String productId,String accessToken,String purchaseToken)
{
try
{
String charset = "UTF-8";
String query = String.format("access_token=%s",URLEncoder.encode(accessToken,charset));
String path = String.format("https://www.googleapis.com/androidpublisher/v1/applications/%s/subscriptions/%s/purchases/%s",packageName,productId,purchaseToken);
URL url = new URL(path + "?" + query);
HttpURLConnection connection = (HttpURLConnection)url.openConnection();
connection.setRequestProperty("Accept-Charset",charset);
connection.setRequestMethod("GET");
BufferedReader reader = new BufferedReader(new InputStreamReader(connection.getInputStream()));
StringBuffer buffer = new StringBuffer();
for(String line = reader.readLine(); line != null; line = reader.readLine())
{
buffer.append(line);
}
JSONObject json = new JSONObject(buffer.toString());
return json.optLong("validUntilTimestampMsec");
}
catch (Exception ex)
{
log.severe("oops! " + ex.getMessage());
}
return 0;
}
Note product id or subscription id is a string found on developer console. Your subscription item appears with name/id column. It looks like this.
Description of item(product id)
Last step (fun part)
Now we have all components to verify subscription is valid or not. I did like this. You need to replace [YOUR PACKAGE NAME], [YOUR PRODUCT ID] with yours.
You need to provide purchase data which you can get with Purchase#getOriginalJson() found in iabHelper code.
private boolean checkValidSubscription(String purchaseData)
{
String purchaseToken;
JSONObject json;
try
{
json = new JSONObject(purchaseData);
}
catch (JSONException e)
{
log.severe("purchaseData is corrupted");
return true; // false positive
}
purchaseToken = json.optString("purchaseToken");
if(purchaseToken.length() == 0)
{
log.severe("no purchase token found");
return true; // false positive
}
String accessToken = getAccessToken();
if(accessToken == null)
{
return true; // false positive
}
long expireDate = getExpireDate([YOUR PACKAGE NAME],[YOUR PRODUCT ID],accessToken,purchaseToken);
if(expireDate == 0)
{
log.severe("no expire date found");
return true; // false positive
}
expireDate += 86400000l; // add one day to avoid mis judge
if(expireDate < System.currentTimeMillis())
{
log.severe("subscription is expired");
return false;
}
// just for log output
long leftDays = (expireDate - System.currentTimeMillis()) / 86400000l;
log.info(leftDays + " days left");
return true;
}
Note for debugging
Google returns JSON string for response. If code won't work as expected, logging JSON string may help understanding what is wrong.
I hope this helps someone.
To piggyback on Jonathan Naguin's great answer, here is a nodejs version of getting the refresh and access token:
//This script is to retreive a refresh token and an access token from Google API.
//NOTE: The refresh token will only appear the first time your client credentials are used.
// I had to delete my client id within api console and create a new one to get the refresh token again.
//This is the downloaded json object from Google API Console. Just copy and paste over the template below.
var googleJson = {"web":{"auth_uri":"","client_secret":"","token_uri":"","client_email":"","redirect_uris":[""],"client_x509_cert_url":"","client_id":"","auth_provider_x509_cert_url":"","javascript_origins":[""]}};
//Retrieved from OAuth
var code = ''; // Retrieved from the response of the URL generated by printGoogleAuthUrl(). You will need to be logged in as your publisher. Copy and paste the generated url. Copy the code parameter into this variable.
var refreshToken = ''; // Retrieved from the printRefreshToken() function call. Requires the code variable to be filled out.
var accessToken = ''; // Retrieved from the printAccessToken() function call. Requires the refreshToken variable to be filled out.
var querystring = require('querystring');
var https = require('https');
var fs = require('fs');
function printGoogleAuthUrl()
{
console.log("https://accounts.google.com/o/oauth2/auth?scope=https://www.googleapis.com/auth/androidpublisher&response_type=code&access_type=offline&redirect_uri=" + googleJson.web.redirect_uris[0] + "&client_id=" + googleJson.web.client_id);
}
function printRefreshToken()
{
var post_data = querystring.stringify({
'grant_type' : 'authorization_code',
'client_id' : googleJson.web.client_id,
'client_secret' : googleJson.web.client_secret,
'code' : code,
'redirect_uri' : googleJson.web.redirect_uris[0]
});
var post_options = {
host: 'accounts.google.com',
port: '443',
path: '/o/oauth2/token',
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': post_data.length
}
};
var post_req = https.request(post_options, function(res) {
res.setEncoding('utf8');
var data = "";
res.on('data', function (chunk) {
data += chunk;
});
res.on('end', function(){
var obj = JSON.parse(data);
if(obj.refresh_token)
{
refreshToken = obj.refresh_token;
}
else
{
console.log("No refresh token found. I had to clear the web client id in Google Api Console and create a new one. There might be a better way here.");
}
console.log(data);
});
});
post_req.write(post_data);
post_req.end();
}
function printAccessToken()
{
var post_data = querystring.stringify({
'grant_type' : 'refresh_token',
'client_id' : googleJson.web.client_id,
'client_secret' : googleJson.web.client_secret,
'refresh_token' : refreshToken
});
var post_options = {
host: 'accounts.google.com',
port: '443',
path: '/o/oauth2/token',
method: 'POST',
headers: {
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': post_data.length
}
};
var post_req = https.request(post_options, function(res) {
res.setEncoding('utf8');
var data = "";
res.on('data', function (chunk) {
data += chunk;
});
res.on('end', function(){
var obj = JSON.parse(data);
if(obj.access_token)
accessToken = obj.access_token;
else
console.log("No access token found.");
console.log(data);
});
});
post_req.write(post_data);
post_req.end();
}
printGoogleAuthUrl();
//printRefreshToken();
//printAccessToken();
For those looking for a more up-to-date answer with AndroidPublisher v3 please look here: https://stackoverflow.com/a/57943483/1028256.
No need to deal with refreshToken & accessToken, and just few lines of code.
For Android client I found this 'official' sample code: https://github.com/googlesamples/android-play-publisher-api/blob/master/v3/java/src/com/google/play/developerapi/samples/AndroidPublisherHelper.java and there are options to get AndroidPublisher both with .p12 file or the app credentials.
I'm pretty sure you have to use your Client ID, not the email address. It looks like this: 37382847321922.apps.googleusercontent.com
See https://developers.google.com/android-publisher/authorization
client_id=<the client ID token created in the APIs Console>
And I'm pretty sure you don't need a P12 file. You only need the
client_secret=<the client secret corresponding to the client ID>
Try doing it manually from the command line first, with 'wget'.

Android HTTPS login

I am attempting to write an Android app which will allow me to read text from a website that holds my work roster https://www.blahblahblahcompany.com/Rostering/exportRoster.aspx
Is this possible? Would I be able to authenticate myself with the website and then download the source code?
The website in question also has the ability to export the roster as an .xls file
screenshot of the page in question
http://img528.imageshack.us/img528/9954/rosterf.png
I don't know if I've get what you really want to do, but I'm sure that at least, you gonna need some Get/Post operations to execute authentication proccess and retrieve data/informations. Please check the following methods:
public StringBuilder post(String url, List<NameValuePair> nvps) {
try {
HttpPost httppost = new HttpPost(url);
// if there is cookies, set then
if (cookies != null && cookies.size() > 0) {
String cookieString = "";
for (int i = 0; i < cookies.size(); ++i) {
cookieString += cookies.get(i).getName() + "=" + cookies.get(i).getValue() + "; ";
}
cookieString += "domain=" + Constants.BaseUrl + "; " + "path=/";
httppost.addHeader("Cookie", cookieString);
}
// connection timeout options
HttpParams httpParameters = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParameters, Constants.timeoutConnection);
HttpConnectionParams.setSoTimeout(httpParameters, Constants.timeoutSocket);
// setup the post method
DefaultHttpClient httpClient = new DefaultHttpClient(httpParameters);
httppost.setEntity(new UrlEncodedFormEntity(nvps, HTTP.UTF_8));
HttpResponse response = httpClient.execute(httppost);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
StringBuilder builder = new StringBuilder();
for (String line = null; (line = reader.readLine()) != null;)
builder.append(line).append("\n");
// set cookies
List<Cookie> incomingCookies = httpClient.getCookieStore().getCookies();
for (int i = 0; incomingCookies != null && i < incomingCookies.size(); i++) {
cookies.add(incomingCookies.get(i));
}
return builder;
} catch (UnsupportedEncodingException e) {
} catch (ClientProtocolException e) {
} catch (IOException e) {
} catch (Exception e) {
}
return null;
}
The above mehod executes a post in the url string parameter, using the pair values nvps as arguments for header. Note that Constants class is the class where you are declaring static strings (such as API entries) for your WebService, and the field cookies is a list of Cookies that gonna hold your session issues. This method will return the result for your POST request as a string builder object. Basically, it is a general-purpose method that can be used in several cases, and you should do little adaptations to fit tasks. This is what you can use to authenticate.
There is another important method, the Http GET:
public StringBuilder get(String url) {
try {
HttpGet httpget = new HttpGet(url);
// if there is cookies, set then
if (cookies != null && cookies.size() > 0) {
String cookieString = "";
for (int i = 0; i < cookies.size(); ++i) {
cookieString += cookies.get(i).getName() + "=" + cookies.get(i).getValue() + "; ";
}
cookieString += "domain=" + Constants.BaseUrl + "; " + "path=/";
httpget.addHeader("Cookie", cookieString);
}
HttpParams httpParameters = new BasicHttpParams();
HttpConnectionParams.setConnectionTimeout(httpParameters, Constants.timeoutConnection);
HttpConnectionParams.setSoTimeout(httpParameters, Constants.timeoutSocket);
DefaultHttpClient httpClient = new DefaultHttpClient(httpParameters);
HttpResponse response = httpClient.execute(httpget);
BufferedReader reader = new BufferedReader(new InputStreamReader(response.getEntity().getContent(), "UTF-8"));
StringBuilder builder = new StringBuilder();
for (String line = null; (line = reader.readLine()) != null;)
builder.append(line).append("\n");
// set cookies
List<Cookie> incomingCookies = httpClient.getCookieStore().getCookies();
for (int i = 0; incomingCookies != null && i < incomingCookies.size(); i++) {
cookies.add(incomingCookies.get(i));
}
return builder;
} catch (UnsupportedEncodingException e) {
} catch (ClientProtocolException e) {
} catch (IOException e) {
} catch (Exception e) {
}
return null;
}
This one do similar sequence of steps, but with very different purposes. The goal of this one is to retrieve informations, considering that you already have authenticated or that the auth process is not needed. It returns the requested data as a string builder.
Please, note that besides these methods are very general, you must closely check what is the proccess used in your requested web page. Hope that it helps you in some way! ;-)
In the Android web browser, go to Tools > Options > Encryption > View Certificates to see a list of supported certificate providers. As long as you buy a cert from a supported provider you will be able to do it. Here is a stack answer that might have some clues.

Apache httpclient returning page before it loads?

I noticed a strange phenomenon when using the apache httpclient libraries and I want to know why it occurs. I created some sample code to demonstrate.
Consider the following code:
//Example URL
String url = "http://www.amazon.com/gp/offer-listing/05961580/ref=dp_olp_used?ie=UTF8";
GetMethod get = new GetMethod(url);
HttpMethodRetryHandler httpHandler = new DefaultHttpMethodRetryHandler(1, false);
get.getParams().setParameter(HttpMethodParams.RETRY_HANDLER, httpHandler );
get.getParams().setCookiePolicy(CookiePolicy.IGNORE_COOKIES);
HttpConnectionManager connectionManager = new SimpleHttpConnectionManager();
HttpClient client = new HttpClient( connectionManager );
client.getParams().setParameter("http.useragent", FIREFOX );
String line;
StringBuilder stringBuilder = new StringBuilder();
String toStreamBody = null;
String toStringBody = null;
try {
int statusCode = client.executeMethod(get);
if( statusCode != HttpStatus.SC_OK ){
System.err.println("Internet Status: " + HttpStatus.getStatusText(statusCode) );
System.err.println("While getting page: " + url );
}
//toString
toStringBody = get.getResponseBodyAsString();
//toStream
InputStreamReader isr = new InputStreamReader(get.getResponseBodyAsStream())
BufferedReader rd = new BufferedReader(isr);
while ((line = rd.readLine()) != null) {
stringBuilder.append(line);
}
} catch (java.io.IOException ex) {
System.out.println( "Failed to get page: " + url);
} finally {
get.releaseConnection();
}
toStreamBody = stringBuilder.toString();
This code prints nothing:
System.out.println(toStringBody); // ""
This code prints the web page:
System.out.println(toStreamBody); // "Whole Page"
But it gets even stranger...
Replace:
get.getResponseBodyAsString();
With:
get.getResponseBodyAsString(150000);
Now we get the error:
Failed to get page: http://www.amazon.com/gp/offer-listing/0596158068/ref=dp_olp_used?ie=UTF8
I was unable to find another website besides for amazon that replicates this behavior but I assume there are others.
I am aware that according to the documentation at http://hc.apache.org/httpclient-3.x/performance.html discourages the use of getResponseBodyAsString(), it does not say that the page will not load, only that you may be at risk of an out of memory exception. Is it possible that getResponseBodyAsString() is returning the page before it loads? Why does this only happen with amazon?
Did you test with any other URL?
The URL in code that you provided redirects with 302 to http://www.amazon.com/dp/05961580/?tag=stackoverfl08-20, which then returns 404 (not found).
HttpClient does not handle redirects: http://hc.apache.org/httpclient-3.x/redirects.html

Using a session with php and Java

I'm currently trying to use the current session of a php web page from an applet. I tought it would be straightforward, but it didn't go as smooth as I tough. From the php man:
session_start() creates a session or resumes the current one based on a session
identifier passed via a GET or POST request, or passed via a cookie.
From there I did some php (simplified here):
// PAGE1.PHP
session_start();
$_SESSION['test'] = true;
echo "sid=" . session_id();
// PAGE2.PHP
session_start();
if ($_SESSION['test'])
$echo "success";
else
$echo "fail";
So, from my applet, I do a request to PAGE1.PHP and it returns me the session id. When I do a new request on the page 2, I pass the session id as a parameter and it seems that the session wasn't kept. I use
URL url = new URL("my/url/PAGE2.php?sid=" + session_id);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter wr = new OutputStreamWriter(conn.getOutputStream());
wr.write(data); // data is the post data created previously
wr.flush();
// Get the response
BufferedReader rd = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String line;
while ((line = rd.readLine()) != null) {
System.out.println(line);
}
I have tried via POST and GET method and it doesn't seem to work.
So I'm wondering if it's possible, and if yes, what do I miss?
thanks.
Accepting session IDs as part of the GET is bad form, and bad idea security wise. I would suggest that you retrieve the session ID from the PHPSESSION cookie with something like:
Following java snippet was shamelessly copied from here – Have a look at that (although it is java 1.4 specific).
public String getCookie() {
/*
** get all cookies for a document
*/
try {
JSObject myBrowser = (JSObject) JSObject.getWindow(this);
JSObject myDocument = (JSObject) myBrowser.getMember("document");
String myCookie = (String)myDocument.getMember("cookie");
if (myCookie.length() > 0)
return myCookie;
}
catch (Exception e){
e.printStackTrace();
}
return "?";
}
public String getCookie(String name) {
/*
** get a specific cookie by its name, parse the cookie.
** not used in this Applet but can be useful
*/
String myCookie = getCookie();
String search = name + "=";
if (myCookie.length() > 0) {
int offset = myCookie.indexOf(search);
if (offset != -1) {
offset += search.length();
int end = myCookie.indexOf(";", offset);
if (end == -1) end = myCookie.length();
return myCookie.substring(offset,end);
}
else
System.out.println("Did not find cookie: "+name);
}
return "";
}
Elsewhere in your code grab the session id using:
getCookie("PHPSESSION"); // replace this with the cookie name in your /etc/php.ini
and set it in your applet.
conn.setRequestProperty("Cookie", "PHPSESSION=value");
Far more current information is available at the sun java cookie page
Your PAGE2.php is not actually using the sid param you're passing via _GET to initiate the session.
In page2.php, try:
session_id($_GET['sid']);
session_start();
instead of plain-old:
session_start();

Categories