I'm not sure if this is possible but is there a way to grab the cookie from yahoo from Firefox's cookies.sqlite file and then use that information in a Java program? When I log into yahoo, I told it to leave me logged in. Since the browser thinks I'm still logged in, that info is stored in a cookie (I assume).
I saw yahoo has their developer's API and some OAuth library. To be able to use OAuth to log in, I would need to register my program but I don't want to register unless I have to. I found this post from SO on how to use sqlite.exe to view the file. However, the file looks like gibberish (to a human) and I can't tell what entry is my cookie.
Is there another way to parse this file to get my yahoo cookie and use it in a Java program? Do I have to register my "secret" program with yahoo to use OAuth properly to log into yahoo? Thanks in advance for any help you can give me.
For Firefox, try this. It's based on using the session recovery file Firefox stores. This is bash syntax, not Java, but can probably be adapted pretty easily.
grep -o '{"host":"<HOSTNAME>"[^}]*}' $HOME/.mozilla/firefox/*.default/sessionstore-backups/recovery.js
That should dump out each cookie as a JSON entry that is associated with whatever you put in for <HOSTNAME>. You can adjust beyond that to extract the specific cookie you want.
Note: If you have more than one FF profile, you may need to adjust the *.default portion. The directory name is stored in .mozilla/firefox/profiles.ini, but extracting it from there is really overkill if only a single profile exists.
Cookies are stored in an sqlite file, so this worked for me:
$ sqlite3 ~/.mozilla/firefox/*.default/cookies.sqlite
sqlite> select name,value from moz_cookies where host="bugs.kde.org" and name LIKE "Bugzill%";
My use case is to extract the bugzilla cookie in order to give it to a script.
For other use cases, adjust the SQL query accordingly, obviously.
Related
I wish to run a simple process on my server/laptop that will upload files to my google drive on a daily basis, once a day. I don't wish to share this, allow other users to use it etc.
All examples I find seem to involve browsing to an address to gain permission from the user (me) and then getting an auth code etc and proceding
ref: Java quickstart
Is there a way/example to do this without need of a browser, getting permission getting unique auth code each time as I only want to do this for my account?
Can I use a bash script with CURL commands rather than having to use Java?
Yes. See How do I authorise an app (web or installed) without user intervention? (canonical ?)
Yes. It becomes a complicated script if your file is large and you are doing resumable uploads, but for small files it's perfectly feasible. You'll need to play around a bit to get the correct encoding, multipart mime body, mime type and content size, but it's all eminently doable. You'll start by calling Google's auth api with your stored refresh token to get an access token. Then you'll set that access token into an Authorization bearer header as part of your content upload call.
I'm trying to get the windows user name of the user in my web application. Can any one suggest how to get it?
I'm developing a web application. So if a user is accessing my application, then I need to get the user's windows id and his host name. I tried a few different ways but it never worked. Any suggestions are highly appreciated.
Windows user details are not sent in plain HTTP requests, which makes it impossible for you to derive them from a user in a web application without additional data.
The host name from which the request is sent is available in the request headers, though.
If you'd still like to get Windows user details, you'll have to do some work, like ask the user to provide them in some form, or, if you have access to the user's Naming/Directory service, you can find things out through his IP/hostname.
This is something you generally don't have readily available in web applications, though.
Im not sure exactly to get the windows user name, but HTML5 The System Information API may provide some useful info. Go through this link once, You may find it helpful.
In YERY OLD(!) version of Internet Explorer the pattern %USER% inside of a URL was replaced by the login username. But with current browsers this doesn't work anymore, and that's good that way.
With JAVA applet you can request it via:
System.getProperty("user.name")
But I don't know if this works for you...
For the intranet case take a look at
How to retrieve the current windows logged on user for Single Sign On purposes in Java
and
Can you get a Windows (AD) username in PHP?
And it seems to be intentionally impossible in common case with modern browsers:
Can your Windows or Linux username be exposed to websites?
I was trying to crawl some of website content, using jsoup and java combination. Save the relevant details to my database and doing the same activity daily.
But here is the deal, when I open the website in browser I get rendered html (with all element tags out there). The javascript part when I test it, it works just fine (the one which I'm supposed to use to extract the correct data).
But when I do a parse/get with jsoup(from Java class), only the initial website is downloaded for parsing. Meaning there are some dynamic parts of a website and I want to get that data but since they're rendered post get, asynchronously on the website I'm unable to capture it with jsoup.
Does anybody knows a way around this? Am I using the right toolset? more experienced people, I bid your advice.
You need to check before if the website you're crawling demands some of this list to show all contents:
Authentication with Login/Password
Some sort of session validation on HTTP headers
Cookies
Some sort of time delay to load all the contents (sites profuse on Javascript libraries, CSS and asyncronous data may need of this).
An specific User-Agent browser
A proxy password if, by example, you're inside a corporative network security configuration.
If anything on this list is needed, you can manage that data providing the parameters in your jsoup.connect(). Please refer the official doc.
http://jsoup.org/cookbook/input/load-document-from-url
I'm trying to get data from a website, but first I need to log in to the site using java. The script worked until now, but now the site installed an anti bot system. Until now the procedure was simple, I've created a HttpStreamWriter and submitted my details to the login.php page, then get the cookies and later, when I want to get data from the site, I resubmit the cookie from the login.php page, but now there is a problem: an anti bot system:
I'm not sure, but I think this is the system:
https://github.com/yuri-gushin/Roboo/blob/master/Roboo.pm
The anti bot system creates a cookie, called anti-bot and I can't access the page without that cookie, the problem is that the cookie is generated by a flash application only after the page loads so I can't get the cookie from the page?
Any ideas how to "hack" this ? Thanks!
Your need is about cookie extraction, here is how to do or on the oracle site
That is you need to connect to the site, browse the headers until Set-Cookie. Having the correct http header, you'll be able to parse it very easily.
After what, you'll have to set it back to your further request.
Edit
Flash cookie or Local Shared Object are stored in AMF. AMF wil be used store anything, the problem with your use case is that you don't know which value (or maybe class instances) have been serialized...
However you could (it will take time, at least for include all necessary libs) try with the AMFConnection to retrieve information. But I won't bet on that.
Could you contact the webmaster to have some information about that ? Or doens't this website any login api ?
Here is my ideal situation. I log into www.philstockworld.com using chrome. Once logged in I start up my java application that uses the cookies just stored by chrome. Then my java application goes to work. So here is my question.
Here is what my program can do now, I can login to the website using whatever browser I want, then look up the value of the PHPSESSID cookie and start up my app using that value. Then my app can do what it needs to. I can also supply my app with my username and password and have it log in, then store the returned PHPSESSID cookie and do what it needs to. However, what I would like to have happen is I login to the website using a browser, then my app starts and uses the PHPSESSID cookie from my browser session, without me having to look it up and copy it.
Is there a way for my java application to get the value of that cookie, without me having to manually type it in?
The location of the Cookies file is:
On Linux:
$HOME/.config/google-chrome/Default/Cookies
For other OS's see the user data page on chromium.org.
However, the file is stored in a binary format, so it's going to be hard for you to load the data within.
Joel's answer tells you where the cookie data is stored. This data is a sqlite3 database file. See this question for how to read a sqlite3 database.
I can't find how/where Chrome stores cookies, on Linux at least. Chances are that they won't be cached as simple plain text files and thus not be easily readable. You say you don't want to hard code your username/password in your java app - but why do you have do this? You could just pass them as arguments to your app?