I 'm getting 620 error response codes back from the google maps geocoding api if i send the request directly from my app engine servlet, so i have no choice but to use a proxy to receive a successful response. I set up a proxy server, and ive tested it from several computers. Now, all I want to do is make a url request from my GAE servlet through my proxy.
I've tried every possible solution out there and none of them work....
-java.net.Proxy isnt supported in the app engine runtime...
-setting properties as follows:
Properties props = System.getProperties();
props.put("http.proxyHost", "proxyhostname");
props.put("http.proxyPort", "proxyhostport");
didnt do anything.
What is the easiest way to send an http GET via a proxy in app engine?
It seems like this is not possible: Google's App Engine APIs don't support it. Using a third-party library (like Apache's HTTPCore/HTTPClient) or writing it yourself is not possible because essential network classes like java.net.Socket are not whitelisted.
Not sure why you can't access the Google Map API, but if that really does not work, your only choice is to write some application on your proxy server that responds to normal HTTP requests and then forwards them to Google Maps.
Update: Googled a bit, seems like a well-known problem: the Map API has a limit of 2500 requests per day and IP, and this is limit is reached quickly on GAE where you share your IP with many other applications. The only thing you can do is move the requests to the client, use some proxy with own IP, or use a different service.
Related
I am trying to block certain websites using a web application. So, when a I type a url suppose "http://www.google.com" it should first check whether google is blocked by my application or not. If not open the website otherwise reject the browser request to open it. I am unable to find a way to capture all HTTP request from browser so that I can process it.
I Know proxies are the most suitable option but is there any alternative solution to this. After some searching I found a library - jpcap (a network packet capture library) and I was wondering if this could help me or not?
What you are trying to create is a proxy-server.
You have to configure the browser to go through the proxy, then you can deny websites, reroute them etc.
There are many proxies already there (open source and commercial) that offer what you want.
For example: Squid http://www.squid-cache.org/
See Wikipedia description of a proxy here: https://en.wikipedia.org/wiki/Proxy_server
Many firewall products offer the service of a transparent proxy, redirecting all http/https traffic going through the firewall into a proxy server. It seems, you have a direct connection but your packages are really filtered. Aka transparent proxy.
If your assignment does not allow this context, you need to check the assignment again, if you really got the scope of filtering right.
You cannot take over the browser's ip communication from a servlet or servlet filter. Using a (servlet) filter, you can only filter requests directed to your application. One step above, using an application server valve (Tomcat uses this term, others may use a different one), you can only filter requests directed at that server. One step above (or below) your application server is the physical server and the network it is running in.
If your client does not share the same network as your server, you can't even apply transparent proxy to it. Since browsers are running on the client computer, most clients in the world do not share the same network zone as the server.
It just does not work as you expect it.
I tried using Muffin's web proxy to record the url's that were hit in the browser.
I am able to track the internet request like google.com,stackoverflow etc.,
But, was unable to track the intranet request like the one which does not need internet. I am not sure how intranet works because those request for the url were not being tracked.
Is there a way to track those request as well(intranet urls).
1) I am able to track the weburls because i am redirecting all the request to the socket i had created in java.But setting it up as a proxy in the settings.
2) Usually intranet sites do not rely on the proxy servers. it will directly communicate though dns server.How to make those request also to go through my socket ?
Note I am trying to achieve it using JAVA sockets.
Use a windows application called FIDDLER.... It can track all type of inbound and outbound connections...
I'm facing an issue with Google Calendar Push Notifications. First I'd like to explain my scenario.
I'm trying to develop a java application to be integrated in a platform that handles sensors and actuators. The goal of this application is to use Google Calendar APIs and Push Notifications to follow a certain kind of events. My first "architecture" that I chose is the following ( I'm open to suggestions and criticism given my young experience as developper ;-) :
A simple Tomcat Servlet which will be the final webhook for Google Servers (A);
A java class that implements a simple Server-Client Socket (B);
A java class that uses Google Calendar API and receive push messages through one of its methods (C);
The logic (maybe not so efficient) behind this structure is that Google Servers send notifications to my Tomcat webhook, which at the same time is the client of my Socket channel. When it receives pushes, it does its job and warns the Socket Server. The latter, thanks to a static object, invokes a method of C. This class will use Google APIs to check what's happened.
First and mandatory resource to study is: https://developers.google.com/google-apps/calendar/v3/push
I've been able to complete Step 1: Verify that you own the domain using Webmaster Tools and add correctly my domain for push notifications as indicated in Step 2: Register your domain These two steps lead me to my first question:
Does Google check whether the listed domains have valid SSL certificates during these two steps?
My domain is something like "domain.com" and the full URL to reach the webhook is hhttps://ab.domain.com/app/notifications. There is a very important note in Google documentation:
Note that the Google Calendar API will be able to send notifications to this >HTTPS address only if there is a valid SSL certificate installed on your web >server. Invalid certificates include:
Self-signed certificates.
Certificates signed by an untrusted source.
Certificates that have been revoked.
Certificates that have a subject that doesn't match the target
hostname.
First thing first, I've configured Tomcat to enable SSL and so HTTPS following the official documentation: tomcat.apache.org/tomcat-6.0-doc/ssl-howto.html . Now I can access tomcat and my servlet through https.
The problem is that I can't receive "correctly" the push notifications. By using a browser plugin for simulating Http/Https request I can reach easily my Tomcat webhook and this one warns correctly my Java application. Therefore , since I can almost exclude problems about the Socket channel, I'm focusing on the communication between Google Servers and my Tomcat webhook.
I'm using Wireshark to see the incoming traffic from Google through the port 8443 (https for Tomcat). Obviously is encrypted, so I can't check a lot, but many packets are marked as PSH(push). However they seem to have a payload, which according to Google doc,they shouldn't because push notifications are just packets with headers info.
So my very first doubt is about the certificate. I don't have one, therefore I'm using this trial one from Verisign.com (which by the way is suggested in the Tomcat documentation for enabling SSL)
The very second doubt is about the validation and registration procedure. Even though I've managed to do it and Google is accepting the domain ab.domain.com/app/notifications as webhook, I don't have tools to prove that everything is correct.
I've found the following post in Stackoverflow: "Unable to receive push notification for Google calender", "Calendar push notification to java app engine backend servlet", "Google Calendar Push Notification watch command" I've already verified the differents answers and suggestions and nothing changed.
What am I missing ? Thanks a lot, I would be grateful for any tip, hint, answer, pointer :-)
If you're trying to debug a webhook, either for mechanics (i.e. proper setup on the sender side) or correctness (examining the post payload) try using RequestBin. If possible, register both a RequestBin URL along with your intended destination URL. RequestBin will give you full visibility of what the payload is, as well as give you assurances that you've set things up properly on the sender (Google) side.
You will need a valid SSL ceritificate. Self-signed is not allowed.
I am trying to develop a Proxy Server (TCP) in Android for YouTubeApp. I am going to use ProxyDroid (the phone is rooted), so that request/response goes through my Proxy Server. As its Android, I am using Java. But there are couple of challenges:
The request from the YouTubeApp has to be parsed. Is there is any existing library for parsing HTTP requests (in Java and can be used with Android)? Can anyone even suggest some snippets of code for this purpose? Or, do I have to do the parsing myself?
When requesting a remote site; e.g. www.google.com, do I have to use URL and openConnection()? Or, can sockets be used also? I am trying to find a way so that I can use sockets and get response from any site?
Hope to hear from u guys soon.
So I want to write a servlet which uploads a video to a youtube channel using the Java API, but I can't seem to find a way of specifying that I want to go through a proxy server. I've seen an example on this site where someone managed to do this using C#, but the Classes they used don't seem to exist in the Java API. Has anybody managed to successfully do this?
YouTubeService service = new YouTubeService(clientID, developerKey);
I'm new here so I'm unable to comment on posts (and a little late on this topic), but Jesper, I believe this is the C# sample that the original poster was talking about: How to upload to YouTube using the API via a Proxy Server
I can see no "direct" way of porting that example to Java though, since the GDataRequestFactory doesn't seem to have any proxy-related fields.
I was also having issues with the Java client Library with proxy in our application. Basically, the library picks up the global Java proxy settings:
System.getProperty("http.proxyHost");
System.getProperty("http.proxyPort");
but for some reason not everywhere. To be more precise, even with a proxy server properly configured in Java, YouTube authentication (calling service.setUserCredentials("login", "pwd")) would use a direct connection and ignore the proxy. But a video upload (calling service.insert(...)) would use the proxy correctly.
With the help of folks at the official YouTube API mailing list, I was able to nail this down. The issue is that the authentication is performed using SSL (HTTPS) and since there is a different set of properties for the HTTPS proxy, this didn't work. The fix is to simply set https.proxy* properties as well (in addition to http.proxy*), so that these point to a valid proxy server too:
System.getProperty("https.proxyHost");
System.getProperty("https.proxyPort");