I have the following code, it works totally fine on my local development server, but when I uploaded to the deployment server, I always hit file not found Exception
String urlStr = "http://" + getContext().getRequest().getServerName() +
getContext().getServletContext().getContextPath() + "test.action";
URL url = new URL(urlStr);
InputStream input = url.openStream(); //Error always occurs here, it gives me the correct URL but it says file not found.
Can anyone help me with this?
Because its a HTTP URL the correct way would be as follows.
String urlStr = "http://" + getContext().getRequest().getServerName() +
getContext().getServletContext().getContextPath() + "test.action";
URL url = new URL(urlStr);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
if (conn.getResponseCode() == HttpURLConnection.HTTP_ACCEPTED) {
InputStream input = conn.getInputStream();
}
I think that #deadlock's comments is probably the key to solving this.
You are getting a FileNotFoundException because the remote server is sending a 404 Not Found response. The most likely explanation is that you are attempting to connect using the wrong URL. Print out the URL string before you try to connect.
All the evidence is pointing to the fact that the server is sending "404 Not Found" responses ... for both versions of the code. This normally means that your URL is wrong. But it is also possible for it to be other things:
You may be using different proxies in the Java and browser cases, resulting in the Java case reaching some server that doesn't understand the URL.
It is conceivable that the server is implementing some anti web scraping mechanism, and sending you 404 responses `cos this thinks (rightly) that your requests aren't coming from a web browser,
Related
I have an android app and REST server made in Spring Tool Suite. REST is connecting to database and returns list of Json objects depending on the query in the GET request.
This is my REST response to GET method
#RequestMapping(value = "/short", method = RequestMethod.GET)
public Map<String,List<Temperature>> shortPeriod(#RequestParam(value = "time",required = false)
#DateTimeFormat(pattern="yyyy:MM:dd HH:mm:ss") Date time){
Map<String, List<Temperature>> results = new HashMap<String, List<Temperature>>();
List<Temperature> temperatureList = new ArrayList<Temperature>();
System.out.println("TEST!");
try {
temperatureList = dataServices.getEntityList(time,"short");
results.put("data", temperatureList);
} catch (Exception e) {
e.printStackTrace();
}
return results;
}
and this is how i connect on android
String tempUrl = new String("http://" + IP + ":8080/" + "RestServer/temperature/" + tableName + "?time=" + urlQueryTimestamp);
String encodedUrlTemp = tempUrl.replace(" ", "%20");
String encodedUrl = encodedUrlTemp.replace("-", ":");
connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("GET");
connection.setDoInput(true);
connection.setDoOutput(true);
connection.connect();
I have put test prints and It's working ok till connection.connect();
The url looks something like this
http://192.168.1.106:8080/RestServer/temperature/short?time=2015:06:05%2010:20:03
I'm printing "url" variable before connecting and when I take that url and paste it into chrome I get normal response as I should. But when I'm trying to connect in my app I get "method not allowed". I have found some solutions about that exception and it's usually problem with POST method not implemented on server, but with same url i get normal response on chrome on my computer and even on my mobile chrome on smartphone.
Other thing is that I have one other server with hardcoded json data for testing and i can connect normally there when there is no query in http request, so only this
http://192.168.1.106:8080/RestServer/temperature/short
So my assumption is that chrome escapes some strings internally or sets up some header infromation by itself when using a query, and android doesn't. Server doesn't even records request as one I pasted in code above, it's like im sending wrong url.
One other thing, I tried using url encoder from android library but it escapes some letters that shouldn't be escaped and I read that it escapes it for html, so I'm doing it manually(maybe I'm doing it wrong).
I turned off the firewall, checked ip adress dozen of times and I have no ideas more.
After trying absolutely every solution I found with escapeing, changing library's and reading about URI's and URL's, I have found the solution on android developer documentation page where it says:
HttpURLConnection uses the GET method by default. It will use POST if
setDoOutput(true) has been called. Other HTTP methods (OPTIONS, HEAD,
PUT, DELETE and TRACE) can be used with setRequestMethod(String).
Hope this helps to someone
I am sending commands to a server using http, and I currently need to parse a response that the server sends back (I am sending the command via the command line, and the servers response appears in my browser).
There are a lot of resources such as this: Saving a web page to a file in Java, that clearly illustrate how to scrape a page such as cnn.com. However, since this is a response page that is only generated when the camera receives a specific command, my attempts to use the method described by Mike Deck (in the link above) have met with failure. (Specifically, when my program requests the page again the server returns a 401 error.)
The response from the server opens a new tab in my browser. Essentially, I need to know how to save the current web page using java, since reading in a file is probably the most simple way to approach this. Do any of you know how to do this?
TL;DR How do you save the current webpage to a webpage.html or webpage.txt file using java?
EDIT: I used Base64 from the Apache commons codec, which solved my 401 authentication issue. However, I am still getting a 400 error when I attempt to connect my InputStream (see below). Does this mean a connection isn't being established in the first place?
URL url = new URL ("http://"+ipAddress+"/axis-cgi/record/record.cgi?diskid=SD_DISK");
byte[] encodedBytes = Base64.encodeBase64("root:pass".getBytes());
String encoding = new String (encodedBytes);
HttpURLConnection connection = (HttpURLConnection) url.openConnection();
connection.setRequestMethod("POST");
connection.setDoInput (true);
connection.setRequestProperty ("Authorization", "Basic " + encoding);
connection.connect();
InputStream content = (InputStream)connection.getInputStream();
BufferedReader in = new BufferedReader (new InputStreamReader (content));
String line;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
EDIT 2: Changing the request to a GET resolved the issue.
So while scrutinizing my code above, I decided to change
connection.setRequestMethod("POST");
to
connection.setRequestMethod("GET");
This solved my problem. In hindsight, I think the server was not recognizing the HTTP because it is not set up to handle the various trappings that come along with post.
I'm trying to get an xml file from a distant server.
The server asks a username and password.
My code works fine on Android 4.x but on Android 2.x i get an Error 400.
400 Bad Request
Your browser sent a request that this server could not understand.
here is my code :
URL url = new URL("http://username:password#xx.xx.xx.xx:80/pathToXML/file.xml");
URLConnection uc = url.openConnection();
String val = (new StringBuffer("username").append(":").append("password")).toString();
byte[] base = val.getBytes();
String authorizationString = "Basic " + new String(Base64.encode(base, Base64.DEFAULT));
uc.setRequestProperty("Authorization", authorizationString);
uc.connect();
InputStream ins = uc.getInputStream(); // it fails on this line
I tried many things (adding Content-Length, SetRequestMethod to POST, set user-Agent) but it doesn't work.
If someone has any idea i'm open to it.
Thanks a lot for your help.
Cheers
Days ago I try to make an application similar to No-IP DUC (Dynamic DNS Update Clients) in java but I have presented a problem and not how to fix ...
Now I can retrieve the domains I have registered with the following line of code:
URLConnection conexion = new URL( "http://dynupdate.no-ip.com/list-hosts.php?email="+USER+"&pass="+PASSWORD ).openConnection();
but when I update with the following URL you recommend on page http://www.no-ip.com/integrate/request, I get an exception ...
URLConnection conexion = new URL( "http://"+USER+":"+PASSWORD+"#dynupdate.no-ip.com/nic/update?hostname="+DOMAIN+"&myip="+IP ).openConnection();
The strange thing is that manually copy the URL in the address, the update is done without problem ...
If anyone can help me I appreciate it a lot ...
Well I faced this situation as well. Found out the reason for this error is that we are using an email address for the username. The "#" present inside it conflicts with the resulting url. Try to use URLEncode to encode the username and the password part.
String username = URLEncoder.encode("abc#xyz.com", "UTF-8");
String password = URLEncoder.encode("password", "UTF-8");
URL url = new URL( "http://"+username+":"+password+"#dynupdate.no-ip.com/nic/update?hostname="+hostname+"&myip="+ip);
URLConnection urlConnection = url.openConnection();
String url = "http://maps.googleapis.com/maps/api/directions/xml?origin=Chicago,IL&destination=Los+Angeles,CA&waypoints=Joplin,MO|Oklahoma+City,OK&sensor=false";
URL google = new URL(url);
HttpURLConnection con = (HttpURLConnection) google.openConnection();
and I use BufferedReader to print the content I get 403 error
The same URL works fine in the browser. Could any one suggest.
The reason it works in a browser but not in java code is that the browser adds some HTTP headers which you lack in your Java code, and the server requires those headers. I've been in the same situation - and the URL worked both in Chrome and the Chrome plugin "Simple REST Client", yet didn't work in Java. Adding this line before the getInputStream() solved the problem:
connection.addRequestProperty("User-Agent", "Mozilla/4.0");
..even though I have never used Mozilla. Your situation might require a different header. It might be related to cookies ... I was getting text in the error stream advising me to enable cookies.
Note that you might get more information by looking at the error text. Here's my code:
try {
HttpURLConnection connection = ((HttpURLConnection)url.openConnection());
connection.addRequestProperty("User-Agent", "Mozilla/4.0");
InputStream input;
if (connection.getResponseCode() == 200) // this must be called before 'getErrorStream()' works
input = connection.getInputStream();
else input = connection.getErrorStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(input));
String msg;
while ((msg =reader.readLine()) != null)
System.out.println(msg);
} catch (IOException e) {
System.err.println(e);
}
HTTP 403 is a Forbidden status code. You would have to read the HttpURLConnection.getErrorStream() to see the response from the server (which can tell you why you have been given a HTTP 403), if any.
This code should work fine. If you have been making a number of requests, it is possible that Google is just throttling you. I have seen Google do this before. You can try using a proxy to verify.
Most browsers automatically encode URLs when you enter them, but the Java URL function doesn't.
You should Encode the URL with URLEncoder URL Encoder
I know this is a bit late, but the easiest way to get the contents of a URL is to use the Apache HttpComponents HttpClient project: http://hc.apache.org/httpcomponents-client-ga/index.html
you original page (with link) and the targeted linked page are not the same domain.
original-domain and target-domain.
I found the difference is in request header:
with 403 forbidden error,
request header have one line:
Referer: http://original-domain/json2tree/ipfs/ipfsList.html
when I enter url, no 403 forbidden,
the request header does NOT have above line referer: original-domain
I finally figure out how to fix this error!!!
on your original-domain web page, you have to add
<meta name="referrer" content="no-referrer" />
it will remove or prevent sending the Referer in header, works both for links and for Ajax requests made