I need to connect to an FTP server and browse all files without using any libraries like apache.commons because I don't have the option to get these libraries at the moment.
I tried using a simple URL connection:
URL url = new URL("username:password#ip/folder/");
URLConnection conn = url.openConnection();
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ( (line = reader.readLine()) != null ) {
System.out.println(line);
}
reader.close();
When I don't include the /folder/ it works, but it prints me many things that I can't even see in the home location of the FTP server, I don't know if it gives me the files or other data.
But when I do include /folder/ i get error CWD /folder/:550 failed to change directory
and I cannot change the directoy's permissions, its read-only and thats what I need.
What is wrong with it? Is it possible to do with just java's default libraries?
First, you need to use an FTP URL:
URL url = new URL("ftp:username:password#ip/folder/");
assuming username and password are substituted with their correct values.
Second, if you have FTP access to folder it will deliver you a directory listing in some format. If you don't, you need to study the exception message you get. If you omit /folder it will give you a listing of whatever the FTP server's default root directory is for that username. The code 550 means either an access problem or that the directory doesn't exist.
Related
When I want to get the source code of a specific web page, I use following code:
URL url = new URL("https://google.de");
URLConnection urlConnect = url.openConnection();
BufferedReader br = new BufferedReader(new InputStreamReader(urlConnect.getInputStream())); //Here is the error with the amazon url
StringBuffer sb = new StringBuffer();
String line, htmlData;
while((line=br.readLine())!=null){
sb.append(line+"\n");
}
htmlData = sb.toString();
The code above works without problems, but when your url is called...
URL url = new URL("https://amazon.de");
...then you might get sometimes a IOException error -> Server error code 503. In my opinion, this doesn't make any sense, because I can enter the amazon web page with the browser without any errors.
When accessing https://amazon.de with curl -v https://amazon.de you either get a 503 or a 301 status code in the response (When following the redirect, you get a 503 from the referenced location https://www.amazon.de/). The body contains the following comment:
To discuss automated access to Amazon data please contact api-services-support#amazon.com.
For information about migrating to our APIs refer to our Marketplace APIs at https://developer.amazonservices.de/ref=rm_5_sv, or our Product Advertising API at https://partnernet.amazon.de/gp/advertising/api/detail/main.html/ref=rm_5_ac for advertising use cases.
I assume Amazon is returning this response when your request is detected as coming from a non browser context (i.e. by parsing the user agent) to hint you towards using the APIs and not crawling the sites directly.
I am trying to develop a Java web application with SSO by following this azure tutorial. I created an account in Azure and created an AD. Developed and deployed the code in Tomcat. When I try to access the page, I am getting the following error
Exception - java.io.IOException: Server returned HTTP response code: 403 for URL: https://graph.windows.net/ppceses.onmicrosoft.com/users?api-version=2013-04-05
I do not find enough answers for this error. I changed the api-version to 1.6. Even then it did not work.
MORE ANALYSIS:
After troubleshooting, I found out that the logged-in user info is fetched and is available in Sessions object. It errors out when its trying to read the response and convert into the String object. Following is the calling method where it errors out.
HttpClientHelper.getResponseStringFromConn(conn, true);
Actual method to write the response into String:
public static String getResponseStringFromConn(HttpURLConnection conn, boolean isSuccess) throws IOException {
BufferedReader reader = null;
if (isSuccess) {
reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
} else {
reader = new BufferedReader(new InputStreamReader(conn.getErrorStream()));
}
StringBuffer stringBuffer = new StringBuffer();
String line = "";
while ((line = reader.readLine()) != null) {
stringBuffer.append(line);
}
return stringBuffer.toString();
}
The actual issue is on the Graphic API where we try to read the response in the String format.
#Anand, According to Microsoft Graph error responses and resource types, the response code 403 means Forbidden below.
Access is denied to the requested resource. The user might not have enough permission.
Please move to the CONFIGURE tab of your application registered in your AAD domain on Azure classic portal, then check whether enable enough permission, please see the figure below.
I got the same error, been struggling with it a few days. What I noticed was that even if I checked ALL permissions for Windows Azure Active Directory I still got the 403. So, I deleted the app in App Registrations and created it again from scratch, generated new application key and readded reply urls. In Required Permissions/Windows Azure Active Directory check:
Sign in and read user profile
Access the directory as the signed-in user
I can now call me/memberOf successfully.
Hope it helps.
The below worked for me.
At the active directory app registrations -> app ->settings->permissions-> enable delegate permissions to read directory data. Save and close the blade. Also Click Grant Permissions and close the blade.
Once the above is done, Log out and Log in back with a fresh token to the application. (Guess the token with prior authorizations will not reflect the latest permission changes and hence the re-login may have worked in my case)
I am reading xml in java via url, this is my code:
String web="example.com";
URL url = new URL(web);
URLConnection conn = url.openConnection();
conn.setDoOutput(true);
OutputStreamWriter writer = new OutputStreamWriter(conn.getOutputStream());
writer.write(ufx);
writer.flush();
BufferedReader reader = new BufferedReader(new InputStreamReader(conn.getInputStream()));
String line;
while ((line = reader.readLine()) != null) {
answer.append(line);
}
writer.close();
reader.close();
} catch (Exception e) {
e.printStackTrace();
}
return answer.toString();
My problem is that the web url that I am using is blocked, and I want to read data from the web via input stream. The web opens successfully in mozilla after removing no proxy in mozilla.
How do I achieve this in java ?
There are system properties which specify the proxy configuration used by java. You can pass them as command line arguments, or set them first thing in your application:
java -Dhttp.proxyHost=1.1.1.1 -Dhttp.proxyPort=1234 -jar somejar.jar
Note that there are more, and you can also set different proxy settings for different protocols like http, https, and you can also specify exceptions.
To define an exception (not to use proxy), you can use the http.nonProxyHosts system property, for example:
java -Dhttp.proxyHost=webcache.example.com -Dhttp.proxyPort=8080
-Dhttp.nonProxyHosts="localhost|host.example.com"
Check more info on Official Oracle documentation.
Since you are not using a programmatic proxy, it is using the system properties:
http.proxyHost
http.proxyPort
http.nonProxyHosts
You can either not set them or update the last one.
UPDATE
Upon rereading your question I'm actually not sure whether you want to use a proxy or you don't want to use one. Can you specify? Either way the properties can help you or you can look at URL.openConnection(Proxy)
Add following lines of code to your source :-
System.getProperties().put("http.proxyHost", "");
System.getProperties().put("http.proxyPort", "");
System.getProperties().put("http.proxyUser", "");
System.getProperties().put("http.proxyPassword", "");
Since you do not want to use proxy , you can provide blank values to the proxy settings
or you can do change settings using java control panel:-
In the Java Control Panel, under the General tab, click on Network Settings.
Select the Use Browser Settings checkbox.
Click OK to save your changes.
Reference for properties
I am trying to download an vcalendar using a java application, but I can't download from a specific link.
My code is:
URL uri = new URL("http://codebits.eu/s/calendar.ics");
InputStream in = uri.openStream();
int r = in.read();
while(r != -1) {
System.out.print((char)r);
r = in.read();
}
When I try to download from another link it works (ex: http://www.mysportscal.com/Files_iCal_CSV/iCal_AUTO_2011/f1_2011.ics). Something don't allow me to download and I can't figure out why, when I try with the browser it works.
I'd follow this example. Basically, get the response code for the connection. If it's a redirect (e.g. 301 in this case), retrieve the header location and attempt to access the file using that.
Simplistic Example:
URL uri = new URL("http://codebits.eu/s/calendar.ics");
HttpURLConnection con = (HttpURLConnection)uri.openConnection();
System.out.println(con.getResponseCode());
System.out.println(con.getHeaderField("Location"));
uri = new URL(con.getHeaderField("Location"));
con = (HttpURLConnection)uri.openConnection();
InputStream in = con.getInputStream();
You should check what that link actually provides. For example, it might be a page that has moved, which gives you back an HTTP 301 code. Your browser will automatically know to go and fetch it from the new URL, but your program won't.
You might want to try, for example, wireshark to sniff the actual traffic when you do the browser request.
I think too that there is a redirect. The browser downloads from ssl secured https://codebits.eu/s/calendar.ics. Try using a HttpURLConnection, it should follow redirects automatically:
HttpURLConnection con = (HttpURLConnection)uri.openConnection();
InputStream in = con.getInputStream();
I am trying to download an xml.gz file from a remote server with HttpsURLConnection in java, but I am getting an empty response. Here is a sample of my code:
URL server = new URL("https://www.myurl.com/path/sample_file.xml.gz");
HttpsURLConnection connection = (HttpsURLConnection)server.openConnection();
connection.connect();
When I try to get an InputStream from the connection, it is empty. (If I try connection.getInputStream().read() I get -1) The file I am expecting is approximately 50MB.
To test my sanity, I aslo tried entering the exact same url in my browser, and it did return the file I needed. Am I missing something? Do I have to set some sort of parameter in the connection? Any help/direction is much appreciated.
Is any exception being logged? Is the website presenting a self-signed SSL certificate, or one that is not signed by a CA? There are several reasons why it might work fine in your browser (the browser might have been told to accept self-signed certs from that domain) and not in your code.
What are the results of using curl or wget to fetch the URL?
The fact that the InputStream is empty / result from the InputStream.read() == -1 implies that there is nothing in the stream to read, meaning that the stream was not able to even be set up properly.
Update: See this page for some info on how you can deal with invalid/self-signed certificates in your connection code. Or, if the site is presenting a certificate but it is invalid, you can import it into the keystore of the server to tell Java to trust the certificate. See this page for more info.
Verify the response code is 200
Check that connection.contentType to verify the content type is recognized
You may need to add a Content-Handler for the GZ mime type, which I can't recall off the top of my head.
After the comment describing the response code as 3xx,
Set 'connection.setFollowRedirects(true)'
Should fix it.
Turns out the download wasn't working because the remote server was redirecting me to a new url to download the file. Even though connection.setFollowRedirects(true) was set, I still had to manually set up a new connection for the redirected URL as follows:
if (connection.getResponseCode() == 302 && connection.getHeaderField("location") != null){
URL server2 = new URL(connection.getHeaderField("location"));
HttpURLConnection connection2 = (HttpURLConnection)server2.openConnection();
connection2.connect();
InputStream in = connection2.getInputStream();
}
After that, I was able to retrieve the file from the input stream. Thanks for all your help guys!