So I have this FTP server with a bunch of folders and files inside.
My program needs to access this server, read all of the files, and display their data.
For development purposes I've been working with the files on my hard drive, right in the "src" folder.
But now that the server is up and running, I need to connect the software to it.
Basically what I want to do is get a list of the Files in a particular folder on the server.
This is what I have so far:
URL url = null;
File folder = null;
try {
url = new URL ("ftp://username:password#www.superland.example/server");
folder = new File (url.toURI());
} catch (Exception e) {
e.printStackTrace();
}
data = Arrays.asList(folder.listFiles(new FileFilter () {
public boolean accept(File file) {
return file.isDirectory();
}
}));
But I get the error "URI scheme is not 'file'."
I understand this is because my URL starts with "ftp://" and not "file:"
However I can't seem to figure out what I'm supposed to do about it!
Maybe there's a better way to go about this?
File objects cannot handle an FTP connection, you need to use a URLConnection:
URL url = new URL ("ftp://username:password#www.superland.example/server");
URLConnection urlc = url.openConnection();
InputStream is = urlc.getInputStream();
...
Consider as an alternative FTPClient from Apache Commons Net which has support for many protocols. Here is an FTP list files example.
if you use URI with file you can use your code but , but when you want to use ftp so you need to this kind of code; code list the name of the files under your ftp server
import java.net.*;
import java.io.*;
public class URLConnectionReader {
public static void main(String[] args) throws Exception {
URL url = new URL("ftp://username:password#www.superland.example/server");
URLConnection con = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(
con.getInputStream()));
String inputLine;
while ((inputLine = in.readLine()) != null)
System.out.println(inputLine);
in.close();
}
}
EDITED Demo Code Belongs to Codejava
package net.codejava.ftp;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
public class FtpUrlListing {
public static void main(String[] args) {
String ftpUrl = "ftp://%s:%s#%s/%s;type=d";
String host = "www.myserver.com";
String user = "tom";
String pass = "secret";
String dirPath = "/projects/java";
ftpUrl = String.format(ftpUrl, user, pass, host, dirPath);
System.out.println("URL: " + ftpUrl);
try {
URL url = new URL(ftpUrl);
URLConnection conn = url.openConnection();
InputStream inputStream = conn.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));
String line = null;
System.out.println("--- START ---");
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
System.out.println("--- END ---");
inputStream.close();
} catch (IOException ex) {
ex.printStackTrace();
}
}
}
Related
The simple Java code below works. Is there an easy way to find out / inspect the HTTP request (not the response) headers actually sent?
import java.net.URL;
import java.io.*;
import javax.net.ssl.HttpsURLConnection;
public class Test {
public static void main(String[] args) throws Exception {
String httpsURL = "https://api.gdax.com/products/BTC-USD/book?level=1";
URL myurl = new URL(httpsURL);
HttpsURLConnection con = (HttpsURLConnection) myurl.openConnection();
InputStream ins = con.getInputStream();
InputStreamReader isr = new InputStreamReader(ins);
BufferedReader in = new BufferedReader(isr);
String inputLine;
while ((inputLine = in.readLine()) != null) {
System.out.println(inputLine);
}
in.close();
}
}
I think you can not do it programaticaly, you could use some kind of proxy like TCP/IP monitor in Eclipse. Or enable the debbug option for java adding these options:
-Djava.util.logging.config.file=logging.properties
And put in logging.properties (by default in JRE_HOME\lib) the following property
sun.net.www.protocol.http.HttpsURLConnection.level = ALL
So in a nutshell, I'm just trying to get a small working skeleton program that I can use to sort of learn about Http communication and "feel" my way around to figure out what I will eventually need for a bigger program I am working on. This particular code here is actually just a chopped up version of an example from the Apache libraries. I could compile the examples listed on the Apache website, but they didn't run properly, giving a "java.net.ConnectException". I figured it had to do with Windows c-blocking a program like this from making a connection, and that I would need to run it as an administrator. I then tried taking the code and throwing it into an executable jar file, but I get a Cannot-find-or-load-main-class error. Am I an idiot or is the Apache library a little outdated/not fit for Win 8/something else?
Code below:
package NewProject;
import java.net.Socket;
import org.apache.http.ConnectionReuseStrategy;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.impl.DefaultBHttpClientConnection;
import org.apache.http.impl.DefaultConnectionReuseStrategy;
import org.apache.http.message.BasicHttpRequest;
import org.apache.http.protocol.HttpCoreContext;
import org.apache.http.protocol.HttpProcessor;
import org.apache.http.protocol.HttpProcessorBuilder;
import org.apache.http.protocol.HttpRequestExecutor;
import org.apache.http.protocol.RequestConnControl;
import org.apache.http.protocol.RequestContent;
import org.apache.http.protocol.RequestExpectContinue;
import org.apache.http.protocol.RequestTargetHost;
import org.apache.http.protocol.RequestUserAgent;
import org.apache.http.util.EntityUtils;
class NewProject
{
public static void main(String[] args) throws Exception
{
HttpProcessor httpproc = HttpProcessorBuilder.create()
.add(new RequestContent())
.add(new RequestTargetHost())
.add(new RequestConnControl())
.add(new RequestUserAgent("Test/1.1"))
.add(new RequestExpectContinue(true)).build();
HttpRequestExecutor httpexecutor = new HttpRequestExecutor();
HttpCoreContext coreContext = HttpCoreContext.create();
HttpHost host = new HttpHost("localhost", 8080);
coreContext.setTargetHost(host);
Out os = new Out("TestOut.txt");
DefaultBHttpClientConnection conn = new DefaultBHttpClientConnection(8 * 1024);
ConnectionReuseStrategy connStrategy = DefaultConnectionReuseStrategy.INSTANCE;
try
{
String[] targets =
{
"http://www.google.com/"
};
for (int i = 0; i < targets.length; i++)
{
if (!conn.isOpen())
{
Socket socket = new Socket(host.getHostName(), host.getPort());
conn.bind(socket);
}
BasicHttpRequest request = new BasicHttpRequest("GET", targets[i]);
os.println(">> Request URI: " + request.getRequestLine().getUri());
httpexecutor.preProcess(request, httpproc, coreContext);
HttpResponse response = httpexecutor.execute(request, conn, coreContext);
httpexecutor.postProcess(response, httpproc, coreContext);
os.println("<< Response: " + response.getStatusLine());
os.println(EntityUtils.toString(response.getEntity()));
os.println("==============");
if (!connStrategy.keepAlive(response, coreContext))
{
conn.close();
}
else
{
os.println("Connection kept alive...");
}
}
}
catch (IndexOutOfBoundsException iob)
{
os.println("What happened here?");
}
finally
{
conn.close();
}
return;
}
}
... they didn't run properly, giving a "java.net.ConnectException"
That could be caused by lots of things. There are clues in the exception message ... which you chose not to share with us.
... "Cannot find or load Main-Class"
Again multiple possible causes, and there are clues in the exception message ... which you chose not to share with us.
But the fact that you have created a JAR file plus the "Main-Class" hint in the error message fragment you provided suggest that you made a mistake in the creation of the JAR file; i.e. you used the wrong name for the "Main-Class" attribute.
Given that source code, the "Main-Class" attribute should be "NewProject.NewProject". I suspect you set it to something else.
A second possibility is that you haven't handled the dependency on the Apache library correctly. The Apache classes need to be on the classpath specified by the JAR file. (You can't use a -cp argument or $CLASSPATH when you launch with java -jar.)
Am I an idiot or is the Apache library a little outdated/not fit for Win 8/something else?
There is nothing wrong with the Apache library.
The code you posted seems a little low level (e.g. interacting directly with Socket connections). The code posted below should give you what it sounds like you are looking for. The classes used also give you a lot of inroads into setting and getting http parameters (e.g. headers, time-outs, etc).
package org.yaorma.example.http.client;
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.OutputStreamWriter;
import java.net.HttpURLConnection;
import java.net.URL;
public class HttpClientExample {
public static void main(String[] args) throws Exception {
String response;
response = get("http://www.google.com");
System.out.println("RESPONSE FROM GET -----------------------------------------");
System.out.println(response);
response = post("http://httpbin.org/post", "This is the message I posted to httpbin.org/post");
System.out.println("RESPONSE FROM POST -----------------------------------------");
System.out.println(response);
}
/**
* Method to post a request to a given URL.
*/
public static String post(String urlString, String message) {
try {
// get a connection
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
// set the parameters
conn.setRequestMethod("POST");
conn.setDoOutput(true);
// send the message
OutputStream os = conn.getOutputStream();
BufferedWriter writer = new BufferedWriter(new OutputStreamWriter(os, "UTF-8"));
writer.write(message);
writer.flush();
writer.close();
os.close();
// get the response
conn.connect();
InputStream content = (InputStream) conn.getInputStream();
// read the response
BufferedReader in = new BufferedReader(new InputStreamReader(content));
String rtn = "";
String line;
while ((line = in.readLine()) != null) {
rtn += line + "\n";
}
return rtn;
} catch (Exception exp) {
throw new RuntimeException(exp);
}
}
/**
* Method to do a get from a given URL.
*/
public static String get(String urlString) {
try {
// get a connection
URL url = new URL(urlString);
HttpURLConnection conn = (HttpURLConnection) url.openConnection();
// set the parameters
conn.setRequestMethod("GET");
conn.setDoOutput(true);
// get the response
conn.connect();
InputStream content = (InputStream) conn.getInputStream();
// read the response
BufferedReader in = new BufferedReader(new InputStreamReader(content));
String rtn = "";
String line;
while ((line = in.readLine()) != null) {
rtn += line + "\n";
}
return rtn;
} catch (Exception exp) {
throw new RuntimeException(exp);
}
}
}
I all but copied the following code from here. I get a java.net.SocketException on line 10 saying "Connection Reset".
import java.net.*;
import java.io.*;
import org.apache.commons.io.*;
public class HelloWorld {
public static void main(String[] x) {
try {
URL url = new URL("http://money.cnn.com/2013/06/07/technology/security/page-zuckerberg-spying/index.html");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
String encoding = con.getContentEncoding();
encoding = encoding == null ? "UTF-8" : encoding;
String body = IOUtils.toString(in, encoding);
System.out.print(body);
} catch (Exception e) {
e.printStackTrace();
}
}
}
I'm worried this may not actually be an issue with the actual code but rather some permission I need to give Java. Is there something wrong with my code or is this an environment issue?
I used your code with small modification cause I don't have IOUtils at hands. And it works as it should. There is no need to set agent. No special privileges also as I run it by normal user.
try {
URL url = new URL("http://money.cnn.com/2013/06/07/technology/security/page-zuckerberg-spying/index.html");
URLConnection con = url.openConnection();
InputStream in = con.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(in));
StringBuilder sb = new StringBuilder();
String line = br.readLine();
while (line != null) {
sb.append(line);
line = br.readLine();
}
System.out.print(sb.toString());
} catch (Exception e) {
e.printStackTrace();
}
I have a html file stored on the server. I have the URL path something like this: <https://localhost:9443/genesis/Receipt/Receipt.html >
I want to read the contents of this html file which would contain tags, from the url i.e. the source code of the html file.
How am I supposed to do this? This is a server side code and can't have a browser object and I am not sure using a URLConnection would be a good option.
What should be the best solution now?
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLConnection;
public class URLContent {
public static void main(String[] args) {
try {
// get URL content
String a = "http://localhost:8080//TestWeb/index.jsp";
URL url = new URL(a);
URLConnection conn = url.openConnection();
// open the stream and put it into BufferedReader
BufferedReader br = new BufferedReader(
new InputStreamReader(conn.getInputStream()));
String inputLine;
while ((inputLine = br.readLine()) != null) {
System.out.println(inputLine);
}
br.close();
System.out.println("Done");
} catch (MalformedURLException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}
Resolved it using spring
added the bean to the spring config file
<bean id = "receiptTemplate" class="org.springframework.core.io.ClassPathResource">
<constructor-arg value="/WEB-INF/Receipt/Receipt.html"></constructor-arg>
</bean>
then read it in my method
// read the file into a resource
ClassPathResource fileResource =
(ClassPathResource)context.getApplicationContext().getBean("receiptTemplate");
BufferedReader br = new BufferedReader(new FileReader(fileResource.getFile()));
String line;
StringBuffer sb =
new StringBuffer();
// read contents line by line and store in the string
while ((line =
br.readLine()) != null) {
sb.append(line);
}
br.close();
return sb.toString();
import java.net.*;
import java.io.*;
//...
URL url = new URL("https://localhost:9443/genesis/Receipt/Receipt.html");
url.openConnection();
InputStream reader = url.openStream();
For exemple :
URL url = new URL("https://localhost:9443/genesis/Receipt/Receipt.html");
URLConnection con = url.openConnection();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String l;
while ((l=in.readLine())!=null) {
System.out.println(l);
}
You could use the inputstream in other ways, not just printing it.
Of course if you have the path to the local file, you can also do
InputStream in = new FileInputStream(new File(yourPath));
Simplest way in my opinion is to use IOUtils
import com.amazonaws.util.IOUtils;
...
String uri = "https://localhost:9443/genesis/Receipt/Receipt.html";
String fileContents = IOUtils.toString(new URL(uri).openStream());
System.out.println(fileContents);
I'm trying to read html code from a URL Connection. In one case the html file I'm trying to read includes 5 line breaks before the actual doc type declaration. In this case the input reader throws an exception for EOF.
URL pageUrl =
new URL(
"http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html"
);
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
DataInputStream dis = new DataInputStream(getConn.getInputStream());
//some read method here
Has anyone ran into a problem like this?
URL pageUrl = new URL("http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
DataInputStream dis = new DataInputStream(getConn.getInputStream());
String urlData = "";
while ((urlData = dis.readUTF()) != null)
System.out.println(urlData);
//exception thrown
java.io.EOFException
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:323)
at java.io.DataInputStream.readUTF(DataInputStream.java:572)
at java.io.DataInputStream.readUTF(DataInputStream.java:547)
in the case of bufferedreader, it just responds null and doesn't continue
pageUrl = new URL("http://www.nytimes.com/2011/03/15/sports/basketball/15nbaround.html");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
BufferedReader br = new BufferedReader(new InputStreamReader(getConn.getInputStream()));
String urlData = "";
while(true)
urlData = br.readLine();
System.out.println(urlData);
outputs null
You're using DataInputStream to read data that wasn't encoded using DataOutputStream. Examine the documented behavior for your call to DataInputStream#readUtf(); it first reads two bytes to form a 16-bit integer, indicating the number of bytes that follow comprising the UTF-encoded string. The data you're reading from the HTTP server is not encoded in this format.
Instead, the HTTP server is sending headers encoded in ASCII, per RFC 2616 sections 6.1 and 2.2. You need to read the headers as text, and then determine how the message body (the "entity") is encoded.
This works fine:
package url;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.net.URL;
/**
* UrlReader
* #author Michael
* #since 3/20/11
*/
public class UrlReader
{
public static void main(String[] args)
{
UrlReader urlReader = new UrlReader();
for (String url : args)
{
try
{
String contents = urlReader.readContents(url);
System.out.printf("url: %s contents: %s\n", url, contents);
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
public String readContents(String address) throws IOException
{
StringBuilder contents = new StringBuilder(2048);
BufferedReader br = null;
try
{
URL url = new URL(address);
br = new BufferedReader(new InputStreamReader(url.openStream()));
String line = "";
while (line != null)
{
line = br.readLine();
contents.append(line);
}
}
finally
{
close(br);
}
return contents.toString();
}
private static void close(Reader br)
{
try
{
if (br != null)
{
br.close();
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
}
This:
public class Main {
public static void main(String[] args)
throws MalformedURLException, IOException
{
URL pageUrl = new URL("http://www.google.com");
URLConnection getConn = pageUrl.openConnection();
getConn.connect();
BufferedReader dis = new BufferedReader(
new InputStreamReader(
getConn.getInputStream()));
String myString;
while ((myString = dis.readLine()) != null)
{
System.out.println(myString);
}
}
}
Works perfectly. The URL you are supplying, however, returns nothing.