How to get list of files from URL - java

I have an URL http ://......../somefolder/ I want to get the names of all the files inside this folder. I have tried this below code but it's showing error.
URL url = new URL("http://.............../pages/");
File f=new File(url.getFile());
String list[]=f.list();
for(String x:list)
{
System.out.println(x);
}
Error :-Exception in thread "main" java.lang.NullPointerException
at Directory.main(Directory.java:25)

It's not possible to do it like this.
HTTP has no concept of a "folder". The thing you see when you open that URL is just another web page, which happens to have a bunch of links to other pages. It's not special in any way as far as HTTP is concerned (and therefore HTTP clients, like the one built into Java).
That's not to say it's completely impossible. You might be able to get the file list another way.
Edit: The reason your code doesn't work is that it does something completely nonsensical. url.getFile() will return something like "/......./pages/", and then you pass that into the File constructor - which gives you a File representing the path /....../pages/ (or C:\......\pages\ on Windows). f.list() sees that that path doesn't exist on your computer, and returns null. There is no way to get a File that points to a URL, just like there's no way to get an int with the value 5.11.

Related

How to convert network path to URL in Java

I have literally searched the whole internet for this question but I have not found an answer. I have a file, in the network and I want to create an Itext image with it and for that, I have to convert its path to URL. The problem is when I use path.toURI().toURL() it appends my project path to the URL such that my URL ends up starting with C:/ which will not work.
Is there a way to just convert a string to file URL in java?
I have tried this:
String paths = "‪\\\\DESKTOP-A11F076\\Users\\Benson Korir\\Desktop\\walgotech\\passport.jpg";
String first = "file:" + paths.replaceAll("\\\\", "//").replaceAll("////", "//");
String second = "file://desktop-a11f076//Users//Benson Korir//Desktop//walgotech//passport.jpg";
System.out.println(first);
System.out.println(second);
The second string I have copied directly from the browser and it works fine. Funny this is these two strings output the same thing but the first string brings an error when I use it here:
Image image1 = Image.getInstance(second);
I am getting the error below:
java.io.FileNotFoundException: ‪\DESKTOP-A11F076\Users\Benson Korir\Desktop\walgotech\passport.jpg (The system cannot find the path specified)
If I got your requirement correctly, your path is a UNC file name, and that is the short form of an SMB path, with DESKTOP-A11F076 being the remote machine, and \Users\Benson Korir\Desktop\walgotech\passport.jpg being the path to the file on that machine.
If I am correct with that assumption, my understanding is that your URL have to look like this: smb://‪DESKTOP-A11F076/Users/Benson Korir/Desktop/walgotech/passport.jpg.
As far I remember is a Java java.io.File object capable to handle a UNC file name (this article implies that, too), but when translating it to a URI, it tries to make it absolute first, and there it fails in your case.
I usually avoid working on Windows whenever possible, therefore I have no environment to verify that.

Why can I get a valid url from getClass().getResource(), but the url that is returned creates a file that doesn't exist

I am trying to load some data into an AWS lambda and am using getClass().getResource() to do so. This returns a nice URL that in logs seemingly prints out a plausible url; however, when I try and make a file based on that path, I get a file that when I call .exists() returns false.
If I run the code bellow, the first print statement gives "returns exists: false"
Meanwhile, the second print statement gives something around the lines of "test path: /file:/var/task/lib/MyLambda-1.0.jar!/com/my/package/folders/file.end
File test = new File(cFile);
System.out.println("exists: " + test.exists());
System.out.println("test path: " + test.getAbsolutePath());
Not sure why this would be. If Java finds a file, then I would assume that the file exists...
Short answer: don't assume that the "path" of a URL is a file system pathname.
I am trying to load some data into an AWS lambda and am using getClass().getResource() to do so. This returns a nice URL that in logs seemingly prints out a plausible url;
Yes. (It would be nice if you showed us what the original URL looks like ... though I can guess.)
However, when I try and make a file based on that path, I get a file that when I call .exists() returns false.
OK, unless the URL has the protocol "file:", I would NOT expect that to work.
The path in a URL is a path that is intended for the protocol handler to resolve. The idea is that you use URL::openStream to open a stream to the resource named by the URL and then read it. The protocol handler takes care of interpreting the path (etc) and setting up the stream.
For a "file:" URL, the protocol handler will resolve the path in the file system, and provide you a stream to read the file.
For a "http:" URL, the protocol handler establishes a connection to the server, sends a GET request, and returns you a stream to read the response body.
For a "jar:" URL, the protocol handler opens the JAR file, finds the entry within the JAR file, and hands you a stream to read it.
And so on.
If you look at these, it is only in the "file:" case that there is a reasonable expectation that treating the path component of the URL as a file system pathname could work.
Looking at the pathname in your question:
file:/var/task/lib/MyLambda-1.0.jar!/com/my/package/folders/file.end
I surmise that the original URL was:
jar:file:/var/task/lib/MyLambda-1.0.jar!/com/my/package/folders/file.end
So what that says to the "jar:" protocol handler is:
Find the resource identified by the URL "file:/var/task/lib/MyLambda-1.0.jar"
Open it as a JAR file stream
Find the entry "/com/my/package/folders/file.end" in the JAR file's namespace
Open a stream to read that entry's content.
The JAR file protocol handler knows how to do that. But (clearly) the File class doesn't ... because that "path" is not a file system pathname.
How you solve this depends on what you really need.
If you just need a stream to read the resource, use getClass().getResourceAsStream(...) instead.
If it must be a file in the file system, you may have to get hold of the stream (see above), copy it to a temporary file, and use a File for the temporary file.
If you are doing the because you want to write to the "file", I would suggest that you give up on that idea. It is a bad idea for an application to try to update its resources. And in some cases it simply won't / cannot work.
Is your File test = new File(cFile), Is your cFile made correctly with a proper path? Maybe the last print statement is just picking up on the incorrect path you made? But in reality you don't actually have a file there. Have you checked manually?

"Access Denied" when trying to access file in web app

I have an xslt file stored in the folder Project/tools. (I'm using Netbeans IDE.)
I try to access this file in my code, but at run time, I get an AccessControlException: access denied.
The code is:
java.net.URI xsltURI = new java.net.URI(myUtil.getUri("xsltFile.xslt"));
Transformer transformer = factory.newTransformer(new StreamSource(new File(xsltURI)));
The myUtil instance must be used to access the URI for reasons not important here. I printed its output, and it correctly gives the relative path of the file.
I have tried to prefix the relative path with file:/// and file:///[fulldomain], but in each of these cases, it actually tries to access a hard drive on the server, even though I did not give a drive name anywhere. (!) It tries to access C:[relative-path], which isn't even where the file is anyway.
If I omit file:/// then I get that the URI is not absolute, and if I just give the full web address of the file I get a NullPointerException.
Any help at all would be greatly appreciated.
UPDATE: Following my comment below, my code resembles
java.net.URI xsltURI = new java.net.URI("https://host" + myB2U.getUri("xsltFile.xslt"));
java.net.URL xsltURL = xsltURI.toURL();
java.net.URLConnection myConnection = xsltURL.openConnection();
myConnection.connect(); //AccessControlException: access denied ("java.net.SocketPermission"...
java.io.InputStream xsltStream = myConnection.getInputStream();
Transformer transformer = factory.newTransformer(new StreamSource(xsltStream));
Is there something obvious that is wrong?
The file:// protocol tells Java to use file access to open the stream. If you don't want file access you should use a different protocol such as http://.
If you're using a relative path the URI should look something like file://./My/Relative/Path. The 3rd slash means that it is relative to the root.
From what I've gathered, I'm supposed to instantiate a URL object with the path of the file. From there, I'm supposed to be able to initialize a URLConnection from the URL. After I call the URL's connect() method, I'm supposed to be able to obtain an InputStream by calling the getInputStream() method.

Files, URIs, and URLs conflicting in Java

I am getting some strange behavior when trying to convert between Files and URLs, particularly when a file/path has spaces in its name. Is there any safe way to convert between the two?
My program has a file saving functionality where the actual "Save" operation is delegated to an outside library that requires a URL as a parameter. However, I also want the user to be able to pick which file to save to. The issue is that when converting between File and URL (using URI), spaces show up as "%20" and mess up various operations. Consider the following code:
//...user has selected file
File userFile = myFileChooser.getSelectedFile();
URL userURL = userFile.toURI().toURL();
System.out.println(userFile.getPath());
System.out.println(userURL);
File myFile = new File(userURL.getFile());
System.out.println(myFile.equals(userFile);
This will return false (due to the "%20" symbols), and is causing significant issues in my program because Files and URLs are handed off and often operations have to be performed with them (like getting parent/subdirectories). Is there a way to make File/URL handling safe for paths with whitespace?
P.S. Everything works fine if my paths have no spaces in them (and the paths look equal), but that is a user restriction I cannot impose.
The problem is that you use URL to construct the second file:
File myFile = new File(userURL.getFile());
If you stick to the URI, you are better off:
URI userURI = userFile.toURI();
URL userURL = userURI.toURL();
...
File myFile = new File(userURI);
or
File myFile = new File( userURL.toURI() );
Both ways worked for me, when testing file names with blanks.
Use instead..
System.out.println(myFile.toURI().toURL().equals(userURL);
That should return true.

Java file IO and "access denied" errors

I have been tearing my hair out on this and thus I am looks for some help .
I have a loop of code that performs the following
//imports ommitted
public void afterPropertiesSet() throws Exception{
//building of URL list ommitted
// urlMap is a HashMap <String,String> created and populated just prior
for ( Object urlVar : urlMap.keySet() ){
String myURLvar = urlMap.get(urlVar.toString);
System.out.println ("URL is "+myURLvar );
BufferedImage imageVar = ImageIO.read(myURLvar);//URL confirmed to be valid even for executions that fail
String fileName2Save = "filepath"// a valid file path
System.out.println ("Target path is "+fileName2Save );
File file2Save = new File (fileName2Save);
fileName2Save.SetWriteable(true);//set these just to be sure
fileName2Save.SetReadable(true);
try{
ImageIO.write (imageVar,"png",file2save)//error thrown here
}catch (Exception e){
System.out.println("R: "+file2Save.canRead()+" W: "+file2Save.canWrite()+" E:"+file2Save.canExecute()+" Exists: "+file2Save.exists+" is a file"+file2Save.isFile() );
System.out.println("parent Directory perms");// same as above except on parent directory of destination
}//end try
}//end for
}
This all runs on Windows 7 and JDK 1.6.26 and Netbeans,Tomcat 7.0.14 . The target directory is actually inside my netbeans project directory in a folder for a normal web app ( outside WEB-INF) where I would expect normally to have permission to write files.
When the error occurs I get one of two results for the file a.) All false b.)all true. The Parent directory permission never change all true except for isFile.
The error thrown ( java.IO.error with "access denied" ") does not occur every time ... in fact 60% of the time the loop runs it throws no error. The remaining 40% of the time I get the error on 1 of the 60+ files it writes. Infrequently the same one. The order in which the URLs it starts from changes everytime so the order in which the files are written is variable. The file names have short concise names like "1.png". The images are small..less then 8k.
In order to make sure the permissions are correct I have :
Given "full control" to EVERYONE from the net beans project directory down
Run the JDK,JRE and Netbeans as Administrator
Disabled UAC
Yet the error persists. Google searches for this seem to run the gamut and often read like vodoo. Clearly I ( and Java and Netbeans etc ) should have permission to write a file to the directory .
Anyone have any insight ? This is all ( code and the web server hosting the URL) on a closed system so I can't cut and paste code or stacktrace.
Update: I confirmed the imageURL is valid by doing a println & toString prior to each read. I then confirmed that a.) the web server hosting the target URL returned the image with a http 200 code b.) that the URL returned the image when tested in a web browser. In testing I also put a if () in after the read to confirm that the values was not NULL or empty. I also put in tests for NULL on all the other values . They are always as expected even for a failure .The error always occurs inside the try block. The destination directory is the same every execution. Prior to every execution the directory is empty.
Update 2: Here is one of the stack traces ( in this case perms for file2Save are R: True W:True E: True isFile:True exists:True )
java.io.FileNotFoundException <fullFilepathhere> (Access is denied)
at java.io.RandomAccessFile.open(Native Method)
at java.io.RandomAccessFile.<init>(RandomAccessFile.java:212)
at javax.imageio.stream.FileImageOutputStream.<init>(FileImageOutputStream.java:53)
at com.sun.imageio.spi.FileImageOutputStreamSpi.createOutputStreamInstance(FileImageOutputStreamSpi.java:37)
at javax.imageio.ImageIO.createImageOutputStream(ImageIO.java:393)
at javax.imageio.ImageIO.write(ImageIO.java:1514)
at myPackage.myClass.afterPropertiesSet(thisClassexample.java:204)// 204 is the line number of the ImageIO write
This may not answer your problem since there can be many other possibilties to your limited information.
One common possibilty for not being able to write a file in web application is the file locking issue on Windows if the following four conditions are met simultaneously:
the target file exists under web root, e.g. WEB-INF folder and
the target file is served by the default servlet and
the target file has been requested at least once by client and
you are running under Windows
If you are trying to replace such a file that meets all of the four conditions, you will not be able to because some servlet containers such as tomcat and jetty will buffer the static contents and lock the files so you are unable to replace or change them.
If your web application has exactly this problem, you should not use the default servlet to serve the file contents. The default servlet is desigend to serve the static content which you do not want to change, e.g. css files, javascript files, background images, etc.
There is a trick to solve the file locking issue on Windows for jetty by disabling the NIO http://docs.codehaus.org/display/JETTY/Files+locked+on+Windows
The trick is useful for development process, e.g. you want to edit the css file and see the change without restarting your web application, but it is not recommended for production mode. If your web application relies on this trick in the production process, then you should seriously consider redesign your codes.
I cannot tell you what's going on or why... I have a feeling that it's something dependent on the way ImageIO tries to save the image. What you could do is saving the BufferedImage by leveraging the ByteArrayOutputStream as described below:
BufferedImage bufferedImage = ImageIO.read(new File("sample_image.gif"));
ByteArrayOutputStream baos = new ByteArrayOutputStream();
ImageIO.write( bufferedImage, "gif", baos );
baos.flush(); //Is this necessary??
byte[] resultImageAsRawBytes = baos.toByteArray();
baos.close(); //Not sure how important this is...
OutputStream out = new FileOutputStream("myImageFile.gif");
out.write(resultImageAsRawBytes);
out.close();
I'm not really familiar with the ByteArrayOutputStream, but I guess its reset() function could be handy when dealing with saving multiple files. You could also try using its writeTo(OutputStream out) if you prefer. Documentation here.
Let me know how it goes...

Categories