We Keep Coding

File list from classpath directory in running jar

I want to load all resource bundle properties from classpath, so that I can show supported languages. I got a reference from here and tried 1st solution. It works file when I run my code from eclipse. But when I create executable jar file, it could not read files. I don't know why behavior is different while running from command java -jar AppName.jar
My Code Is:
public static List<String> getResourceFiles(String path) throws IOException
{
List<String> filenames = new ArrayList<>();
InputStream in = getResourceAsStream(path);
BufferedReader br = new BufferedReader(new InputStreamReader(in));
System.out.println("br = " + br.readLine());
String resource;
while ((resource = br.readLine()) != null)
{
filenames.add(resource);
}
return filenames;
}
private static InputStream getResourceAsStream(String resource)
{
final InputStream in = getContextClassLoader().getResourceAsStream(resource);
System.out.println("input stream = " + in);
return in == null ? FileUtil.class.getResourceAsStream(resource) : in;
}
private static ClassLoader getContextClassLoader()
{
return Thread.currentThread().getContextClassLoader();
}
Here I noticed that InputStream is null when I run from command, but while running from eclipse InputStream is not null.
How to solve this problem so that I can read resource files when running from command also?

I found solution. Below code worked for me:
public static String[] getResourceListing(Class clazz, String path) throws URISyntaxException, IOException
{
URL dirURL = clazz.getClassLoader().getResource(path);
if (dirURL != null && dirURL.getProtocol().equals("file"))
{
/* A file path: easy enough */
return new File(dirURL.toURI()).list();
}
if (dirURL == null)
{
/*
* In case of a jar file, we can't actually find a directory. Have to assume the
* same jar as clazz.
*/
String me = clazz.getName().replace(".", "/") + ".class";
dirURL = clazz.getClassLoader().getResource(me);
}
if (dirURL.getProtocol().equals("jar"))
{
/* A JAR path */
String jarPath = dirURL.getPath().substring(5, dirURL.getPath().indexOf("!")); // strip out only the JAR file
JarFile jar = new JarFile(URLDecoder.decode(jarPath, "UTF-8"));
Enumeration<JarEntry> entries = jar.entries(); // gives ALL entries in jar
Set<String> result = new HashSet<String>(); // avoid duplicates in case it is a subdirectory
while (entries.hasMoreElements())
{
String name = entries.nextElement().getName();
if (name.startsWith(path))
{ // filter according to the path
String entry = name.substring(path.length());
int checkSubdir = entry.indexOf("/");
if (checkSubdir >= 0)
{
// if it is a subdirectory, we just return the directory name
entry = entry.substring(0, checkSubdir);
}
result.add(entry);
}
}
return result.toArray(new String[result.size()]);
}
throw new UnsupportedOperationException("Cannot list files for URL " + dirURL);
}

Get hidden files in FTP server

I have the local server mockftpserver, and in the server there are couple of files and they are protected with a prefix '._' and the method to protect from getting those files is the following :
protected String getRealPath(Session session, String path) {
String currentDirectory = (String) session.getAttribute(SessionKeys.CURRENT_DIRECTORY);
String result;
if (path == null) {
result = currentDirectory;
}
else if (getFileSystem().isAbsolute(path)) {
result = path;
}
else {
result = getFileSystem().path(currentDirectory, path);
}
return result.replace("._", "");
}
I tried to list the files in the FTP server I got them but the protected ones like '._passwrd' I was not able to see it.
I used the normal method to get the file list:
boolean login = ftpClient.login("user", "password");
if (login) {
System.out.println("Connection established...");
FTPFile[] files = ftpClient.listFiles();
for (FTPFile file : files) {
if (file.getType() == FTPFile.FILE_TYPE) {
System.out.println("File Name: "
+ file.getName()
+ " File Size: " );
}
}
String[] fil = ftpClient.listNames();
if (files != null && fil.length > 0) {
for (String aFile: fil) {
System.out.println(aFile);
}
}
BufferedReader reader = null;
String firstLine = null;
try {
InputStream stream =
ftpClient.retrieveFileStream("._"+"._passwd");
reader = new BufferedReader(new InputStreamReader(stream, "UTF-8"));
firstLine = reader.readLine();
} finally {
if (reader != null)
try {
reader.close();
} catch (IOException logOrIgnore) {}
}
}
But thinking that the method will only check the name once, so if I added the ._ once again it should work. Although it did not or I could not apply it in the right way.

i don't know about Java but in Python i solved similar task in the following way:
i used: FTP server, Python 2.7.12, library 'ftplib'
so i show just needed part with comments:
#while customer list not empty
while self.customerDirs:
#create connect to root
self.connection.cwd("/")
#choose customer
customer = self.customerDirs.pop()
try:
#go inside to customer's folder
self.connection.cwd(customer)
#for all folders inside
for dir in self.connection.nlst():
#go inside
self.connection.cwd(dir)
#create empty list
hiddenList = []
#create variable which contains path
pathDir = self.connection.pwd()
#write to list hidden files
self.connection.retrlines("LIST -a", hiddenList.append)
for entry in hiddenList:
#split value and take file name
entrySplit = entry.split(' ')[-1]
#cheсk file name
if entrySplit not in ['.', '..'] and entrySplit.startswith('.'):
#all necessary files are sent to method which will delete it (lool like: /customer/folder/.hidden_file.hid)
self.ftp_delete_file('/'.join([pathDir, entrySplit]))
#return to step up
self.connection.cwd("..")
that all, i hope it will be helpful information

You should set your ftpClient to list hidden Files before listing the files, so:
ftpClient.setListHiddenFiles(true);
FTPFile[] files = ftpClient.listFiles();

Read text files from source folder OR JAR depending on execution runtime

My program displays elements read from a text file. The text files will be stored in a folder found in the package folder containing the .java and .class files so they can be embedded in the jar.
I'm trying to get the application to read the text files properly for both situations
Running from the IDE (Netbeans)
Running from the JAR
Currently I can do point one with no problem, but the code reads using File where as the way I am seeing how to do it with Jars is using InputStream.
The functions which work for the IDE runs
public void loadWidgets() {
ArrayList<String> results = new ArrayList<>();
String dir = new File("").getAbsolutePath() + "/src/Creator/textFiles/widgets/;
System.out.println(dir);
getWidgetFiles(dir, results);
results.stream().forEach((s) -> {
readFile(s); // given a string and it opens the file using a Scanner
});
updateWidgetVariables(); // gui updates
}
public void getWidgetFiles(String dirName, ArrayList<String> filePaths) {
File directory = new File(dirName);
File[] files = directory.listFiles();
for (File file : files) {
if (file.isFile()) {
filePaths.add(file.getName() + "," + file.getAbsolutePath());
} else if (file.isDirectory()) {
getWidgetFiles(file.getAbsolutePath(), filePaths);
}
}
}
So I have a bunch of text files organized by the type of widget it is, so I am running through the /widgets/ directory to find all the text files.
The problem I'm having is how I can go through the directories and files of a Jar? Can they be converted to a file, or read into a string?
I got this code from this question and it can read the files, but I dont know how I can open them using a new Scanner(file); code
CodeSource src = WidgetPanel.class.getProtectionDomain().getCodeSource();
try {
System.out.println("Inside try");
List<String> list = new ArrayList<>();
if (src != null) {
URL jar = src.getLocation();
ZipInputStream zip = new ZipInputStream(jar.openStream());
ZipEntry ze = null;
System.out.println(jar.getPath());
System.out.println(zip.getNextEntry());
while ((ze = zip.getNextEntry()) != null) {
String entryName = ze.getName();
System.out.println("Entry name: " + entryName);
if (entryName.startsWith("Creator/textFiles/widgets") && entryName.endsWith(".txt")) {
list.add(entryName);
System.out.println("Added name: " + entryName);
}
}
list.stream().forEach((s) -> {
readFile(s);
});
updateWidgetVariables();
} else {
System.out.println("Src null");
}
}catch (IOException e) {
System.out.println(e.getMessage());
}

Try and obtain a FileSystem associated with your source; the matter then becomes pretty simple. Here is an illustration of how to read, as text, all files from a given FileSystem:
private static final BiPredicate<Path, BasicFileAttributes> FILES
= (path, attrs) -> attrs.isRegularFile();
private static void readAllFilesAsText(final List<Path> paths)
throws IOException
{
for (final Path path: paths)
try (
final Stream<String> stream = Files.lines(path);
) {
stream.forEach(System.out::println);
}
}
private static List<Path> getAllFilesFromFileSystems(final FileSystem fs,
final String pathPrefix)
{
final Path baseDir = fs.getPath(pathPrefix);
try (
final Stream<Path> files = Files.find(baseDir, Integer.MAX_VALUE,
FILES);
) {
return files.collect(Collectors.toList());
}
}
Why the mix of "old style" for loops and "new style" lambdas: it is because lambdas just don't handle checked exceptions... Which means you have to do it. For a workaround, see here.
But the gist of it here is to be able to create a FileSystem out of your sources, and you can do it; yes, you can read jars as such. See here.

I was able to find a solution using the functions I already had.
I first check to see if the text file can be found normally (Run from an IDE). If a file not found exception occurs, it sets ide = false so it tries to read from the jar.
public void loadWidgets() {
boolean ide = true;
String path = "textFiles/widgets/";
ArrayList<String> results = new ArrayList<>();
String dir = new File("").getAbsolutePath() + "/src/Creator/" + path;
try {
getWidgetFiles(dir, results);
results.stream().forEach((s) -> {
readFile(s);
});
updateWidgetVariables();
} catch (FileNotFoundException e) {
System.out.println(e.getMessage());
ide = false;
}
if (!ide) {
CodeSource src = WidgetPanel.class.getProtectionDomain().getCodeSource();
try {
List<String> list = new ArrayList<>();
if (src != null) {
URL jar = src.getLocation();
ZipInputStream zip = new ZipInputStream(jar.openStream());
ZipEntry ze = null;
while ((ze = zip.getNextEntry()) != null) {
String entryName = ze.getName();
if (entryName.startsWith("Creator/textFiles/widgets") && entryName.endsWith(".txt")) {
// Wouldnt work until i added the "/" before the entryName
list.add("/"+ entryName);
System.out.println("Added name: " + entryName);
}
}
list.stream().forEach((s) -> {
readJarFile(s);
});
updateWidgetVariables();
} else {
System.out.println("Src null");
}
} catch (IOException e) {
System.out.println(e.getMessage());
}
}
}
I then created a new readFile function for reading from a Jar
public void readJarFile(String result) {
String name = result.substring(result.lastIndexOf("/") + 1);
String entireWidget = "";
String line;
ArrayList<String> vars = new ArrayList<>();
int begin, end;
InputStream loc = this.getClass().getResourceAsStream(result);
try (Scanner scan = new Scanner(loc)) {
while (scan.hasNextLine()) {
line = scan.nextLine();
entireWidget += line;
while (line.contains("`%")) {
begin = line.indexOf("`%");
end = line.indexOf("%`") + 2;
vars.add(line.substring(begin, end));
//System.out.println("Variable added: " + line.substring(begin, end));
line = line.substring(end);
}
}
}
System.out.println(name + ": " + entireWidget);
Widget widget = new Widget(name, vars, entireWidget, result);
widgetList.put(name, widget);
}
Most of this answer is thanks to this question

How can I count the number of files in a folder within a JAR?

I've spent a bit of time trying to find a way to count the number of files in a folder within a JAR. I put together several examples of code that served different purposes to make this work. It counts just fine when I run the code through Eclipse but after exporting to a JAR it fails and returns 0. In this case, my folder path I use is just "rules/". I would appreciate any recommendations or samples. Thanks.
public static int countFiles(String folderPath) throws IOException { //Counts the number of files in a specified folder
ClassLoader loader = ToolSet.class.getClassLoader();
InputStream is = loader.getResourceAsStream(folderPath);
try {
byte[] c = new byte[1024];
int count = 0;
int readChars = 0;
boolean empty = true;
while ((readChars = is.read(c)) != -1) {
empty = false;
for (int i = 0; i < readChars; ++i) {
if (c[i] == '\n') {
++count;
}
}
}
return (count == 0 && !empty) ? 1 : count;
} finally {
is.close();
}
}
EDIT:
The following doesn't exactly match my original question but thanks to MadProgrammer I was able to reduce my code and eliminate the need to even count the files. The code blow searches every file in my JAR looking for those that end with ".rules", opens the file, searches the file for a string that matches "searchBox.getText()", appends results, and continues on to the next ".rules" file.
StringBuilder results = new StringBuilder();
int count = 0;
JarFile jf = null;
try {
String path = ToolSet.class.getProtectionDomain().getCodeSource().getLocation().getPath();
String decodedPath = URLDecoder.decode(path, "UTF-8");
jf = new JarFile(new File(decodedPath));
Enumeration<JarEntry> entries = jf.entries();
while (entries.hasMoreElements()) {
JarEntry entry = entries.nextElement();
if (entry.getName().endsWith(".rules")) {
String name = entry.getName();
InputStream in = ToolSet.class.getResourceAsStream(name);
InputStreamReader isr = new InputStreamReader(in);
BufferedReader bf = new BufferedReader(isr);
String line;
while ((line = bf.readLine()) != null) {
String lowerText = line.toLowerCase();
if(lowerText.indexOf(searchBox.getText().toLowerCase()) > 0) {
results.append(line + "\n");
count++;
}
}
bf.close();
}
}
} catch (IOException ex) {
try {
jf.close();
} catch (Exception e2) {
}
}
if(count>0) {
logBox.setText(results.toString());
} else {
logBox.setText("No matches could be found");
}

A Jar file is essentially a Zip file with a manifest.
Jar/Zip files don't actually have a concept of directories like disks do. They simply have a list of entries that have names. These names may contain some kind path separator and some entries may actually be marked as directories (and tend not to have any bytes associated with them, merely acting as markers)
If you want to find all the resources within a given path, you're going to have to open the Jar file and inspect it's entries yourself, for example...
JarFile jf = null;
try {
String path = "resources";
jf = new JarFile(new File("dist/ResourceFolderCounter.jar"));
Enumeration<JarEntry> entries = jf.entries();
while (entries.hasMoreElements()) {
JarEntry entry = entries.nextElement();
if (!entry.isDirectory()) {
String name = entry.getName();
name = name.replace(path + "/", "");
if (!name.contains("/")) {
System.out.println(name);
}
}
}
} catch (IOException ex) {
try {
jf.close();
} catch (Exception e) {
}
}
Now, this requires you to know the name of the Jar file you want to use, this may be problematic, as you may wish to list resources from a number of different Jars...
A better solution would be to generate some kind of "resource lookup" file at build time, which contained all the names of the resources that you might need, maybe even keyed to particular names...
This way you could simple use...
BufferedReader reader = null;
try {
reader = new BufferedReader(new InputStreamReader(getClass().getResourceAsInputStream("/resources/MasterResourceList.txt")));
String name = null;
while ((name = br.readLine()) != null) {
URL url = getClass().getResource(name);
}
} finally {
try {
br.close();
} catch (Exception exp) {
}
}
For example...
You could even seed the file with the number of resources ;)

this is a simple solution :
InputStream is = loader.getResourceAsStream(folderPath);
//open zip
ZipInputStream zip = new ZipInputStream(is);
//count number of files
while ((zip.getNextEntry()) != null ) {
UnzipCounter++;
}

Reading a zip file within a jar file

Previously we had some zip files within our web application. We would want to pares a specific text document within the zip file. This wasn't a problem:
URL url = getClass().getResource(zipfile);
ZipFile zip = new ZipFile(url.getFile().replaceAll("%20", " "));
Entry entry = zip.getEntry("file.txt");
InputStream is = zip.getInputStream(entry);
BufferedReader reader = new BufferedReader(new InputStreamReader(is));
String line = reader.readLine();
while (line != null) {
// do stuff
}
However we've moved these zip files into another module and want to package them within a jar. Unfortunately, creating the ZipFile now fails. I can get an InputStream for the zip: but I have no way of getting an input stream for the entry itself.
InputStream is = getClass().getResourceAsStream(zipfile);
ZipInputStream zis = new ZipInputStream(is);
ZipEntry entry = zis.getNextEntry();
while (entry != null && !entry.getName().equals("file.txt")) {
entry = zis.getNextEntry();
}
but I have no way of getting an input stream for the entry itself. I tried finding the length of the entry and getting the next n bytes from the ZipInputStream but this didn't work for me. It seemed all bytes read were 0.
Is there a way round this or am I going to have to move the zip files back into the core project?

How about TrueZip? Using it you could simply open the zipped file as if it was located inside a directory.
new FileOutputStream("/path/to/some-jar.jar/internal/zip/file.zip/myfile.txt");
According to the docs, infinite nesting is also supported. I have yet to actually use this project, but it's been on my radar for a while and it seems applicable to your problem.
Project Site: http://truezip.java.net/ (edited)

entry can give you the inputstream of the inner-zip file.
InputStream innerzipstream = zip.getInputStream(entry);
So you can use
new ZipInputStream(innerzipstream);
and ask the ZipInputStream to retrieve the content of the inner-zip-file (in an ordered fashion, you don't have random access because it's a ZipInputStream)
Look at http://download.oracle.com/javase/1.4.2/docs/api/java/util/zip/ZipInputStream.html
Sequential zip access
As ZipInputStream is reading a zip file from an input stream it has to do things in order:
// DO THIS for each entry
ZipEntry e = zipInputStreamObj.getNextEntry();
e.getName // and all data
int size = e.getSize(); // the byte count
while (size > 0) {
size -= zipInputStreamObj.read(...);
}
zipInputStreamObj.closeEntry();
// DO THIS END
zipInputStreamObj.close();
Note: I don't know if ZipInputStream.getNextEntry() returns null when end of zip file is reached or not. I hope so because I don't know other way to realize when there are no more entries.

I have modified the Sequential Zip access code provided above:
File destFile = new File(destDir, jarName);
JarOutputStream jos = new JarOutputStream(new FileOutputStream(destFile));
JarInputStream jis = new JarInputStream(is);
JarEntry jarEntry = jis.getNextJarEntry();
for (; jarEntry != null ; jarEntry = jis.getNextJarEntry()) {
jos.putNextEntry(new JarEntry(jarEntry.getName()));
if(jarEntry.isDirectory()) {
continue;
}
int bytesRead = jis.read(buffer);
while(bytesRead != -1) {
jos.write(buffer, 0, bytesRead);
bytesRead = jis.read(buffer);
}
}
is.close();
jis.close();
jos.flush();
jos.closeEntry();
jos.close();
In the above code, I am trying to copy a Jar file inside another Jar file to a folder in file system.
'is' is the input stream to the jar file inside another jar file (jar.getInputStream("lib/abcd.jar"))

it is also possible to parse the string and open an ZipInputStream on another ZipInputStream and set the entry to the file inside.
e.g. you have the String like above "path/to/some-jar.jar/internal/zip/file.zip/myfile.txt"
private static final String[] zipFiles = new String[] { ".zip", ".jar" };
public static InputStream getResourceAsStream(final String ref) throws IOException {
String abstractPath = ref.replace("\\", "/");
if (abstractPath.startsWith("/")) {
abstractPath = abstractPath.substring(1);
}
final String[] pathElements = abstractPath.split("/");
return getResourceAsStream(null, pathElements);
}
private static InputStream getResourceAsStream(final ZipInputStream parentStream, final String[] pathElements)
throws IOException {
if (pathElements.length == 0) return parentStream;
final StringBuilder nextFile = new StringBuilder();
for (int index = 0; index < pathElements.length; index++) {
final String pathElement = pathElements[index];
nextFile.append((index > 0 ? "/" : "") + pathElement);
if (pathElement.contains(".")) {
final String path = nextFile.toString();
if (checkForZip(pathElement)) {
final String[] restPath = new String[pathElements.length - index - 1];
System.arraycopy(pathElements, index + 1, restPath, 0, restPath.length);
if (parentStream != null) {
setZipToEntry(parentStream, path);
return getResourceAsStream(new ZipInputStream(parentStream), restPath);
} else return getResourceAsStream(new ZipInputStream(new FileInputStream(path)), restPath);
} else {
if (parentStream != null) {
setZipToEntry(parentStream, path);
return parentStream;
} else return new FileInputStream(path);
}
}
}
throw new FileNotFoundException("File not found: " + nextFile.toString());
}
private static void setZipToEntry(final ZipInputStream in, final String name) throws IOException {
ZipEntry entry;
while ((entry = in.getNextEntry()) != null) {
if (entry.getName().equals(name)) return;
}
throw new FileNotFoundException("File not found: " + name);
}
private static boolean checkForZip(final String ref) {
for (final String zipFile : zipFiles) {
if (ref.endsWith(zipFile)) return true;
}
return false;
}