We Keep Coding

Monitoring Zip4J extractAll() method progress monitor

I am using Zip4J for extracting zip file and I am able to do it. However, I want to use progress monitor provided in Zip4J but not able to use it successfully.
The documentation only says that it should have run in thread mode true. I did it and my console stuck on this on command line. Any working example of extractAll() with progress monitor.
public String unzipFile(String sourceFilePath, String extractionPath) {
String extractionDirectory = "";
FileHeader fileHeader = null;
if (FileUtility.isPathExist(sourceFilePath) && FileUtility.isPathExist(extractionPath)) {
try {
ZipFile zipFile = new ZipFile(sourceFilePath);
LOG.info("File Extraction started");
List<FileHeader> fileHeaderList = zipFile.getFileHeaders();
if (fileHeaderList.size() > 0)
fileHeader = (FileHeader) fileHeaderList.get(0);
if (fileHeader != null)
extractionDirectory = splitFileName(fileHeader.getFileName());
long totalPercentage = 235;
long startTime = System.currentTimeMillis();
zipFile.extractAll(extractionPath);
LOG.info("File Extraction completed.");
System.out.println();
} catch (ZipException e) {
LOG.error("Extraction Exception ->\n" + e.getMessage());
}
} else {
LOG.error("Either source path or extraction path is not exist.");
}
return extractionDirectory;
}

Don't know, works fine if you add enough files, that there actually is a progress to see. I added some really fat ones for the purpose.
#Test
public void testExtractAllDeflateAndNoEncryptionExtractsSuccessfully() throws IOException {
ZipFile zipFile = new ZipFile(generatedZipFile);
List<File> toAdd = Arrays.asList(
getTestFileFromResources("sample_text1.txt"),
getTestFileFromResources("sample_text_large.txt"),
getTestFileFromResources("OrccTutorial.pdf"),
getTestFileFromResources("introduction-to-automata-theory.pdf"),
getTestFileFromResources("thomas.pdf")
);
zipFile.addFiles(toAdd);
zipFile.setRunInThread(true);
zipFile.extractAll(outputFolder.getPath());
ProgressMonitor mon = zipFile.getProgressMonitor();
while (mon.getState() == BUSY) {
System.out.println(zipFile.getProgressMonitor().getPercentDone());
try {
Thread.sleep(10);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
}
ZipFileVerifier.verifyFolderContentsSameAsSourceFiles(outputFolder);
verifyNumberOfFilesInOutputFolder(outputFolder, 5);
}

testAddFilesWithProgressMonitor.java in the the project's test cases shows how to use ProgressMonitor.

how to scan multiple directory using java multi threading

The problem statement is, you have to list the name of the files from the given directory, you have given one directory structure which has some subdirectory and some file in them.
I did some part of the code but it is not working can you please help me what is the correct way of doing it.
code
public class Test {
public static void main(String[] args) {
RunableExample run = new RunableExample();
Thread th = new Thread(run, "thread1");
String directoryName = "C:\\Users\\GUR35893\\Desktop\\CleanupMTM";
File directory = new File(directoryName);
File[] fList = directory.listFiles();
RunableExample.MyList = new ArrayList<File>();
for (File file : fList) {
RunableExample.MyList.add(file);
}
try {
th.start();
} catch (Exception e) {
}
}
}
public class RunableExample implements Runnable {
public static List<File> MyList;
int count = 0;
File filepath;
public void run() {
try {
while (count < MyList.size()) {
System.out.println(Thread.currentThread().getName() + ">>>>"
+ MyList.size() + " >>>>> " + count);
filepath = MyList.get(count);
if (filepath != null && filepath.isFile()) {
System.out.println(Thread.currentThread().getName() + " >>"
+ filepath.getAbsolutePath());
} else {
synchronized (this) {
if (filepath != null) {
// System.out.println("Else");
RunableExample run3 = new RunableExample();
Thread th3 = new Thread(run3, "thread" + count);
File[] fList = filepath.listFiles();
// System.out.println("Else1");
for (File file : fList) {
MyList.add(file);
}
th3.start();
}
}
}
count++;
}
} catch (Exception e) {
e.printStackTrace();
System.out.println(e);
}
}
}

If you have a directory (including sub-directories) and you want list all files.
The simplest yet effective approach would be iterate through a directory, there will be just 2 options either its a file or its a directory.
If it's a file, simply name it, don't spawn a new thread for it.
If it's a directory, spawn a new thread and re-use the same code for traversing the files or sub-directories in that directory in the newly spawned thread.
If you could give a sample output then maybe we can help further. But till then, I don't see any use of synchronization in the code.

Implementation of #Himanshu Answer.
import java.io.File;
class Lister extends Thread{
String basepath;
Lister(String basepath){
this.basepath = basepath;
}
#Override
public void run(){
File rootDir = new File(basepath);
for(File f : rootDir.listFiles()){
if(f.isDirectory())
new Lister(f.toString()).start();
else
System.out.println(f);
}
}
}
class Main {
public static void main(String[] args) {
new Lister("/").start();
}
}
This code works, but make sure it don't memory overflow for huge directory trees. For that you can add extra checks to spawn only directory you need.

Moving a directory in java throws java.nio.file.FileAlreadyExistsException

I am creating a rollback feature and here is what I have and wanna achieve:
a tmp folder is created in the same location as the data folder;
before doing any operation I copy all the contents from data folder to tmp folder (small amount of data).
On rollback I want to delete the data folder and rename tmp folder to data folder.
This is what I tried
String contentPath = "c:\\temp\\data";
String tmpContentPath = "c:\\temp\\data.TMP";
if (Files.exists(Paths.get(tmpContentPath)) && Files.list(Paths.get(tmpContentPath)).count() > 0) {
FileUtils.deleteDirectory(new File(contentPath));
Files.move(Paths.get(tmpContentPath), Paths.get(contentPath), java.nio.file.StandardCopyOption.REPLACE_EXISTING);
}
but this throws FileAlreadyExistsException even though I deleted the target directory in the same method.
Once the program exits I don't see the c:\temp\data directory, so the directory is actually deleted.
Now if I try StandardCopyOption.ATOMIC_MOVE it throws an java.nio.file.AccessDeniedException.
What is the best way to move tmp dir to data dir in these kind of situations?

Actually in java 7 or above you can just use the Files to achieve the folder moving even there is a conflict, which means the target folder already exists.
private static void moveFolder(Path thePath, Path targetPath) {
if (Files.exists(targetPath)) { // if the target folder exists, delete it first;
deleteFolder(targetPath);
}
try {
Files.move(thePath, targetPath);
} catch (IOException ignored) {
ignored.printStackTrace();
}
}
private static void deleteFolder(Path path) {
try {
if (Files.isRegularFile(path)) { // delete regular file directly;
Files.delete(path);
return;
}
try (Stream<Path> paths = Files.walk(path)) {
paths.filter(p -> p.compareTo(path) != 0).forEach(p -> deleteFolder(p)); // delete all the children folders or files;
Files.delete(path); // delete the folder itself;
}
} catch (IOException ignored) {
ignored.printStackTrace();
}
}

Try This
public class MoveFolder
{
public static void main(String[] args) throws IOException
{
File sourceFolder = new File("c:\\temp\\data.TMP");
File destinationFolder = new File("c:\\temp\\data");
if (destinationFolder.exists())
{
destinationFolder.delete();
}
copyAllData(sourceFolder, destinationFolder);
}
private static void copyAllData(File sourceFolder, File destinationFolder)
throws IOException
{
destinationFolder.mkdir();
String files[] = sourceFolder.list();
for (String file : files)
{
File srcFile = new File(sourceFolder, file);
File destFile = new File(destinationFolder, file);
copyAllData(srcFile, destFile); //call recursive
}
}
}

Figured out the issue. In my code before doing a rollback, I am doing a backup, in that method I am using this section to do the copy
if (Files.exists(Paths.get(contentPath)) && Files.list(Paths.get(contentPath)).count() > 0) {
copyPath(Paths.get(contentPath), Paths.get(tmpContentPath));
}
Changed it to
try (Stream<Path> fileList = Files.list(Paths.get(contentPath))) {
if (Files.exists(Paths.get(contentPath)) && fileList.count() > 0) {
copyPath(Paths.get(contentPath), Paths.get(tmpContentPath));
}
}
to fix the issue

Android Filesystem

I've been trying to get this working, but I'm stumbling somewhere.
Here is the major source of the whole thing. And the logcat result...some is a bit redundant. I have "startingDirectory" above the onCreate, so I copied and removed the private...
String startingDirectory = Environment.getExternalStorageDirectory().getAbsolutePath();
String directory = startingDirectory;
if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED) == false) {
Log.d(TAG, "Media is not mounted.");
} else {
Log.d(TAG, "Loading File Directory.");
Log.d(TAG, "Directory listing for " + directory);
File path = new File(directory);
//File path = new File(directory);
if (path.isDirectory()) {
if (path.canRead()) {
Log.d(TAG, "Readable directory");
} else {
Log.d(TAG, "Non-readable directory");
if (Environment.getExternalStorageState().equals(Environment.MEDIA_MOUNTED)) {
Log.d(TAG, "Media is mounted");
} else {
Log.d(TAG, "Media is not mounted");
}
}
} else {
Log.d(TAG, "Is not a directory.");
}
}
And here is the logcat of the whole thing...
07-13 01:13:45.705 4361-4361/? D/MainActivity: Loading File Directory.
07-13 01:13:45.705 4361-4361/? D/MainActivity: Directory listing for /storage/emulated/0
07-13 01:13:45.705 4361-4361/? D/MainActivity: Non-readable directory
07-13 01:13:45.706 4361-4361/? D/MainActivity: Media is mounted
I'm trying to get the directory structure of current directory (staringDirectory) and place into an class...but it doesn't seem to stay from non-readable

This is my permitFileAcess method using setWritable and setReadable method of java.io.File class whenever I modify some file and directory's permission.
Well, I still have got a message but it works for me
public static void permitFileAcess(final File file) {
if (!file.setWritable(true, true)) {
log.e(new Object[] { "Could not set Writable permissions" });
}
if (!file.setReadable(true, true)) {
log.e(new Object[] { "Could not set Readable permissions" });
}
if (!file.setExecutable(true, true)) {
log.e(new Object[] { "Could not set Executable permissions" });
}
}
So, if you want to create a file use it before the operation.
private static boolean createFile(File parent, String fileName, boolean createDir) {
boolean returnValue = false;
if (parent.isDirectory()) {
File newFile = new File(parent.getAbsolutePath(), fileName);
permitFileAcess(newFile);
if (createDir) {
returnValue = newFile.mkdir();
} else {
try {
returnValue = newFile.createNewFile();
} catch (IOException e) {
e.printStackTrace();
}
}
}
return returnValue;
}
Good luck.

I'm trying to retrieve all of the files and directories in a specific dir. To make a simple file file manager. If I don't SEE the file, I wouldn't know which one to load in.

How to verify file is finished being written before attempting to process it?

Hard to give a good title to my problem but it is as follows. First I'm doing this on windows and it could possibly be used on a linux box also so I'd need the fix to work on both systems. I am monitoring a directory for new files. I basically looking at the directory's files and comparing them over and over and only processing the new files. Problem is I keep getting an error where the file isn't finished being written before I attempt to process.
public class LiveDetectionsProvider extends DetectionsProvider {
protected LiveDetectionsProvider.MonitorDirectory monitorDirectory = null;
protected TimeModel timeModel = null;
private ArrayList<String> loaded = new ArrayList();
private File topLayerFolder = null;
public LiveDetectionsProvider(String directory, String id) {
super(directory, id);
timeModel = super.timeModel;
}
/**
* Initialize the data provider.
*/
public void initialize() {
try {
topLayerFolder = new File(directory);
File[] dir = topLayerFolder.listFiles();
for (File file : dir) {
loaded.add(file.getName());
}
monitorDirectory = new MonitorDirectory();
monitorDirectory.execute();
}
catch (Exception ex) {
Logger.getLogger(LiveDetectionsProvider.class.getName()).log(Level.SEVERE, "Failed to read detection\n{0}", ex.getMessage());
}
super.initialize();
}
/**
* Un-initialize the data provider.
*/
public void uninitialize() {
super.uninitialize();
if (monitorDirectory != null) {
monitorDirectory.continuing = false;
}
}
/**
* The class that is used to load the detection points in a background
* thread.
*/
protected class MonitorDirectory extends SwingWorker<Void, Void> {
public boolean continuing = true;
/**
* The executor service thread pool.
*/
private ExecutorService executor = null;
/**
* The completion service that reports the completed threads.
*/
private CompletionService<Object> completionService = null;
#Override
protected Void doInBackground() throws Exception {
int count = 0;
executor = Executors.newFixedThreadPool(1);
completionService = new ExecutorCompletionService<>(executor);
while (continuing && topLayerFolder != null) {
File[] dir = topLayerFolder.listFiles();
Thread.sleep(10);
ArrayList<File> filesToLoad = new ArrayList();
for (File file : dir) {
if (!loaded.contains(file.getName())) {
long filesize = 0;
boolean cont = true;
while (cont) {
if (file.length() == filesize) {
cont = false;
Thread.sleep(3);
filesToLoad.add(file);
}
else {
filesize = file.length();
Thread.sleep(3);
}
}
Thread.sleep(3);
}
}
for (File file : filesToLoad) {
timeModel.setLoadingData(LiveDetectionsProvider.this.hashCode(), true);
completionService.submit(Executors.callable(new ReadDetection(file, false)));
while (completionService.take() == null) {
Thread.sleep(2);
}
loaded.add(file.getName());
count++;
Logger.getLogger(LiveDetectionsProvider.class.getName()).log(Level.SEVERE, "Detection Message Count:" + count);
}
detectionsModel.fireStateChanged(DetectionsModel.CHANGE_EVENT_DETECTIONS);
timeModel.setLoadingData(LiveDetectionsProvider.this.hashCode(), false);
}
return null;
}
}
}
The file is processed at the line with
completionService.submit(Executors.callable(new ReadDetection(file, false)));
The file at this point still hasnt finished being written and thus fails. I've tried sleeping my thread to slow it down, and I've tried verifying the file size hasn't changed. My test case for this is I'm unzipping a tar file which contains tons of 1,000 KB files.

Usually I solve this issue by create a temporary file while the file is being written. Once finish I rename the file and only the renamed file can be process.

Use a "flag file": once file.txt is "finished", indicate this by creating a file.flg - consuming process should wait for .flg to appear.

First yes it compiles look a the solution I posted in relation to the code In my question. You substitute
For(File file: dir){
while(!file.renameTo(file)){
Thread.sleep(1)
}
// In my code I check to see if the file name is already in the list which
// contains files that have been previously loaded if its not I add it to a list
// of files to be processed
}
in for
for (File file : dir) {
if (!loaded.contains(file.getName())) {
long filesize = 0;
boolean cont = true;
while (cont) {
if (file.length() == filesize) {
cont = false;
Thread.sleep(3);
filesToLoad.add(file);
}
else {
filesize = file.length();
Thread.sleep(3);
}
}
Thread.sleep(3);
}
}
sorry I forgot to put comment tags // in on the line that said do what every you need to do here.
What it does is it looks at each file in the directory and checks to see if you can rename it if the rename fails it sleeps and continues checking till it sucessfully is able to rename at which point you can do what you need to with the file which in my case was everything after the for loop that was replaced. I'm curious why my awnser was viewed as sub par deleted and locked. This solution does work and solved my problem and would anyone else who's having the same issue attempting to process a file that still being written or copied to a directory thats being monitored for changes.