We Keep Coding

Guava's Files.move() does it move the files or copy it

I have to move huge number of file(almost 60 MB each) from folder A to folder B. Folder B is a i/p folder for a a Spring scheduled task. it picks those files and start processing in parallel. I am using Guava's file utility methods to move the files.
Files.move(sourceFile,targetFile,Charsets.UTF_8);
I do see error in my TaskScheduler class that the file is not there to read
at org.apache.commons.io.FileUtils.openInputStream(FileUtils.java:299) ~[commons-io-2.4.jar:2.4]
at org.apache.commons.io.FileUtils.lineIterator(FileUtils.java:1856) ~[commons-io-2.4.jar:2.4]
at com.varun.processor.readFile(Processor.java:342) ~[classes/:?]
My hunch is that File is in copying and hence Spring scheduled thread couldn't acquire the lock to read it.As per Guava's doc it says its move method behave like mv command on unix but I see code as copy in guava jar.
Can anyone suggest a better way to move files on unix system from spring app Where o/p directory is i/p for another downstream process.

moving files with Guava works fine. you just have to keep trying to read the file until it succeeds.
I tested moving a file with size 525 MB and Guava moved it in less than 1 second.
see below for an example (I intentionally added a delay before moving the file, so the file processor will try to open the file before and while it was being moved) :
import com.google.common.io.ByteSource;
import com.google.common.io.Files;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* <dependency>
* <groupId>com.google.guava</groupId>
* <artifactId>guava</artifactId>
* <version>22.0</version>
* </dependency>
*/
public class GuavaFileMoveExample {
private static final Logger LOGGER = Logger.getLogger("GuavaFileMoveExample");
public static void main(String[] args) throws IOException, InterruptedException {
GuavaFileMoveExample a = new GuavaFileMoveExample();
a.doTheWork();
}
private void doTheWork() throws InterruptedException {
ExecutorService executorService = Executors.newFixedThreadPool(2);
executorService.submit(new FileMover());
executorService.submit(new FileProcessor());
executorService.shutdown();
executorService.awaitTermination(10, TimeUnit.SECONDS);
}
}
class FileMover implements Callable<Void> {
private static final Logger LOGGER = Logger.getLogger("FileMover");
#Override
public Void call() throws Exception {
Thread.sleep(1000);
moveFile();
return null;
}
private void moveFile() {
final File sourceFile = new File("/tmp/origin/ideaIU-2016.3-no-jdk.tar.gz");
final File targetFile = new File("/tmp/destination/ideaIU-2016.3-no-jdk.tar.gz");
try {
LOGGER.log(Level.INFO, "started moving file");
Files.move(sourceFile, targetFile);
LOGGER.log(Level.INFO, "finished moving file");
} catch (IOException e) {
LOGGER.log(Level.WARNING, "ioexception while moving file ", e);
}
}
}
class FileProcessor implements Callable<Void> {
private static final Logger LOGGER = Logger.getLogger("FileProcessor");
#Override
public Void call() throws Exception {
readBinaryFile("/tmp/destination/ideaIU-2016.3-no-jdk.tar.gz");
return null;
}
private byte[] readBinaryFile(String aFileName) {
File file = new File(aFileName);
ByteSource source = Files.asByteSource(file);
byte[] result = null;
while (result == null) {
try {
LOGGER.log(Level.INFO, "started reading file");
result = source.read();
LOGGER.log(Level.INFO, "finished reading file. size: " + result.length);
} catch (FileNotFoundException e) {
// expected if file is not yet at destination
} catch (IOException e) {
LOGGER.log(Level.SEVERE, "error reading file", e);
break;
}
}
return result;
}
}

Making changes in the properties file of Java project

I need to make a change in .properties file in my Java project. This is later deployed as a jar and used by other Java project. But according to this, I see that we should not directly make the change instead create a new object. Where should we create that new object and how can we make sure that its changes are visible?

Yes that's correct if your properties files is inside a jar then you won't be able to directly change that properties file since its packaged and zipped up in an archive. Instead you can create/change the file placed on a drive and read it, I used "user.home" for an example which you can change it as your need, below is the code for the same:
package com.test.util;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Properties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class PropertyFileReader {
private static final Logger LOGGER = LoggerFactory
.getLogger(PropertyFileReader.class);
private static Properties properties;
private static final String APPLICATION_PROPERTIES = "application.properties";
private static final String workingDir = System.getProperty("user.home");
private static File file = new File(workingDir, APPLICATION_PROPERTIES);
static {
properties = new Properties();
}
public static void main(String[] args) {
write("hello", "2");
System.out.println(read("hello"));
}
public static String read(final String propertyName) {
try (InputStream input = new FileInputStream(file)) {
properties.load(input);
} catch (IOException ex) {
LOGGER.error("Error occurred while reading property from file : ",
ex);
}
return properties.getProperty(propertyName);
}
public static void write(final String propertName,
final String propertyValue) {
try (OutputStream output = new FileOutputStream(file)) {
properties.setProperty(propertName, propertyValue);
properties.store(output, null);
} catch (IOException io) {
LOGGER.error("Error occurred while writing property to file : ", io);
}
}
}

How to refer relative paths for code outside pom project?

I have a very different situation to deal with. Something never seen before.
I have a codebase which is not a maven based project. It basically is set of Pig Script that are executed on Hadoop Cluster.
Now there is requirement to test these scripts using PigUnit, so I created a maven based project with all dependencies needed for the project.
Visually it looks like
user_mapper/
src/main/
user.pig
other.pig
test/
pom.xml
src/java/
/UserTest.java
/OtherTest.java
As you could see, test is a maven based project in itself.
What I need
In UserTest.java I want to refer to relative path of user.pig
How can I provide a relative path in UserTest.java?

Try the following code (internally uses commons-io jar)
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.logging.Level;
import java.util.logging.Logger;
public class FileReader {
Logger logger = Logger.getLogger(FileReader.class.getName());
static String webAppPath;
private static final boolean IS_WINDOWS = System.getProperty( "os.name" ).contains( "indow" );
private InputStream inputStream;
private static FileReader fileReader;
public String getAbsolutePath(Class appClass, String relativePath) {
try {
String parentPath = "";
if (StringUtils.isNotBlank(webAppPath)) {
parentPath = webAppPath;
} else {
parentPath = appClass.getProtectionDomain().getCodeSource().getLocation().getPath();
}
String osAppropriatePath = IS_WINDOWS ? parentPath.substring(1) : parentPath;
String absolutePath = osAppropriatePath + relativePath;
File file = new File(absolutePath);
if (!file.exists()) {
FileUtils.writeStringToFile(file, IOUtils.toString(readFile(relativePath), "UTF-8"));
}
return absolutePath;
} catch (IOException ioe) {
logger.log(Level.SEVERE, null, ioe);
return relativePath;
}
}
public void closeFileReader() {
synchronized (this) {
try {
inputStream.close();
} catch (IOException ex) {
Logger.getLogger(FileReader.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
private FileReader() {
}
public static FileReader getInstance() {
return new FileReader();
}
public static String getWebAppPath() {
return webAppPath;
}
public static void setWebAppPath(String webAppPath) {
FileReader.webAppPath = webAppPath;
}
}
And call the class to get the relative path as follows
FileReader.getInstance().getAbsolutePath(user.pig, "user.pig");

I solved this issue by using java.io.File as
final String filePath = new File("../src/user.pig").getAbsolutePath();

Java i18n without RessourceBundle

I am trying to get the i18n properties file out of my BuildPath. If you are trying to get the PropertiesFile the ResourceBundle.getBundlewill throw a java.util.MissingResourceException. Is there a method to load i18n files from outside the BuildPathbut still has the comfort of detecting your locale?
EDIT:
Here is the solution I was able to create with the Help of Paweł Dyda. Maybe somebody will need it. Probably there could be some improvements made, but it works ;)
import java.io.File;
import java.io.FileNotFoundException;
import java.io.UnsupportedEncodingException;
import java.net.URLDecoder;
import java.util.List;
import java.util.Locale;
import java.util.MissingResourceException;
import java.util.ResourceBundle.Control;
import org.apache.commons.configuration.ConfigurationException;
import org.apache.commons.configuration.PropertiesConfiguration;
import org.apache.commons.io.FilenameUtils;
import org.apache.log4j.Logger;
public class GlobalConfigurationProvider {
Logger logger = Logger.getLogger(GlobalConfigurationProvider.class);
private static GlobalConfigurationProvider instance;
PropertiesConfiguration i18n;
private GlobalConfigurationProvider() {
String path = GlobalConfigurationProvider.class.getProtectionDomain().getCodeSource().getLocation().getPath();
String decodedPath = "";
try {
decodedPath = URLDecoder.decode(path, "UTF-8");
// This ugly thing is needed to get the correct
// Path
File f = new File(decodedPath);
f = f.getParentFile().getParentFile();
decodedPath = f.getAbsolutePath();
} catch (UnsupportedEncodingException e) {
e.printStackTrace();
this.logger.error("Failed to decode the Jar path", e);
}
this.logger.debug("The Path of the jar is: " + decodedPath);
String configFolder = FilenameUtils.concat(decodedPath, "cfg");
String i18nFolder = FilenameUtils.concat(configFolder, "i18n");
File i18nFile = null;
try {
i18nFile = this.getFileForLocation(new File(i18nFolder), Locale.getDefault());
} catch (FileNotFoundException e) {
e.printStackTrace();
this.logger.error("Can't find the LocaleFile", e);
}
if (!i18nFile.exists()) {
// If this can't be found something is wrong
i18nFile = new File(i18nFolder, "eng.i18n");
if (!i18nFile.exists()) {
this.logger.error("Can't find the i18n File at the Location: " + i18nFile.getAbsolutePath());
}
}
this.logger.debug("The Path to the i18n File is: " + i18nFile);
try {
this.i18n = new PropertiesConfiguration(i18nFile);
} catch (ConfigurationException e) {
this.logger.error("Couldn't Initialize the i18nPropertiesFile", e);
}
}
private File getFileForLocation(File i18nFolder, Locale locale) throws FileNotFoundException {
Control control = Control.getControl(Control.FORMAT_DEFAULT);
List<Locale> locales = control.getCandidateLocales(this.getBaseName(), locale);
File f = null;
for (Locale l : locales) {
String i18nBundleName = control.toBundleName(this.getBaseName(), l);
String i18nFileName = control.toResourceName(i18nBundleName, "properties");
f = new File(i18nFolder, i18nFileName);
this.logger.debug("Looking for the i18n File at: " + f);
if (f.exists()) {
return f;
}
}
// Last try for a File that should exist
if (!locale.equals(Locale.US)) {
return this.getFileForLocation(i18nFolder, Locale.US);
}
throw new FileNotFoundException("Can't find any i18n Files in the Folder " + i18nFolder.getAbsolutePath());
}
private String getBaseName() {
// TODO: Get this from the Settings later
return "messages";
}
public static GlobalConfigurationProvider getInstance() {
if (GlobalConfigurationProvider.instance == null) {
GlobalConfigurationProvider.instance = new GlobalConfigurationProvider();
}
return GlobalConfigurationProvider.instance;
}
public String getI18nString(String key) {
try {
return this.i18n.getString(key);
} catch (MissingResourceException e) {
return '!' + key + '!';
}
}
}

Of course there are methods to do that. Anyway, I believe your problem is the wrong path to the resource you are trying to load.
Nonetheless, for sure you are looking the way to use Locale fall-back mechanism to load very specific resource. It can be done. You may want to take a look at ResourceBundle.Control class. For example you can get the list of fall-back locales:
Control control = Control.getControl(Control.FORMAT_DEFAULT);
List<Locale> locales = control.getCandidateLocales("messages",
Locale.forLanguageTag("zh-TW"));
From there, you can actually create names of the resource files you are looking for:
for (Locale locale : locales) {
String bundleName = control.toBundleName("messages", locale);
String resourceName = control.toResourceName(bundleName, "properties");
// break if resource under given name exist
}
Then, you need to load the resource somehow - you may want to use ClassLoader's getResourceAsStream(String) to open the InputStream for you. The last step could be actually use the stream as an input to PropertyResourceBundle:
ResourceBundle bundle = new PropertyResourceBundle(inputStream);
You can alternatively pass a Reader rather than InputStream, which has at least one advantage - you may actually allow properties file to be encoded in UTF-8, rather than regular ISO8859-1.

Trying to implement a manipulatable zip file system - failing

I need to find a solution to be able to manipulate a zip / jar directly (without unpacking) and without using third-party libraries. However I can't seem to grasp how the FileSystem ties in with Path and URI.
The URI that I'm trying to copy to is jar:file://E:/Projects/ZipDir/dist/test_folder/test.zip!/test_file.txt
The exception I'm getting is:
FileSystemNotFoundException but that zip file definitely does exist.
Using Java 7, this is what I have so far:
...
ZipDirectory zip = new ZipDirectory("test_folder/test.zip");
Path copyTo = zip.getPath("/test_file.txt");
Path copyFrom = Paths.get("test_file.txt");
Files.copy(copyFrom, copyTo, StandardCopyOption.REPLACE_EXISTING);
...
//
import java.io.IOException;
import java.net.URI;
import java.nio.file.*;
import java.util.HashMap;
public class ZipDirectory {
private Path path;
private FileSystem fileSystem;
public ZipDirectory(String path){
this.path = Paths.get(Paths.get(path).toUri());
create();
}
private void create(){
HashMap<String, String> env = new HashMap<>();
env.put("create", "true");
try {
fileSystem = FileSystems.newFileSystem(path, null);
} catch (IOException ex) {
System.out.println(ex);
}
}
public Path getPath(String relativePath){
return Paths.get(URI.create("jar:file:/" + path.toUri().getPath() + "!" + fileSystem.getPath(relativePath)));
}
public Path getRoot(){
return Paths.get(URI.create(path.toUri().getPath() + "!/"));
}
public void close(){
try {
fileSystem.close();
} catch (IOException ex) {
System.err.println(ex);
}
fileSystem = null;
}
}

I never thought I'd answer my own question, but I've got it working:
Treating an archive like a directory with Java 7