I have a very different situation to deal with. Something never seen before.
I have a codebase which is not a maven based project. It basically is set of Pig Script that are executed on Hadoop Cluster.
Now there is requirement to test these scripts using PigUnit, so I created a maven based project with all dependencies needed for the project.
Visually it looks like
user_mapper/
src/main/
user.pig
other.pig
test/
pom.xml
src/java/
/UserTest.java
/OtherTest.java
As you could see, test is a maven based project in itself.
What I need
In UserTest.java I want to refer to relative path of user.pig
How can I provide a relative path in UserTest.java?
Try the following code (internally uses commons-io jar)
import org.apache.commons.io.FileUtils;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
import java.io.File;
import java.io.IOException;
import java.io.InputStream;
import java.util.logging.Level;
import java.util.logging.Logger;
public class FileReader {
Logger logger = Logger.getLogger(FileReader.class.getName());
static String webAppPath;
private static final boolean IS_WINDOWS = System.getProperty( "os.name" ).contains( "indow" );
private InputStream inputStream;
private static FileReader fileReader;
public String getAbsolutePath(Class appClass, String relativePath) {
try {
String parentPath = "";
if (StringUtils.isNotBlank(webAppPath)) {
parentPath = webAppPath;
} else {
parentPath = appClass.getProtectionDomain().getCodeSource().getLocation().getPath();
}
String osAppropriatePath = IS_WINDOWS ? parentPath.substring(1) : parentPath;
String absolutePath = osAppropriatePath + relativePath;
File file = new File(absolutePath);
if (!file.exists()) {
FileUtils.writeStringToFile(file, IOUtils.toString(readFile(relativePath), "UTF-8"));
}
return absolutePath;
} catch (IOException ioe) {
logger.log(Level.SEVERE, null, ioe);
return relativePath;
}
}
public void closeFileReader() {
synchronized (this) {
try {
inputStream.close();
} catch (IOException ex) {
Logger.getLogger(FileReader.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
private FileReader() {
}
public static FileReader getInstance() {
return new FileReader();
}
public static String getWebAppPath() {
return webAppPath;
}
public static void setWebAppPath(String webAppPath) {
FileReader.webAppPath = webAppPath;
}
}
And call the class to get the relative path as follows
FileReader.getInstance().getAbsolutePath(user.pig, "user.pig");
I solved this issue by using java.io.File as
final String filePath = new File("../src/user.pig").getAbsolutePath();
Related
I would like to obtain the most accurate File typed representation of a String that is supposed to refer to a local (existing) file in one of several forms like:
String file0 = "/home/my_user/file.txt"
String file1 = "file:///home/my_user/file.txt"
String file2 = "file.txt"; // assuming that the working dir is /home/my_user.
Is there a (quasy) single liner using the standard library or perhaps a common third party like apache-commons that would do the trick?
Thanks.
You can define your own function for this purpose. Given below is the function definition and test code:
import java.io.File;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
public class Main {
public static void main(String[] args) {
String file0 = "/Users/arvind.avinash/file.txt";
String file1 = "file:///Users/arvind.avinash/file.txt";
String file2 = "file.txt"; // assuming that the working dir is /Users/arvind.avinash.
System.out.println(getFile(file0).exists());
System.out.println(getFile(file1).exists());
System.out.println(getFile(file2).exists());
}
static File getFile(String pathOrUri) {
URI uri;
File file = null;
try {
uri = new URL(pathOrUri).toURI();
} catch (MalformedURLException e) {
return new File(pathOrUri);
} catch (URISyntaxException e) {
return new File(pathOrUri);
}
if (uri != null) {
file = new File(uri);
}
return file;
}
}
Output:
true
true
true
[Update]
Given below is a more simplified version:
import java.io.File;
import java.net.MalformedURLException;
import java.net.URI;
import java.net.URISyntaxException;
import java.net.URL;
public class Main {
public static void main(String[] args) {
String file0 = "/Users/arvind.avinash/file.txt";
String file1 = "file:///Users/arvind.avinash/file.txt";
String file2 = "file.txt"; // assuming that the working dir is /Users/arvind.avinash.
System.out.println(getFile(file0).exists());
System.out.println(getFile(file1).exists());
System.out.println(getFile(file2).exists());
}
static File getFile(String pathOrUri) {
URI uri;
try {
uri = new URL(pathOrUri).toURI();
} catch (MalformedURLException | URISyntaxException e) {
return new File(pathOrUri);
}
return new File(uri);
}
}
You be able to call new File(x) on examples 1 and 3 and it should work.
As for #2, you can create a URI, and then create File from that. In fact I think they all probably will work using URI
String fileStr = "file:///home/my_user/file.txt";
try {
URI uri = new URI(fileStr);
File f = new File(uri);
} catch (URISyntaxException ex) { ...}
I have my XML file under the src/main/resources directory. My spring code looks like
import java.io.IOException;
import java.util.concurrent.atomic.AtomicLong;
import com.google.common.base.Charsets;
import com.google.common.io.Files;
import org.springframework.core.io.ClassPathResource;
import org.springframework.integration.xml.transformer.XsltPayloadTransformer;
import org.springframework.messaging.Message;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.ResponseBody;
#Controller
public class BdeApplicationController {
#GetMapping("/ping")
#ResponseBody
public String ping(#RequestParam(name="name", required=false, defaultValue="Stranger") String name) {
return myFlow();
}
private String myFlow() {
XsltPayloadTransformer transformer = getXsltTransformer();
return transformer.transform(buildMessage(getXMLFileString())).toString();
}
private String getXMLFileString() {
try {
return Files.toString(new ClassPathResource("XML1.xml").getFile(), Charsets.UTF_8);
} catch (IOException e) {
e.printStackTrace();
}
return "";
}
private XsltPayloadTransformer getXsltTransformer() {
return new XsltPayloadTransformer(new ClassPathResource("XSLT1.xsl"));
}
protected Message<?> buildMessage(Object payload) {
return MessageBuilder.withPayload(payload).build();
}
}
On running this code I get the following exception: -
java.io.FileNotFoundException: class path resource [XML1.xml] cannot be resolved to absolute file path because it does not reside in the file system: jar:file:/Users/user/Documents/project/target/bde-0.0.1-SNAPSHOT.jar!/BOOT-INF/classes!/XML1.xml
Can you please suggest how can I fix this?
When you use resource.getFile() you look for the file in the file system, thats why it does't work when you runnit as a jar.
Try with a InputStream
String data = "";
ClassPathResource resource = new ClassPathResource("/XML1.xml");
try {
byte[] dataArr = FileCopyUtils.copyToByteArray(resource.getInputStream());
data = new String(dataArr, StandardCharsets.UTF_8);
} catch (IOException e) {
// do whatever
}
You have no File inside jar archive: use InputStream
Once you have the resource (via ClassPathResource) you should use getInputStream() to get its contents independently of where it is located. This way will work inside your IDE (actually a File there) and also when running the jar on the server (inside the jar archive, not exactly a File).
You need only modify your getXMLFileString() method so it uses the InputStream instead of the File:
private String getXMLFileString() {
String xml;
try {
BufferedReader reader = new BufferedReader(new InputStreamReader(resource.getInputStream()));
xml = reader.lines().collect(Collectors.joining("\n"));
reader.close();
} catch (IOException e) {
e.printStackTrace();
xml = null;
}
return new String(xml, Charsets.UTF_8);
}
I need to make a change in .properties file in my Java project. This is later deployed as a jar and used by other Java project. But according to this, I see that we should not directly make the change instead create a new object. Where should we create that new object and how can we make sure that its changes are visible?
Yes that's correct if your properties files is inside a jar then you won't be able to directly change that properties file since its packaged and zipped up in an archive. Instead you can create/change the file placed on a drive and read it, I used "user.home" for an example which you can change it as your need, below is the code for the same:
package com.test.util;
import java.io.File;
import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.util.Properties;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class PropertyFileReader {
private static final Logger LOGGER = LoggerFactory
.getLogger(PropertyFileReader.class);
private static Properties properties;
private static final String APPLICATION_PROPERTIES = "application.properties";
private static final String workingDir = System.getProperty("user.home");
private static File file = new File(workingDir, APPLICATION_PROPERTIES);
static {
properties = new Properties();
}
public static void main(String[] args) {
write("hello", "2");
System.out.println(read("hello"));
}
public static String read(final String propertyName) {
try (InputStream input = new FileInputStream(file)) {
properties.load(input);
} catch (IOException ex) {
LOGGER.error("Error occurred while reading property from file : ",
ex);
}
return properties.getProperty(propertyName);
}
public static void write(final String propertName,
final String propertyValue) {
try (OutputStream output = new FileOutputStream(file)) {
properties.setProperty(propertName, propertyValue);
properties.store(output, null);
} catch (IOException io) {
LOGGER.error("Error occurred while writing property to file : ", io);
}
}
}
package com.teamsite.client;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.util.Hashtable;
import com.interwoven.cssdk.common.CSClient;
import org.apache.commons.lang.StringUtils;
import com.interwoven.cssdk.access.CSUser;
import com.interwoven.cssdk.common.CSClient;
import com.interwoven.cssdk.common.CSException;
import com.interwoven.cssdk.filesys.CSAreaRelativePath;
import com.interwoven.cssdk.workflow.CSExternalTask;
import com.interwoven.cssdk.workflow.CSTask;
import com.interwoven.cssdk.workflow.CSURLExternalTask;
import com.interwoven.cssdk.workflow.CSWorkflow;
import java.util.logging.*;
public class ApplicationEdition implements CSURLExternalTask{
private static String userid;
private static String cssdkconfigfile;
private String applicationName;
private String applicationEditionPath;
private static CSClient csClient;
public static final String KEY_TARGET_TASK_NAME = "target_task_name";
private String transitionComment = "Auditing for deployed files ";
private String transition = "";
public static String getCssdkconfigfile() {
return cssdkconfigfile;
}
public static void setCssdkconfigfile(String cssdkconfigfile) {
ApplicationEdition.cssdkconfigfile = cssdkconfigfile;
}
private static CSClient getCsClient() {
return csClient;
}
private static void setCsClient(CSClient csClient) {
ApplicationEdition.csClient = csClient;
}
private static String getUserid() {
return userid;
}
private static void setUserid(String userid) {
ApplicationEdition.userid = userid;
}
private String getApplicationName() {
return applicationName;
}
private void setApplicationName(String applicationName) {
this.applicationName = applicationName;
}
private String getApplicationEditionPath() {
return applicationEditionPath;
}
private void setApplicationEditionPath(String applicationEditionPath) {
this.applicationEditionPath = applicationEditionPath;
}
#Override
public void execute(CSClient client, CSExternalTask currentTask, Hashtable params) throws CSException {
String userId = client.toString();
String cssdkconfigfile = "D:\\iw-home\\TeamSite\\cssdk\\cssdk.cfg";
setUserid(userId);
setCssdkconfigfile(cssdkconfigfile);
String targetTaskName = currentTask.getVariable(KEY_TARGET_TASK_NAME);
CSUser thisTaskOwner = currentTask.getOwner();
String thisTaskOwnerAddress = thisTaskOwner.getEmailAddress();
String branchName = currentTask.getArea().getBranch().getName();
CSAreaRelativePath[] files = currentTask.getFiles();
String Area = currentTask.getArea().getName();
System.err.println("*********************************************************");
System.err.println("Target task name"+targetTaskName);
System.err.println("Task owner's address"+thisTaskOwnerAddress);
System.err.println("Area name"+Area);
System.err.println("*********************************************************");
} private static CSTask getTaskByName(CSWorkflow job, String name) throws CSException {
if (name == null) {
return null;
}
CSTask[] tasks = job.getTasks();
for (int i=0; i<tasks.length; i++) {
if (name.equals(tasks[i].getName())) {
return tasks[i];
}
}
return null;
}
public static void showFiles(String string1,String string2,String string3 ) {
try {
File filename = new File ("C:\\temp\\ApplicationEditions_dynamic.txt");
BufferedWriter writer = new BufferedWriter(new FileWriter(filename,true));
writer.write(string1+"\n");
writer.write(string2+"\n");
writer.write(string3+"\n");
writer.newLine();
writer.close();
}
catch (Exception e)
{
System.out.println("Error occurred due to branch, refer output file");
}
finally {
}
} }
In this code, following files are imported in this source file
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileWriter;
import java.util.Hashtable;
import com.interwoven.cssdk.common.CSClient;
import org.apache.commons.lang.StringUtils;
import com.interwoven.cssdk.access.CSUser;
import com.interwoven.cssdk.common.CSClient;
import com.interwoven.cssdk.common.CSException;
import com.interwoven.cssdk.filesys.CSAreaRelativePath;
import com.interwoven.cssdk.workflow.CSExternalTask;
import com.interwoven.cssdk.workflow.CSTask;
import com.interwoven.cssdk.workflow.CSURLExternalTask;
import com.interwoven.cssdk.workflow.CSWorkflow;
Here source file is in location "package com.teamsite.client". So other files that are being imported from location com.interwoven.cssdk.common.CSClient, should have common path upto "com" folder and within "com" dir there should be dir "interwoven" and within this dir there should be other dir.
But when I check dir on server, I don't see any other dir than teamsite. This code workd fine without any problem.
So, how are these other files are getting imported in here ? Our environment is bit complex, but still files need to be in the path for being imported. We have repositories where jar is kept.
Thanks
When the class loader looks for a class e.g. com.interwoven.cssdk.workflow.CSExternalTask, it scans the entire classpath, looking for a directory branch like com/intervowen/cssdk/workflow. The above statement covers also the exploded jars that may be on the classpath.
The classpath usually contains more directories than your runnable jar file. Obviously, your jar is not expected to contain all classes contained in packages starting with com.*, so these can be imported from any location on the classpath.
You probably have a .jar file with the com.interwoven.cssdk. packages on your classpath somewhere.
.jar files behave like a zip file with it's own directory structure in it.
I need to find a solution to be able to manipulate a zip / jar directly (without unpacking) and without using third-party libraries. However I can't seem to grasp how the FileSystem ties in with Path and URI.
The URI that I'm trying to copy to is jar:file://E:/Projects/ZipDir/dist/test_folder/test.zip!/test_file.txt
The exception I'm getting is:
FileSystemNotFoundException but that zip file definitely does exist.
Using Java 7, this is what I have so far:
...
ZipDirectory zip = new ZipDirectory("test_folder/test.zip");
Path copyTo = zip.getPath("/test_file.txt");
Path copyFrom = Paths.get("test_file.txt");
Files.copy(copyFrom, copyTo, StandardCopyOption.REPLACE_EXISTING);
...
//
import java.io.IOException;
import java.net.URI;
import java.nio.file.*;
import java.util.HashMap;
public class ZipDirectory {
private Path path;
private FileSystem fileSystem;
public ZipDirectory(String path){
this.path = Paths.get(Paths.get(path).toUri());
create();
}
private void create(){
HashMap<String, String> env = new HashMap<>();
env.put("create", "true");
try {
fileSystem = FileSystems.newFileSystem(path, null);
} catch (IOException ex) {
System.out.println(ex);
}
}
public Path getPath(String relativePath){
return Paths.get(URI.create("jar:file:/" + path.toUri().getPath() + "!" + fileSystem.getPath(relativePath)));
}
public Path getRoot(){
return Paths.get(URI.create(path.toUri().getPath() + "!/"));
}
public void close(){
try {
fileSystem.close();
} catch (IOException ex) {
System.err.println(ex);
}
fileSystem = null;
}
}
I never thought I'd answer my own question, but I've got it working:
Treating an archive like a directory with Java 7