We Keep Coding

checkmarx - How to resolve Stored Absolute Path Traversal issue?

Checkmarx - v 9.3.0 HF11
I am passing env value as data directory path in docker file which used in dev/uat server
ENV DATA /app/data/
In local, using following Environment variable
DATA=C:\projects\app\data\
getDataDirectory("MyDirectoryName"); // MyDirectoryName is present in data folder
public String getDataDirectory(String dirName)
{
String path = System.getenv("DATA");
if (path != null) {
path = sanitizePathValue(path);
path = encodePath(path);
dirName = sanitizePathValue(dirName);
if (!path.endsWith(File.separator)) {
path = path + File.separator;
} else if (!path.contains("data")) {
throw new MyRuntimeException("Data Directory path is incorrect");
}
} else {
return null;
}
File file = new File(dirName); // NOSONAR
if (!file.isAbsolute()) {
File tmp = new File(SecurityUtil.decodePath(path)); // NOSONAR
if (!tmp.getAbsolutePath().endsWith(Character.toString(File.separatorChar))) {
dirName = tmp.getAbsolutePath() + File.separatorChar + dirName;
} else {
dirName = tmp.getAbsolutePath() + dirName;
}
}
return dirName;
}
public static String encodePath(String path) {
try {
return URLEncoder.encode(path, "UTF-8");
} catch (UnsupportedEncodingException e) {
logger.error("Exception while encoding path", e);
}
return "";
}
public static String validateAndNormalizePath(String path) {
path = path.replaceAll("/../", "/");
path = path.replaceAll("/%46%46/", "/");
path = SecurityUtil.cleanIt(path);
path = FilenameUtils.normalize(path); // normalize path
return path;
}
public static String sanitizePathValue(String filename){
filename = validateAndNormalizePath(filename);
String regEx = "..|\\|/";
// compile the regex to create pattern
// using compile() method
Pattern pattern = Pattern.compile(regEx);
// get a matcher object from pattern
Matcher matcher = pattern.matcher(filename);
// check whether Regex string is
// found in actualString or not
boolean matches = matcher.matches();
if(matches){
throw new MyAppRuntimeException("filename:'"+filename+"' is bad.");
}
return filename;
}
public static String validateAndNormalizePath(String path) {
path = path.replaceAll("/../", "/");
path = path.replaceAll("/%46%46/", "/");
path = SecurityUtil.cleanIt(path);
path = FilenameUtils.normalize(path); // normalize path
return path;
}
[Attempt] - Update code which I tried with the help of few members to prevent path traversal issue.
Tried to sanitize string and normalize string, but no luck and getting same issue.
How to resolve Stored Absolute Path Traversal issue ?

Your first attempt is not going to work because escaping alone isn't going to prevent a path traversal. Replacing single quotes with double quotes won't do it either given you need to make sure someone setting a property/env variable with ../../etc/resolv.conf doesn't succeed in tricking your code into overwriting/reading a sensitive file. I believe Checkmarx won't look for StringUtils as part of recognizing it as sanitized, so the simple working example below is similar without using StringUtils.
Your second attempt won't work because it is a validator that uses control flow to prevent a bad input when it throws an exception. Checkmarx analyzes data flows. When filename is passed as a parameter to sanitizePathValue and returned as-is at the end, the data flow analysis sees this as not making a change to the original value.
There also appears to be some customizations in your system that recognize System.getProperty and System.getenv as untrusted inputs. By default, these are not recognized in this way, so anyone trying to scan your code probably would not have gotten any results for Absolute Path Traversal. It is possible that the risk profile of your application requires that you call properties and environment variables as untrusted inputs, so you can't really just remove these and revert back to the OOTB settings.
As Roman had mentioned, the logic in the query does look for values that are prepended to this untrusted input to remove those data flows as results. The below code shows how this could be done using Roman's method to trick the scanner. (I highly suggest you do not choose the route to trick the scanner.....very bad idea.) There could be other string literal values that would work using this method, but it would require some actions that control how the runtime is executed (like using chroot) to make sure it actually fixed the issue.
If you scan the code below, you should see only one vulnerable data path. The last example is likely something along the lines of what you could use to remediate the issues. It really depends on what you're trying to do with the file being created.
(I tested this on 9.2; it should work for prior versions. If it doesn't work, post your version and I can look into that version's query.)
// Vulnerable
String fn1 = System.getProperty ("test");
File f1 = new File(fn1);
// Path prepend - still vulnerable, tricks the scanner, DO NOT USE
String fn2 = System.getProperty ("test");
File f2 = new File(Paths.get ("", fn2).toString () );
// Path prepend - still vulnerable, tricks the scanner, DO NOT USE
String fn3 = System.getProperty ("test");
File f3 = new File("" + fn3);
// Path prepend - still vulnerable, tricks the scanner, DO NOT USE
String fn4 = System.getProperty ("test");
File f4 = new File("", fn4);
// Sanitized by stripping path separator as defined in the JDK
// This would be the safest method
String fn5 = System.getProperty ("test");
File f5 = new File(fn5.replaceAll (File.separator, ""));
So, in summary (TL;DR), replace the file separator in the untrusted input value:
String fn5 = System.getProperty ("test");
File f5 = new File(fn5.replaceAll (File.separator, ""));
Edit
Updating for other Checkmarx users that may come across this in search of an answer.
After my answer, OP updated the question to reveal that the issue being found was due to a mechanism written for the code to run in different environments. Pre-docker, this would have been the method to use. The vulnerability would have still been detected but most courses of action would have been to say "our deployment environment has security measures around it to prevent a bad actor from injecting an undesired path into the environment variable where we store our base path."
But now, with Docker, this is a thing of the past. Generally the point of Docker is to create applications that run the way same everywhere they are deployed. Using a base path in an environment likely means OP is executing the code outside of a container for development (based on the update showing a Windows path) and inside the container for deployment. Why not just run the code in the container for development as well as deployment as is intended by Docker?
Most of the answers tend to explain that OP should use a static path. This is because they are realizing that there is no way to avoid this issue because taking an untrusted input (from the environment) and prefixing it to a path is the exact problem of Absolute Path Traversal.
OP could follow the good advice of many posters here and put a static base path in the code then use Docker volumes or Docker bind mounts.
Is it difficult? Nope. If I were OP, I'd fix the base path prefix in code to a static value of /app/data and do a simple volume binding during development. (When you think about it, if there is storage of data in the container during a deployment then the deployment environment must be doing this exact thing for /app/data unless the data is not kept after the lifetime of the container.)
With the base path fixed at /app/data, one option for OP to run their development build is:
docker run -it -v"C:\\projects\\app\\data":/app/data {container name goes here}
All data written by the application would appear in C:\projects\app\data the same way it does when using the environment variables. The main difference is that there are no environment-variable-prefixed paths and thus no Absolute Path Traversal results from the static analysis scanner.

It depends on how Checkmarx comes to this point. Most likely because the value that is handed to File is still tainted. So make sure both /../ and /%46%46/ are replaced by /.
checkedInput = userInput.replaceAll("/../", "/");
Secondly, give File a parent directory to start with and later compare the path of the file you want to process. Some common example code is below. If the file doesn't start with the full parent directory, then it means you have a path traversal.
File file = new File(BASE_DIRECTORY, userInput);
if (file.getCanonicalPath().startsWith(BASE_DIRECTORY)) {
// process file
}
Checkmarx can only check if variables contain a tainted value and in some cases if the logic is correct. Please also think about the running process and file system permissions. A lot of applications have the capability of overwriting their own executables.

If there is one thing to remember it is this
use allow lists not deny lists
(traditionally known as whitelists and blacklists).
For instance, consider replacing /../ with / suggested in another answer. My response is to contain the sequence /../../. You could pursue this iteratively, and I might run out of adversarial examples, but that doesn't mean there are any.
Another problem is knowing all the special characters. \0 used to truncate the file name. What happens to non-ASCII characters - I can't remember. Might other code be changed in future so that the path ends up on a command line with other special characters - worse, OS/command line dependent.
Canonicalisation has its problems too. It can be used to some extent probe the file system (and perhaps beyond the machine).
So, choose what you allow. Say
if (filename.matches("[a-zA-Z0-9_]+")) {
return filename;
} else {
throw new MyException(...);
}
(No need to go through the whole Pattern/Matcher palaver in this situation.)

For this issue i would suggest you hard code the absolute path of the directory that you allow your program to work in; like this:
String separator = FileSystems.getDefault().getSeparator();
// should resolve to /app/workdir in linux
String WORKING_DIR = separator + "app"+separator +"workdir"+separator ;
then when you accept the parameter treat it as a relative path like this:
String filename = System.getProperty("test");
sanitize(filename);
filename = WORKING_DIR+filename;
File dictionaryFile = new File(filename);
To sanitize your user's input make sure he does not include .. and does not include also \ nor /
private static void sanitize(filename){
if(Pattern.compile("\\.\\.|\\|/").matcher(filename).find()){
throw new RuntimeException("filename:'"+filename+"' is bad.");
}
}
Edit
In case you are running the process in linux you can change the root of the process using chroot maybe you do some googling to know how you should implement it.

how about using Java's Path to make the check("../test1.txt" is the input from user):
File base=new File("/your/base");
Path basePath=base.toPath();
Path resolve = basePath.resolve("../test1.txt");
Path relativize = basePath.relativize(resolve);
if(relativize.startsWith("..")){
throw new Exception("invalid path");
}

Based on reading the Checkmarx query for absolute path traversal vulnerability (and I believe in general one of the mitigation approach), is to prepend a hard coded path to avoid the attackers traversing through the file system:
File has a constructor that accepts a second parameter that will allow you to perform some prepending
String filename = System.getEnv("test");
File dictionaryFile = new File("/home/", filename);
UPDATE:
The validateAndNormalizePath would have technically sufficed but I believe Checkmarx is unable to recognize this as a sanitizer (being a custom written function). I would advice to work with your App Security team for them to use the CxAudit and overwrite the base Stored Path Traversal Checkmarx query to recognize validateAndNormalizePath as a valid sanitizer.

JavaFX Image from resources folder

For some reason I keep getting an NPE in a gradle javafx project.
My folder structure is very basic. I have a package with my java files in the main/java folder. I also have my resources in the main/resources folder. When I try to load image.png it gives me an NPE.
public static Image createImage(Object context, String url) {
Image m = null;
InputStream str = null;
URL _url = context.getClass().getResource(url);
try {
m = new Image(_url.getPath());
} catch (NullPointerException e) {
e.printStackTrace();
}
return m;
}
This is a helper class.
From the Scene I call: Image image = Helper.createImage(this, "image.png");
The absolute path to the image would be main/resources/images/image.png.
I checked every tutorial on the internet but I couldn't find any solution for this. I also tried it with the path to the image as parameter and also with an InputStream but it never worked.

Resources
The Class#getResource(String) and related API are used for locating resources relative to the class path and/or module path. When using Class to get a resource you can pass an absolute name or a relative name. An absolute name will locate the resource relative to the root of the class path/module path; an absolute name starts with a /. A relative name will locate the resource relative to the location of the Class; a relative name does not start with a leading /.
In a typical Maven/Gradle project structure, the src/main/java and src/main/resources are roots of the class path/module path. This means all resource names are relative to those directories. It's slightly more complicated than that because the files under those directories are moved to the target/build directory and it's that location that's put on the class path/module path, but for all intents and purposes consider the source directories as the root. There's a reason a get-resource API exists in the first place, to provide an application-location-independent way of obtaining resources.
Issues in Your Code
From your question I gather your project structure looks something like:
<project-dir>
|--src/
|--main/
|--java/
|--resources/
|--images/
|--image.png
And you're calling your method with an Object and a resource name of image.png. The problem here is that, since you're passing a relative name, the resource is located relative to the Class of the passed Object (i.e. context). I doubt your class is located in a package named images which means the resource will not be found relative to said class. You need to pass an absolute name: /images/image.png.
The other problem is your use of URL#getPath(). The URL you obtain from Class#getResource(String) will, if the resource were to be found, look something like this:
file:/path/to/gradle/project/build/resources/main/images/image.png
But the result of URL#getPath() will give you:
/path/to/gradle/project/build/resources/main/images/image.png
This causes a problem due to the way Image works. From the documentation:
All URLs supported by URL can be passed to the constructor. If the passed string is not a valid URL, but a path instead, the Image is searched on the classpath in that case.
Notice the second sentence. If the passed URL does not have a scheme then it's interpreted as a resource name and the Image will locate the image file relative to the classpath. In other words, since you're passing the value of URL#getPath() to the Image constructor it searches for the resource image.png in the package path.to.gradle.project.build.resources.main.images. That package does not exist. You should be passing the URL as-is to the Image constructor via URL#toString() or URL#toExternalForm().
Solution
If you're going to use the URL returned by Class#getResource(String) to load the Image then no matter what you need to use URL#toString() or URL#toExternalForm() instead of URL#getPath().
public static Image createImage(Object context, String resourceName) {
URL _url = context.getClass().getResource(resourceName);
return new Image(_url.toExternalForm());
}
Then you have at least two options:
Pass the absolute resource name (i.e. "/images/image.png") to your #createImage(Object,String) method since the image.png resource is not in the same package as the passed Object (i.e. context).
Move the resource to the same package as the class of the passed in Object (i.e. context). For instance, if the context object's class is com.foo.MyObject then place the resource under src/main/resources/com/foo and it will be in the same package as MyObject. This will allow you to continue passing the relative resource name.
Of course, as noted by the documentation of Image you can pass a scheme-less URL and it's interpreted as a resource name. In other words, you could do:
Image image = new Image("images/image.png");
And that should work. A note of caution, however: When using modules the above will only work if the resource-containing package is opens unconditionally or if the module itself is open.

Try using the path /images/image.png.
The resources always get referenced from the class root, in your case src/main/resources, so from there going to /images/image.png should be the correct path.

this is how I am passing the images in my application. ivSerialAssignmentLogo is a FXML element (ImageView).
ivSerialAssignmentLogo.setImage(new Image(getClass().getResourceAsStream("/img/serialAssignment.svg")));
In your case, you could use something like that
public static Image createImage(Object context, String url) {
Image m = null;
InputStream str = null;
URL _url = context.getClass().getResource("/images/" + url);
try {
m = new Image(_url.getPath());
} catch (NullPointerException e) {
e.printStackTrace();
}
return m;
}

Cannot find protocol 'resource' in URL constructor

I've implemented a class to read from RSS 2.0 and Atom 1.0 feeds. I want to write some unit tests in order to verify functionality. Here is the feed reader section of my code:
private String readFeed(final String url) throws IOException
{
final StringBuilder builder = new StringBuilder();
final URL feedUrl = new URL(url);
final BufferedReader in = new BufferedReader(
new InputStreamReader(feedUrl.openStream()));
String input;
while ((input = in.readLine()) != null)
{
builder.append(input);
}
in.close();
return builder.toString();
}
After some research, I figured the best way to test would be to have a sample feed as an XML file in my project resources directory.
I've created a example file "resources/rss2-0.xml"
I'm sending in the following value to the readFeed function, "resource:///rss2-0.xml",
and I keep receiving java.net.MalformedURLException: unknown protocol: resource
This is my first time using a URL pathway to load from a resource. From what I can tell, resource seems like it should be a valid protocol. Anyone have any ideas what I may be doing wrong or other ways to go about this?

If you want to deal with path using your local file system, the Path class is best suited for this task.
An object that may be used to locate a file in a file system. It will
typically represent a system dependent file path.
You can use it like so :
Path path = FileSystems.getDefault().getPath("/resources/rss2-0.xml");
BufferedReader reader = Files.newBufferedReader(path, StandardCharsets.UTF_8);
If your really want to deal with URL, the protocol you're looking for is simply "file". So it would be file:///rss2-0.xml instead of resource:///rss2-0.xml and even file:/resources/rss2-0.xml to be exact.
Note that in your case, you will indeed have to deal with URLs sooner or later, but when working on local tests, using the Path class will save you troubles. If you want another alternative, try the URI class. Since an URI is an identifier (see difference between URI and URL) it can identify either an URL or a Path an may serve as a bridge between your production code which will ultimately deal with URLs and your test code where the Path class could be best put in use.
For example :
public interface FeedReader {
String readFeed(final URI uri);
}
And 2 implementations, one for testing locally :
public class LocalFeedReader implements FeedReader {
#Override
public String readFeed(final URI uri) {
// URI -> Path
// then dealing with Path to target local rss2-0.xml file
}
}
And one for production code :
public class WebFeedReader implements FeedReader {
#Override
public String readFeed(final URI uri) {
// URI -> URL
// then dealing with URL to target real resources
}
}

The java docs say that only http, https, file, and jar are "guaranteed" to exist on the search path for protocol handlers. Others only "may" be supported.
http://docs.oracle.com/javase/8/docs/api/java/net/URL.html#URL-java.lang.String-java.lang.String-int-java.lang.String-
It looks like if you want a custom handler that isn't supported in your java distribution, you'll have to create one.
http://mjremijan.blogspot.com/2012/02/create-your-own-java-url-handlers.html

Why do I get ProviderMismatchException when I try to .relativize() a Path against another Path?

[note: self answered question]
I have opened a FileSystem to a zip file using java.nio. I have gotten a Path from that filesystem:
final Path zipPath = zipfs.getPath("path/into/zip");
Now I have a directory on the local filesystem which I have obtained using:
final Path localDir = Paths.get("/local/dir")
I want to test whether /local/dir/path/into/zip exists, so I check its existence using:
Files.exists(localDir.resolve(zipPath))
but I get a ProviderMismatchException. Why? How do I fix this?

This behaviour is documented, albeit it is not very visible. You have to delve into the java.nio.file package description to see, right at the end, that:
Unless otherwise noted, invoking a method of any class or interface in this package created by one provider with a parameter that is an object created by another provider, will throw ProviderMismatchException.
The reasons for this behaviour may not be obvious, but consider for instance that two filesystems can define a different separator.
There is no method in the JDK which will help you there. If your filesystems use the same separator then you can work around this using:
path1.resolve(path2.toString())
Otherwise this utility method can help:
public static Path pathTransform(final FileSystem fs, final Path path)
{
Path ret = fs.getPath(path.isAbsolute() ? fs.getSeparator() : "");
for (final Path component: path)
ret = ret.resolve(component.getFileName().toString());
return ret;
}
Then the above can be written as:
final Path localPath = pathTransform(localDir.getFileSystem(), zipPath);
Files.exists(localDir.resolve(localPath));

How can I get a URL from relative path under a jar?

What I have done so far is :
/** Default location of help files folder */
private static final String DEFAULT_PATH = "../../../../resources/files/help/";
/** A 404 error file */
private static final String ERROR_404 = "../../../../resources/files/help/404.html";
/**
* #param fileName
* #return The URL of file, if found at default location, otherwise a 404 error page URL.
*/
public static URL getURL(String fileName) throws MalformedURLException{
URL url = (new File(ERROR_404)).toURI().toURL();
System.out.println(url);
url = (new File(DEFAULT_PATH + fileName)).toURI().toURL();
System.out.println(url);
return url;
}
Output:
file:/H:/My%20Project/Project%20Name%20Module/../../../../resources/files/help/404.html
file:/H:/My%20Project/Project%20Name%20Module/../../../../resources/files/help/plazaCode.html
Folder Hierarchy in the JAR created through NetBeans:
I am on Windows 7, JDK 7.
UPDATE:
Actually I want this URL for a JTextPane to show a HTML page by method:
textPane.setPage(URL url);
Can I have any better solution than this? and with the same Folder Heirarchy.. ?

404.html since this is an application resource, it will probably end up embedded in a Jar.
Resources in archives cannot be accessed using a File object.
URL getURL(String fileName) To help avoid confusion, change that to URL getURL(String resourceName).
Use Class.getResource(String) much as discussed on your previous questions.
'Relative' URLs become dangerous by that stage, since they depend on the package of the class that calls them, I generally make them 'absolute' by prefixing the entire path with a single / which effectively means 'look for this resource, from the root of the run-time class-path'.
So that String might read (adjusting the rest of the code as already advised):
private static final String ERROR_404 = "/resources/files/help/404.html";

You can use URL url = getClass().getResource("..."). Probably "/files/help/404.html".

When you create a File in this way you will get a relative file (and therefore a relative URL). To get a absolute URL from that path you need to get an absolute File first.
Try this:
new File(ERROR_404).getCanonicalFile().toURI().toURL()
(EDIT: After you JTextPane information above)
I haven't ever tried this, so I have no clue on why that's not working. But its a different question, you should ask a new Question.