Directory existence check results NPE in hadoop - java

i am trying to verify that given path is a directory or file in HDFS but it results NPE at fs.getFileStatus(path).isDir() line. i don't understand what's the issue here , even i verified the path that's not null.
public static class RegexExcludePathFilter extends Configured implements
PathFilter {
private String path;
String patterns = "hdfs://localhost:9100/user/input-new/ncdc/filterdata/2007.[0-1]?[0-2].[0-9][0-9].txt" ;
Configuration conf;
Pattern pattern;
FileSystem fs;
#Override
public boolean accept(Path path) {
System.out.println(path);
try {
if(fs.getFileStatus(path).isDir()){
System.out.println(path);
return true;
}
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return path.toString().matches(patterns);
}
}
line 108 points to if statement in accept method.
thanks in advance

Have you inisialiezed FileSystem fs? I think that's the one that is null
make a constructor
public FileSystem(){
fs = new FileSystem();
}
Edit: i just noticed you are using a static class so i dont know if making a constructor will work.

Well you've already established that path is not null. The only thing that can be null now, is the FileStatus object returned. Check if that is null.
When chaining methods together like this, you should always be sure that somewhere in the middle a method cannot return null. If that may happen, you should never chain. Basically the method that can return null, breaks the chain.

Related

Why do I get a Reflection exception NoSuchMethodException if the tests runs fine

I'm using Reflection to Mock a private method (I don't want to discuss if that makes sense or not).
Anyone know why? I'll let my testClass source code here it may help. I've tryed much of the Internet helps and ways to solve this but none have worked for me.
public class testProtexManagerProcessRequiredFile {
#Mock
ProtexManager PxManager;
#Before
public void inicializa() {
MockitoAnnotations.initMocks(this);
}
#Test
public void processRequiredFileTest() throws ClassNotFoundException, IllegalAccessException, IllegalArgumentException, InvocationTargetException, NoSuchMethodException, SecurityException, InstantiationException {
Method method;
try {
method = ProtexManager.class.getDeclaredMethod("processRequiredFile", File.class);
method.setAccessible(true);
File FileExample = new File();
String NameExample = "Nome";
File outputs = new File();
outputs = (File) Mockito.when(method.invoke(PxManager, FileExample,NameExample)).thenReturn(FileExample);
assertNotNull(outputs);
assertEquals(outputs, method.invoke(PxManager, FileExample,NameExample));
} catch (NoSuchMethodException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("Teste Concluido.");
}
}
That's the method code:
private File processRequiredFile(File file, String name) {
if (!file.exists()) {
this.message = name + " file not found at location: " + file;
this.msgResponse.addMsgList(MsgCode.FAILURE, MsgLevel.ERROR, this.message, StringUtils.EMPTY);
}
return file;
}
And thank you all for helping me in my doubts.
To answer your question,
Because you caught the NoSuchMethodException. To get a test failure you have to somehow get some exception or error during the test execution
To follow up on the comments, here's how one can test this method:
// let's assume there are getter for this.message / this.msgResponse
// and this method is in the class foo.bar.Foobar
protected File processRequiredFile(File file, String name) {
if (!file.exists()) {
this.message = name + " file not found at location: " + file;
this.msgResponse.addMsgList(MsgCode.FAILURE, MsgLevel.ERROR, this.message, StringUtils.EMPTY);
}
return file;
}
In a test class foo.bar.FoobarTest:
#Mock
private File file;
private Foobar foobar = new Foobar();
#Test
public void testWithNonExistingFile() {
Mockito.when(this.file.exists()).thenReturn(false); // this is to illustrate, you could also use some non existent file: new File("/does-not-exists.foo")
File result = this.foobar.processRequiredFile(this.file, "some name");
assertThat(result).isEqualTo(this.file);
assertThat(foobar.getMsgResponse()).isNotEmpty(); // TODO: better assertion
assertThat(foobar.getMessage()).isEqualTo( "some name file not found at location: " + this.file);
}
#Test
public void testWithExistingFile() {
Mockito.when(this.file.exists()).thenReturn(true);
File result = this.foobar.processRequiredFile(this.file, "some name");
assertThat(result).isEqualTo(this.file);
assertThat(foobar.getMsgResponse()).isEmpty();
assertThat(foobar.getMessage()).isNull();
}
The class under test (i.e. Foobar) is really tested, this uses a real instance of it and call its method. A mock is used to replace something we don't have (here it's a file to illustrate but it's usually something more complicated)
What is your actual question? Why the testcase succeeds? That's already answered in the comments. You catch the exception and essentially ignore it. If you want to see the stacktrace on STDERR and let the testcase fail, you have to initiate the failing procedure yourself, e.g by calling
throw (AssertionFailedError) new AssertionFailedError("method not found").initCause(e);
This construct looks strange but JUnit 3 (I assume you're using that given your code) doesn't come with an AssertionFailedError with a constructor allowing to pass a cause. This way you see the stacktrace in your IDE as well and will be visible in JUnit-reports created during build processes.
Or is your question why the particular method is not found? One reason can be that someClass.getDeclaredMethod only returns a result if the method is declared in that particular class. If that class has a super class inheriting this method, you have to use the superclass when calling getDeclaredMethod to get the method.
If you don't know what class actually contains a method you have to iterate over all superclasses until reaching the "end":
Class<?> clazz = ProtexManager.class;
while (clazz != null) {
try {
return clazz.getDeclaredMethod("processRequiredFile", File.class);
catch(NoSuchMethodException e) {
clazz = clazz.getSuperClass();
}
}
That code block swallows the NoSuchMethodException but I don't want to do things more complicated than necessary to illustrate the idea.
Another reason why the method is not found might be that the class in question has a method processRequiredFile(java.io.File) and not processRequiredFile(com.blackducksoftware.sdk.codecenter.deeplicense.data.File). Also you later call the method by method.invoke using three parameters (PxManager, File, String) so either your call of getDeclaredMethod is missing parameter classes or your call of invoke will later fail due to the differences between declaration of the method and passed parameters.

How can I test java NIO static methods?

How can I test java NIO static methods?
I have a method
public Stream<String> getFileInformation(Provider provider) {
Path feedDirectoryPath;
DirectoryStream<Path> stream;
List<Path> files = new ArrayList<>();
try {
feedDirectoryPath = getFeedDirectoryPath();
if (!isFileExists(feedDirectoryPath)) {
return null;
}
stream = getDirectoryStream(feedDirectoryPath);
for(Path p : stream) {
files.add(p);
}
return Files.lines(Paths.get(files.stream().toString()));
} catch (Exception e) {
return null;
}
}
I have to test this method, but once my test arrives at Files.lines(..), I am stuck. I don't know how to assert this, and of-course if I were to pass a filePath like Path filePath = Paths.get("dummy.txt");
A NoSuchFileException would be thrown. Any help in testing this method would be greatly appreciated.
You could either supply it with an actual file (for instance in the src\test\resources folder if you're using maven), or mock the static method using PowerMock. I would recommend the first, it should be possible to load a file (for instance relative to your test class using classloader) and use this as input to the method under test.

Dynamically loading method from external class

I am trying to load methods Customer.cypher and Customer.cypherCBC method from my class Configuration. Customer class is rendering from different environments so few environmets are having cypherCBC() and cypher() method and few are having only cypher() method.
Now i want to check if cypherCBC if not there in Customer class then load cypher() method. My function is so far;
try {
Class<?> customerClass = Class.forName("com.myapp.impl.service.Customer");
Object obj = customerClass.newInstance();
//here getting "NoSuchMethodException" exception
Method methodCBC = customerClass.getDeclaredMethod("cypherCBC", String.class); //line - 7
if(methodCBC.getName().equals("cypherCBC")){
methodCBC.invoke(obj, new String(dbshPass));
System.out.println("CYPHER_CBC: "
+ methodCBC.invoke(obj, new String(dbshPass)));
}else{
Method method = customerClass.getDeclaredMethod("cypher", String.class);
method.invoke(obj, new String(dbshPass));
System.out.println("CYPHER: " + method.invoke(obj, new String(dbshPass)));
}
}catch (Exception e){
e.printStackTrace();
}
Getting an error at line 7.
NoSuchMethodException:
com.myapp.impl.service.Customer.cypherCBC(java.lang.String)
that means for particular environment class Customer doesn't having cypherCBC() method, but ideally it should come in else part and execute cypher() method.
Class<?> client = null;
Object obj = null;
try{
client = Class.forName("com.myapp.impl.service.Client");
obj = client.newInstance();
}catch (InstantiationException ex) {
System.err.println("Not able to create Instance of Class");
} catch (IllegalAccessException ex) {
System.err.println("Not able to access Class");
} catch (ClassNotFoundException ex) {
System.err.println("Not able to find Class");
}
try {
Method methodCBC = client.getDeclaredMethod("cypherCBC", String.class);
System.out.println("CYPHER_CBC: " + methodCBC.invoke(obj, new String(dbshPass)));
}catch (NoSuchMethodException ex) {
System.err.println("Not able to find Method on class");
ex.printStackTrace();
} catch (Exception e){
e.printStackTrace();
}
That is exactly what is to be expected: getDeclaredMethod() throws that exception when no method exists that meets your specification. And you are wondering that it throws an exception if the required method is missing? Hint: better read the javadoc next time. Don't assume that something does something, but verify your assumptions!
Besides: read your code again. What is it doing? You are asking "give me the method named 'foo'". And then, your next step is to ask that method "is your name 'foo'". So even without reading javadoc, it should become clear that your logic is flawed.
As solution, you can implement a non-throwing lookup yourself, like
private Method lookupCypher(Class<?> client, String methodName) {
for (Method declaredMethod : client.getDeclardMethods()) {
if (declaredMethod.getName().equals(methodName)) {
Class<?>[] parameterTypes = declaredMethod.getParameterTypes();
if (parameterTypes.length == 1 && parameterTypes[0].equals(String.class)) {
// so declaredMethod has the given name, and takes one string as argument!
return declaredMethod;
}
}
// our search didn't reveal any matching method!
return null;
}
Using that helper method, you can rewrite your code to:
Method toInvoke = lookupCypher(client, "cypherCBC");
if (toInvoke == null) {
toInvoke = lookupCypher(client, "cypher");
}
toInvoke(obj, new String ...
Or, with the idea from hunter in mind; a much more "OO like" version:
interface CustomerCypherWrapper {
void cypher(String phrase);
}
class NewCustomerWrapper() implements CustomerCypherWrapper {
#Override
void cypher(String phrase) {
new Customer.cypherCBC(phrase);
}
}
class oldCustomerWrapper() implements CustomerCypherWrapper {
#Override
void cypher(String phrase) {
new Customer.cypher(phrase);
}
}
And your client code boils down to:
CustomerCypherWrapper wrapper =
(lookupCypher(..., "cypherCBC") == null)
? new NewCustomerWrapper()
: new OldCustomerWrapper();
wrapper.cypher();
[ I hope you notice that my version A) is easier to read and B) doesn't contain any duplicated code any more. ]
And yes, an alternative implementation of the lookup method could just go like
private Method lookupCyper(Client<?>, String methodName) {
try {
return client.getDeclaredMethod(methodName, String.class);
} catch ....
and return null;
}
... return your public cypherCBC method
But that is an "uncommon practice" in Java. In Java, we ask for permission; instead of forgiveness. Unlike other languages
if you compile the application with a Customer class which has both method,you can use reflection once to check whether the cypherCBC method available or not at runtime, then you can keep that status, you can call the method without using reflection
if(newVersion)
{
customer.cypherCBC(arg);
}
else
{
customer.cypher(arg);
}
But to write a better application,you should use two version baselines.
even though this is a small code fragment you should setup a another module to hide this Customer class and its interactions,that module should have two versions. but your main module has only single version.Now when you you deliver the application , product should be packaged with right version baseline based on compatibility for the target environment.
Although reflection works (as explained in the other answers). if you have control over the Customer class, you can try a non-reflection approach.
interface CBCCypherable {
public String cypherCBC(String pass);
}
You can now have two versions of Customer class, one that implements CBCCypherable and one that doesn't. And when you call it, it looks like this:
Customer c = new Customer();
if (c instanceof CBCCypherable) {
((CBCCypherable)c).cypherCBC(pass);
} else {
c.cypher(pass);
}
What you get with this solution is much simpler code, and as a bonus the compiler will check that you use the correct method name and parameter types. Unlike with reflection, where that's all your job, and you have to run the code to find out if something's wrong.
P.s.: I don't know if this is just sample code or you are really encrypting/hashing passwords here, but it's generally considered a bad idea to roll your own security code.

Use of try/catch inside constructor calling another constructor

I m having a problem of calling a constructor with arguments from default constructor.
Class A {
private static Properties properties;
A(Properties property){
// do some checks
try{
load(property, fileName)
} catch(IOException e) {
throw new RuntimeException();
}
}
A(){
this(load(properties));
}
private static Properties load(Properties properties, String fileName ) throws IOException {
try {
properties.load(A.class.getClassLoader()
.getResourceAsStream(fileName));
} catch (IOException ioException) {
throw new IOException("Unable to process the properties File. " + fileName, ioException);
}
return properties;
}
}
My problem is: In Default constructor I wanted to use try/catch block and do the same operation of throwing a run time exception. Can you help me out as what can be done in this?
WRT this post: chaining constructors in Java without throwing exceptions from the default constructor
I have an option of putting try/catch inside another method. But is there any other way?
P.S: I do not want to use 'throws'
Java does not allow chained constructor calls to be enclosed within a try block, since such constructs could if not restricted allow an object whose base object threw an exception to end up being returned to calling code. This makes it difficult to express certain concepts involving resources like files [e.g. it would be helpful to have a constructor open a file before chaining to the parent and close it afterward, but there's no way to safely have a constructor take responsibility for a file which is opened before chaining to the parent]. The best one can do in Java is avoid public constructors which might throw exceptions and instead use factory methods which can be better equipped to handle them.
Option 1: Pass the other constructor a new empty instance of Properties:
class A
{
public A()
{
this(new Properties());
}
// rest of code...
}
Option 2: Pass the other constructor a null instance of Properties. You'll then have to guard against null in load(...), but you probably should be anyway:
class A
{
public A()
{
this(null);
}
// rest of code...
}
Option 3: Pass the other constructor a default instance of Properties:
class A
{
private static final Properties defaultProperties;
static
{
defaultProperties = new Properties();
// populate here however you wish
}
public A()
{
this(defaultProperties);
}
// rest of code...
}

A point of style with try/catch

I was wondering if this approach was correct :
public ITask getState()
{
statePredicate[Some predicate definition];
ITask nextRunnable = null;
try {
nextRunnable = Iterables.find((Iterable)queue, statePredicate);
}
catch (NoSuchElementException e)
{}
return nextRunnable;
}
The points on which I am wondering are :
should the predicate be cached as a member of the class ?
I do nothing with the catch, I do not even log it because it is
normal for my app to not find anything.
t return null because I do a final return.
Thank you for your input !
-
1) If the predicate is always the same, I would make it a static final class member.
2) There is also a version of Iterables.find that you can specify a default value to (assuming you're using Google Guava). Then you don't need to deal with the NoSuchElementException at all.
3) Is there a reason to cast queue to Iterable? If this is not necessary, then don't cast.
class MyClass {
private static final Predicate STATE_PREDICATE = new Predicate<ITask>() {
#Override
public boolean apply(ITask input) {
// ... your code here
}
};
public ITask getState() {
return Iterables.find(queue, STATE_PREDICATE, null);
}
}
If the exception is really the usual case in your approach than you should put at least a comment into the catch area to make clear for everyone who reads the code that it was intentional and not a mistake. In my opinion returning Null is something different, but it some circumstanced not avoidable.

Categories