Related
I am trying to solve the issue where one of unit tests fails only when I run all the tests in the project.
Thus I would like to come inside some common code only when my failing test is executed. I was trying to find out how to achieve it in Intellij IDEA because there is a feature of condition breakpoints, but for now I fail because don't understand how I can write this type of condition plus it seems that other possibilities don't really allow this.
Is it possible?
Put a breakpoint in the test case that fails (in the image its line 28). Put another break point(in the image its line 42) in the common-code and make that as a conditional one that is disabled till the break point in the test case is hit. To get this window, right click the breakpoint (sometimes you'll have to disable and then re-enable the suspend checkbox).
Hope it helps!
I'm not aware of an IDEA feature which lets you do this, however there are some manual steps that you can take to do this type of debugging:
Add a breakpoint to the first line of the unit test that is failing.
When this breakpoint hits, add another breakpoint in the common class you're interested in.
Alternatively, you can do this in a more automated, repeatable way by making some minor modifications to your unit test code.
Add a public static boolean field to your Unit test class which is set to false by default.
set this to true in the first line of the failing test
Use this field in your test condition. e.g. if your unit test was "ThingTest" and your field "footest" - you're condition will be "ThingTest.footest == true"
Set it to false at the end of your failing test (if you want the follow on tests to run normally)
How about declaring a boolean value, set it to false by default, and only set it to true when you test is failing.
Then you can set a condition on your in your breakpoint, to watch only when the boolean is set to true.
In IntelliJ, you add your breakpoint as per normal, then right click on it, and select View Breakpoint. Another window open, and you can enter your condition.
Hope this helps.
You have two options:
You can like you said use an conditional breakpoint if there is some state (value) in the common code, that is specific to the test you are trying to check. For example a specific string parameter.
You can put an extra breakpoint in the test you are trying to check
before the common code is called from this test. Once this breakpoint
is hit you activate the original breakpoint. You can do this
manually or using the option "Disabled until selected breakpoint is
hit" in the breakpoint dialog of the original breakpoint by
choosing your extra breakpoint in the combo box. If you use the
"automatic"-way you can even uncheck "Suspend" for the extra
breakpoint. This way it will only halt/suspend on your original breakpoint.
I have searched and found that indeed Eclipse does not support this 'direct' feature. But, Did I stil miss something? and Is it present in other IDEs?
Let me elaborate my question more -
if a statement falls under execution flow based on an expression evaluation, then why can't we force execute it? (without the execution of the expression).
For example consider this -
... if(bool returnsABoolean) {
<execute some statement>;
}
...
Can the execution of 'if' be skipped and the statement be executed as a 'next statement'? (I obviously can control the value of 'returnAsBoolean' in the Variables view; but can I not skip (in a controlloed manner) all the statements until a particular statement in the execution?)
Highlight the code you want to run and right-click/Execute or press Ctrl+U.
Alternatively to "Execute", use "Display" (Ctrl+Shift+D) or "Inspect" (Ctrl+Shift+I) to see the result.
Looks like you want the 'Display' view - in the Debug perspective, do :
Window -> ShowView -> Display.
You can enter Java statements to execute there (you have to select the bit of text each time that you want to execute)
http://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.jdt.doc.user%2Freference%2Fviews%2Fdisplay%2Fref-display_view.htm
Debugging allows you to run a program interactively while watching the source code and the variables during the execution.
So debugging is nothing but executing program but inspecting the elements during execution but you can not jump into something which is not there in execution.
You can use keyboard buttons F5,F6 ,F8 etc. (For Eclipse) and other Shortcuts during debugging for your convinience but you can't jump to something directly which is not in the execution sequence.
Debugging Shortcuts:
F5 Step into
F6 Step over
F8 Resume and will take you to the break point
Ctrl+Shift+B Toggle breakpoint
Ctrl+Shift+D Display Info of current statement
Ctrl+Shift+I Inspect the selected element
You can Skip some code by the use of breakpoint you can directly jump to specific point and avoid debugging of code which you believe works fine.Or you can jump out code snippet if you want to.
The question really was to set the Instruction pointer at will. This has been discussed and through url's i pasted on the comments above - this is not an eclipse feature (yet).
I wonder if a lot of people program in Java with assertions. I think this can be very useful on large projects without enough written contracts or outdated contracts. Particularly when you use webservices, components, etc.
But I have never seen any project using assertions (except in JUnit/testing tests...).
I've noticed that the thrown class is an Error and not an Exception. Why do they choose an error? Can it be because an exception could be unexpectedly caught and not logged/rethrown?
If you develop an application with components, I wonder where you put the assertions:
On the component side, just before returning the data through the public API?
On the component client side? And if the API is called everywhere you set up a facade pattern that will call the assertion mechanism? (Then I guess you put your assertions and facade on some external project and your client projects will depend on this assertion project?)
I understand how to use assertions, and when use them, but just wonder if some people have recommendations based on a real experience of assertions.
By the way, do you refer to assert in Java?
I personally find assertions especially useful for invariants.
Take into account that assertion checking is turned off by default in Java. You have to add the -ea flag to enable assertion checking.
In other words, you can test your application in a kind of debug mode, where the program will halt once an assertion is broken. On the other hand, the release application will have its assertion turned off and will not incur time penalty for assertion checking. They will be ignored.
In Java, assertions are far less powerful than exceptions and have totally different meanings. Exceptions are there when something unexpected happens and you have to handle it. Assertions are about the correctness of your code. They are here to confirm that what 'should be' is indeed the case.
My rough policy, especially when working with many developers:
public methods: always check arguments and throw IllegalArgumentException when something is wrong
private methods: use assertions to check arguments against null pointers and so on
complex methods: intermediate assertions to ensure that the intermediate results satisfy requested properties
...but actually, I use them sparsingly. Just where it's critical or error-prone places.
About the minor usage of asserts, I think that was a bad decision to make assertions disabled by default.
About extending Error I suppose it extends Error because Errors are exceptions that are not expected to be caught. And that way, when in your code you have catch(Exception), the assertion isn't cached.
And about usage, the best place is in precoditions, postconditions or in the middle of the code in any invariant you want to check.
In my opinion, errors in Java should be treated as an Exception. Therefore I would enable assertions in development and in private methods to check that my code is running fine and don't pass invalid values to private methods.
Since those checks should be made in public methods, I wouldn't not check again in private methods.
In order to disable assertions:
-da flag in compiler
In my opinion, in public methods you should check and manage the exception or log them yourself.
Assertions should not be used outside tests because they can be turned off in a production environment, which may cause serious problems due to the lack of proper checking.
However, I've seen statement that it is allowed to use them to check parameters in private methods. That's because you assume that data which managed to reach to your private method is correct and if it isn't application may fail hard.
I'd like to know if it is "ok" to write a test without any "assert" in it. So the test would fail only when an exception / error has occured.
Eg: like a test which has a simple select query, to ensure that the database configuration is right. So when I change some db-configuration, I re-run this test and check if the configuration is right. ?
Thanks!
It is perfectly valid to make sure a unit test runs without encountering an exception.
As per Matt B's suggestion, be sure to document what the test is actually testing to be clear and precise.
As #Kyle noted, your test case is valid. In fact the opposite would also be a valid: when you write a test case to confirm that a certain call with specific parameter(s) results in an exception.
Sure you can do that.
It is also perfectly fine to write a test without assertions where the expected outcome is an exeption. I know testng will let you specify an exception that should be thrown and the test will fail if the expected exception isn't thrown.
Testing is a really subjective discussion. Some people will say no, you should always have AAA syntax. Personally I've written tests that do things very similar to what your talking about so I'd say, sure go ahead - if it helps you build a more stable app then why not.
For example in NUnit i consider [ExpectedException typeof(XXXX)] to be logically equivalent to an Assert.
Also in some tests you might not assert anything but expect a particular order of execution via Mocks and Expects.
It is surely acceptable to write a unit test that doesn't have any assertions. You could do this for:
Testing a case that ends without an exception. In this case, if you can, it's nice to dress the test with the specific type of the exception, as in [ExpectedException(MyException)].
Testing a feature is there. Even there isn't a possibility that the test may generate an exception, you may want to make this test fail if someone decides to remove that feature. If the test uses a method and the method is removed, the test will simply fail to build.
the purpose of the test is to check if "X" has an "expected something", in order to check that "expected something" is correct to assert, to expect or to verify. This is why most frameworks implements those methods in a way or another
While poking around the questions, I recently discovered the assert keyword in Java. At first, I was excited. Something useful I didn't already know! A more efficient way for me to check the validity of input parameters! Yay learning!
But then I took a closer look, and my enthusiasm was not so much "tempered" as "snuffed-out completely" by one simple fact: you can turn assertions off.*
This sounds like a nightmare. If I'm asserting that I don't want the code to keep going if the input listOfStuff is null, why on earth would I want that assertion ignored? It sounds like if I'm debugging a piece of production code and suspect that listOfStuff may have been erroneously passed a null but don't see any logfile evidence of that assertion being triggered, I can't trust that listOfStuff actually got sent a valid value; I also have to account for the possibility that assertions may have been turned off entirely.
And this assumes that I'm the one debugging the code. Somebody unfamiliar with assertions might see that and assume (quite reasonably) that if the assertion message doesn't appear in the log, listOfStuff couldn't be the problem. If your first encounter with assert was in the wild, would it even occur to you that it could be turned-off entirely? It's not like there's a command-line option that lets you disable try/catch blocks, after all.
All of which brings me to my question (and this is a question, not an excuse for a rant! I promise!):
What am I missing?
Is there some nuance that renders Java's implementation of assert far more useful than I'm giving it credit for? Is the ability to enable/disable it from the command line actually incredibly valuable in some contexts? Am I misconceptualizing it somehow when I envision using it in production code in lieu of statements like if (listOfStuff == null) barf();?
I just feel like there's something important here that I'm not getting.
*Okay, technically speaking, they're actually off by default; you have to go out of your way to turn them on. But still, you can knock them out entirely.
Edit: Enlightenment requested, enlightenment received.
The notion that assert is first and foremost a debugging tool goes a long, long way towards making it make sense to me.
I still take issue with the notion that input checks for non-trivial private methods should be disabled in a production environment because the developer thinks the bad inputs are impossible. In my experience, mature production code is a mad, sprawling thing, developed over the course of years by people with varying degrees of skill targeted to rapidly changing requirements of varying degrees of sanity. And even if the bad input really is impossible, a piece of sloppy maintenance coding six months from now can change that. The link gustafc provided (thanks!) includes this as an example:
assert interval > 0 && interval <= 1000/MAX_REFRESH_RATE : interval;
Disabling such a simple check in production strikes me as foolishly optimistic. However, this is a difference in coding philosophy, not a broken feature.
In addition, I can definitely see the value of something like this:
assert reallyExpensiveSanityCheck(someObject) : someObject;
My thanks to everybody who took the time to help me understand this feature; it is very much appreciated.
assert is a useful piece of Design by Contract. In that context, assertions can be used in:
Precondition checks.
Postcondition checks.
Intermediate result checks.
Class invariant checks.
Assertions can be expensive to evaluate (take, for example, the class invariant, which must hold before and after calling any public method of your class). Assertions are typically wanted only in debug builds and for testing purposes; you assert things that can't happen - things which are synonymous of having a bug. Assertions verify your code against its own semantics.
Assertions are not an input validation mechanism. When input could really be correct or wrong in the production environment, i.e. for input-output layers, use other methods, such as exceptions or good old conditional checks.
Java's assertions aren't really made for argument validation - it's specifically stated that assertions are not to be used instead of dear old IllegalArgumentException (and neither is that how they are used in C-ish languages). They are more there for internal validation, to let you make an assumption about the code which isn't obvious from looking at it.
As for turning them off, you do that in C(++), too, just that if someone's got an assert-less build, they have no way to turn it on. In Java, you just restart the app with the appropriate VM parameters.
Every language I've ever seen with assertions comes with the capability of shutting them off. When you write an assertion you should be thinking "this is silly, there's no way in the universe this could ever be false" -- if you think it could be false, it should be an error check. The assertion is just to help you during development if something goes horribly wrong; when you build the code for production you disable them to save time and avoid (hopefully) superfluous checks
Assertions are meant to ensure things you are sure that your code fulfills really are fulfilled. It's an aid in debugging, in the development phase of the product, and is usually omitted when the code is released.
What am I missing?
You're not using assertions the way they were meant to be used. You said "check the validity of input parameters" - that's precisely the sort of things you do not want to verify with assertions.
The idea is that if an assertion fails, you 100% have a bug in your code. Assertions are often used for identifying the bug earlier than it would have surfaced otherwise.
I think its the way assert usage is interpreted and envisioned.
If you really want to add the check in your actual production code, why not use If directly or any other conditional statement?
Those being already present in language, the idea of assert was only to have developer's add assertions only if they don't really expect this condition to ever happen.
E.g checking an object to be null, let's say a developer wrote a private method and called it from two places (this is not ideal example but may works for private methods) in the class where he knows he passes a not null object, instead of adding unnecessary check of if since as of today you know there is no way object would be null
But if someone tomorrow calls this method with null argument, in developer's unit testing this can be caught due to presence of assertion and in final code you still don't need an if check.
Assertions are really a great and concise documentation tool for a code maintainer.
For example I can write:
foo should be non-null and greater
than 0
or put this into the body of the program:
assert foo != null;
assert foo.value > 0;
They are extremely valuable for documenting private/package private methods to express original programmer invariants.
For the added bonus, when the subsystem starts to behave flaky, you can turn asserts on and add extra validation instantly.
This sounds about right. Assertions are just a tool that is useful for debugging code - they should not be turned on all the time, especially in production code.
For example, in C or C++, assertions are disabled in release builds.
If asserts could not be turned off, then why should they even exist.
If you want to performa validity check on an input, you can easily write
if (foobar<=0) throw new BadFoobarException();
or pop up a message box or whatever is useful in context.
The whole point of asserts is that they are something that can be turned on for debugging and turned off for production.
Assertions aren't for the end user to see. They're for the programmer, so you can make sure the code is doing the right thing while it's being developed. Once the testing's done, assertions are usually turned off for performance reasons.
If you're anticipating that something bad is going to happen in production, like listOfStuff being null, then either your code isn't tested enough, or you're not sanitizing your input before you let your code have at it. Either way, an "if (bad stuff) { throw an exception }" would be better. Assertions are for test/development time, not for production.
Use an assert if you're willing to pay $1 to your end-user whenever the assertion fails.
An assertion failure should be an indication of a design error in the program.
An assertion states that I have engineered the program in such a way that I know and guarantee that the specified predicate always holds.
An assertion is useful to readers of my code, since they see that (1) I'm willing to set some money on that property; and (2) in previous executions and test cases the property did hold indeed.
My bet assumes that the client of my code sticks to the rules, and adheres to the contract he and I agreed upon. This contract can be tolerant (all input values allowed and checked for validity) or demanding (client and I agreed that he'll never supply certain input values [described as preconditions], and that he doesn't want me to check for these values over and over again).
If the client sticks to the rules, and my assertions nevertheless fail, the client is entitled to some compensation.
Assertions are to indicate a problem in the code that may be recoverable, or as an aid in debugging. You should use a more destructive mechanism for more serious errors, such as stopping the program.
They can also be used to catch an unrecoverable error before the application fails later in debugging and testing scenarios to help you narrow down a problem. Part of the reason for this is so that integrity checking does not reduce the performance of well-tested code in production.
Also, in certain cases, such as a resource leak, the situation may not be desirable, but the consequences of stopping the program are worse than the consequences of continuing on.
This doesn't directly answer your question about assert, but I'd recommend checking out the Preconditions class in guava/google-collections. It allows you to write nice stuff like this (using static imports):
// throw NPE if listOfStuff is null
this.listOfStuff = checkNotNull(listOfStuff);
// same, but the NPE will have "listOfStuff" as its message
this.listOfStuff = checkNotNull(listOfStuff, "listOfStuff");
It seems like something like this might be what you want (and it can't be turned off).