Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Although I have programmed with Java for roughly 3 years + now (not day-to-day but at least I understand the fundamentals), haven't really come into the field of Unit Testing...
My work is now more Testing / Problem Analysis oriented, so I reckon a good Java Unit Testing Framework will be quite helpful to this role.
Obviously there is no formal rule of which framework to go with in my Team. So just wondering, as a beginner, which framework is a good one to get started quickly?
Junit?
TestNG?
or something else?
Edit: it seems JUnit has more printed books as references compared to TestNG on Amazon. But I don't know their popularity / trend in Java Industry as for now.
I'm the creator of TestNG and author of the "Next Generation Testing" book, so here are a few (obviously biased) thoughts.
For functional (or end to end / integration / system, whatever term you prefer) testing, TestNG offers a few features that JUnit doesn't have and that users have found very useful:
Groups. Once you have compiled your tests, you can just ask TestNG to run all the "front-end" tests, or "fast", "slow", "database", etc...
Support for multithreaded testing. This has two different meanings:
1) You can tell TestNG to run all your tests concurrently in thread pools that you define (one line change in an XML file). This is very configurable so that tests that are not multithreaded can still be run in single threads.
2) You can tell TestNG to invoke your test methods n times from p threads. This gives you a good idea of how thread safe the code you are testing is.
Test dependencies and deterministic ordering. This is very useful for integration testing where it's very common to see test methods that need prior test methods to succeed before they can even run. TestNG is very popular in the Selenium community for this specific reason ("don't bother testing this page of my web site if the login test failed").
Another immediate advantage of test dependencies is that TestNG can tell you "1 method failed, 99 methods skipped", which is a much more accurate result than "100 methods failed".
TestNG has many, many more features and I'm happy to answer questions here or on our mailing-list (http://groups.google.com/group/testng-users ).
I would learn TestNG with Unitils. Covered all my needs all the time. Add XMLUnit and DBUnit and you should be settled for a long time.
To be perfectly honest I JUnit is way more popular than TestNG, at least here where I work and live. I know this might be a strange argument, but if I were you I'd scan through some job adds and see which framework does the industry in your area/country favour (for example I'm going for JUnit because of that, I mean sure I could learn TestNG, it might turn out to be better, but what good would it do to me if all the employers require JUnit knowledge?).
Although it may seem to be a petty criterion, all things being equals, it looks like job trends heavily favour JUnit.
http://www.indeed.com/jobtrends?q=TestNG%2C+JUnit&l=
Both do their jobs equally well, so there is no right or wrong. You can use both in combination with numerous frameworks to help support your development.
If you need to work for a client, you can learn the one that you are expected to use. If not, the best idea is to try them both out and find out what works best for you.
I've used both and found that for unit testing only jUnit works very well, but in terms of System testing I would recommend TestNG. TestNG offers a few extra features (like parametric testing) and is highly flexible. For example, in jUnit one test failing in a suite usually means you have to re-run your entire suite, but in TestNG you can simply rerun the test that failed. Heres a really good article outlining both (its a bit biased towards TestNG though)
http://www.ibm.com/developerworks/java/library/j-cq08296/index.html
JUnit is very well supported in many IDE's including Eclipse, and JUnit 4 with annotations is quite nice to work with.
Unless you have specific needs then go for tool support, i.e. jUnit.
TestNG with Unitils is really great. It covers almost all the requirements. Unitils support lot of assertion utils, Spring, Hibernate, mocking frameworks, dbutils etc.
To me, it is just a matter of taste. I've personnaly used jUnit daily for more than 5 years now and I'm very happy with it, especially with the last version and its #Rule feature that prevents my team from writting the same code over and over.
It integrates well with all the tools I use daily : IDE, build, continuous intégration...
Junit is famous and best unit test tools, I think you and me both know Junit is so important so that Java developer changes their coding pattern as junit contribution, You can learn some Junit as basic unit test technology as well as learn testng as second unit test technology.
The following link maybe be useful for you!
http://www.javarmi.com/2010/08/junit-4-tutorial-and-example-get-starting/
Junit is the best place to start. Junit 4.x has many rich features like annotations etc as well its get good support from popular frameworks like Spring.
I would go for JUnit.
Related
I have to write a java program, and prove that i used TDD while writing the program.
I need to find some method to prove the use of Test Driven Development, meaning to somehow log the failures of some tests, then log writing of some code, then log the tests passing, and so on.
Any help is welcome.
Also any java related TDD documentation sources would help!
Thank you!
Personally, I disagree that you should judge developers based on whether they used TDD or not. It’s like judging a Striker by how skilful they are in dribbling. The only thing that matters is whether they helped their team to win or scored enough goals. As with dribbling, TDD is a mean, not a destination. As it is not a goal on its own, there is no need to test whether you have used TDD while developing you app or not. You decide to use TDD if you believe that this helps you with creating a maintainable, tested and future-proof solution and this should be your goal. If you can achieve the same results using some other technique, the results should speak for themselves. Having said that, TDD is by far the most popular and proven technique to achieve above mentioned qualities. Therefore, you can test if someone skilfully used TDD (or equally efficient technique) by measuring the quality of the result. TDD is just a tool and as with any other tool, you need practice to use it properly. Poor TDD can lead to a poor design with a bunch of useless tests.
To test if someone used TDD or equally efficient process:
Run static analysis (such as sonar, qulice etc.) and compare the results with some reference projects (with good and bad design, as you need some benchmark). TDD provides a feedback loop that helps you deliver a well-designed software. Poor design is a clear indicator that TDD hasn't been used or the person didn't know how to use the feedback that TDD provided.
Run mutation tests (such as pitest) and, as above, compare the results with some reference project that is well tested and poorly tested. 100% coverage is not a TDD goal. However, one of its goals is to give you confidence to change and improve your code by providing a valuable test suite and self-documented code. The side effect of applying Three Laws of TDD is that you have very high code coverage. I encourage you to use a mutation framework, not merely a coverage analysis. Mutation tests check if the test suite actually tests the behaviour (and not merely executes lines of code to achieve high coverage).
If, according to the check above, the project has good design and is well tested, you can ask the author whether they used TDD out of curiosity, but this should not affect your judgement. On the contrary – if the design is good and the project is well tested despite not using TDD, it is an opportunity for you to discover and add new tools to your toolbox that can help you to achieve your goals as a developer.
Test Driven Development is more a kind of development methodology.
The reference book, by Kent Beck, defines TDD by example. Literally speaking, you can't prove the use of TDD. To cite the author, TDD is a sets of rules that imply "Red/green/refactor-the TDD mantra". That is to say:
Red-Write a little test that doesn't work, and perhaps doesn't compile at first
Green-Make the test work quickly, committing whatever sins necessary in the process
Refactor-Eliminate all of the duplication created in merely getting the test to work
This would not be a prove bot only a track of your work: for the points 1 and 2 you could log the result of JUnit tests to show the sequence between red and green (assuming, on point 1, that your tests compile).
For the third point, this is about the measure of refactoring. A tool such as Sonar could give you measure of duplication. For that, the simple way is a project managed with Maven, what should help writing the JUnit tests. Usually, the project is built with a continuous integration tool such as Jenkins or Hudson.
I can suggest that you should read Test-Driven Java Development - Second Edition
,written by Viktor Farcic, Alex Garcia, . It's amazing book.
Explore the tools and frameworks required for effective TDD development
Perform the Red-Green-Refactor process efficiently, the pillar around which all other
TDD procedures are based
Master effective unit testing in isolation from the rest of your code
Design simple and easily maintainable code by implementing different techniques
Use mocking frameworks and techniques to easily write and quickly execute tests
Develop an application to implement behavior-driven development in conjunction with
unit testing
Enable and disable features using feature toggles
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
Currently we are in the process of finalizing which language out of Java, Python should be used for Automation using selenium webdriver and a suitable unit testing frameworks. I have made use of Junit, TestNG and webdriver while using with Java and have designed frameworks without much fuss before. I am new to python though I came across pyhton's unit testing frameworks like unittest, pyunit, nose e.t.c but I have doubts if they would be as successful as testNG or Java. I would like to analyze point by point when used with selenium webdriver as below:
1)I have read that as Python is an interpreted language hence it's execution is slower, so say if I have to run 1000 test cases which take about 6 hours to run in Java, would python take considerably longer time for the same test cases like 8 hours?
2)Can the Python unit testing framework be as flexible as a Java unit testing framework like testNG in terms or Grouping the tests, parallel execution, skipping test. e.t.c
3)Also one point that I think of is that Python with selenium webdriver doeasn't have as big or learned community as we have for Java with webdriver, say if I run into trouble with something I am more likely to find an answer for Java as compared to python?
4)Somewhat related to point 3, is it safe to rely on tools, plugins or even webderiver's python's binding as a continuously well maintained?
5)One major drawback as I see while using python's unit testing framework is lack of boilerplate code or libraries for nicely illustrative HTML reports preferably historical reports with Pie charts, bar graphs and timelines as we have in case of Java like Allure, TestNG's default reports, reportNG or Junit reports with the help of ANT as shown below
Allure reports:
Historical Junit Reports:
Also I would like to emphasize on the fact if there is a way for one to write the framework in java nd make libraries or utilities according to out application in webdriver which can easily be called or integrated in with python code or modules? That would actually solve the problem for us as the client would be able to use the code we write in Java and make use of the same or call it from their python modules?
I don't think that decision is that important you think it is. You can go either way, what you feel more comfortable with. Here's why in my opinion:
1) If you're tests are running 6 or 8 hours your tests are slow. It's not the language that makes them slow but what they do. Neither java nor python will magically make the tests run faster or slower (on this scale).
If there is a performance impact from the choice of your languages, I would suspect it to be like +- 1 minute...
2) Sorry, I can't answer. I don't know pythons test frameworks.
(But the whole rest of the answer doesn't rely on that)
3) Regarding WebDriver, do you know how it works? It's an extension in the webbrowser. You talk to it via a JSON protocol in your code. Your test code is just a client, generating that JSON commands and sending it to the browser.
The webdriver core doesn't care if the JSON is generated by python or java.
The sole thing that can go wrong is the client code and I would suspect both to be stable and well used.
4) See answer 3. It is only the client part you switch here, but yes, it is well maintained in my opinion.
5) Isn't that the job of the container (e.g. Jenkins / Hudson). All the test frameworks do is normally generate a report of the failed / passed tests. Most likely in ANT / JUnit XML Format. For details see this question. That may depend on the plugin you're using to generate the diagrams though. But not on the test framework generating the resulting report.
If you feel more comfortable with Java just use it. In favor of python is the shorter syntax. You can write the tests a bit shorter with less boilerplate code. But that wouldn't let me learn python just for the sake of shorter (in the sense of source code) tests.
Dont trust random people on the internet to make critical business decisions!
As to 2) nose has nice test grouping and parralel execution.
The basic question is "How should one start with writing unit and integration testing for a untested project? Especially considering the fact that the person is not familiar with the code and has not done integration testing before."
Consider the scenario where unit tests and integration tests have to be written for a project. The project uses Java/J2EE technology does not have any tests written at all.
The dilemma that I face is since I have not written the code, I don't want to refactor the code immediately to write tests. I also have to select a testing framework. I am thinking of using Mockito and Powermock.
I also have to estimate code coverage for the tests. And then perform integration testing. I will have to research on integration testing tools and select one. I have not done any integration testing or estimated acceptable level of code coverage for a project before.
Since I am working independently, if there are some strategies, tips, suggestions on what should I start with and tools that one can recommend, I will appreciate it.
First comes first:
Understand the architecture, what are the main components?
If you have no good overview of the features and functions the program offers, make a list of them and create a hierarchy of them
Get familiar with the code, I recommend the following approach:
after you understand where the code of its different components are started, try to figure out the method invocation hierarchy (in Eclipse you can easily jump the source code definitions by pressing F3)
later you can do the same, while debugging the code, this way it will jump automatically to the definitions, plus you can observe how the state of the program changes
For Unit Testing itself, I can recommend Clean Code Chapter 9 (circa 12 pages) for starters. It uses JUnit for the example and gives a very good introduction how good testing is done.
There you will learn things like the F.I.R.S.T. principle, that Unit Tests should be:
Fast, Independent, Repeatable, Self-Validating and Timly
Some clarifications, JUnit is the most used and accepted test framework itself. Mockito and Powermock are mocking frameworks, they are used together with JUnit when you want to do integration tests.
For code coverage I can only recommend Cobertura, but there are many more.
Start with unit tests before you dive into integration tests (bottom-up), you can also do it the other way around (top-bottom), but since you say you are not so much experienced I would stay stick to the first.
Finally, just go for it and get started. You will learn the most and fastest while actually writing the test code.
Stop. "..not familiar with the code..". First get familiar with the code and most importantly its expected functionality. You can't refactor or unit test a code that you are not comfortable with.
Since you have not done unit-tests before, I would suggest learning and getting convenient with unit-tests.
Important: Bad/Wrong unit-tests are worse than no unit-tests. This is because the next guy who will maintain your code will misinterpret
the functionality.
There are bunch of Code Coverage tools out there. You can use which ever seduces you better.
Adding tests to legacy code that has no tests is a difficult task. As #Suraj has mentioned, get familiar with the code base and the expected functionality. You can't test it if you don't know what it is supposed to do.
In terms of choosing which areas of the code to test. Start with the high business value areas. Which functionality is most important? You want to make sure you have a strong test set for that code.
Since you don't have any unit/integration tests, I would start with some high level end to end tests that at least ensure that given some inputs to the system you get some expected outputs. This doesn't ensure correctness but at least ensures consistency.
Then as you develop a test suite you can be confident that the refactorings you are doing are not changing the behavior of the code (unless you find bugs of course that are being fixed).
For testing frameworks, JUnit is the standard unit testing framework. Note that the frameworks Mockito and Powermock are not testing frameworks themselves, but they can be used within JUnit.
For acceptance tests, there are also a variety of frameworks to help. For web UI testing, Selenium is pretty standard. There are also tools like Fitnesse for more table driven testing.
There are also some common frameworks to help with code coverage - Cobertura, Emma, Clover come to mind.
I would also set up an automated build (Jenkins build server is pretty simple to set up). This will allow you to run your tests on every checkin. Even though your code coverage is going to be low to start, getting in this habit is a good one.
I have googled a lot and there are several options,
Arquillian
JUnitEE
Cactus
Does anyone have experience with one of these (or other frameworks)? Do they depend on JUnit only or does TestNG work as well? Please give me your suggestions. Thanks.
I haven't had direct experience with any of them, but after some reading, it seems that Arquillian is the newest one, it has an experienced company backing it, and it has a lot of hype surrounding it. Criteria that I have enumerated don't necessarily say Arquillian is quality wise the best out there, since that would be a pretty subjective argument. I'd say it's worth checking out.
JUnitEE's last release was about 6 years ago, and from what I've read, Cactus is a bit complicated and "heavyweight". If I were to use one of Java EE unit testing frameworks you've listed, I'd try Arquillian first.
I'm working on an existing Java EE project with various maven modules that are developed in Eclipse, bundled together and deployed on JBoss using Java 1.6. I have the opportunity to prepare any framework and document how unit testing should be brought to the project.
Can you offer any advice on...
JUnit is where I expect to start, is this still the defacto choice for the Java dev?
Any mocking frameworks worth setting as standard? JMock?
Any rules that should be set - code coverage, or making sure it's unit rather than integration tests.
Any tools to generate fancy looking outputs for Project Managers to fawn over?
Anything else? Thanks in advance.
Any tools to generate fancy looking outputs for Project Managers to fawn over?
Be careful. A fancy tool for displaying metrics on unit test counts, coverage, code quality metrics, line counts, check-in counts and so on can be dangerous in the hands of some project managers. A project manager (who is not in touch with the realities of software development) can get obsessed with the metrics, and fail to realize that:
they don't give the real picture of the project's health and progress, and
they can give a completely false picture of the performance of individual team members.
You can get silly situations where a manager gives the developers the message that they should (for example) try to achieve maximal unit test coverage for code where this is simply not warranted. Time is spent on pointless work, the important work doesn't get done, and deadlines are missed.
Any rules that should be set - code coverage, or making sure it's unit rather than integration tests.
Code coverage is more important for parts of the code that are likely to be fragile / buggy. Acceptable coverage levels should reflect this.
Unit tests versus integration tests depends on the nature and complexity of the system you are building.
Adding lots of unit level tests after the fact is probably a waste of time. It should only be done for class identified as being problematic / needing maintenance work.
Adding integration level tests after the fact is useful, especially if the projects original developers are no longer around. A decent integration test suite helps to increase your confidence that some change does not break important system functionality. But this needs to be done judiciously. A test suite that tests the N-th degree of a website's look and feel can be a nightmare to maintain ... and impediment to progress.
Concerning the unit testing framework, there are mainly two of them : jUnit and TestNG. Both have theuir advantages, and both are equally performant. The main dvantage of jUnit is (to my mind) its default incoproration of an Eclipse plugin allowing easy tests calling.
Concerning the mocking framework, I don't find them to be a required part of your testing approach. Of course they're useful, but they solve a specific purpose : testing a behaviour (as opposite to testing an interface - what jUnit allows. With mocking frameworks, you're able to test how a specific class implements a specific interface. Will you need it ? Obviously. Will you need it first ? I don't know.
Concerning the rules, the only one I've found to be useful is simple (as always) : "always test code that broke at least once.". Consider your bug tracker. Each time a bug is encountered, there must be a unit test ensuring there is no regression. It's, to my mind, the faster way to have quality code.
Concerning the fancy- and efficient - output, I can recommend you enough to install a continous integration server (Hudson, obviously). It will run all your test suite each time code is commited, to ensure there are no side effects. it will generate graphs shoiwing the number of test run, and so on. it also can integrate code coverage tools and graphs. This continuous integration server will really become fast your testing buddy.
This is a complex question, so just a few notes about our practice at $work:
JUnit is indeed still the standard. Most documentation and literature treats JUnit.
Mockito seems to be the new star in Java mocking, although we still use JMock and think it's fine for our needs.
We use the EclEmma Eclipse plugin for checking our test coverage, and like it.
If you haven't done so already, read Working Effectively with Legacy Code by Michael Feathers.
I've been retrofitting unit tests to a C++ project and it is not pleasant.
First thing I did was to identify where most of the 'action' occurs. Then use that to start putting unit tests on the functions that can be test easily.
Then once you have the easier ones you can start looking at expanding the coverage virally - attack the functions that have fewer dependancies, run through them a few times in a debugger seeing what values are passed in and then write unit tests with those values to make sure you don't break anything.
Don't expect a quick fix - it's taken 3 weeks (6hr days, 5 days a week) to get 20% coverage but the code spends 80% of the time in that code so I think it has been time well spent and has uncovered quite a few bugs.
Regarding test coverage, I think that when you're bringing in unit testing to an existing project it's too early to start setting coverage expectations. You should start by ensuring that you actually can integrate the test framework and get reports from the coverage tools. Once you've done that you can start monitoring coverage, and then you can consider targets.