I've been asked a question by one of our developers here. They are asking whether it is possible for a selenium test suite to be tied in with a TFS build in visual studio to the test server? So that way, as soon as a build is done it can kick off a run of the regression test suite, directly after that.
Bear in mind that my scripts have been written in Eclipse, NOT Visual Studio, so I'm not sure if this will cause restrictions.
You can call UI scripts as you describe and I would additionaly recommend that you use Release Management to do this rather than build. It makes much more sense to use a deployment engine rather than a compilation engine to maintain this. One does not usually have an instance of ones application running on the build server.
http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
You need to get a few things lined up, but it worked pretty good..
Related
For past 4 years, I have been programming with Eclipse (for Java), and Visual Studio Express (for C#). The IDEs mentioned always seemed to provide every facility a programmer might ask for (related to programming, of course).
Lately I have been hearing about something called "build tools". I heard they're used almost in all kind of real world development. What are they exactly? What problems are they designed to solve? How come I never needed them in past four years? Are they kind of command-line stripped down IDEs?
What are build tools?
Build tools are programs that automate the creation of executable
applications from source code (e.g., .apk for an Android app). Building
incorporates compiling,linking and packaging the code into a usable or
executable form.
Basically build automation is the act of scripting or automating a
wide variety of tasks that software developers do in their day-to-day
activities like:
Downloading dependencies.
Compiling source code into binary code.
Packaging that binary code.
Running tests.
Deployment to production systems.
Why do we use build tools or build automation?
In small projects, developers will often manually invoke the build
process. This is not practical for larger projects, where it is very
hard to keep track of what needs to be built, in what sequence and
what dependencies there are in the building process. Using an
automation tool allows the build process to be more consistent.
Various build tools available(Naming only few):
For java - Ant,Maven,Gradle.
For .NET framework - NAnt
c# - MsBuild.
For further reading you can refer following links:
1.Build automation
2.List of build automation software
Thanks.
Build tools are tools to manage and organize your builds, and are very important in environments where there are many projects, especially if they are inter-connected. They serve to make sure that where various people are working on various projects, they don't break anything. And to make sure that when you make your changes, they don't break anything either.
The reason you have not heard of them before is that you have not been working in a commercial environment before. There is a whole lot of stuff that you have probably not encountered that you will within a commercial environments, especially if you work in software houses.
As others have said, you have been using them, however, you have not had to consider them, because you have probably been working in a different way to the usual commercial way of working.
Build tools are usually run on the command line, either inside an IDE or completely separate from it.
The idea is to separate the work of compiling and packaging your code from creation, debugging, etc.
A build tool can be run on the command or inside an IDE, both triggered by you. They can also be used by continuous integration tools after checking your code out of a repository and onto a clean build machine.
make was an early command tool used in *nix environments for building C/C++.
As a Java developer, the most popular build tools are Ant and Maven. Both can be run in IDEs like IntelliJ or Eclipse or NetBeans. They can also be used by continuous integration tools like Cruise Control or Hudson.
Build tools are generally to transform source code into binaries - it organize source code, set compile flags, manage dependencies... some of them also integrate with running unit test, doing static analysis, a generating documentation.
Eclipse or Visual Studio are also build systems (but more of an IDE), and for visual studio it is the underlying msbuild to parse visual studio project files under the hood.
The origin of all build systems seems like the famous 'make'.
There are build systems for different languages:
C++: make, cmake, premake
Java: ant+ivy, maven, gradle
C#: msbuild
Usually, build systems either using a propriety domain specific language (make, cmake), or xml (ant, maven, msbuild) to specify a build. The current trend is using a real scripting language to write build script, like lua for premake, and groovy for gradle, the advantage of using a scripting is it is much more flexible, and also allows you the to come up with a set of standard APIs(as build DSL).
These are different types of processes by which you can get your builds done.
1. Continuous Integration build: In this mainly developers check-in their code and right after their check-in a build initiates for building of the recent changes so we should know whether the changes done by the developer has worked or not right after the check-in is done. This is preferred for smaller projects or components of the projects. In case where multiple teams are associated with the project or there are a large no. of developers working on the same project this scenario becomes difficult to handle as if there are 'n' no. of check-in’s and the build fails at certain points it becomes highly difficult to trace whether all the breakage has occurred because of one issue or with multiple issues so if the older issues are not addressed properly than it becomes very difficult to trace down the later defects that occurred after that change. The main benefit of these builds is that we get to know whether a particular check-in is successful or not.
2. Gated check-in builds: In this type of check in a build is initiated right after the check in is done keeping the changes in a shelve sets. In this case if the build succeeds than the shelve-set check-in gets committed otherwise it will not be committed to the Team Foundation Server. This gives a slightly better picture from the continuous integration build as only the successful check-in's are allowed to get committed.
3. Nightly builds: This is also referred as Scheduled builds. In this case we schedule the builds to run for a specific time in order to build the changes. All the previous uncommitted changes from the last build are built during this build process. This is practiced when we want to check in multiple times but do not want a build every time we check in our code so we can have a fixed time or period in which we can initiate the build for building of the checked-in code.
The more details about these builds can be found at the below location.
Gated-check in Builds
Continuous Integration Builds
Nightly Builds
Build Process is a Process of compiling your source code for any errors using some build tools and creating builds(which are executable versions of the project). We(mainly developers) do some modifications in the source code and check-in that code for the build process to happen. After the build process it gives two results :
1. Either build PASSES and you get an executable version of your project(Build is ready).
2. It fails and you get certain errors and build is not created.
There are different types of build process like :
1. Nightly Build
2. gated Build
3. Continuous integration build etc.
Build tools help and automates the process of creating builds.
*So in Short Build is a Version of Software in pre-release format used by the Developer or Development team to gain confidence for the final result of their Product by continuously monitoring their Product and solving any issues early during the development process.*
You have been using them - IDE is a build tool. For the command line you can use things like make.
People use command line tools for things like a nightly build - so in the morning with a hangover the programmer has realised that the code that he has been fiddling with with the latest builds of the libraries does not work!
"...it is very hard to keep track of what needs to be built" - Build tools does not help with that all. You need to know what you want to build. (Quoted from Ritesh Gun's answer)
"I heard they're used almost in all kind of real-world development" - For some reason, software developers like to work in large companies. They seem to have more unclear work directives for every individual working there.
"How come I never needed them in past four years". Probably because you are a skilled programmer.
Pseudo, meta. I think build tools do not provide any really real benefit at all. It is just there to add a sense of security arising from bad company practices, lack of direction - bad software architectural leadership leading to bad actual knowledge of the project. You should never have to use build tools(for testing) in your project. To do random testing with a lack of knowledge of the software project does not give any sort of help at all.
You should never ever add something to a project without knowing it's purpose, and how it will work with the other components. Components can be functional separate, but not work together. (This is the responsibility of the software architect I assume).
What if 4-5 components are added into the project. You add a 6th component. Together with the first added component, it might screw up everything. No automatic would help to detect that.
There is no shortcut other than to think think think.
Then there is the auto download from repositories. Why would you ever want to do that? You need to know what you download, what you add to the project. How do you detect changes in versions of the repositories? You need to know. You can't "auto" anything.
What if we were to test bicycles and baby transports blindfolded with a stick and just randomly hit around with it. That seems to be the idea of build tool testing.
I'm sorry there are no shortcut
https://en.wikipedia.org/wiki/Scientific_method
and
https://en.wikipedia.org/wiki/Analysis
I am a beginner with Java and Eclipse. I am trying to create an automated framework with both Selenium (Web application) and Robotium (Android Application). I have both project completed, one is a Java Project (Selenium) that uses the JDK compiler to run Junit test cases, and the other one is an Android Test Project that uses the Android Compiler to run android test cases.
My questions is: is there a way to combine an Android project onto a Java project?
I have tried adding the essential files/changes of an android project onto the java project but when I run each test it fails because it uses different compilers.
Is there a way around this? I am open to any suggestions.
Thank you in advanced!
After reading your comment I believe i can say that you should keep them separate. If you want to share some functionality between them all (I cannot imagine you could share all that much between all 3, between android and selenium maybe as they are at least in the same language) then you can create another project and use this as a shared source folder between the two. You will find having them separate makes a lot more sense. The most glaring example is with version controlling your tests, you will find you will want to make tags when you release the actual application, with a single project you will be tagging your individual project for any release of any of the three applications when they will not reflect the actual state of release. That hopefully makes sense. I am pretty tired.
I'm interested in developing android applications. I've a background of Java/Swing/C++/ajax developer so I think I may find myself at home. As I installed the SDK I noticed that I can't follow my usual java development scheme: building base libraries, then the final app. In fact I'd like to develop libs and test them in a more convenient environment like, Swing. There is no way I can use android libs in pure swing apps...
Do you have any suggestion about these topics and what's your development process?
If you have written a lib that does call functions from the android jar, you can write JUnit 4 tests in a separate Project and execute them to test your lib functions. This forces you to keep some of your functionality separated from the android specific things and you can then be sure that all the errors you tested for are gone before you introduce the problems that can arise if you are running on the device. You can do JUnit Tests on the emulator too but that is a little bit more complex.
I'm trying to find a practical unit testing framework for JSF.
I know about JSFUnit, but this is very impractical to me. I need to include about 10 JARs to my project, and jump through many other hoops just to get it running.
I realize that -- due to the need to simulate a platform and a client -- unit testing web applications is difficult. But is there a better way?
Have you thought about doing integration testing with Selenium or another tool? Selenium allows you to record and run tests directly in the browser. You can also run tests in multiple browsers and on multiple platforms with Selenium Remote Control.
Writing unit tests is good, but it might provide more to create some functional integration tests rather than unit-testing the presentation layer code.
On the project I'm working on at the moment we dabbled with using selenium. We actually spent a lot of time writing these selenium tests, but found that they added little value because the UI changes so much and you just end up doubling your effort for very little return on investment.
Another problem with selenium is that it requires your code to be deployed, which means it doesn't play well with unit test frameworks eg maven.
What I would say is that writing really good unit tests for your managed beans is invaluable.
Have you taken a look at the jsfunitwar Ant task or alternatively the Maven plugin provided by JSFUnit? Both greatly reduce the complexity of generating the .war file to be tested. I'm using JSFUnit on my current project and find the combination of white box and black box testing capabilities to be very powerful. Because JSFUnit uses HtmlUnit under the covers, you can very easily and effectively examine the generated HTML, or conversely, verify the state of your internal JSF backing beans. I was able to incorporate the JSFUnit tests into my Continuous Integration process and have been quite pleased with the outcome.
HttpUnit can also be an alternative. It provides apis so you have a choice to automate the tests.
http://httpunit.sourceforge.net/index.html
Selenium is superficial, jsfunit is inward. I recommend that use jsfunit if project is not simple. Because team member can change jsf managedbean names or etc, you can catch that with jsfunit.
I'm with Paul on Selenium being very easy to setup and start working with. I use Selenium IDE in Firefox with some customization at that level, then you can export these to other platforms such as Java JUnit tests. It was quite easy to download and launch the selenium-server.jar, add the selenium-java-client-driver to my existing Eclipse Maven POM driver project; then launch the same exported JUnit test in Eclipse. I mainly wanted to use Java just for looping which the basic Selenium IDE didn't support.
I have configured JSF Unit for my project too which does require more time to configure... more importantly though with in-container tests like JSFUnit changes to the test require rebuilding the WAR, redeploying in the container and then executing from Eclipse or via a browser. So for quickly trying a small change this is time consuming. Of course with JSFUnit you have access to all the internals of the JSFSession etc so it depends what granularity of testing you need I guess.
I'd be interested if anybody knows a faster way to turnaround changes to a JSFUnit test and execute it. Definitely Selenium tests feel more like JUnit tests in that regard.
I've been looking for a code coverage viewer aimed at inspecting live Java applications, mostly webapps running inside an application container like Tomcat. Sure, there are a number of decent tools for getting automatic reports of unit test coverage, but my aim is more like learning in real time what an unfamiliar Java app does e.g. on a specific user interaction.
The Eclipse Java debugger (with JPDA for remote debugging) is really useful, but only if you are already familiar with the application's architecture. And in theory, I could take some coverage tool and set it up to auto-refresh static HTML coverage reports every two seconds, but this is far from optimal.
For Adobe Flex, FlexCover does just what I want by providing a coverage viewer tool that visualizes the coverage in nearly real time, and it's relatively simple to set up at least for somebody who knows the stuff. So is there a similar easy-to-set-up GUI tool, free or non-free, for Java?
You can view Clover's coverage data generated by a web application, in Eclipse, without the need to start the web server from Eclipse.
The trick is to configure the initString in the Eclipse Clover Config screen to point to the same clover.db that your webapp is using:
And - you need to ensure you are using a threaded flushpolicy. The clover-maven2-plugin uses one by default. If you are using Ant, you will need to set it explicitly on <clover-setup/>.
You must also ensure you are using the same version of Clover in both Eclipse and your build tool.
I have tested this locally - and it works quite well!
Please let me know how you go.
Have a look at clover. It may be what you are looking for. Not free, but nice.