First time poster, long time lurker. I've gotten a lot of great advice to problems from this site, but I haven't found anything here for the topic of this question. Normally I would bug our SME at the office but he's indisposed.
So, we use Selenium Web Driver to do automated tests. I'm working on an application with some mapping and demographics features, so my tests are very function vs. form oriented.
My tests are written such that I have classes/methods that are a part of the puzzle (the site is essentially one workflow where you go from page 1 to page 5 and the same actions need to be performed in steps 2-3, for example, but test A might do something different on page 4 to see the result in page 5. Clear as mud?
Anyways, during manual tests, I can sometimes see an error message pop up on the site (a hidden div that will become visible if it detects an error, but it's usually a very generic/vague error). This error sometimes pops up even if you're able to go through the flow with no other ill-effects. However, I want to capture when these errors happen so I can look for patterns - if this means just logging it to console or failing the test...I can figure that out later.
The immediate problem is having a persistent check in place that will always look for this error during every test. I could create a method and call it in my "action" methods, though this would leave gaps and slow the tests down. Is there any clever way of implementing something like this without slowing the tests down or calling this check every time I do a step in the process? Also, forgive me, I'm still learning Java and the selenium web driver, so if I've said anything stupid, that's why.
Since this message is persistent if it is there, you might try adding a check for it in your test case teardown method. (I would recommend that you reduce the implicit wait time before you do that check, though, otherwise each test will take an extra amount of time waiting for an error message that isn't there.)
Another possible option is to define your own listener on your own test runner and update the testFinished() method to go check for your error message. See this for some ideas.
Since it sounds like the error messages are always in known locations on each page, I would create a method (or methods, depending on how many error message locations there are on a given page) that looks to see if an error exists and then log it before leaving the page. It sounds like you might be using the page object model. If so, you can add these methods to the each relevant page object for easy access.
NOTE: Checking for errors once before you leave the page may not be enough. You may need to check each time you do some action that might cause an error. This is probably not a bad practice anyway because it will help in debugging errors because you will notice an error closer to the time it was triggered, thus narrowing down what caused the error.
If you have the ability, do something like log it as a warning so that it doesn't fail your test but stands out (and is searchable) in your logs.
You seem concerned that checking for all these errors will significantly slow your script. If properly written, it shouldn't add a significant delay. One significant delay you might run into is if you have implicit waits turned on and are checking for elements that don't exist (e.g., unless there's an error). This will cause the implicit wait to be applied each time you search for the missing element and will likely add significant time to the run time. My suggestion is to turn off implicit waits and add explicit waits only where needed. Searching for any element will add some time but 25ms here and there should be negligible in an overall script run.
Have you tried using EventFiringWebDriver?
There is an answer here on what it does:
What is EventFiringWebDriver in selenium?
Newer selenium versions have more types of events present in the interface, which can broaden its use on these types of tests.
Related
while running the script, when the code finds the xpath is changed.
Is there any way to update the xpath automatically during run time and proceed with the next steps without fail.
John, I think what you're asking is "how do I implement a crystal ball that foresees changes in HTML and auto-adapts to them".
The answer is - you don't, the best realistic way is to talk to developers to implement sensible IDs and classes on every important element and ask them to communicate changes before you run your tests, see them break and wonder what happened.
If your a dev yourself and you try to practice TDD, then:
1) implement a failing test with a dummy Xpath, see it fail
2) implement the UI
3) Update the Xpath in your test, run, see the test pass
I am writing a test suite using WebDriver in Java. Importantly, the tests are functional tests, not unit tests. Often, the same test case will be run a few times in sequence with different data - for example, "create application" with different names and components for every application.
The test case executon path includes a few dialogs. At each dialog, an error can happen (for example, "component not found"). Currently, my code handles the errors right next to where they happen - for example: (this is a simplified example not a piece of production code, it was not tested, so please pardon trivial mistakes)
WebElement component;
try {
component = componentsDialog.findElement(By.xpath("#class='component' and #componentId = '" + componentId + "']"));
} catch ( NoSuchElementException nse ) {
log.error("Component not found");
driver.switchTo().activeElement().sendKeys(Keys.ESCAPE);
stoppedOnError = true;
return;
}
component.click();
WebElement buttonAdd =
componentsDialog.findElement(By.className("addbutton"));
buttonAdd.click();
This is not very Java-like for error handling. It might be hard to integrate into TestNG if I choose to use that in the future.
But I can't just leave this to the general NoSuchElementException handler for the test case. Most of the time, NoSuchElementException means that the UI has changed (or I made a mistake in the test code). In this case, it means the particular configuration for this instance of the test case is wrong. And that configuration is set by the user. It is a different error and should be reported differently.
So I could just catch the exception and raise another, with the right message... But note the part where the Escape key is pressed. I close the component selection dialog, so that the state of the UI is he same as if the component selection was successful. In the rest of this test case, this particular dialog is not open. So how would the exception handler (at the end of the test case method or in the caller) know what state the UI is currently in and what it needs to do to recover?
(Autodetection is possible but flaky, as it would rely on detecting the presence of some element unique for every possible dialog).
So what do I do here, in order to enable error handling outside of the immediate execution stream? Keep some state tracking variable somewhere? This seems awfully error prone.
I could of course try to switch to the Page Object Model. The model strikes me as very heavyweight, requiring an increase of lines of code by up to an order of magnitude, and only paying off if many diverse test cases use the very same controls. (Typically, in my cases so far different use cases use different UI elements, so I don't understand how the model would pay off).
Perhaps this impression is mistaken. But even if I do use the model, every page is an unrelated object - how do I know which page is actually active at the time? Calling methods of a page when another page is active will only lead to meaningless exceptions (in the absence of complicated detection logic).
If I were to write this code, I would write it like this.
List<WebElement> component = componentsDialog.findElements(By.xpath("#class='component' and #componentId = '" + componentId + "']"));
if (component.isEmpty())
{
log.error("Component not found");
driver.switchTo().activeElement().sendKeys(Keys.ESCAPE);
componentsDialog.findElement(By.className("addbutton")).click();
}
else
{
component.get(0).click();
}
You don't need to (and IMO shouldn't) throw any exceptions here. If you look at the docs it states
findElement should not be used to look for non-present elements, use findElements(By) and assert zero length response instead.
Oh... and to your comment about page objects. It shouldn't require much extra code... definitely not orders of magnitude more, if done right. What it does get you is better organization, better reuse of code, lessens the maintenance burden, and so on. Putting all your code for a page or dialog into a single class makes it so much easier to update scripts when things change or when a bug needs to be fixed. It will drastically reduce your maintenance effort.
This is one of the questions that involves crossing what I call the "Hello World Gulf" I'm on the "Hello world" I can use SQLite and Content Providers (and resolvers) but I now need to cross to the other side, I cannot make the assumption that onUpgrade will be quick.
Now my go-to book (Wrox, Professional Android 4 development - I didn't chose it because of professional, I chose it because Wrox are like the O'Reilly of guides - O'Reilly suck at guides, they are reference book) only touches briefly on using Loaders, so I've done some searching, some more reading and so forth.
I've basically concluded a Loader is little more than a wrapper, it just does things on a different thread, and gives you a callback (on that worker thread) to process things in, it gives you 3 steps, initiating the query, using the results of the query, and resetting the query.
This seems like quite a thin wrapper, so question 1:
Why would I want to use Loaders?
I sense I may be missing something you see, most "utilities" like this with Android are really useful if you go with the grain so to speak, and as I said Loaders seem like a pretty thin wrapper, and they force me to have callback names which could become tedious of there are multiple queries going on
http://developer.android.com/reference/android/content/Loader.html
Reading that points out that "they ought to monitor the data and act upon changes" - this sounds great but it isn't obvious how that is actually done (I am thinking about database tables though)
Presentation
How should this alter the look of my application? Should I put a loading spinning thing (I'm not sure on the name, never needed them before) after a certain amount of time post activity creation? So the fragment is blank, but if X time elapses without the loader reporting back, I show a spiny thing?
Other operations
Loaders are clearly useless for updates and such, their name alone tells one this much, so any nasty updates and such would have to be wrapped by my own system for shunting work to a worker thread. This further leads me to wonder why would I want loaders?
What I think my answer is
Some sort of wrapper (at some level, content provider or otherwise) to do stuff on a worker thread will mean that the upgrade takes place on that thread, this solves the problem because ... well that's not on the main thread.
If I do write my own I can then (if I want to) ensure queries happen in a certain order, use my own data-structures (rather than Bundles) it seems that I have better control.
What I am really looking for
Discussion, I find when one knows why things are the way they are that one makes less mistakes and just generally has more confidence, I am sure there's a reason Loaders exist, and there will be some pattern that all of Android lends itself towards, I want to know why this is.
Example:
Adapters (for ListViews) it's not immediately obvious how one keeps track of rows (insert) why one must specify a default style (and why ArrayAdapter uses toString) when most of the time (in my experience, dare I say) it is subclasses, reading the source code gives one an understanding of what the Adapter must actually do, then I challenge myself "Can I think of a (better) system that meets these requirements", usually (and hopefully) my answer to that converges on how it's actually done.
Thus the "Hello World Gulf" is crossed.
I look forward to reading answers and any linked text-walls on the matter.
you shouldnt use Loaders directly, but rather LoaderManager
I am new to JUnit and I got a sample java project in which I need to write unit tests for all the methods.
Unfortunately the code is poorly designed and some of the methods are done from the UI. Furthermore, some of the methods pop up a messagebox and do not return a return value.
I have two questions: First, without modifying the existing code, is there a way I can suppress the message boxes and not press enter every time I run the unit tests?
Second question: can a test function expect a message box and assert failure\success upon it's string content?
I appreciate any help, I know the best solution is to fix the code itself - separate the BusinessLogic completely from the UI and to test expected result, or even if message boxes are somehow mandatory use modal message boxes (like humble dialog boxes) but unfortunately I am not allowed to change anything in the code.
Thanks :)
Nili
There are all sorts of ways you could get started if only you were allowed to edit the code, so my first approach would be to see if you can get this restriction relaxed, and to read Working Effectively With Legacy Code.
Failing that you could try using a GUI testing framework like FEST-Swing to check the contents of the message boxes are as expected.
Not allowed to change the code, you say? First thought it to have a look at JMockit which really opens up a lot of possibilities when you are severely constrained by code that was not written with much concern about how it should be tested. It should enable you to, without modifying any code, substitute your preferred implementation of bothersome parts while your test is running--so only in the context of testing would you have altered the test subject (be careful to write a meaningful test!) or its dependencies. Other mock object frameworks can be useful, too, but the investment to learn JMockit is really time well-spent.
unfortunately I am not allowed to change anything in the code.
There's all sorts of stuff on Google about how to automate Swing testing with JUnit. Unfortunately, there's no way to get around this problem when testing.
I tried to do several searches before posting this question. If this is a duplicate, please let me know and I will delete it.
My question revolves around the proper way to handle errors produced through our web application. We currently log everything through log4j. If an error happens, it just says "An error has occurred. The IT Department has been notified and will work to correct this as soon as possible" right on the screen. This tells the user nothing... but it also does not tell the developer anything either when we try to reproduce the error. We have to go to the error log folder and try finding this error. Let me also mention that the folder is full of logs from the past week. Each time there is an error, one log file is created for that user and email is sent to the IT staff assigned to work on errors. This email does not mention the log file name but it is a copy of the same error text written that is in the log file.
So if Alicia has a problem at 7:15 with something, but there are 10 other errors that happen that same minute, I have to go through each log file trying to find hers.
What I was proposing to my fellow co-workers is adding an Error Log table into the database. This would write a record to the table for each error, record who it is for, the error, what page it happened on, etc. The bonus of this would be that we can return the primary key value from the table (error_log_id) and show that on the page with a message like "Error Reference Id (1337) has been logged and the proper IT staff has been notified. Please keep this reference id handy for future use". When we get the email, it would tell us the id of the error for quick reference. Or if the user is persistent, they can contact us with the id and we can find the error rather quickly.
How do you setup your error logging? By the way, our system uses Java Servlets that connect to a SQL Server database.
I answered a similar question here, but I will adapt that answer to your question.
We use requestID for this purpose - assign a request ID to each incoming (HTTP) request, at the very beginning of processing (in filter) and then log that on every log line, so you can easily grep those logs later by that ID and find all relevant lines.
If you think it is very tedious to add that ID to every log statement, then you are not alone - java logging frameworks have made it transparent with the use of Mapped Diagnostic Context (MDC) (at least log4j and logback have this).
RequestID can also work as a handy reference number, to spit out, in case of errors (as you already suggested). However, as others have commented, it is not wise to load those details to database - better use file-system. Or, the simplest approach is to just use the requestID - then you do not need to do anything special at the moment error occurs. It just helps you to locate the correct logfile and search inside that file.
How would one requestID look like?
We use the following pattern:
<instanceName>:<currentTimeInMillis>.<counter>
In consists of the following variables:
instanceName uniquely identifies particular JVM in particular deployment environment / .
currentTimeInMillis is quite self-explanatory. We chose to represent it in human-readable format "yyyyMMddHHmmssSSS", so it is easy to read request start time from it (beware: SimpleDateFormat is not thread-safe, so you need to either synchronize it or create a new one on each request).
counter is request counter in that particular millisecond - in the rare case you might need to generate more than one request ID in one millisecond
As you can see, the ID format has been set up in such a way that currentTimeInMillis.counter combination is guaranteed to be unique in particular JVM and the whole ID is guaranteed to be globally unique (well, not in the true sense of "global", but it is global enough for our purposes), without the need to involve database or some other central node. Also, the use of instanceName variable gives you the possibility to limit the number of log files you later need to look in to find that request.
Then, the final question: "that is fine and dandy in single-JVM solution, but how do you scale that to several JVMs, communicating over some network protocol?"
As we use Spring Remoting for our remoting purposes, we have implemented custom RemoteInvocationFactory (that takes request ID from context and saves it to RemoteInvocation attributes) and RemoteInvocationExecutor (that takes request ID from attributes and adds it to diagnostic context in the other JVM).
Not sure how you would implement it with plain-RMI or other remoting methods.
If multiple servers are running and each server leaves log messages on itself, it is really difficult to trace them. So,somebody or a tool should gather and sort them in time order.
It is a good way to have a central point where all messages are sent.
A possible solution, have your error page include a 'send email to whatever' link. When the user clicks this email the body of the e-mail might start with a few blank lines followed by something like:
----Please do not modify the information below this line.---
Error details
Any users complaining via this link will automatically send you the info you need and if you are reproducing the error you have quick access to the error message. You might even have a form for sending the e-mail so that the user never sees this (which may be important to some) but then you are relying on your system being at least able to send an e-mail.
Actually I find it useful to print the error details in an HTML comment on error pages like this so that I can always get at them myself.
I do agree with david above that I do not like storing this kind of information in a DB.
For the strategies of logging you can see the discussion Logging best practices.
I have used an approach like the one you're suggesting ( log to a db ) in the past and it has been very helpful.
Not only you cat get the error via SQL but you can also generate reports of what's the most recurring errors and attend them first.
On the design we did, equals stacktraces belong to the same records ( since they were originated exactly in the same place )
We had an small app that pooled that db and we knew then a new exception was generated instead of getting an e-mail that summed with the rest of the previous weeks were ignored altogether.
Of course this database design was very specific for the application we had and additional identifications were possible, we had software version, build, some times input parameters , etc. etc.
With time, the system administrators get to know what to do with each kind of exception and proceed accordingly.
But! Your application may not be that big anyway. Probably you can have the same just parsing the log files.
I'd oppose the idea of storing error logs in a database. The logging system should be as simple as possible and not involve components that are not 100% necesary to write a log record.
Things can get pretty complex when logging into a DB - e.g. you can having troubles logging any database-related errors (how to log errors that occured because DB not responding, e.g. because of a heavy load or a infrastructure error); other issue I'd see is a potential need to have separate transactions for logging, etc.
On the other hand, having a reference ID for an error is not a bad idea, but again, this it also means to increase complexity of logging system ( e.g. how would you propagate the ref. ID through all layers of your application when a error occurs? )
In projects I'm involved to, the general guideline is to log errors as verbosely as possible, and to include as much context information as possible (to write the logs, we use a 'conventional' approach usually - log4j or simillar). Usually, this works well even for heavy loaded systems.