Archive for February, 2010
Being able to to run individual tests repeatedly and also being able to run them consecutively is a serious challenge when it comes to automation. I think all automation engineers want it, but some fall prey to not having it due to time constraints, setup and teardown complexity, or system access limitations.
What good is automation when it can’t be ran repeatedly, further more running them all at once? Seems like a dumb question, but I’ve seen plenty of non-repeatable tests or suites written by others, and occasionally have fallen prey myself.
How do I ensure repeatability?
- When planning the automation time budget, I allocate at least 15% for building infrastructure. Can’t get the time? Fight for it, as an automation engineer it’s your job. You have to sell the fact that if you don’t have the time to build in the infrastructure for repeatability your tests will be less useful, take longer to setup due to manual intervention, and will eventually crumble into uselessness over time because manual intervention is not something you can easily hand off to another automation engineer or developer.
- I start building in repeatability with test #1. This test can often take the longest to write because I’m often times building in the infrastructure needed to run all my tests repeatedly (reusable functions, system access functions (db connections, etc), setup and teardown functions).
- When the test is done being developed, I run the test, I run it again, I run it again. repeat and rinse until it passes every time.
How do I ensure I can run all my tests consecutively?
- I build each test so that it can be ran independently, which means that when I run them all from a list, no matter which order they are in, they will not interfere with each other. This is why extracting common code into methods, setup and teardown is instrumental; I will reuse it in almost every test.
- I try to run all tests after each new test is developed and completed to make sure the new test plays nice with the others. This isn’t always possible if you have long running tests, but if they aren’t, I definitely do this.
- I run them all at least once a day.
The first copy of the Software Testing Club magazine (STC) is out! I had a chance to review it before the release and I have to say it’s a really cool magazine. It’s fun, different, and not so stuffy. SQA talk can get so boring, but STC breaks out of that box. Between the articles written by the community, the artwork, the comic strips, and the smart-ass QA and development quotes on the bottom-left of every other page, I found myself having a great time reading it. I really enjoyed the comics by Andy Glover, and found myself laughing out loud over them.
I’d like to also point out that my Do Loop Until 0 comic is in the magazine. Not necessarily funny, but a quick view of the realities of testers and developers in the software development environment. I’ll admit Do Loop Until 0 can be a little deep at times, but if you study the details closely the irony will hit you like a sledgehammer. The more I do the strip, the more I realize how Spy vs. Spy influenced me as a child.
Take a look yourself here, I really do think you’d enjoy it.
It’s on the horizon. a new, unique and fun magazine written by software testers for software testers: STC Magazine. Keep you eyes peeled this weekend for it on the Software Testing Club site. I’m highly confident you’re going to like it.