Archive for March, 2007

Is QA better at writing product specifications?

0

“Is QA better at writing product specifications?” This thought passed through my head while reading a post a while back written by QA Girl. I asked this question to myself because in the past I was part of a group where QA successfully helped to write the product specifications. With past experiences under my belt my first thought was “Yes, QA is better at writing specifications”, but after thinking a bit more about it I came to the conclusion of “not necessarily”. To explain, I’ll revisit how our QA group ended up writing specifications.


In the beginning… documents didn’t contain the things that testers needed to write test cases and test. Mostly specifics, things like a textbox’s allowed characters, min lengths, max lengths, defaults, or a button’s behavior when clicked. Important specifics when it came to Web page design, functionality and security. Through the course of several projects the lack of specifics were expressed through defects, and those defects would be duplicated from one project to the next. Consistently. Consistent because the test cases were detailed and consistently shared across all projects. Verbal frustrations started flowing…”This document sucks, what am I testing here?”, and then a new tag-line was born “Specification definition through defect input”. Developers were adding to and rewriting specifications like crazy just so they could get their head around what was supposed to be developed (far too late in the process too!) Then, one fine day, in a company that was slowly moving to a more agile environment, QA’s voice was heard and the marketing and design teams started asking “what do you want to see in the specification documents”?


Finally, the clouds were gloriously parted and a direct beam of sunlight was shining blissfully on QA’s face. “What do we want? We want you to provide the product details that we have built into our test cases!” “You have the details in your test cases?” they asked with astonishment. “Yes, we defined them through defect input over the course of several projects.” the QA team replied with pride. “Wow, can we get a copy of those cases?” was the sheepish response. But delivering that copy of test cases didn’t fix the problem entirely. The next release and next document contained the first attempt at delivering those needs but somehow was still lacking the details. It seemed to QA that the design team couldn’t translate QA talk/test cases. This was scary, since the test case were well designed, clear and concise. What was the hang-up? Did they not understand QA talk? The answer was not that but instead: lack of time. The design team had enough trouble keeping up with other details in the specification document, adding the new details (the right way) didn’t fit into their design schedule. Ouch…But, how can you develop a product that is not designed?


What came next is what led to my first impression of “Is QA better at writing specifications” as being a big fat “Yes”. The design team said “Can QA add those details to the specification document?” It is that we did, and by doing so, QA moved themselves earlier into the project (design phase) and ultimately satisfied the specification need.


Could the design team have done the same? Yes. Did they have time to do so? No. Thus, “Is QA better at writing product specifications?” Not necessarily. BUT…QA contribution is required during the design process. If the QA team can’t help add details to the specification document during design then they must at least provide and translate those needs to the design team. In my experience, translation can be tough, something is always lost along the way, and it seems to be easier to cut out the middleman. With that said…. “Is QA better at writing product specifications?” Yes, for the purpose of testing needs.


To recap: “Is QA better at writing product specifications?” Yes. Not necessarily. Yes. Got it? Maybe it’s better said: “Should QA contribute to the product specification?” and to that we say a big, fat, gihugey “YES”.


Now, one’s inner QA mind should be screaming “Danger! Danger! You can’t do both, write the requirements and test them!” Ah, yes. You are right, you, a solo Test Engineer should not, but a Test team can. One QA individual can write them, another can test them, mitigating the risk while also giving the tester what he/she needs.


Now let’s throw away my example and past experience and weigh a few pros and cons. I’m sure you can come up with more but when you add them and reweigh I’ll bet your conclusion is the same as mine:


The Pros of QA contributing to specifications



  • Specifications are written early, forcing QA to be involved early.
  • Specifications will contain exactly what the Test Engineer needs during the Test phase.
  • QA teams that are required to test documents aren’t getting them in a waterfall fashion. You now have agile documentation testing going on! 
  • The QA team will be more knowledgeable of the product. Better knowledge of the product will result in better tests, better test estimates, and earlier definition and delivery of QA requirements (e.g. environments).
  • Defects (design, functional, and detail) can be found before the product is developed saving costly redesign time later down the road.
  • Fewer defects are found during the Testing Phase, giving the tester more time to focus on finding the harder, better hidden defects.
  • The QA and Tester mind-set forces Design teams to think outside of the box.
  • Fewer specification document updates occur in later project phases.

The Cons of QA contributing to specifications



  • The specification writer can’t test too. Testing needs fresh and unbiased eyes. If the tester writes the requirements they are biased when testing them.
  • Lack of QA resources. You can’t afford to take a resource from running test cases on another project(I’m really reaching here, see below)

By my calculations the pros outweigh the cons! If we mitigate the cons, QA contribution to specifications is a no-brainer. Let’s mitigate those cons:



  • The QA/specification writer can’t test too.

    • Mitigation: A minimum of two man test teams.

  • Lack of QA resources

    • Mitigation: This is NOT a real problem! If your waterfall process has tricked you into thinking this is a problem then you can’t afford not to! I promise you, you will see the savings in subsequent project phases by finding defects early vs. late. Worst case; imagine that you are taking a Developer’s time from testing phase and putting that into the design phase for the Test Engineer. You break even, just less head-ache and stress at the end of the project. This con has now been converted to a pro!

Makes sense doesn’t it? So….what are you going to do now? In case you missed the point, let’s put this in a pretty TO DO list for you:



  • Include QA where needed in the requirements gathering process.
  • Include QA in the specifications writing process.
  • Budget less Developer resources in the testing phase and budget a QA resource to the requirements gathering/writing phase.
  • Find issues early, before development begins, and avoid costly redesign.
  • Smile with satisfaction because you have a less hectic and higher quality project.


PC won’t boot with ReadyBoost

1

Jeff Atwood and Scott Hanselman convinced me that running ReadyBoost on Vista was the way to go and they were right. Running ReadyBoost has increased my system performance quite a bit. I set myself up with the Apacer 2GB Handy Steno and saw the gains immediately. ReadyBoost works and it… feels better.  My click and wait is gone.


After about a week of running and loving ReadyBoost I rebooted my PC. It was not happening. Reboot denied. Nothing but a crisp, black & white, you lose, sorry, no way message of: “Invalid or damaged bootable partition”. Being a week later, ReadyBoost wasn’t in the front of my mind so the heart skipped a few beats while my troubleshooting mind raced to and fro. Seconds later, I re-reminded myself of the ReadyBoost change, yanked the drive, and rebooted. No more error message. What the?


My fix… I turned off the USB Boot feature in the boot options of my system BIOS. The other USB drives attached to my PC don’t have this problem but the ReadyBoost drive does for some reason. Oh well. Easy fix.


Find installer defects quickly by reviewing key MSI database tables

0

I often find defects in the small details that go along with an installer. So many things are configurable, and simple things can be easily missed due to the overwhelming amount of things to look for. Walking through installation wizards can be visually overwhelming when testing. Each step presents it’s own window title, header text, product descriptions, button names, error text, events, and so on. Each wizard step can also be a complex form full of variables that need testing. To avoid the visual overwhelm, that often results in missed defects, you can get an overview of all those details by peering into the msi database within the msi installer. Using the tool MSI Database Reader & Reporter you can get that overview and quickly identify many defects. After loading an msi database in the tool you can find a majority of detail oriented, low severity defects by looking at key database tables. I find the following tables to be informative and a potential defect haven:


Property table: contains various properties found throughout the installer (e.g. form default values, upgrade guid/code).


File table: contains the file list and all file details (e.g. size, version).


ActionText table: contains a list of the text that appears above the progress bar during install.


Feature table: contains features’ description text.


Control table: contains a list of the form controls used in the installer, their properties. The Text column lists all form text strings.


Error table: contains installer error text (with the exception of custom errors defined in the property table).


MsiFileHash table: contains a hash for every file in the installer. This is great for detecting file changes between new builds of installers.


Keep in mind this is a quick and dirty overview of the guts in the installer, and not a replacement for the actual testing of the install, since the hoards of information in the database is not easily tied to a step in the installer wizard.


Make testing of the details in an installer easier on yourself, use MSI Database Reader & Reporter . Create installer reports with it and do report diffs between builds to detect changes. It’ll find you installer defects. I promise.


MSI Installer Testing Tool: MSI Database Reader and Reporter

0

I had a need to view and create a report on an MSI database (from an MSI installer) and another tool was born. MSI Database Reader & Reporter enables you to view the contents of an MSI database (a feature already handled by Orca) but….then create an html report of that database. This is valuable for installer testing since the report is essentially an .msi installer snapshot, and you can view differences between installers  by doing a file difference report between two the html reports. Download MSI Database Reader & Reporter here (25.02 KB). Just unzip and run. View an example .msi database report here (224.2 KB, created from WatirNUT Setup.msi).


Screenshot:





Sample Section of an MSI Database Report:




Update to User Agent Import for User Agent Switcher

2

Update 05/17/2007: The list has been updated and can be found here.



Somebody asked for an update to my outdated XML import file for User Agent Switcher. Honoring that request, I have added the latest user agent strings for each browser. Updated or added strings are marked with green below. I also added a new section for RSS reader user agent strings. Download the User Agent List import file here: AgentStrings20070304.xml (16.51 KB).


Internet Explorer
MSIE 7 (Win Vista)
MSIE 7 (Win XP)
MSIE 7 (Win Storage Server)

MSIE 6 (Win XP)
MSIE 5.5 (Win 2000)
MSIE 5.5 (Win ME)
MSIE 5.0 (Win 95)
MSIE 4.01 (Win 95)
MSIE 4.0 (Win NT)

FireFox

FireFox 2.0 (Win XP)
FireFox 1.5.0.6 (Fedora Core 5/Linux)
FireFox 1.5.0.4 (Debian Linux)
FireFox 1.5.0.3 (Mac OSX)
FireFox 1.5.0.1 (Win Vista)
FireFox 1.5 (Win XP)
FireFox 1.4 (Win XP)
FireFox 1.0.7 (Win XP)
FireFox 1.0.6 (Win XP)
FireFox 1.0.5 (SlackWare/Linux)
FireFox 1.0.4 (FreeBSD)
FireFox 1.0 (Win XP)
FireFox 0.9.3 (Win XP)

Opera

Opera 9.1 (Win XP)
Opera 8.0 (Win 2000)
Opera 7.6 (Server 2003)
Opera 7.51 (Win XP)
Opera 7.5 (Win XP)
Opera 7.5 (Win ME)
Opera 4
Opera 3
Opera 2
Opera 1


Netscape
Netscape 8.1 (Win XP)
Netscape 8.0.1 (Win XP)
Netscape 7.2 (Win 2000)
Netscape 7.1 (Win 98)
Netscape 4.8 (Win XP)
Netscape 4.76 (Win 98)
Netscape 4.5 (Win 98)
Netscape 4.05 (Win 95)
Netscape 3.01 gold (Win 95)
Netscape 2.02 (Win 95)

Safari

Safari 3.0 (v521)
Safari 2.0 (v418.9.1)
Safari 2.0 (v412)
Safari 2.0 (v401)
Safari 1.3 (v162)
Safari 1.3 (v158)
Safari 1.3 (v146)
Safari 1.2 (v125.8; OSX 10.3.4)
Safari 1.1 (v100.1; OSX 10.3.2)
Safari 1.0 (v85)
Safari 1.0 (v74 beta 2)
Safari 1.0 (v73 beta 2)
Safari 1.0 (v64 beta)
Safari 1.0 (v51 update 1.0 beta)
Safari 1.0 (v48 public beta)

AOL
AOL 9.0/MSIE 6.0 (Win XP)
AOL 8.0/MSIE 6.0 (Win XP)
AOL 7.0/MSIE 5.5 (Win 98)
AOL 6.0/MSIE 5.5 (Win 98)
AOL 5.0/MSIE 5.0 (Win 98)
AOL 5.0 (OS X)
AOL 5.0 (Mac PPC)
AOL 4.0/MSIE 3.02 (Win 95)
AOL 4.0/MSIE 4.01 (Win 95)
AOL 4.0 (Mac PPC)
AOL 4.0 (Mac 68k)
AOL 4.0/MSIE 3.0 (Win 3.1)
AOL 3.0/MSIE 3.02 (Win 95)
AOL 3.0/MSIE 4.01 (Win 95)

Mozilla

Mozilla 1.7.8 (Linux)

Spiders/Bots

Googlebot 2.1 (New version)
Msnbot 1.0
Msnbot 1.0 Beta (v0.11)
Yahoo Slurp
Ask Jeeves/Teoma

RSS Readers

NewsGator 2.5 (Win XP)
Feed Demon 1.6 (Win XP)
Bloglines 2.1
SharpReader 0.9.6.0 (Win XP)


Extrem Virtual PC Slowness Due to VPC Window Scrolling

0

The Environment: Doing a remote desktop connection (RDC) into a machine that runs your Virtual PC 2004 images.


The Issue: EXTREME VPC slowness, window title bars are missing, the mouse leaves artifacts and is about 5 seconds delayed, and the VPC is not maximized (has scrollbars present).


The Fix: Make the VPC large enough so that it does not have scrollbars or maximize it (alt+enter).


A seriously annoying and confusing issue… Below is a screenshot of the craziness. Note the mouse artifacts and the top of the Windows Start menu is missing:


VPC slowness and artifacting


Post navigation