Our Corporate Security group recently was blessed with a 2nd QA person…MR. ROHIT MATHUR (big booming voice there). The guy is freakin’ great at QA, testing, and is also a good friend. I’ve been willingly (big, huge, willingly) sharing my QA responsibilities with him for over a week now and sharing often turns up little QA and test things that are part of my everyday, going through the motions thing that can save a new person on the scene 5 minutes, 15 minutes, 1 hour… The scene, and my point today: Virtual PC 2004, what to do when you can’t drag-n-drop copy from the hosting machine to the VPC.
When you can’t drag-copy a file from the host system to your VPC …
Drag-copy a file from the VPC to the host system! By reversing the process, it somehow fixes the issue. Nice defect. Nice work-around.
Note: This only works if the VPC can be resized by dragging the edges. If you can’t resize the VPC by dragging then you have a bigger copy-paste problem.
It wasn’t long ago that I started using the free tool Install Analyzer by VeriTest-Rational to validate directory and registry changes during install. The tool is ultra helpful to monitor changes caused by an installer under test and has the plus of being fast. I had tried other system change monitoring tools but the speed of gathering 3 snapshots was unbearably slow (before install, after install, after uninstall). With unbearably slow not an option Install Analyzer quickly found a snug little place in my testing toolbox. Over the course of a few weeks I realized that Install Analyzer was missing one key piece of information: assembly version numbers. Hmmm, no biggy I can work around that…or can I… As my installer tests evolved due to a defect after defect of file property issues, I found that an overview of the files installed and their properties was needed direly. What better way to guarantee getting the perfect tool for your needs than to build it yourself? File Property Reporter was born.
File Property Reporter analyzes a directory and all underlying subdirectories, reports the directory structure, all the files names, and all the properties of the files. Using all the file properties available in the classes FileVersionInfo and FileInfo the tool is capable of gathering and reporting on 40 different file properties (CompanyName, FileBuildPart, FileDescription, FileMajorPart, FileMinorPart, FileName, FilePrivatePart, FileVersion, InternalName, IsDebug, IsPatched, IsPrivateBuild, IsPreRelease, IsSpecialBuild, Language, LegalCopyright, LegalTrademarks, OriginalFilename, PrivateBuild, ProductBuildPart, ProductMajorPart, ProductMinorPart, ProductName, ProductPrivatePart, ProductVersion, SpecialBuild, Name, Length, DirectoryName, Directory, Exists, FullName, Extension, CreationTime, CreationTimeUtc, LastAccessTime, LastAccessTimeUtc, LastWriteTime, LastWriteTimeUtc, Attributes). The tool can also detect and report non-compliant property strings for “ProductVersion” and “CompanyName”. In addition, files reported on can be narrowed by using file patterns (e.g. *.dll, *.aspx, *.*).
See an example File Property Report ran against a FireFox 1.0.8 directory here (8 properties enabled).
Extract contents of Installshield installers to a directory (using the /admin command line parameter) and run the tool to get an overview report of all the files and directory structure. Or in other words, create a file manifest.
Use a file differencing tool to compare resulting reports between two installer builds to expose changes.
Snapshot your file system pre-install, post-install, post-uninstall and compare the reports using a file differencing tool to see the disk changes.
Enjoy, it’s free. No installation is required, just drop the exe and the config file in a directory and run it. The options are pretty self explanatory. The configuration file keys are documented in the file. Email me if you have questions or comments.
Future plans include:
Making it multi-threaded so the GUI doesn’t freeze up while running against huge directory structures (e.g. all of C:\).
Microsoft has released Virtual PC 2007 and it is available for download here. I never had the chance to run the beta and hope to put the release to work soon enough. As blogged by Ben the following are a few key features of Microsoft Virtual PC 2007:
Support for x64 Windows as a host operating system
Support for hardware virtualization
Support for Windows Vista as a guest and host operating system
Support for PXE network booting of virtual machines
Support for the use of fullscreen virtual machines on multi-monitor systems (VM still stays on just one monitor though)
I’ll post some comments once I’ve put it through the paces.
The other day I took some time to read a case-study written by RSnake entitled Death by 1000 Cuts. The study was interesting in the fact that a small pile of 10 minor and medium defects turned into 1 large security issue. It’s a good read and I suggest it if you have 5 minutes on your hands.
Before I even started reading the study the term “Death by 1000 cuts” hit me. This is the perfect term that I had looked for so many times in the past to explain an out of control project with tons of minor defects, the release date around the corner, and the implementation that was going live no matter what. The 1000 cuts in this case being the dozens and dozens of low severity defects that when put together made a pretty crappy looking and feeling Web site.
When the project has a customer weighing in like 800 pound guerilla holding tight on promised release dates, the over-worked development team tends to get the high severity defects fixed first and the low severity defects are quick to be ignored by developers and project managers by abusing the leniency in the defined QA Exit Criteria:
Severity 1: Zero remaining defects. Severity 2: Zero remaining defects. Severity 3: To be agreed on by the QA Lead, Project Manager and customer. Depending on the agreement of the disposition of these defects, this status could possibly delay QA exit. Severity 4: To be agreed on by the QA Lead, Project Manager and customer. However, defects of this severity will not delay QA exit.
As mentioned before, a large pile of Severity 3s and 4s in a Web site can simply suck. Sure, for the most part it works and the user can do what they need to do, but the overall user experience can be just plain horrible. Don’t let your project die the Death of 1000 Cuts due to guerilla tactics and bad QA exit criteria.Create strong exit criteria (stronger than listed above) and stick to your guns. I did, and the defects at go-live turned from several dozen to a few.
I’m currently looking for a SQL data differencing tool that has the features:
Support for SQL 2000 and 2005
For the purpose of database migration testing, allow data snapshot comparisons. Tools I’ve encountered so far allow you to compare two, up and running databases which isn’t what I want to do. I want to snapshot a database, run my database upgrade installer, snapshot after the upgrade and then compare the two snapshots.
I’ve been recommended and have reviewed the following but they don’t meet my 2nd criteria:
Nearly every minor and major release of our product ends up with changes to the database. Sometimes to the schema, sometimes to the data, sometimes both. Schema changes are obviously important to be aware of from a testing perspective, so a tool to catch these changes is needed in the testing toolbox. When working with with products that use SQL 2000, I use two tools to detect the changes: SQL Enterprise Manager and a file differencing tool (I use Beyond Compare). Detecting and analyzing changes using the two tools is quite simple and only takes a few minutes. Here’s how:
The magic starts with Enterprise Manager’s feature “Generate SQL Script”. Using this feature outputs the selected database to a SQL file in DDL. The first step in the process is to take a snapshot of your existing database using this feature. To do so:
In Enterprise Manager right-click the database, select item “Generate SQL Script”:
The Generate SQL Script window will open, on the first tab labeled “General” check the Script all objects checkbox:
On the 2nd tab labeled “Formatting”, select the following checkboxes:
Generate the CREATE <object> command for each object.
Generate scripts for all dependant objects.
Include extended properties.
On the 3rd tab labeled “Options”, select the following checkboxes:
Script object-level permissions
Script full-text indexes
Script PRIMARY keys, FOREIGN keys, default and check constraints
You’re all configured to dump the important stuff now (you deem what its important for your product though, feel free to check something that you think applies to your database change). Now that you’re configured, click the “OK” button and you’ll be prompted to save the file. Save the file. You now have a snapshot of your current schema!
With the snapshot of your schema in place, you can now conduct your database upgrade or whatever database altering voodoo that the developer has asked you to validate. Once the upgrade is complete, run the above set of instructions again to create a DDL dump of your newly modified database. Once the .sql file is saved, it’s time to view the differences, do this using your file differencing tool. The differences in the report that your diff tool creates are obviously the differences between database schemas. Create test cases accordingly…
Unfortunately, SQL 2005 has ruined this feature as far as I can tell. The feature exists but the output is wildly different. Differences in schemas create DDL files that are far different and hard to line up. This appears to be due to some new ordering of how things are created for the DDL. Needless to say, I’m in the process of finding a good tool or work-around trick for this issue.
When testing Web Services it’s important to know if the service/contract has changed so that you can conduct testing accordingly. From experience, when developers build Web Services using .NET, changes can be made to the service unknowingly. Because of this, a tester must add another test case to his arsenal to check for those changes. Fortunately detecting changes is really simple and can be done with two tools you most likely already have in your testing toolbox: a Web browser and a file differencing tool like Beyond Compare.
Below, I’ll show you how I detect Web Service changes. For the purpose of these instructions I’ll use two example product versions Product 1.0 and Product 1.1:
Simply put, we need to compare one Web Service contract to the other; the contract being the WSDL.
I’m still up to my neck with installer testing and I’ve started to dig into some of the command line parameters of Installshield and msiexec.exe to better understand how things work and to help Matt with troubleshooting. If you’re new to the Installshield thing then you’ll need to know that an InstallShield .exe wraps a .msi and each can be passed command line parameters (.the .exe takes them, and the exe can pass them to the .msi). Problems can be cryptic and logging can be your best friend. However, making the logging work for you isn’t intuitive and requires some investigation. Here is a little diddy I discovered today that helped us through a defect:
To force the .msi to use its verbose logging feature you would use the following at the command line:
where MySexyInstallShieldInstaller.exe = your InstallShieldInstaller name
where /v = passes everything in the following quotes as arguments to the .msi, or rather msiexec.exe
where “/lv = l for logging, v for verbose; for the .msi
where C:\myverboselog.txt “ = the directory and name of the file where the logging is done
The output, according to Matt is “really verbose”. Perfect if the defect is easily discovered and fixed! If you’re in the same installer world as me, the following parameter definitions could be helfpul to you:
I have to admit, my faith in God these days is pretty..how can I say it… lacking. I’ll spare you the gory details of my frustrations, but in a nutshell I’m just kind of frustrated with humanity, how we treat each other, and the feeling like we’re hard wired to fail.
My loving Grandma, who has served nearly her whole life to the lord, and holds her own the Internet (go Grandma!) emailed me a video URL the other day that reminded me that there is something up there/out there that is driving us. The video is about the child prodigy Beck Akiane who has had an astonishing talent for art (starting at age 6), poetry, and music. Simply amazing. A faith refresher. Check out Beck’s site.
I’ve always admired fellow coworker and QA engineer Phillip Forteza for his enthusiastic and positive attitude. Sitting in my cube 10 feet away, I could hear Phillip interact with coworkers and customers; his interaction always upbeat, caring, and real. I really admire that. I wish we could always ooze positive emotion like Phillip, especially myself.
Perusing my blog logs I encountered that Phillip had started a blog. Excited to see another fellow coworker’s musings, I found my way to his site and read all of his posts. I LOVE IT. It’s Phillip to the T (Phillip to the P?). So far Phillip’s musings are life lessons and each post a positive, uplifting addition to your day. Go check him out at A Day in the Life of PF3.