Sunday, November 7, 2010

Quality Counts

Quality matters much more in a SaaS offering than with an on-premise offering.

QA managers may disagree with me -- after all, it's their job to get quality right regardless of the delivery model.  But in my experience, SaaS gives a vendor far less room for sloppy quality than with an on-premise solution.  Some of this is really a question of velocity.  With most SaaS solutions, an upgrade is done for all customers at once or for large blocks of customers in server "pods", which makes it a fast and efficient model for putting new capabilities into customer hands.  Unfortunately, it also means you're putting bugs into your customers' hands quickly and efficiently!

In an on-premise model, when a new release is shipped, there's generally a nice, gradual roll-out, as customers take the upgrade on their own schedule.  If 10% of a customer base adopts a new release in the first month or two, that's not a bad take-up rate at all, and problems will tend to start trickling in, not gushing.  That gradual pace often gives the vendor time to fix reported problems before too many customers are impacted.  And that's often why customers choose to wait, of course -- give it a couple of months, and someone else will suffer through the release's teething pains.

But with a SaaS model, the entire installed base may go to bed on one release, and wake up to another.  If there are quality problems, there are no gradual roll-outs -- no early adopters to find the problems on behalf of the rest of the pack.  The effects of poor quality will be multiplied, accordingly -- from Support, to Engineering, to Account management to the Executive team.  And remember, delighting customers and securing renewals are the name of the game with SaaS.  Poor quality, in this case, can be terminal.

There are many ways to keep quality problems at bay.  But I think the most important is cultural.  QA is often under-resourced relative to Development, and it's not uncommon for developers look down their noses at their QA counterparts, instead of looking to them for partnership.  An organization serious about stamping out quality problems could do worse than to look for and eliminate these two causes (I'm betting they'll be found more often than not).

As Product Managers, we need to champion that cause, and also acknowledge our own contribution to quality problems.  It's tempting to ask for one more feature, change or bug fix late in the game.  It's easy to look at our backlog of enhancement requests and conclude that we can't live without way more than can reasonably be done in a high quality manner.  We need to remember that the software has to work, and work well.  There are no more important requirements, and shame on us if we put the business at risk for something lesser.

There's a good case in here for agile development approaches, as well -- more on that another time.

All for now,

J