How to make Agile work for product development… It’s all about the people

Some background before I get into the detail, I’ve been developing small, medium and large ‘shrink wrapped’ products for the past 12 years and before that I worked on large multi-million pound in house projects.

Why do I give this background? Well it’s to set context in relation to this post from @Martijnlinssen where Martijn questions whether Agile can be used effectively for product development. I read into Martijn post that whilst Agile might work for project development it doesn’t work as well (if at all?) for Product development. I chose to disagree with Martijn on this point and responded with the comment:

I’m not sure from reading this that you actually get agile (sorry Martijn).. Whilst I agree Agile is not for everything, Agile can be a highly effective way to deliver software (and even commercial business) projects.

What it isn’t is throw all control in the bin and fly by the seat of your pants… What it is is a model whereby accountability lies with the dev team and not some project manager miles away from reality of day to day decision making.

The controls are very adequate and personally I’ve used it on dev teams of 80 or so people… Based around scrum of scrum hierarchy … However, that said I also like high level governance in place for really complex implementations.

We debated this via twitter some more and a few more comments were added to the original post. I think we got to the stage where we agreed on some parts but not all, i.e. Martijn came up with some very interesting follow up questions/concerns:

How much money gained during dev is squandered in test and maintenance, compared to waterfall?

When does the benefit of not writing everything down turn into a concern?

How is the transition made from build to test to product for maintenance?

Can you just outsource your finite product once Live?

So, I’m writing this post to share my thoughts (and learning’s) around these questions. I say leaning’s as I have the very same concerns in the past. If you don’t know what Agile is then look here for a very basic overview of Agile.

Now, if you just took the overview and went off to develop products you’d be in a big mess in no time at all. In my experience, there are some other very key components that are essential for success. Assuming that you have your development backlog, velocity and your sprint schedule defined, I’d advise that you also need:

  1. A solid system architecture in place before you begin – my approach is to deliver the blueprint and prove it out if necessary with a proof of concept (POC) in Sprint Zero… you won’t see mention of sprint zero in many (if any) Agile books, but it’s my preferred way to prime the Agile process. Start building without this and you’ll hit dead ends time after time and end up reworking huge chunks of the product.
  2. Design patterns and framework for the user experience and interaction, i.e. it’s no good if different parts of the product look, feel and work differently. Usability is absolutely key and should be built in from the start as opposed to painting it on at the end.
  3. Good coding guidelines around solid engineering principles such as defensive programming and more recently test driven development, without these it’s very easy to write poor quality code the costs a fortune to maintain using Waterfall or Agile.
  4. And the secret ingredient… A team of highly proficient people who understand how customers work and what problems the Agile stories are trying to solve… I can’t stress how critical it is that developers and testers understand the needs of the customer, otherwise they just build what they are told and we all know that’s a recipe for disaster. It’s also a good idea on complex systems to supplement the developers and testers knowledge with highly specialist domain experts (Business Analysts) for things such as compliance and complex processes.

Some of the techniques I really like in Agile are daily meetings, which are fantastic to keep everyone on track, the burn down charts to show you exactly where you are, and the retrospective reviews, which are a brilliant way to look at what you’ve just done and how you did it and improve it for next time around. All of the best athletes do this, the military do this, and it’s a great way to continually learn and increase the capability of the team.

So, coming back to Martijns questions…

How much money gained during dev is squandered in test and maintenance, compared to waterfall?

None of it if you follow the principles above… and if you structure the sprint teams correctly you can save money by being more effective, i.e. the formula I use is:

Developers and Testers working together as part of a 6 person scrum team, normally to a ratio of 3 devs, 2 testers and a scrum master (Team Leader)… this means that you test as you go, can fix issues as they are found and deliver better quality software at the end of each sprint.

For larger systems, I also prefer a separate system/integration test team that don’t work as an agile team. This team doesn’t retest everything, they specialise on integration, regression and beta testing.

As for maintenance I don’t use agile per say for that either. I prefer a separate sustained engineering team that are dedicated to respond to any issues found post release quickly and efficiently.

If I compare this to my experience of waterfall, where all the specs are written up front by BA’s who understand the customer, handed to development to code up who don’t understand the customer, then passed to testing for QA who try and piece things together. It’s much less efficient and makes no attempt to build quality in as you go, and if the project is a long running development there’s a real chance the requirements have changed since the BA wrote the original spec. The system is out of date the minute its delivered.

I’ve seen many more waterfall projects fail than I have Agile, generally people are more connected with what the goal is and what needs to be done to ensure delivery.

When does the benefit of not writing everything down turn into a concern?

Great question, of course nobody says you don’t have to write anything down, you document the system in a number of ways, first you have the architectural blueprint and the interaction patterns, then you document complex business rules and compliance and finally you document in the code. From my experience, documentation is used to define and not to maintain, i.e. I’ve never met a developer who goes back to original spec to fix a problem, the best ones read the code and trace the problem through a debugger.

How is the transition made from build to test to product for maintenance? Can you just outsource your finite product once Live?

Okay, so the sprint teams deliver running code, by running code I mean it is developed, working and tested within the team. For multi-team and multi sprint projects each sprint deliverable is then assembled (using a versioned build process) and handed over to the system test team who check all the component parts delivered by the sprints work together. This takes the form of an initial smoke test, thorough integration testing, and full regression test to ensure the new deliverable hasn’t broken the system. Any bugs that are found are captured and these are fed into subsequent sprints, or more likely that you have a one or more bug fixing only sprints only once you’ve reached feature complete lockdown…. This may also include a period of external beta testing, but the process is the same.

When the bugs have been resolved the product deemed fit for purpose, it is versioned and shipped.

So then you start working on the next version of the product and repeat the process from the top. In the meantime you have to sustain the product you have just released. It’s even more important here that the sustainability team understands the customer and the domain. Whilst Agile could be used here to produce a scheduled service pack which contains a defined body of work, Its not good for producing a single hot patch which are normally a few days in duration.

Can you outsource the product once live?…

I don’t, but it’s possible as long as the sustainability team has a certain amount of product/domain knowledge and can read the blueprint and the code. I’ve tried this in the past based on specs only and it fails miserably, even with waterfall, it fails miserably… as at the very least you need a link back to someone who understands the customer and the domain.  

Agile is about people… and not about the Project Manager

All of that said… the most important part of Agile for me is Ownership, Accountability and Empowerment. In a high performing Agile team, the traditional Project Manager feels uneasy and disempowered, they may feel out of control, as the team are empowered to make their own decisions. The old style command and control ‘tell do’ Project Manager will never succeed with Agile and will fight tooth and nail against its adoption. Whereas the new breed of Project Manager who can trust his people and continually develop them through coaching, will soon reap the benefits of delivering much more through their people than they ever could through command and control.  

How do I know that?

This is my own story…  I am that person.

Technocrats put your soap box away and give the Cloud a chance

If you’ve read any of my recent posts you might think that I’m not supportive of Cloud. Some have even insinuated that I might even be a detractor, deliberately trying to slow down the adoption of Cloud.

Well, it’s time for me to come clean and state for the record that I’m actually pro Cloud, yes that’s right I’m actually pro Cloud. I believe that the Cloud is a good thing that can deliver many user benefits and that the Cloud will change our industry forever.

However, I’m not a Cloud technocrat … I’m not prejudiced or biased towards Cloud vs. other delivery models. And even though I’m a technologist by profession, my real passion is for using the most appropriate technology, whatever that may be, to deliver solutions that drive customer benefit and solve real business problems.

I predict that no single model will dominate the technology roadmap and therefore I don’t support the theory that everything has to run on the Cloud. Stand alone on device applications and on device applications connected to Cloud services will continue to provide excellent pragmatic solutions to many problems for years to come. It’s clear for everyone to see their ascendency and this trend won’t change anytime soon. I also feel that the desktop as a platform for on premise applications is far from dead and that client server architectures will continue to prevail in the future in many guises.

But coming back to Cloud, I find it interesting not because of the hype, but because it offers new opportunities spanning many different technologies and operating models. The Cloud allows easier sharing of information between entities, and therefore fuels collaboration. It enables anytime any place anywhere accessibility. It lends itself to a “pay for what you eat” utility, and it offers a conduit between connected and disconnected systems and thus enables composite Business Process Outsourcing (BPO).

Today, the delivery model we hear most about in relation to Cloud is Software as a Service (SaaS) and it may come as a surprise to some to hear that I also understand and support the benefits of the SaaS model of software delivery. Its great for some things but its not a panacea for all things and so I’ve been a tad critical of those who try to ram the “SaaS is the only Cloud delivery model” down everyone’s throat. To those people I would suggest they:  

Go speak to an IT director and they may tell you they’re looking to move some of their internal systems to the cloud in conjunction with one of the many Infrastructure as a Service (IaaS) platform providers. What’s the most pragmatic way to approach this… demand that they must throw out all of the systems that they have acquired and customised over a period of time to meet their business needs and replace them with a whole suite of “not anywhere near like for like” SaaS equivalents… or help them on the journey by suggesting they look at virtualisation as a first step, i.e. their existing systems work as they always have… no retraining, no costly redoing of system integration, they’re just hosted in the cloud, and then gradually move to SaaS and other forms of Cloud on a business needs basis.

Or go speak to an ISV developing mobile software or an iPad developer who is accessing data and a business service running on the Cloud and they may tell you the future as far as they are concerned is all about utilising the best of both worlds. It’s all about applications developed specifically for a device that leverages all the power of the device (not just a web browser running on the device), connected to highly scalable services running in the Cloud that fully utilise the power of the Cloud.

These are only two examples of how businesses want to use the Cloud that don’t involve SaaS, and there are many more. You don’t have to look far to see there is lots of great work and innovation going on around Cloud. So, how come I keep coming across the suggestion that the Cloud isn’t being adopted fast enough and how this is down to traditional vendors of on-premise software deliberately slowing down the adoption of Cloud?

Well, more and more I wonder if Cloud adoption is actually being hindered rather than helped by those who believe they are trying to promote Cloud by demanding that “if you want to be Cloud you have to be SaaS”, and that you have to be prepared to share your “sensitive business” information in the same database as others whether you like it or not.

I’m not normally one for controversy, but… “Demanding you have to be SaaS to be Cloud is a crazy suggestion”… even if it was sensible, and I for one don’t think it is, it would take decades to convert the plethora of applications that businesses use today to SaaS applications, by which time the cloud will have passed us by… and we’ll be onto the next major trend.

So, technocrats … you know who you are … put your soap box away and give the Cloud a chance.

Is this the future, is #Apple defining the future here??

Is this the future, is this type of web store the provisioning model of the future, is Apple defining the future here?

Apple has already changed the music landscape with iTunes and the iPod, they have changed the mobile landscape with the iPhone and they have changed they have unshackled workers from the office with the iPad. Are they now changing the landscape of the PC user? Does this put a firm brake on the notion of some (I wasn’t one of them) that the PC, and applications running on the PC are dying?

Does this seamless on-line provisioning disperse with all of those age old headaches and arguments about on-premise CD’s and time consuming installations? Is the appeal of this limited to consumers or will it appeal to the business community as well?

Just as others have followed Apple into the new world of music, mobile and tablet computing. Surely the question now is not whether, but how long before, others follow suit?

SaaS and Cloud… one in the same or completely different???

I was intrigued to see over on Accountingweb a whole list of replies to what on the surface was a very simple question … “What is cloud Computing?” the rationale behind this question was also simple:

Once we agree the definition of cloud computing, then we can have a more productive discussion in 2011!

However, what followed were a whole series of answers, some good, some not so good, and some way off topic as is the norm for an open forum. I’m not criticising any of the respondents here, to the contrary many different views are healthy.

What did emerge though was a really interesting sub debate (and some confusion) as to whether the Cloud and SaaS are one in the same thing?

Richard Messik suggested this very simple definition of Cloud:

The definition of Cloud Computing has changed somewhat over the years but I still think that the simplest one is the “Hotel” Test… if you can walk into an Hotel lobby anywhere in the world and access your data and applications – without the need to download any plugins etc – then that is truly Cloud Computing.

Now, this sounds completely feasible until you read this definition of SaaS by John Patterson :

Next time you are on holiday, walk into the hotel lobby and log on to your application using whatever machine and browser they have. If you can access all the data and all the functionality in your SaaS application immediately, without having to download any extra software, it’s a true SaaS product.

The close similarity between these statements might lead one to believe that Cloud and SaaS are indeed the same thing, but some great insight is provided by Duane Jackson, he comments:

SaaS and Cloud are not synonyms… the terms SaaS and Cloud are not interchangeable. SaaS is a software delivery and business model. Cloud isn’t. I used to think SaaS was a subset of Cloud (with P[latform]aaS, D[ata]aaS, etc being other subsets – along with a load of other stuff that isn’t anything as-a-service). Now I’m not so sure. It depends on how cloud is defined. KashFlow is definitely a SaaS application. I’m not sure if it’s “cloud” though. It’s all on servers in the cloud. But it’s dedicated hardware. To some, cloud computing must involve virtualised hardware and elasticity. We don’t have that.

My twopenneth on this for what it’s worth… is that I agree with Duane, and disagree with Richard here… in that SaaS and Cloud are not one in the same. However, the market is being conditioned (confused) by some SaaS vendors who are leveraging the Cloud bandwagon for the benefit of their own product. I’m not necessarily saying there is anything wrong with that, after all that is what Marketing departments are paid to do, but I admire Duane’s honesty for calling it as it really is here. 

The Cloud is much much bigger than SaaS and the majority of applications running in the Cloud today are not pure play SaaS, unless of course you limit your thinking to one or two SaaS vendors ‘private’ clouds.

SaaS applications (some but not all) tick the cloud box, but so do many AppStore type mobile applications that access information and services on the cloud. Then you have the online gaming industry who arguably make much more use of the cloud today than the business sector. 

No doubt this debate will run on for some time… and if you have a different view I’d love to hear it?

In the meantime, next time you are on holiday, don’t go into a hotel lobby and try this… grab a beer and chill out…

You can only come to my cloud party if…..

I lost count of the number of times I heard or read that intro during 2010,

You can only be Cloud if you’re a pure play SaaS application. You can only be Cloud if you run multitenant. You can only be Cloud if you run public cloud. You can only be Cloud if you’re application independent. You can only be Cloud if you run in a web browser. You can only be Cloud if you don’t use smart clients. You can only be Cloud if you aggregate data. You can only be Cloud if… you get the picture

During the holidays, I had a couple of conversations on the difference between “sell side” and “buy side” perspectives from industry players. Now, if I asked the question, which side are the above statements coming from? The answer is easy; they are all sell side statements originating from SaaS vendors. How many customers would ask for any of this stuff?

So, I went in search of a more balanced view and picked up a tweet pointing me at this post on the hidden value from SaaS deployments from Ray Wang

Here Ray talks about the traction of SaaS and discusses seven characteristics of SaaS applications and I’d like to pick out just one for this post… the user experience:

Richer user experience – SaaS apps bring Web 2.0 usability to the enterprise world through rich internet applications using Adobe Air, HTML 5, Microsoft Silverlight, and other tools.

This struck a chord as it highlights a buy side customer need for Rich Internet Applications (RIA). It also begged the question, “could a technology, such as Adobe Air or Silverlight that requires a download to the customer’s device be classed as SaaS?”

Around the same time I read this entartaining post from Martjin Linssen who suggests that having to download Silverlight to use the Cloud was a big no no in SaaS terms.

I had a brief follow up twitter conversation with Ray, but we didn’t reach a conclusion…

Coincidentally, just before Christmas I’d been having another twitter conversation, this time with Gary Turner on whether a smart client, in this case a particular RIA that is found in the on-line game World of Warcraft (hence the picture) could be classed as a SaaS application? Despite a hefty initial download WoW runs in a web browser and connects millions of users (12 million to be precise) to a farm of servers (some of the most powerful machines in the world) in the Cloud?

Our conversation went along the lines of:

If it wasn’t desktop software and it wasn’t cloud software, then what was it… again, we didn’t reach a conclusion…

At the end of the day to the 12 million players of WoW, you might ask does it really matter? as long as the experience they get makes the gameplay enjoyable… and the more entranced in the game they are, the happier the vendor (Blizzard Entertainment) $$$ kerching.

Now that’s an example of where buy side and sell side are completely aligned…

2010 blog review

The stats helper monkeys at WordPress.com mulled over how this blog did in 2010, and here’s a high level summary of its overall blog health:

Healthy blog!

The Blog-Health-o-Meter™ reads This blog is on fire!.

Crunchy numbers

Featured image

A helper monkey made this abstract painting, inspired by your stats.

A Boeing 747-400 passenger jet can hold 416 passengers. This blog was viewed about 1,400 times in 2010. That’s about 3 full 747s.

In 2010, there were 20 new posts, not bad for the first year! There were 16 pictures uploaded, taking up a total of 2mb. That’s about a picture per month.

The busiest day of the year was October 8th with 85 views. The most popular post that day was Maybe the so called dinosaurs (desktop vendors) of the industry aren’t so long in the tooth after all….

Where did they come from?

The top referring sites in 2010 were twitter.com, linkedin.com, accmanpro.com, bnet.com, and lmodules.com.

Some visitors came searching, mostly for stuart lynn, stuart lynn blog, airprint compatible printers, airprint, and sage ebis-xml invoice.

Attractions in 2010

These are the posts and pages that got the most views in 2010.

1

Maybe the so called dinosaurs (desktop vendors) of the industry aren’t so long in the tooth after all… October 2010
4 comments

2

When the industry is crying out for standards, why reinvent the wheel? October 2010
12 comments

3

My beloved iPad will never replace my beloved Laptop November 2010
4 comments

4

Softworld change or die… ?? October 2010
4 comments

5

The world is changing and the web is changing too! October 2010
3 comments and 2 Likes on WordPress.com