Posts Tagged Agile
I spent an excellent couple of days this week with a group of CIO’s from some of the largest companies across Europe and one of the hot topics we debated was around adoption of the Cloud… what’s the business case for cloud? who’s driving the cloud agenda? are businesses ready for cloud? which applications are best suited to cloud? and what’s the general feeling around the public and private cloud debate?
It was refreshing to have such an open debate without the hype and cloud washing by cloud vendors, and despite the fact there were a few cloud vendors present, they were politely asked to stop selling and start listening to what businesses really need:
- The smart business is much more interested in using the cloud to drive business benefits and increase revenue as opposed to saving costs. (the vendor argument that the cloud reduces your operating cost simply doesn’t matter if it doesn’t help business growth).
- The much publicised CAPEX vs. OPEX argument was also dismissed as a benefit as most felt it easier to secure CAPEX funding as opposed to OPEX.
- The speed and agility of procuring cloud servers and services and the elasticity of increasing compute and storage capacity around demand peaks were seen as attractive benefits.
- Getting data (not necessarily applications) to the cloud is seen as a positive move as it opens up a new range of business opportunities around collaboration, e.g. customer/employee self-service and supply chain digitisation (what a fantastic word!).
- System integration and interoperability has replaced security fears as the biggest concern around cloud adoption… getting disparate systems to work together remains as one of the most difficult businesses issues and it is felt the cloud could complicate this further.
- Whilst there was a bias towards private cloud, businesses were open to adopting all types of public, private and hybrid cloud on the basis that a “one size fits all” model is unlikely to suit all enterprise business models.
- It was generally accepted that using the cloud for consumer applications is much more popular than using the cloud for the Enterprise today. The few cloud usage examples shared were all consumer facing operations of the Enterprise as opposed to back office operations.
I was very impressed with the level of knowledge and debate at this session and I have to conclude by saying Enterprise business leaders definitely understand the cloud, they see through the cloud wash, and they are more than capable of deciding how and when to adopt cloud in their business.
A few weeks back I wrote this post on How to make Agile work for Product Development. As a build on that post I’d like to share how I’ve used Agile to successfully deliver ‘Business’ challenges outside of Product Development.
Consider this scenario, there is a problem in the business and despite many efforts to solve the problem the situation continues to get worse. I’m sure you’ve come across this before and it normally happens when a lot of people are involved in solving the problem but they aren’t working for a common purpose, they are trying to resolve the symptoms rather than the cause, and very often their efforts cancel each other out or make the problem worse.
So how can Agile help?
Well the Scrum process works just as well in this situation as it does for Product Development; here is the process I’ve used:
- Establish a Framework: Break the problem down into its component parts, identify the deliverables and the priority of each deliverable. Sound familiar, yes? This is our backlog.
- Identify Clear Focus and Accountability: Chunk the backlog up into a number of time bound sprints, identify clear accountability for who’s going to lead each sprint, who’s going to work on each sprint, and what needs to be accomplished in each sprint.
- Trust the process and the people: Let the sprint team get on with it. If you’ve given accountability to the right people then you don’t need to do their job. Instead provide support and help them by removing any barriers and keep distractions away.
I know what you’re thinking… Does, this mean you’re abdicating accountability? Absolutely not, you still have accountability for the entire thing and you need to be there if the sprint team need you. Does this mean that you won’t know what’s going on? Absolutely not, Scrum is an open process… anyone can drop into a sprint meeting to hear what’s going on, but remember not to interfere if you’re not part of the sprint team.
As with Product Development, It’s a good habit for the sprint team to use their Friday review meeting for more formal progress update, i.e. if you have a short project with one week sprints, this will be the end of sprint retrospective meeting, if you have two weekly sprints then this would be the mid sprint review, and so on. Personally, I favour Friday as it’s a good way to celebrate progress at the end the week and to retain focus on what the plan is for the next week. This is also a good technique to kill two birds with one stone, e.g. if you need to provide visibility to an external or higher level sponsor.
This really is a very simple and effective process which I’ve used this countless times to very good effect. However, as with Product Development, the formula for success is all down to having a clear framework that everyone understands, a strong focus always, and good people who are willing to take accountability.
There is of course a fourth ingredient to the formula that I haven’t mentioned here, anyone like to tell me what it is?
Around the same time that I wrote my last post “How to make Agile work for product development… It’s all about the people” a colleague sent me a link to an interesting and insightful post by @yeeguy on “How Facebook Ships Code”
I found this post fascinating and I’d recommend you take a few minutes to read it. So why not just retweet the link? Well, I wanted to add my own observations. On whether Facebook have turned Agile into some sort of Ultra Agile concept or if this is more akin to Anarchy?
As an engineer myself, it’s interesting to hear that 50% of the Facebook workforce work in engineering and Operations:
The two largest teams are Engineering and Ops, with roughly 400-500 team members each. Between the two they make up about 50% of the company.
So, they are heavily investing in the platform which is a good, or is it? I have to tell you that when I look at Facebook I can’t help wondering what a team of this size is actually delivering, unless of course one assumes they are working behind the scenes on the next generation. Anyway, let’s not get hung up on that, when you value the company at $50Billion you can afford to set fire to a pile of cash along the way. This next bit is even more interesting:
Very engineering driven culture. ”product managers are essentially useless here.” is a quote from an engineer. Engineers can modify specs mid-process, re-order work projects, and inject new feature ideas anytime
WHAT!!!!!!!, now whilst I openly profess that developers should take accountability for their own work, if we take this insight on face value what Facebook engineers are doing here is not Agile, it’s scary, very scary, especially if it’s pervasive across such a large workforce… I can only assume that anarchy prevails in parts… or that for every piece of code that makes it to the live environment a bucket load of wasted development goes down the drain.
I would also strongly contest the statement that “product managers are essentially useless here.” On the grounds that if people think that, and unfortunately I’ve met many engineers who do, then they don’t understand the function or purpose of Product Management.
In my experience this normally stems from an internal power struggle whereby the Product Manager believes they call the shots, and are opposed by an Engineering Manager who believes they call the shots. I’ve even heard Product Managers say, “we are the conscience of R&D” sounds great doesn’t it… erm no!!! A “we are watching you comment in any guise” says loud and clear that we don’t trust you and it kills relationships. My stern advice to these types of people on both sides would be to grow up and get over yourselves… My belief is that Product Management isn’t about the role it’s about the function, in exactly the same way that Development Management isn’t about the role it’s about the function.
However, all is not lost with Facebook, as another quote seems to support Product Management:
Product managers have a lot of independence and freedom. The key to being influential is to have really good relationships with engineering managers. Need to be technical enough not to suggest stupid ideas
I really like the good relationships piece, albeit the last bit seems to make this somewhat of a backhanded compliment. Whilst I believe there is no such thing as a stupid idea, I also believe that Product Managers and Development Managers need intimate knowledge of their customers, their platforms and their organisation… if they have this then they work really well together and you tend not to get stupid ideas.
So all that said, someone somewhere in Facebook must believe in Product Management, because get this:
Product manager to engineer ratio is roughly 1-to-7 or 1-to-10
Wow, for a company that doesn’t seem to value Product Management they sure have a lot of them on the payroll. From my experience American companies tend to have a higher ratio of Product Managers to developers than European Companies, but this seems high even by that standard.
Facebook sounds like an interesting place to work, I wonder what the holidays are like?
Some background before I get into the detail, I’ve been developing small, medium and large ‘shrink wrapped’ products for the past 12 years and before that I worked on large multi-million pound in house projects.
Why do I give this background? Well it’s to set context in relation to this post from @Martijnlinssen where Martijn questions whether Agile can be used effectively for product development. I read into Martijn post that whilst Agile might work for project development it doesn’t work as well (if at all?) for Product development. I chose to disagree with Martijn on this point and responded with the comment:
I’m not sure from reading this that you actually get agile (sorry Martijn).. Whilst I agree Agile is not for everything, Agile can be a highly effective way to deliver software (and even commercial business) projects.
What it isn’t is throw all control in the bin and fly by the seat of your pants… What it is is a model whereby accountability lies with the dev team and not some project manager miles away from reality of day to day decision making.
The controls are very adequate and personally I’ve used it on dev teams of 80 or so people… Based around scrum of scrum hierarchy … However, that said I also like high level governance in place for really complex implementations.
We debated this via twitter some more and a few more comments were added to the original post. I think we got to the stage where we agreed on some parts but not all, i.e. Martijn came up with some very interesting follow up questions/concerns:
How much money gained during dev is squandered in test and maintenance, compared to waterfall?
When does the benefit of not writing everything down turn into a concern?
How is the transition made from build to test to product for maintenance?
Can you just outsource your finite product once Live?
So, I’m writing this post to share my thoughts (and learning’s) around these questions. I say leaning’s as I have the very same concerns in the past. If you don’t know what Agile is then look here for a very basic overview of Agile.
Now, if you just took the overview and went off to develop products you’d be in a big mess in no time at all. In my experience, there are some other very key components that are essential for success. Assuming that you have your development backlog, velocity and your sprint schedule defined, I’d advise that you also need:
- A solid system architecture in place before you begin – my approach is to deliver the blueprint and prove it out if necessary with a proof of concept (POC) in Sprint Zero… you won’t see mention of sprint zero in many (if any) Agile books, but it’s my preferred way to prime the Agile process. Start building without this and you’ll hit dead ends time after time and end up reworking huge chunks of the product.
- Design patterns and framework for the user experience and interaction, i.e. it’s no good if different parts of the product look, feel and work differently. Usability is absolutely key and should be built in from the start as opposed to painting it on at the end.
- Good coding guidelines around solid engineering principles such as defensive programming and more recently test driven development, without these it’s very easy to write poor quality code the costs a fortune to maintain using Waterfall or Agile.
- And the secret ingredient… A team of highly proficient people who understand how customers work and what problems the Agile stories are trying to solve… I can’t stress how critical it is that developers and testers understand the needs of the customer, otherwise they just build what they are told and we all know that’s a recipe for disaster. It’s also a good idea on complex systems to supplement the developers and testers knowledge with highly specialist domain experts (Business Analysts) for things such as compliance and complex processes.
Some of the techniques I really like in Agile are daily meetings, which are fantastic to keep everyone on track, the burn down charts to show you exactly where you are, and the retrospective reviews, which are a brilliant way to look at what you’ve just done and how you did it and improve it for next time around. All of the best athletes do this, the military do this, and it’s a great way to continually learn and increase the capability of the team.
So, coming back to Martijns questions…
How much money gained during dev is squandered in test and maintenance, compared to waterfall?
None of it if you follow the principles above… and if you structure the sprint teams correctly you can save money by being more effective, i.e. the formula I use is:
Developers and Testers working together as part of a 6 person scrum team, normally to a ratio of 3 devs, 2 testers and a scrum master (Team Leader)… this means that you test as you go, can fix issues as they are found and deliver better quality software at the end of each sprint.
For larger systems, I also prefer a separate system/integration test team that don’t work as an agile team. This team doesn’t retest everything, they specialise on integration, regression and beta testing.
As for maintenance I don’t use agile per say for that either. I prefer a separate sustained engineering team that are dedicated to respond to any issues found post release quickly and efficiently.
If I compare this to my experience of waterfall, where all the specs are written up front by BA’s who understand the customer, handed to development to code up who don’t understand the customer, then passed to testing for QA who try and piece things together. It’s much less efficient and makes no attempt to build quality in as you go, and if the project is a long running development there’s a real chance the requirements have changed since the BA wrote the original spec. The system is out of date the minute its delivered.
I’ve seen many more waterfall projects fail than I have Agile, generally people are more connected with what the goal is and what needs to be done to ensure delivery.
When does the benefit of not writing everything down turn into a concern?
Great question, of course nobody says you don’t have to write anything down, you document the system in a number of ways, first you have the architectural blueprint and the interaction patterns, then you document complex business rules and compliance and finally you document in the code. From my experience, documentation is used to define and not to maintain, i.e. I’ve never met a developer who goes back to original spec to fix a problem, the best ones read the code and trace the problem through a debugger.
How is the transition made from build to test to product for maintenance? Can you just outsource your finite product once Live?
Okay, so the sprint teams deliver running code, by running code I mean it is developed, working and tested within the team. For multi-team and multi sprint projects each sprint deliverable is then assembled (using a versioned build process) and handed over to the system test team who check all the component parts delivered by the sprints work together. This takes the form of an initial smoke test, thorough integration testing, and full regression test to ensure the new deliverable hasn’t broken the system. Any bugs that are found are captured and these are fed into subsequent sprints, or more likely that you have a one or more bug fixing only sprints only once you’ve reached feature complete lockdown…. This may also include a period of external beta testing, but the process is the same.
When the bugs have been resolved the product deemed fit for purpose, it is versioned and shipped.
So then you start working on the next version of the product and repeat the process from the top. In the meantime you have to sustain the product you have just released. It’s even more important here that the sustainability team understands the customer and the domain. Whilst Agile could be used here to produce a scheduled service pack which contains a defined body of work, Its not good for producing a single hot patch which are normally a few days in duration.
Can you outsource the product once live?…
I don’t, but it’s possible as long as the sustainability team has a certain amount of product/domain knowledge and can read the blueprint and the code. I’ve tried this in the past based on specs only and it fails miserably, even with waterfall, it fails miserably… as at the very least you need a link back to someone who understands the customer and the domain.
Agile is about people… and not about the Project Manager
All of that said… the most important part of Agile for me is Ownership, Accountability and Empowerment. In a high performing Agile team, the traditional Project Manager feels uneasy and disempowered, they may feel out of control, as the team are empowered to make their own decisions. The old style command and control ‘tell do’ Project Manager will never succeed with Agile and will fight tooth and nail against its adoption. Whereas the new breed of Project Manager who can trust his people and continually develop them through coaching, will soon reap the benefits of delivering much more through their people than they ever could through command and control.
How do I know that?
This is my own story… I am that person.