Case for starting with longer sprints

At last something reasonably sensible here, from a big Indian services company regarding Scrum. I say reasonably, because the conclusions arrived at are largely correct, whereas a couple of points made in support reveal an underlying thinking which is slightly questionable. I shall proceed to answer the Q of an “ideal” sprint length before returning to take apart the article referred to above.

Most Scrum teams use two week sprints and a few even one week sprints.  In some organizations where iterations have been about 3 months, or more in length or even worse a classic waterfall (Before Scrum) teams have a real problem starting with two week sprints. Much of this is in the mind. It is common for people in such projects to feel (or even ‘know’) that nothing can be achieved in two weeks. So, there is considerable reluctance to pick a two week sprint. Now the question is what should be the length of the first few sprints?

Something of a storm in a tea cup, is how I see it; There is no doubt in my mind that a two week sprint (with suitably small units functionality as backlog items) is what the doctor ordered. However in these times of diminished authority, quite a few people seem to duck clinical instructions, let alone heed good advice. So, a logical (at least superficially) decision is made where in a much longer sprint length is chosen.  If it were of four weeks, that is still within the rules of Scrum, but not a good choice. I’ve even seen a two month sprint taken up as how to proceed with Scrum, which  really is a marvelous illustration of how NOT to do Scrum. Basically the underlying problem is once again the lack of realisation that Scrum is really a rather radical departure from the traditional waterfall way of completing projects. Not an embellishment of the old ways of working with the latest jargon, minus the real intent, which in turn simply results in Scrum-but.

So why is a long sprint three weeks or more a problem?

1) Loss of feedback (course correction opportunity): Essentially there are fewer opportunities for the PO and functional stakeholders to get to know the growing system and provide feedback.

2) difficulty of holding on to the “NO CHANGE RULE”, particularly the part where the sprint goal has to be held fixed; i.e no changes to the stories picked up by the team at time of SPM (sprint planning meeting) are to be allowed, is increasingly difficult to maintain as the length of sprint increases.

3) Effectiveness of retrospective: One of the most important environmental conditions for a good retrospective is that people should remember what happened during the sprint. The only other even more important condition is openness and assurance of safety. Most people cannot recall many (unremarkable, but important) details beyond last week. That’s a two week span. So once a sprint is longer than two weeks, we have a memory retention problem, unless the team maintains a detailed activity log. Yeah right!

4) Reduced planning effectiveness: As the time horizon increases, it becomes more difficult to plan for the entire timebox. It is substantially more difficult to plan a longer period of activity realistically than a short one. A weakness of long term planning ability exists in most software project groups.

In fact all the above means, it is imperative that teams improve their ability to plan (a major underlying factor is estimation, but that is another long story), make progress, correct and then build upon what has already been achieved.  Basically do the “inspect and adapt”. So in a way more opportunities the better. Hence a shorter cycles are better.

OBERVATIONS on the Cognizant article

(Overall I think the article is well written and has many aspects covered well)

Pl refer to table in the article.

A flawed line of thinking here is “Adaptation to uncertainty in story creation”, which talks about stories crossing sprint boundaries. This assumes that stories are going to be large. Actually they can be quite small. Short cycles can be handled if the organisation and team learn to break functionality into sufficiently small blocks (here is an interesting thought on this matter). This is a skill that needs to be developed, and time after time many people have discovered that the most complex application can be broken down into block that need a couple of people to work on it for a couple of days. So ideally for a two week sprint a team of about seven people about 8 to 15 stories (or sub-stories, if you prefer) can be picked up.

An argument made by armchair planners against short sprints is that the overheads will take up a greater percentage of time as a whole (as possibly being hinted by the above article; see 32,16,11 and 8 days). However for shorter sprint the planning, review and retrospective times will also come down;  Of course time spent arranging for these meetings may go up, since number of meeting per month will increase. However this is a matter of habit, if everyone soon gets used to weekly meetings (each meeting of shorter duration) this will actually be a negligible overhead, with a tremendous amount of learning and correcting being generated. So a shorter sprint is better than a longer sprint.  So item 1 “Ceremonial days” in the table isn’t quite right.

The other (more worrying) point made is that about procrastination;  Use of sprint burndown chart correctly will actually circumvent this! In fact very long sprints may just increase pressure on the team since there is a real possibility that team will struggle to meet sprint goals towards the end of a sprint. There is a real danger of falling into a serious waterfall mode every long sprint. Teams don’t seriously examine their way of working and change to get as far away as possible from the waterfall. In fact in this matter the article seems to lie on a resource (read people) utilization perspective and the worry that people are not going to be busy all the time, easily confusing activity for progress; which is fairly anti-Scrum in approach.

Another issue with the article is the point about “minimum number of days spent testing”! This is really puzzling.  Is less “testing” days better, or worse!?! I’ve no idea what this article considers better, let alone what the thinking behind this is. In reality it depends heavily on the context of the project, the abilities of the team members,  automation involved (rightly or wrongly)… so many ifs and buts. In fact Scrum does not even attempt to prescribe this.

So the main root point in favour of a long sprint is the comfort of the team and managers involved coming from a waterfall habit! This is really a question of willingness, understanding and realization of the people involved in the project. Not something to be sneered at, maybe a coach can help.  He/She must show sensitivity to this, and do all possible to show the team how to break down user stories into smaller blocks of functionality.

In closing…everything else points to shorter sprints. A two week sprint is strongly recommended and indeed used by many, many successful Scrum teams.

PS: The points made above may seem like a merciless slating of a deeply flawed article, but not so, the article itself makes many good points from a certain level of Scrum understanding.  I think it analyses the various factors of making the sprint length decision fairly well. Here I am simply attempting to provide a deeper understanding of how to think through this matter.

Advertisement

The VOC, but who is the customer?

A recent post encouraging proactive steps to gather the VOC leaves the issue of “who is the customer?” well alone, since a generic answer would be vague, it is heavily context sensitive. Many good ideas are presented here.

An intriguing point made here is that, these are solutions to a situation “…particularly where there is little time to do any early user research”. You must consider what is underlying such a train of thought carefully.   I hear this sentiment sometimes and it is almost surely leading to building of the wrong product! So lot of activity will ensue, but progress in terms of generating ROI may is certainly not ensured! In other words it is a case of leap before you look. This is not to say what’s written in the above mentioned post is wrong, or that it is recommending ignoring user research. It just is trying to work around a poor decision.

As usual since my blog is about Scrum,  here I’ll try to relate this situation with the product owner’s role. To cut a long story short, it’s the PO’s responsibility to bring the VOC into the project. Identifying and engaging with customer(s) or potential customers and the difficult balancing act that may be needed is in the PO’s court. Again it is difficult to provide general advice that is practical but it certainly helps to make as much activity around this as visible as possible. The transparency of such interactions will mean a very significant reduction in dysfunctional behaviour. So as far as the teams are concerned the PO is the customer, albeit,  one who follows the rules of Scrum.

Sprint Review undone: Part-I

I was recently present at an attempted Sprint Review Meeting for a team I was coaching. They were struggling to get finished by the time of the meeting, and by the appointed time could not get the functionality ready. The problem was that they had picked eight items, and could not even demonstrate one item, and this was their second sprint! They had earlier done a sprint zero, with many of it’s attendant problems which are described here. Now my overall  analysis of how this situation came to pass is at two levels:

1. Poor habits carried over from the waterfall days.

2. Lack of prioritization within the sprint.

Over and above this is the matter of a somewhat diluted DoD (a separate topic in itself) which is bad news being stored for the future. This team was working with a weak DoD, which was seemingly clear to them (For some one who reads tea leaves in his spare time, this is an ominous sign)

In this post I’ll explain the  first point. Within the sprint, each story was done strictly in a waterfall manner, including hand-offs. This not withstanding all the training the team and managers were given. Which meant, that the burndown chart, while warning about bad news, actually did not quite manage to reveal the really bad news. That nothing was actually ‘done’ ( this does not mean the team did not do anything). Suddenly towards the end of the sprint 4  user stories  were 90% done, and as the old saw goes, the remaining 10% takes the other 90% of time. The other 4 items were in even lower levels of completion. However what was the actual completion is moot.  Even more importantly, trying to find out the true completion percentage is missing the point. The team at the time of the sprint review, should try to keep the done/undone as binary as brutally as possible. Try to see it, standing in the PO’s shoes.

So what is to be done? Avoid waterfall. How? Especially given that the team is already facing too much change for comfort (One reason why many teams don’t see successful sprints early on).  Assuming they haven’t changed their technical practices yet (very common for teams starting off on Scrum), the first step is to try to break-down user stories to even smaller units of functionality, whose completion should be staggered across the sprint. Another approach that can be simultaneously used is  to write acceptance tests as the stories are picked up. This means that the team can test the stories (to some extent) at a very early stage, hence testing early and (hopefully) often.  A very good idea.

The above ideas are a reason why teams aiming to succeed have to change their ways of working and learn to breakdown stories and change the approach to testing among other things.

In the next post I’ll explain the other practical step to take (if you haven’t guessed already).

Handling changes in Scrum

Srinivas is the co-author of “Essentials of Scrum practice“- a mini-book
currently in a draft stage. This is an extract from the book.

 

All s/w development projects have greater or fewer changes to be incorporated after the start of development. So how are changes handled in Scrum? and more importantly why are they handled in the manner described/recommended by Scrum?

Basically in Scrum the top few items of a product backlog (the highest priority ones) are chosen for any given sprint. The remaining items, can be modified by the product owner as he/she pleases at any time. As an aside, there has been some recent talk about the backlog not being prioritised, but ordered. I’m less than impressed by that explication.

Since Scrum is supposed to take care of changing functional/non-functional needs over time, the germane question is “How do we handle changes within a sprint/iteration?” in other words “how do we handle changes to the sprint goal?”. This is a recurring question, and the answer lies in correctly understanding the intent of Scrum.  Scrum’s intent is to provide a means, an island of stability where a complex project can be run smoothly, even though the environment around is in flux. This island of stability  is ensured by the “NO CHANGE RULE”. Which means no change in duration, goal or team for any given sprint.

See the following post for the reasoning on keeping the team fixed. Here I’ll essentially cover how and why of keeping the goal fixed.

There are two cases which cover changes to sprint goal

1. Urgent bug-fixes which may turn up at any time, hence cannot be foreseen at the time of the Sprint Planning Meeting (SPM)

2. Change requests to the selected product backlog items

These two cases are to be handled in radically different manner. For the first case the team can inspect and adapt to choose a means of dealing with bug fixes, by either keeping a percentage of available time (say 10 to 15%)  aside as contingency or by rotation having one or two team members on stand-by to handle the bug-fixes as and when they turn up. In other words this is being prepared for the unpredictable. The second case is severely discouraged and is  handled by not taking it up in the current sprint! This may seem extreme, but there is sound reasoning underlying this approach.

First and foremost, if change requests are allowed at any time within a sprint, there is no line drawn as to how much of change is allowed. Given human nature this will result in any amount of change being allowed any number of times with any sprint. What is the implication of this on the usefulness of the SPM? We are very possibly on a slippery slope by allowing a change in sprint goal, mid-sprint. Essentially, by having this rule, we are creating an island of stability.

Secondly, the reason we have sprints with items being pulled into any given sprint, is to accommodate change. This means at a project level change to functionality is handled in an orderly fashion as well as a welcoming manner.

Thirdly, and this will be difficult to stomach for some stakeholders,  a significant amount of change is due to lazy thinking early on. In other words quite a few of the change requests are a result of not thinking through what is needed. While it is recognized that the PO/customer cannot always know everything that is required in detail, up-front, what this rule does is encourage reasonably detailed analysis of the items which are at the top of the backlog. When the PO is confronted, with a situation where he/she/it (How PC am I becoming!?!) has thought of a change and has to wait till the next sprint, this is a stimulus for more careful thinking before and during the following SPMs.

The encompassing principle above is that Scrum is a system that encourages corrective behaviour from the stakeholders, while maintaning a balance between accommodating change and reducing avoidable waste due to lazy change.

Two further points:

1. Scrum does NOT need (in fact discourages)  to have the functionality described in every detail at the starting of the sprint! Then, how in the blue blazes are we supposed to keep change at bay, as clarifications are made during the sprint!?! There is another simple rule in play “If in doubt, it is the team that decides whether it is a change or clarification”. In case of clarification the team simply implements, in case of change, it gets put away for following sprints.

2. What if there is an utterly, unavoidably, compellingly, the-sky-is-falling-on-our-collective-heads, URGENT change of sprint goal, in the middle of the sprint!?! Either a change to an item picked in the current sprint, or an altogether new item added.

Well, indeed if it is THAT important, follow Scrum by the book!

i. Terminate the sprint

ii. Inform all stakeholders

iii. Immediately start a new SPM for a sprint, of what-ever length, that makes sense for the explosive circumstance. Does one need to do a full SPM (of 4 hrs?), not really. Exceptional circumstances can mean that just the new/changed item is to be dealt with in the planning.

Any loss of time/money/delay due to this decision is made in full knowledge by the PO.

Again note, that this is a highly disruptive, but necessary change, and is widely visible. Not to be undertaken lightly. After all if there is a serious earthquake, the sprint goal will change.

Manager! Leave the team alone!

We don’t need no disruption
We don’t need no time control
No breathing down the neck in the server room
Manager, leave the team alone.

What do we do if a senior manager is asking for a ‘resource’ from within the team, in the middle of a sprint, to deal with some urgent matter?

As with a lot of things, this is better preempted, by informing as much of the surrounding organisation as possible before the start of a Scrum project, that we have to keep a team constant for at least the duration of a sprint. In other words the organisation should understand that the ‘NO CHANGE RULE’ is critical and central to the Scrum way of working.

However, senior managers may not pay sufficient attention to this or feel that whatever is occupying their minds at the moment is of utmost importance, and they can pull ‘resources’ (actually people) out without too much consideration. It is the Scrum Master’s job to protect the team, and as such does throw the Scrum Masters into a very difficult situation. The organisation culture will be a dominating factor in how people (SM included) behave. In many organisations the hierarchical structure combined usually with a command and control culture means that Scrum Masters are either powerless in face of such disruption or get into a downward spiral of de-motivation. This effect should be highlighted after the sprint end, to the management including any effects on how the team struggled to meet the sprint goal.

The Scrum Master in such circumstances should talk to the team and PO, and if applicable to the PO’s manager (who could be the senior manager in question) about the side-effects of pulling someone out mid-sprint.

Organisations where everyone is well-educated about Scrum, will seldom face such a situation without being able to make corrections and recover fast. This situation is particularly difficult in organisations which are largely ignorant of Scrum and the attendant principles. So it is important to respond to this situation without using jargon in such settings.

Speak in terms of disruption, de-motivation and relative importance of the sprint goal versus the work the pulled out ‘resource’ will do, versus the importance of adhering to the rules of the game. Such a conversation should also guide the persons making this decision to think in terms of ramifications of such requests and induce an inclination for senior managers to talk to the PO and SM before making any such requests/orders to pull out people, in the future.

All this engenders a culture where disruptions are kept to a minimum and only for good, genuine reasons. A relevant point here is the case when in response to a management edict coming from much higher-up, and in essentially a knee-jerk reaction, someone is pulled out even before it is really clear what exactly needs to be done. In other words the person pulled out cannot really begin work, since certain clarifications are still being sought. In such cases the useful practical steps to take consist of the Scrum Master taking a close look, spending sometime to discuss this with relevant team member, so they are prepared, wait till the end of the Sprint, so there is minimal disruption. Then after the sprint ends, the team can be reset, with a freshly updated backlog, which may include this ‘urgent’ piece of work; alternatively the person in question will not be part of the team for the new sprint, and works independently.