Staying lean!
Posted by Marcel Offermans in Uncategorized on 21-10-2009
A while ago I posted here about the role of the architect when building lean software. Inspired by posts by Kirk, Hal and Joel, I’d like to build a case for selecting the right tools and frameworks in order to be able to sustain your agility in the long run.
Starting with the opening argument that Hal makes, we’ve all been in projects for long amounts of time, where at one point you start wondering what became of that once beautiful and comprehensible design, or those fast build and test cycles you started out with. As products evolve over time, they all seem to loose a lot of their initial “elegance” up to the point where people seriously start considering starting from scratch “because it’s easier than bolting on new features to the existing product”. Apart from the fact that that is very counter-productive from the point of view of the customer, you’re also throwing away a lot of experience, especially if the original authors of that code are no longer around (which sometimes is stated as a reason to start over). Joel made a case for not starting from scratch and raises some very valid points there.
So, if it’s not smart to start from scratch, and you really want to sustain that agility, how are you going to make that happen? There, going back to one of the points that Kirk makes, a lot depends on your architecture, especially if you look at it from his point of view:
The goal of architecture is to minimize the impact and cost of change. To do this requires we create flexible systems with the ability to adapt and evolve. But with flexibility comes complexity, presenting architects with a paradox. As we design more adaptable systems, the additional complexity reduces their ability to survive. Eventually, the software rots, the architecture crumbles, and the system fails. Therefore, we must find a way to increase adaptability without decreasing survivability.
Combining observations here it becomes increasingly important to stay modular and to really try hard to loosely couple your components because that has tremendous benefits in the long run. Doing that is not easy though, as it requires a lot of thought, and you might need to make some other trade-offs in favor of modularity and loose coupling. However, it is one of the few ways that allow you to make decisions that are always reversible, as ripping out a single component in an environment where there are as little dependencies as possible and where each of these dependencies have small and substitutable interfaces remains as easy as it was when you started the project. In practice it also means that it is worthwhile ensuring that these qualities in the overall architecture do not deteriorate and that you spend enough time making sure they don’t by refactoring and reviewing code as part of the normal development cycle.
Scrum at University
Posted by Wout Lemmens in Development Process, scrum on 28-08-2009
A lot of companies are currently developing software in an agile way. Quite often universities stay behind in technologies and methodologies that are taught with respect to what’s hot in the working field. Luckily quite some universities do try to keep up with the ‘real world’, and try to deliver graduates with up-to-date knowledge that fits the needs of companies.
At Avans University of Applied Sciences in ’s Hertogenbosch, the project ‘Game Design’ was started, and the idea was to apply an agile development process to introduce this to the students as well. This project had a duration of about 14 weeks, fixed six-person teams, and quite some availability of all team members each week. Besides that, the lecturers were willing to play the role of Product Owner, preventing the students from creating whatever they liked. Good starting point of a project to apply Scrum to.
And as luminis has quite some experience in applying Scrum at various client projects, I was asked to give a lecture on the Scrum basics, and share some of our experiences.
Just before the summer holiday the project was finished, and the following experiences were the result:
- Applying Scrum was quite hard in the beginning. They didn’t meet every day, and especially in the beginning they were working on large tasks like the design document and getting familar with technologies. It was difficult to split that up into smaller pieces.
- In the second part of the project (more focused on implementation), the students were able to apply Scrum in a better way. Even though they didn’t meet at a daily basis, they were able to make well estimated tasks, and execute these in a one-week sprint. This provided a clear overview of the project and the progress. And the detailed planning worked out great.
Concluding: as an introduction to Scrum, it wasn’t a real success applying it in the first phase of the project (sprint 0 type of work). For the students it felt quite artificial, and they did not experience any benefits. On the other hand, looking at the second phase (implementation) of the project it was a success as they really experienced the usefulness of practices like detailed planning and weekly deliveries.
Result; the invitation for next years ‘Games’ project already stands.
The right people are not afraid
Posted by Angelo van der Sijpt in Development Process on 23-08-2009
Quite some time ago, I read Jaibeer Malik’s excellent writeup on the right kind of people for staffing you agile project. I totally agree with his analysis that you need people that are “willing to make it happen”.
The right kind of people for your project have another trait in common: they are not afraid. What is there to be afraid of?
- Change It is in the nature of people to prefer it when things stay the same. Yet, change is the lifeblood of agile processes: we changed to get agile, we update our own process, and just when you think we figured it out, we change again. Better get used to it.
- Failure As John Holt wrote in Celebrate your mistakes, “If you’re not making mistakes, you’re not taking risks, and that means you’re not going anywhere. The key is to make mistakes faster than the competition, so you have more changes to learn and win.” Sure, failing is no fun, but fearing it will freeze you, making sure you can never make things happen.
- Visibility Tying into failure, visibility is about showing what you do. People tend to be very protective about “their job”, which they know bloody well how to do, thank you very much. Still, the open communication in an agile project forces us all to show what we’re doing, every minute we’re working on the project. I think it’s inexcusable to say “Well, I’m working on it now, so just come back tomorrow, and I’ll show you then”. Just show me now, I understand that not everything is OK yet; I would be surprised if it was OK, and you were still spending time on it.
The Agile Manifesto contains the same elements of collaboration, change and interaction. These can, indeed, be kind of intimidating when you are used to signing of work to be done, working under someone else’s responsibility, and consider yourself an expert. Then again, remember that we’re in this together, as a a team. Stop being afraid, and start making things happen!
Things the books don’t tell you
Posted by Premek Brada in Development Process, scrum on 18-08-2009
It seems there are things which somehow managed to bypass the attention of Scrum book authors. In this sequel to my previous post on Scrum Luminis edition, I’ll try to sum up a few pieces of advice which I’ve picked up during my Luminis “learn Scrum” stay and which may come handy to other Scrum beginners.
Project bootstrap (context = fixed-price contracts):
- Initialize backlog – use RFP, client-supplied requirements doc, customer talks, whatever.
- Sketch hi-level design with functionality blocks, so you know which blocks will handle what requirements.
- Optionally re-phrase requirements to increase information value (if client-supplied is incomplete and/or technically too weak).
- From all these sources, create product backlog.
- Set up the team – either you have your own people (easy
or the team gets assigned by client; in that case the Scrum master must work hard to get the right people for the job.
As a corollary, there is no formal project finalization stage—it is done as normal work planned for intermediate/final sprints.
Sprint goals and release not achieved
The sprint release is defined as “potentially shippable” ie not necessarily “deployed live”. The Scrum master or product owner sometimes needs to make sure there is a “product champion” on the client side who will push towards complete transition into daily use.
Also, it may happen that not all sprint goals / scenarios are achieved by sprint end. Whether this upsets the product owner depends on lot of things—priority of the unfinished, how team performed otherwise, etc. Should be however spotted and dealt with during the sprint, not only at demo meeting.
Of course, if team fails to meet high-priority needs (and spends sprint time on easy low-priority stuff) then they will, and should, have hard time at end-of-sprint demo and retrospective meetings…
Where are the bug reports?
Bugs tend not to appear in sprint backlog as stories / tasks. They mainly pop up during “in progress” or “testing” state of a task and since they are easy to repair (remember the benefits of unit testing and small increments?) they are dealt with straight away by the developer—why bother with bugtracker overhead when the bug is gone by sundown. (This is basically true even if a bug slips through after the task is “done” and surfaces sometime later.)
Or they are on functional / acceptance level and thus presumably coarser-grained. In this case they get caught by the product owner and put into backlog as improvement / rework items or possibly as reasons for re-opening a bugtracker item.
On re-planning…
Common reason for re-negotiating during planning meeting are resource constraints (not effort estimate mismatch): “For all these things to implement, we would need 5 Java engineers but there are only 3 on the team. Let’s shuffle things a bit so we can match work required with people available.”
For all changes of the original sprint plan, resulting from daily standups, you better have them documented (e.g. on sprint wiki page) – provides great traceability between burndown chart hiccups and the reasons behind them.
… and (non)estimation techniques
As one of the Scrum masters put it, “The best teams I’ve seen don’t do explicit estimates” – they use gut feeling when planning the sprint. The point is that when all is said, team commits to their plan and it does not matter how they arrive at that plan.
Yet it takes 2-3 sprints for the team to arrive at the level “we know how much we can take”, and it took Luminis about a year to get from detailed estimation and time logging to the story point and gut feeling estimates.
Scrum Luminis edition
Posted by Premek Brada in Development Process, scrum on 18-08-2009
First, a short background: I’m a university lecturer, and software development process has been core of my teaching as well as practical duties for the last 10 years or so. Jumped on the agile bandwagon couple of years ago, and this year got a small grant to stay a week with Luminis in order to see Scrum at work—with the aim to transfer (some) of the knowledge into my courses.
This post is a summary of the differences between the “book edition” (Schwaber & Beedle, Kniberg’s “From the trenches”) and “Luminis edition” of the Scrum process.
Obviously, Luminis have been doing Scrum for upteen years and know their job well. Obviously still, no Scrum incarnation is the same by definition—there are the non-prescribed things you must find your way through, and there are the times when local context makes you breaking the rules.
So let me state: all core stuff prescribed by Scrum is there and there’s no point reiterating them in this post. The differences follow.
Local tweaks and optimizations
These are the things which Scrum says “decide on your own what suits you”. So Luminis have:
- Sprint length mostly shorter than 30 days – one project used 5-week sprints, one converged towards 2-week sprints. The driving factors are complete product integration needs (other teams’ development pace) and higher ability to embrace change.
- Sprint backlog maintained in Jira – not as Scrum board post-its or similar – and contains literally anything that has to be done, including daunting (“refactor”, “document”) and management (“prepare for demo”, “scrummaster”) tasks. Enables traceable project and change management, documentation, and off-site customer visibility.
- Some cherry-picked things from Rational Unified Process (RUP) are used, e.g. the need for product vision, use cases with their template, quite detailed architectural documentation along the 4+1 views (but kept as Wiki page to stay agile
) - Explicit “requirements analyst” and “system architect” roles – both are managed by the product owner and available to the team to provide details into backlog items and implementation; analyst often staffed from client’s side, architect as team member. This offloads some burden from the product owner who can concentrate on other things, see below.
- Requirement analysts and system architects run 1-2 sprints ahead of team – so that they can feed the backlog early enough.
- Planning and estimation is simplified – done in pairs and wrapped-up on whiteboard (no task cards and poker), normal hours often used in estimation (not necessarily scenario points), standard “priority” field used for backlog items (not relative importance value).
Notable diversions
Here comes the real interesting stuff. You could call these “violations of Scrum” if you wanted but then listen to the local circumstances which drive the process modifications. Roughly in decreasing priority:
- There is no well-established product log – its stories get listed as sprint goals but not picked up from a backlog list. Also, sprint goals are composed as a list of scenario names (ie not a single sprint goal + a list of scenarios). This is acknowledged as current process weakness, OTOH it does not seem to have negative effect on project’s convergence to their goals.
- Product owner has a more managerial role in addition to the prioritization duty – is responsible for project’s success from client’s point of view and is not “requirements analyst” for the team. His role entails mainly acting as a firewall to take on disturbances and “brilliant new ideas” from client, managing requirements analysts and architects, and perform stakeholder management i.e. negotiate/interface with client management.
- Consequently, product owner need not be responsible for product backlog management – basically explains/discusses goals, priorities, hi-level requirements with scrum master who then maintains the backlog.
Put concisely, the product owner makes sure the project is effective (in business terms), Scrum master makes sure the project is efficient (in sprint velocity terms).
This is perhaps the most interesting point of Luminis’ Scrum edition. Their experience is that Scrum masters – staffed by more junior members – may not be perceived by client’s management as “senior enough”. This would in effect block their ability to firewall the team on a sufficiently high level, influencing client management goals and local politics (and boy do these factors steer the projects—most issues which hamper work aren’t technological but concern human nature and communication).
The product owner’s role is therefore changed. They must know client’s “board members” needs—the timeline and priorities they live in (“outside world”). Staffed most often by a Luminis senior level member, their experience makes it easier to align internal development with client’s processes, and their rank plus the fact they are outside client’s internal politics make it easier to do/say certain things (e.g. negotiate with top-level client management, make “impossible” things happen).
The parting words
I’d like to thank the guys from Luminis for being very helpful, open and friendly—it was a great week
Daily scrum token
Posted by Jorg Hollenberg in scrum on 03-08-2009
In my team I introduced a toy football as a token in the daily standups. The ball is intended for energizing the daily standup. The teammember who receives the ball should answer the 3 questions:
- What did you accomplish yesterday?
- What are going to do today?
- Any problems, bottlenecks, impediments?
And pass the ball on the next member of, you should really pay attention, you might be the next to get the ball!
Bruised and battered, I left it with the team to take care of it. In retrospective:
- age: 24 months
- cost: $ 3
- served in 2 internal en 2 customer teams
- thrown by 35 people (didn’t count vandals trying to steal the ball)
- number of throws (est.) : 6620 x
- rescued from the bin (overactive facility department): 2 x
- misused by 1 Software architect and 1 engineer (don’t ask me what they did with it, it makes me cry)
Now I am in search for a new token….any ideas out there?
Lean Software, the role of the Architect
Posted by Marcel Offermans in Architecture, Development Process, scrum on 29-07-2009
When we first started doing Scrum a couple of years ago, there was some discussion about the role and place of the architect in an agile process. Some of the initial literature we read did not mention an architect at all. Building and delivering software in short sprints seemed to encourage a very hands-on approach to software development, trying, building and refactoring in a very iterative way.
However, developing in sprints did not mean that we could all just “dive in” and design as we pleased. On the contrary, by adopting a process where the requirements can change every sprint, it becomes even more important to have some kind of architecture that is just as flexible. In fact, it is the ultimate test for any architecture: a couple of sprints down the road, how easy is it to go in a different direction, to really change core aspects of the system?
Some important lessons we learned here are:
- Build your software out of reusable, loosely coupled components. Components can easily be refactored, rewritten or thrown away with consequences that are easy to oversee.
- In the age of libraries and frameworks, be very selective about using big frameworks, especially if you only use 20% of them. Usually at some point, the added complexity of the other 80% will bite you.
- Have components communicate using service interfaces that are as “small and simple” as possible, and implement them in the simplest way that makes sense. Like with performance optimizations, don’t prematurely optimize until you know something is a problem.
- Prototype the most critical or complex components in your application. This is where the experience of senior engineers and architects can really make a difference.
- Keep things simple. For example, building and deploying an application should be as easy as possible. I’ve seen too many cases where you had to follow a long list of steps to get “something that works”. Besides, in your continuous build, you want to produce working artifacts automatically, on every build.
In the end, it’s the role of the architect in this process to make sure the whole team shares the same technical vision. This can be achieved by working closely with the team (an architect should really also have all the engineering skills to talk with any member of the team and really understand what they’re doing), explaining the important aspects of the decisions that were made. It makes sense to document the architecture and design a wiki, using any means necessary, such as taking photos from whiteboards. Make sure these documents are actively read and maintained by the whole team. One thing you definitely should not do is sit in a room for a couple of months and write “the architecture document” in splendid isolation. That is the best way to ensure it will never be read or used.
To Scrum or not to Scrum, or ScrumBut?
Posted by Wout Lemmens in Development Process, scrum on 15-07-2009
“We are doing ScrumBut… means you aren’t doing Scrum. Not doing Scrum means you likely won’t get the benefits other teams that are doing Scrum have claimed.”, says Dennis Stevens in his blog.
This is often interpreted as there is only black and white. You should either Scrum and stick to all the key principles, or don’t Scrum at all. But doing ScrumBut does not mean you don’t get any of the benefits of Scrum at all.
Quite some organizations still run projects in a traditional, non-iterative, non-agile way, without having roles with clear responsibilities, and defining large tasks with only few details. For these organizations it’s quite hard to fully adapt Scrum, and thus it should be introduced carefully. Downside to this is, not all key-principles can be followed from the start.
But does this mean that it doesn’t deliver any benefits? Not at all!
As described on the ControlChaos website: “ScrumButs are reasons why they can’t take full advantage of Scrum to solve the problems and realize the benefits.”. Having ScrumButs only results in not taking full advantage of scrum. Some examples from the field:
- Team
For the current project, the team consists of two full time members, and one part time member. This is rather small for a Scrum team, but the project isn’t bigger, neither is the budget. And even with this small team, we have a daily standup and a retrospective each sprint, which is really valuable to us.
- Two week iterations
We defined iterations of two weeks. Making it smaller did not give us the feeling we could really deliver anything, but making it larger results in being less agile than we want in this project. The specification and roadmap are not clear enough to determine what we’re going to pick up in the near future.
- Scrumboard
We have a paper Scrum board with post-its. This makes it possible, even with a team of three members, to pick up tasks from the highest prio sprint backlog item. For us it really makes clear who is working on which task, and what the overall progress is.
- Burndown
Each post-it has an original and remaining estimate in hours. This is used to daily update the burndown. Even estimating in hours (task points or equivalent would be better) tells us if we’re reaching the sprint goals in time or not.
- Testing
Each tasks done by an engineer is being tested by another engineer. Testing a task includes checking: code, functionality, unit testing, documentation, running the application (and perform some simple actions just to check deployment still works) and test against the real system as well (not only the test environment).
And that’s in fact our current definition of ‘done’.
- Specification
The first specification was a memo with just some requirements listed. This was accompanied by a first set of user stories, and later on also UI and interaction design was added. Before each sprint (preferably at least one sprint before starting on the actual work), it should be clear what to develop. All dependencies to other people and systems should be identified and taken care of before starting the sprint. In practice, not all impediments are resolved, but we’re trying to deal with this in an agile way as well.
The items above are principles we try to apply, but quite often can’t apply to the full extend. Still, applying ScrumBut provides a deliverable each two weeks, allows us to inform the product owner of any delays in an early stage, gives an overview of the work in progress, provides focus and commitment, and also of great importance; we’re having fun in doing our job.
The hardest part for people new to Scrum, is getting used to this way of working. It makes everything really transparant, from the engineers to the management. It’s really a new way (or at least a different) of working, but once used to it, you’re going to like it.
Concluding:
Scrum provides you with quite some benefits. ScrumBut provides less benefits, but isn’t bad at all.
Devnology Open Spaces session: an inspiring bunch
Posted by Angelo van der Sijpt in Uncategorized on 07-07-2009
Last Saturday, I visited the Devnology open space session, hosted at VX Company in Baarn. I am not a .NET guru, but I have an opinion about everything, so I went there for the agile theme.
Open spaces is the agile shape of a conference: we start out with no agenda whatsoever, and anyone who has a good idea puts it on a post-it. If you’re interested in a session, note this in some way, and we end up with an ad-hoc planning for the day. More information about open spaces can be found at Open Space World.
Some highlights that caught my eye:
- ‘Scrum’ and ‘agile’ are definitely the hot thing in town. Lots of companies have some sort of agile process, and most call it Scrum. Although I am rather skeptical of using to-be-signed-off requirements documents (I believe this hampers agility) and teams that enforce silence in the office (I think this will give some productivity boost in the short run, but I also think it harms learning, communication, and perhaps even team spirit), I’m glad to see there is so much interest. We might really be on to something here…
- The next hot thing in the agile nation, Kanban, was virtually unheard of.
- Pair programming is not as hot a topic as I thought it would be. We have been using pair programming for quite some time now, with great results (I won’t give you the numbers, other people have gathered those way better than I can). Several people I spoke either wanted to try it but never had the chance, or are outright against wasting two people to do a one-man job.
- Open space sessions are highly self-selecting: all people I spoke have a very clear opinion, and are willing to discuss anything that comes along. It’s been a long time since I found so many interesting people in the basement of a big corporation on a Saturday.
All in all, a very inspriring day. It always fun to be surrounded by enthousiastic people, discussing the thing we all strive to do best: create great software.
Kanban for a week?
Posted by Angelo van der Sijpt in Development Process, kanban, scrum on 01-07-2009
Starting this month, the development budget for my current project has run out. Our last sprint ended little over a week earlier, and with two-week sprints, we did not feel comfortable planning and starting a new, much shorter sprint. Instead, we switched to a different mode of operation, working in a Kanban-like fashion, fixing minor annoyances and small bugs all around.
For our team, the transition was remarkably smooth. We did away with planning, story points and committing to sprint goals, and replaced it with a list of issues (each roughly the same size that we had in our sprints) and a priority set by the product owner. All the practices we already had in place have stayed the same: we still use pair programming, have an interaction designer on board, and only declare an issue closed when at least one other pair of eyes has looked at it. We did not even have to alter our scrum board, we just removed the headings we used for sprint goals.
With the fixed deadline coming up, we do not plan big features, and keep the product continuously in a shippable state: each issue we handle is one in and of its own, and not part of a bigger feature. If we cannot finish an issue, we just don’t, and nothing gets hurt. This allows us to cope with the interruptions that such a last week brings, without having to worry about leaving the product in an inconsistent state.
All in all, it has worked out perfectly for us. If you are looking at a fixed deadline, and have a few days left that don’t make up an entire sprint, you should try it!
Recent Comments