Feeds:
Posts
Comments

Archive for the ‘Agile’ Category

Software Estimates

“We will ask for estimates and then treat them as deadlines.”

Estimates are typically a necessary evil in software development. Unfortunately, people tend to assume that writing new software is like building a house or fixing a car and that as such the contractor or mechanic involved should be perfectly capable of providing a reliable estimate for the work to be done in advance of the customer approving the work. With custom software, however, a great deal of the system is being built from scratch, and usually how it’s put together, how it ultimately works, and what exactly it’s supposed to do when it’s done are all moving targets. It’s hard to know when you’ll finish when usually the path you’ll take and the destination are both unknown at the start of the journey.

I realize that estimates are a hard problem in custom software development, and I am certainly not claiming to be the best at producing accurate estimates. However, I have identified certain aspects of estimates that I believe to be universally (or nearly) true.

Estimates are Waste

Time spent on estimates is time that isn’t spent delivering value. It’s a zero-sum game when it comes to how much time developers have to get work done – worse if estimates are being requested urgently and interrupting developers who would otherwise be “in the zone” getting things done. If your average developer is spending 2-4 hours per 40-hour week on estimates, that’s a 5-10% loss in productivity, assuming they were otherwise able to be productive the entire time.

A few years ago, a Microsoft department was able to increase team productivity by over 150% without any new resources or changes to how the team performed software engineering tasks (design, coding, testing, etc.). The primary change was in when and how tasks were estimated. Ironically, much of this estimating was at the request of management, who, seeking greater transparency and hoping for insight into how the team’s productivity could be improved, put in place policies that required frequent and timely estimates (new requests needed to be estimated within 48 hours). Even though these estimates were only ROMs (Rough Orders of Magnitude), the effort they required and the interruptions they created destroyed the team’s overall productivity.

Estimates are Non-Transferable

Software estimates are not fungible, mainly as a corollary to the fact that team members are not fungible. This means one individual’s estimate can’t be used to predict how long it might take another individual to complete a task.

The transfer ability of estimates is obviously improved when the estimator and the implementer have similar experience levels, and even more so when they work together on the same team. Some techniques, like planning poker, will try to bring in the entire team’s experience when estimating tasks, ensuring estimates don’t miss key considerations known to only some team members or that they’re written as if the fastest coder would be assigned every task. This can help produce estimates, or estimate ranges, that are more likely to be accurate, but it does so by multiplying the time spent on estimating by the entire team’s size.

Estimates are Wrong

Estimates aren’t promises. They’re guesses, and generally the larger the scope and the further in the future the activity being estimated is, the greater the potential error. This is known as the Cone of Uncertainty.

Nobody should be surprised when estimates are wrong; they should be surprised when they are right. If estimates were always accurate, they’d be called exactimates.

Since smaller and more immediate tasks can be estimated more accurately than larger or more future tasks, it makes sense to break tasks down into small pieces. Ideally, individual sub-features that a user can interact with and test should be the unit of measuring progress, and when these are built as vertical slices, it is possible to get rapid feedback on newly developed functionality from the client or product owner. Queuing theory also suggests that throughput increases when the work in the system is small and uniform in size, which further argues in favor of breaking things down into reasonably small and consistent work items.

Estimates of individual work items and projects tend to get more accurate the closer the work is to being completed. The most accurate estimate, like the most accurate weather prediction, tells you about what happened yesterday, not what will happen in the future.

Estimates are Temporary

Estimates are perishable. They have a relatively short shelf-life. A developer might initially estimate that a certain feature will take a week to develop, before the project has started. Three months into the project, a lot has been learned and decided, and that same feature might now take a few hours, or a month, or it might have been dropped from the project altogether due to changes in priorities or direction. In any case, the estimate is of little or perhaps even negative value since so much has potentially changed since it was created.

To address this issue, some teams and development methodologies recommend re-estimating all of the items in the product backlog on a regular basis. However, while this does address the perishable nature of estimates, it tends to exacerbate the waste. Would you rather have your team estimate the same backlog item, half a dozen times, while never actually starting work on it, or would you rather they deliver another feature every week?

We know that estimates tend to grow more accurate the later they’re made (and the closer they are to the work actually being done). Thus, the longer an estimate can be reasonably delayed, the more accurate it is likely to be when it is made. This ties in closely with Lean Software Development’s principle of delaying decisions until the last responsible moment. Estimates, too, should be performed at the last responsible moment, to ensure the highest accuracy and the least need to repeat them.

Estimates are Necessary

Despite of all the drawbacks, estimates are often necessary. Businesses cannot make decisions about whether or not to build software without having some idea of the cost and time involved. Service companies frequently must provide an estimate as part of any proposal they make to build an application or win a project. Just because the above words are true doesn’t magically mean estimates can go away. However, one can better manage expectations and time spent on estimating if everybody involved, from the customer to the project manager to the sales team to the developer, understands these truths when it comes to custom software estimates.

Conclusion

If you’re in a position where you want a reliable estimate for a software project, and you’re having a hard time getting one from your developer/team, remember this quote: “You can’t find someone who knows how long this will take, but you can probably find someone who will lie to you.”

Essentially: The more difficult it is for you to get an estimate, the more likely it is that when you finally do, it’s not terribly accurate.

Advertisements

Read Full Post »

Recently organizations are embracing DevOps which is a great thing. However the whole adoption is causing a lot of confusion as well. Some of you might have heard the term “Agile and DevOps”. With that it looks like Agile and DevOps are different. To over-simplify further people assume Agile is all about processes (like Scrum and Kanban) and DevOps is all about technical practices like CI, CD, Test Automation and Infrastructure Automation.

This is causing a lot of harm as some organizations now have Agile and DevOps as two separate streams as part of their enterprise Agile transformation. Agile by definitions disrupts silos and you see, in this case people are creating new silos in the name of Agile and DevOps.

With that background in mind, let’s try to understand what exactly DevOps is all about:

  • DevOps is mainly the widening of Agile’s principles to include systems and operations instead of stopping its concerns at code check-in. Apart from working together as a cross-functional team of designer, tester and developer as part of an Agile team, DevOps suggests to add operations as well in the definition of cross-functional team.
  • DevOps strives to focus on the overall service or software fully delivered to the customer instead of simply “working software”.
  • It emphasizes breaking down barriers between developers and operations teams, and getting them to collaborate in a way where they benefit from combined skills.
  • Agile teams used automated build, test automation, Continuous Integration and Continuous Delivery.
  • With DevOps that extended further to “Infrastructure as Code”, configuration management, metrics and monitoring schemes, a tool chain approach to tooling, virtualization and cloud computing to accelerate change in the modern infrastructure world. DevOps brings some tools on the block as well like configuration management (puppet, chef, ansible), orchestration (zookeeper, noah, mesos), monitoring, virtualization and containerization (AWS, OpenStack, vagrant, docker) and many more.
  • So you see DevOps is not a separate concept but a mere extension of Agile to include operations as well in the definition of cross-functional Agile team, collaborate together and work as one team with an objective to delivery software fully to the customer.

Creating separate Agile and DevOps horizontals in any organization just defeats the whole purpose (removing silos) of DevOps.

Read Full Post »

Measuring Agile Maturity

There are many efficient way to measure the maturity and performance of an agile team and one of them is by measuring cycle time. In a traditional project, we have the project cycle time, which is the time between start of the project and the final delivery or release. In an agile project, the cycle time can be measured on the level of user stories.

Lean: The above reasoning is supported by two main principles of a lean mindset:

Reducing cycling time is one very important principle of a lean mindset: we get faster feedback and test results, we prevent work from clogging our development queues and we improve morale because the team sees that the work ‘disappears’ faster from the backlog.

‘Avoiding waste’ is the primary objective of a lean approach. Any wait time between the different process steps can be considered as waste. Reducing the wait time can therefore be considered as a good way to get rid of ‘waste’, and also reduces the overall cycle time.

How to measure: The following image shows a simplified view of the life cycle of a user story in an agile project, including the potential wait times:

Measuring Agile Maturity 1.gif

The cycle time is the time between ‘Ready to Start’ and ‘Release’. This time can be significantly reduced by reducing the wait times (WT1, WT2 and WT3).

Depending on how we organize our sprint backlog, we can start by noting on each user story card the date on which the story became ready for development by our agile team. This might be the start date of our sprint.

As soon as we meet all the ‘definitions of done’ for that particular user story, note the actual date again on the story card. The difference between the two dates is the user story cycle time.

How to calculate the average cycle time: At the end of each iteration, calculate the average of the cycle times and divide the result by the average complexity of the user stories of the sprint.

The result of this division gives us the average cycle time of that particular sprint. The division by the average complexity is necessary to standardize on complexity.

Over time, when our agile team matures and becomes more efficient, we should see a decrease of the average cycle time. If this is not the case, then we are doing something wrong, and there are probably some bottlenecks in our process that we want to resolve.

Advanced mode: Cycle time variance: Reducing cycle time is one thing, but what about cycle time variance?

Let’s see these two different cycle time-related metrics on a graph:

Measuring Agile Maturity 2.png

The red curve show the distribution of cycle times of team 1, whilst the green curve shows the distribution of cycle times of team 2. The green team has a higher maturity level because of two reasons:

  • The mean/average cycle time CT2 is smaller than the mean/average cycle time of team 1, CT1.
  • The standard deviation of the measured cycle times of team 2 (B) is smaller than the standard deviation of the cycle times of team 1 (A).

This means that over multiple sprints or projects, team 2 is able to produce better predictable output.

Why cycle time is the right indication for maturity:

  • It focuses on the final goal (customer value) and not on the process (the customer does not care).
  • It is very easy to collect the relevant data.
  • It is a very simple and understandable metric.
  • We do not need additional tools, only a pencil and a calendar.

Read Full Post »

Every project has a certain level of complexity in it. When we say a project is simple, we actually mean that its degree of complexity is very less or can be considered as negligible, but nevertheless, it does exist even if in a minute magnitude. Project management methods deal with project complexities depending upon how flexible they are, and what kind of provisions they offer to deal with them. Tradition management methods such as Waterfall are often rigid owing to their staged working processes, which are also often irreversible. Using pre 1990 era methods, one can try to address the complexity in a project to a lesser or greater degree of success, but typically such management methods do not make it possible to reduce or minify the actual level of difficult in executing the project and neither its complexity. This is not necessarily true in case of Agile. With Agile we can actually try to control the level of complexity in a project provided we have the correct level of experience in implementing Agile principles and techniques.

What is “Complexity” in a project?

Broadly speaking the term ‘complexity’ can be best understood as difficult or complicated conditions arising due to the availability of multiple options, or options which make us simultaneously focus upon different directions at a given time, and which result in a multi-dimensional scenario that is hard to understand and resolve. Complexities can be of different types in a project. Business level complexities arise due to uncertain market conditions, technological advancements, and other such factors which affect the business logic contained in the project. The project level complexities can be of two types: Project complexity and Requirements complexity.

Project complexity

Project related complexity can be different for different types of organizations. Several factors contribute to it; however, the most important ones are uniqueness, limiting factors, and uncertainty.

Uniqueness: Every project is unique and has its own attributes and requirements. As the project commences it gains maturity over time and benefits through the learning process. This is most significant when the organization is running a project which is the first of its kind, or if it has no prior experience dealing with projects.

Limiting factors: Projects are subjected to certain factors which can affect its execution or commencement such as budget constraints, the technical knowhow of the team, working schedule, and at times even cultural differences.

Uncertainty: Uncertainty in a project can be due to external or internal factors. External factors may include government imposed rules and regulations, uncertain market changes, and fluctuating economic climate. Internal factors may comprise of the levels of management’s participation in the project, constantly changing company policies, stakeholder’s involvement in the project, etc. All these factors affect the scope of the project.

It is a known fact that a project’s complexity affects its success. The manner in which a business anticipates, fully understands, and addresses the complexity determines whether a project is going to be successful or not.

Requirements complexity

Requirement analysis is the journey to discover the ‘unknowns’. It is an understanding of the business problem, needs and what it takes to address them. Requirements complexity is defined by two key factors: The level of ‘unknowns’ and Volatility.

The level of ‘unknowns’: At the start of the project, how much is ‘known’ about the problem statement? How much is known about the business processes? The level of ‘unknown’ must be assessed at a very granular level, particularly pertaining to business rules, systems, functions etc.

Volatility: What is the expected level of requirements volatility once the project is launched? ‘Volatility’ in requirements emerges due to frequent changes, starting from the design phase all the way through implementation. Project management methodologies often assume that, when requirements move onto the design phase, they stand ‘complete’ and are not subject to change. However, that is not always the case as there is always some level of uncertainty and unpredictability. Requirements volatility leads to significant risk and its consequent uncertainty.

Can Agile reduce complexity?

In traditional project management methods, the complexity in a project is often managed by investing a certain amount of time in the analysis phase with the sole objective of analyzing the levels of complexity and making plans to deal with it. It is based upon the assumption that the time invested in the analysis activity will help to reduce complexity and increase the chances of developing a successful project by using various methods and processes. The investment in time should be considered worthwhile since the analysis can help managements to make informed decisions.

Unlike traditional project management methods, in Agile there are no special stages to deal with project complexity. The product owner who is responsible and who oversees the entire project tries to address the complexity levels based upon his or her experience in the subject, in addition to what the team can contribute in terms of efforts and suggestion to deal with complexity. However, there is a big plus point in how the Agile process works and how the inherent product incremental model makes it possible to reduce project complexity to a great level.

In Agile we don’t work with the entire project at any given time rather we select a few important features and develop them in short bursts of activity known as sprints. The time spent in developing the features varies from team to team depending upon the team’s level of maturity and its hold over the technology used for developing the project. A project may appear to be complex when its overall complexity is considered, but in Agile since we don’t have to deal with the entire project during the sprints, the levels of complexity can be easily addressed to by estimating the levels of difficulty in the features and developing them individually. That way we only encounter a fraction of the actual complexity, which can be easily tackled by the team.

Project complexity is inevitable and should be acknowledged to enhance the team’s ability to respond and adapt to change while staying focused on the end objective. Agile practices and methodology promote the capability to drive and manage change through an understanding of the inherent complexity in projects.

Read Full Post »

It is pretty incredible how often we complain about our best employees leaving, and we really do have something to complain about — few things are as costly and disruptive as good people walking out the door.

We tend to blame our turnover problems on everything under the sun while ignoring the crux of the matter: People do not leave jobs; they leave their managers.

Here manager does not mean only the boss or the owner. For a junior developer it can be his senior one. For a senior developer it can be lead developer. For a lead developer it can be development manager. For a development manager it can be scrum master or product owner.

The sad thing is that this can easily be avoided. All that is required is a new perspective and some extra effort on both ends.

Let us discuss few things that send good people packing:

Overwork: Nothing burns good employees out quite like overworking them. It is so tempting to work your best people hard that we frequently fall into this trap. Overworking good employees is perplexing; it makes them feel as if they are being punished for great performance. Overworking employees is also counterproductive.

New research from Stanford shows that productivity per hour declines sharply when the workweek exceeds 50 hours, and productivity drops off so much after 55 hours that you do not get anything out of working more.

If we must increase how much work our talented employees are doing, we better increase their status as well. Talented employees will take on a bigger workload, but they would not stay if their job suffocates them in the process. Raises, promotions, and title-changes are all acceptable ways to increase workload.

If we simply increase workload because people are talented, without changing a thing, they will seek another job that gives them what they deserve.

Not recognize contributions and not reward good work: It is easy to underestimate the power of a pat on the back, especially with top performers who are intrinsically motivated. Everyone likes kudos, none more so than those who work hard and give their all.

We need to communicate with our people to find out what makes them feel good (for some, it is a raise; for others, it is public recognition) and then to reward them for a job well done.

Not care about employees: Study shows that more than half of people who leave their jobs do so because of their relationship with their boss. We need to balance being professional with being human.

We should celebrate an employee’s success; empathize with those going through hard times, and challenge people, even when it hurts. If we fail to really care then we will always have high turnover rates. It is impossible to work for someone nine-plus hours a day, five days a week, fifty two weeks a year when we are not personally involved with each other and do not care about anything other than your production yield.

Not let people pursue their passions: Talented employees are passionate. Providing opportunities for them to pursue their passions improves their productivity and job satisfaction. But sometimes we want people to work within a little box. Sometimes we think that productivity will decline if we let people expand their focus and pursue their passions. This fear is unfounded. Studies show that people who are able to pursue their passions at work experience flow, a euphoric state of mind that is five times more productive than the norm.

Fail to develop people’s skill: Management may have a beginning, but it certainly has no end. When we have a talented employee, it is up to us to keep finding areas in which they can improve to expand their skill set. The most talented employees want feedback — more so than the less talented ones — and it is our job to keep it coming. If we do not, our best people will grow bored and complacent.

Finally, I want to use one line to summarize the whole discussion: We want to create such a team that everyone wants to join.

Read Full Post »

Why The Who Matters Less

Theory of the perfect team

Google is known for being obsessed with studying their own performance and then adjust accordingly. At one time Larry Page decided to get rid of all the middle management but then quickly realized that the increased amount of direct reports slowed the company down immediately. So, they brought them back. With its Aristotle project Google wanted to figure out what are great teams made of. At the beginning, they focused all on the individual skills of the team members and their performance but the results were sobering. Having all superstars on the team did not necessarily made a great team. After hundreds of teams studied, a pattern emerged. In all the high performing teams the members felt psychological safety. That means they were not scared to admit an error and talk about their ideas. Furthermore, these teams had all empathy for each other (social sensitivity) and equal voices, which mean that everybody’s opinion is heard when a decision must be made. This does not mean democracy, still somebody can make the decision but people feel that their opinion is considered. For our Scrum teams that means, it is more a cultural challenge rather than a skill challenge and the culture should incorporate those values:

  • Psychological safety
  • Social sensitivity
  • Equal voices

A popular thought for hiring a new employee regarding this: Hire for fit not for skill.

Multifunctional teams

A frequent question I get is: Does multi-functional team mean everybody on the team has to be an expert in everything? This is probably due to 2 facts:

  • We want the team to be independent with a few dependencies as possible to the outside. Therefore, we need all skills inside the team.
  • We advertise that there are no more specialists rather everybody has to be able to help out with other tasks.

Now if there is for example a data base expert on the team, it does make sense that he helps with the database issues but if he has idle time he should help out other team members. On the other hand, when there are a lot of database related tasks and he becomes the bottle neck others should be able to help him out as well.

The approach of A-teams (small engagement units) from the Special Forces is this: Each team member is cross trained. That means he is usually an expert in 1 or 2 specialties e.g. medic, radio, etc. but is also able to do general tasks in other areas. They have what is called T shaped people, a person that is expert in 1 or 2 areas but has general knowledge about all the other topics. Furthermore, every specialty exists twice (in case you lose the medic, there is still another one).

How can we apply that to our Scrum teams? First, it is important that all necessary skills are on the team to be independent. Then, we want every expertise twice, for example 2 persons with highly developed database skills. But we want also T-shaped people that can work on other parts as well (not only their expertise).

Bringing it all together

In sum, it is not so important to have superstars on the team rather than how the team is operating. The number one indicator is psychological safety followed by social sensitivity and equal voices. Secondly, culture has a huge impact on team performance. Even though, culture is probably the most difficult thing to change, moving to a commitment culture is one the most performance improving steps. Lastly, when it comes to who is on the team it is important to have T-shaped people that do have the necessary skills to work independent from the outside. However, they do not have to be superstars.

Takeaways

  • Define team ground rules that lead to psychological safety, social sensitivity and equal voices.
  • Implement a commitment culture.
  • Create a team of T-shaped people covering the necessary skills.

Read Full Post »

Organize Agile Team

We always feel like our teams could be organized better. How to organize teams in an optimal way is a common question in Agile organizations. A question we should always discuss and answer together with the people in the actual teams.

Here I will try to provide an overview of 5 possibilities for organizing teams and the main factors to take into account. Our main considerations should be based on product complexity and maturity, as well responsibility, coordination efforts and sustainability. Depending on our specific context one will suit our situation better than the other, or maybe we will come to the conclusion that we are so Agile we can leave the concept of team all together.

Component based teams: Component based teams focus on specific components in the product that do not deliver customer value in it. They are also sometimes called focus teams for that reason. The focus is usually necessary because of the use of very specific or exotic techniques. Learning the skills and expertise to work in these teams takes years so it is more logical to put experts together. We can compare this kind of teams with functionally specialized teams.

Feature based teams: Feature based teams focus on a specific customer feature or feature areas within a product. For example: search optimization in a web-shop, or payments and conversion. Feature teams are necessary if the product is too large for a single team to optimize or when a single larger base product is differentiated into various market segments in retrospect. For instance, the same web-shop product can be differentiated for selling kitchen appliances, or tailored for selling TV sets.

Product based teams: Product teams carry responsibility for the entire product or product/market combination. This means all of the skills and expertise necessary to develop and run the product reside within the team. Product teams are therefore suited for smaller products with a relatively small feature set. That is why we often see these types of teams in start-up settings. When developing our new product, we do not want it to be complex and we will have a limited feature set if not a single one we want to validate first. Once the product matures, team orientation eventually changes towards feature teams to excel in feature performance and fulfilling customer needs.

Other factors to take into account: There are a number of other things to take into account when discussing team organization with the teams. These are: responsibility, coordination and sustainability.

Responsibility does not decrease or increase given we work on a component or product as a whole. The person working on the component probably wants to do a good job just as well as the person working on ordering functionality. But the thing is, it is just easier to feel responsible for an end-result. The other question is, if we only have component teams, who feels responsible for the end-result?

Coordination is also something to take into account. The need for coordination increases from product towards component teams. Just think about all the planning, hand-overs and dependencies before a group of component teams manages to deliver the final product.

The final thing is sustainability. What if technology becomes end-of -life? Are we going to fire the entire component team? Do we want to split them up over other teams? Could they adopt a totally new technology, and would not this undermine the whole reason for forming the component team in the first place? Technological advance is moving faster than ever. We know that new tech will come and go so better stay as flexible as possible.

One good conclusion so far can thus be to avoid component teams as much as possible and move from product to feature teams when product scope and/or maturity increase.

Customer journey teams: Customer journeys are becoming more and more important in differentiating our digital product or services from the competition. It is not just the core product that matters, but more and more how customers can experience added value from that digital offering; from problem recognition, to orientation, to acquiring, to aftercare, to discarding or switching to a new offering. Teams aimed towards optimizing the customer journey, can therefore be very effective. Pretty much the same goes for a customer journey teams than for product teams though; if complexity increases we can have multiple teams, but eventually we will need team specialized in certain parts of the journey, specific actors, or channels to differentiate further.

No teams: But wait, there is more! Yes, people, because real Agility would be to self-organize in shapes and forms that simply work. Should not we let go of the whole concept of “team”? Should not we trust people to form groups that just tackle the challenges at hand, no matter what form or shape? Like a swarm of bees or a flock of birds. I sure hope to see this sometime! I would expect to see it in an organization operating in a highly volatile and competitive market where constant change is necessary to keep up.

To recap, our main considerations should be based on product complexity and maturity, as well responsibility, coordination efforts and sustainability. Do we want to push the boundary towards customer journey teams or maybe even dare to let go of teams? Whatever we decide, we should decide it together.

Read Full Post »

Older Posts »