Feeds:
Posts
Comments

Recently organizations are embracing DevOps which is a great thing. However the whole adoption is causing a lot of confusion as well. Some of you might have heard the term “Agile and DevOps”. With that it looks like Agile and DevOps are different. To over-simplify further people assume Agile is all about processes (like Scrum and Kanban) and DevOps is all about technical practices like CI, CD, Test Automation and Infrastructure Automation.

This is causing a lot of harm as some organizations now have Agile and DevOps as two separate streams as part of their enterprise Agile transformation. Agile by definitions disrupts silos and you see, in this case people are creating new silos in the name of Agile and DevOps.

With that background in mind, let’s try to understand what exactly DevOps is all about:

  • DevOps is mainly the widening of Agile’s principles to include systems and operations instead of stopping its concerns at code check-in. Apart from working together as a cross-functional team of designer, tester and developer as part of an Agile team, DevOps suggests to add operations as well in the definition of cross-functional team.
  • DevOps strives to focus on the overall service or software fully delivered to the customer instead of simply “working software”.
  • It emphasizes breaking down barriers between developers and operations teams, and getting them to collaborate in a way where they benefit from combined skills.
  • Agile teams used automated build, test automation, Continuous Integration and Continuous Delivery.
  • With DevOps that extended further to “Infrastructure as Code”, configuration management, metrics and monitoring schemes, a tool chain approach to tooling, virtualization and cloud computing to accelerate change in the modern infrastructure world. DevOps brings some tools on the block as well like configuration management (puppet, chef, ansible), orchestration (zookeeper, noah, mesos), monitoring, virtualization and containerization (AWS, OpenStack, vagrant, docker) and many more.
  • So you see DevOps is not a separate concept but a mere extension of Agile to include operations as well in the definition of cross-functional Agile team, collaborate together and work as one team with an objective to delivery software fully to the customer.

Creating separate Agile and DevOps horizontals in any organization just defeats the whole purpose (removing silos) of DevOps.

It depends on what you are passionate about. I would choose passion over all other considerations or else you will burn out and the money won’t help. Any position you pursue in tech is going to require your complete dedication to achieve success.

However, I think the DevOps area has a lot of growth potential in the future. While I cannot speak to your interests I can tell you why I am drawn to the field after being a web based software developer.

I have learned through that if you are operations staff you’d better be automating your job, and if you are a developer you have to face the inevitability of getting down and dirty with operations if you are to stay relevant. Developers who won’t administer/monitor and admins who won’t develop will increasingly become less and less valuable to organizations needing to stay competitive.

DevOps is exciting because you are always working with and integrating new technologies and solving new challenges. Essentially your job is to find a happy balance between operations and developers. This relationship is delicate and can blow up if not regulated. As a devops specialist your job is to integrate these two different mindsets. This requires that aspects of IT be securely shared so that you don’t have the blame game. Developers need to continually push code and operations want to keep everything running smoothly. The more integrated the systems and processes in use, the easier it is for each to do their job.

I personally like to think of IT as three separate phases that all contribute to the ultimate success of the enterprise tech ecosystem; packaging, automation, and scaling.

Packaging:

DevOps is great if you like to explore and work with a variety of technologies and processes. I think one of the first things to consider is the packaging of IT that the tech teams use to provide the organizations products and services. The better packaged and more malleable the packaging the easier it is to keep everything standardized and reusable.

If you like playing with configuration management systems (Puppet, Chef, Ansible, etc…) and digging into imaging systems such as Docker you will like DevOps. I would caution that it is very important to create highly configurable packaging of the IT systems in use so that they can evolve as the organization’s needs change. This also makes it easier to modify for production, QA, staging, and development environments.

If you think about it, the amount of new technologies and services being released into the market is growing exponentially (especially with the add-on potential of all the open source frameworks in existence). In DevOps no technology is off limits and you find yourself continuously working with, integrating, and automating different technologies. As the amount of tech and services grow, so is the demand for people who can put it all together into golden images (configuration managed images on different environments).

Automation:

Your automation potential is only as good as your ability to package the infrastructure in a form machines can work with effectively. If you come from a development background you most likely have had to deal with brittle environments (at least in testing new technologies, which you should be continuously doing).

The DevOps specialist makes it easy for programmers and operations to automate their jobs so that we don’t have to reinvent the wheel over and over again. Ultimately, if the automation is good enough we can realize a scalable architecture (which is the end goal).

You should like scripting a lot. You don’t need to be the best programmer to accomplish this but the more integrated your approach the easier it will be to build on your previous work (which I like). Automation brings the machines to life and if you like seeing a bunch of moving pieces come together to achieve some measurable outcome you will like this part of the job.

I would recommend that you know at least one glue language: Python, Ruby, Go, etc. The more flexible the language, the better. Although, the beauty of automation is that many different languages can be brought together to create a unified system. If something needs to be built for speed, it’s easy enough to design that part in a language like C or Go while allowing other tasks that need more flexibility to be written in a higher level language. You definitely want to become very good at shell scripting which many times ties everything together.

I personally cannot see automation becoming less in demand in the future. The promise of the cloud is built on automation, and enterprise usage of the cloud is growing rapidly throughout organizations of all sizes and types.

Scaling:

If reusability is a passion of yours, I think you would definitely like DevOps. I believe the biggest factor in the successful tech organizations of the future will be their ability to scale rapidly while being able to deflate when not needed to minimize costs in downtime. Customers want speed. They don’t care about the tech behind the application as long as the application is reliable, zippy, and meets their needs.

If you can create packages of IT that can be easily automated in a portable fashion then I think you will have great prospects in the tech world in the future. Companies like Google and Facebook would never have gotten as popular as they are if they had not learned to scale their IT effectively.

Scalability is not easy to achieve and many would rather not have to worry about it, which explains the growth of scalability as a service offerings. But somebody has to know how. Think about the problems of the future: Data analysis, AI, internet of things, mobile consumption, scalable web driven apps, etc. While all of these tech areas require different skills to develop on their own, each is absolute garbage without the same fundamental building blocks. Want to jump from mobile to AI? DevOps could allow that. Want to play with that new SaaS service that is all the rage these days? DevOps can allow that.

DevOps is about being the glue that holds everything and everyone together, and to me that is what makes it so exciting. The possibilities are limitless and the technologies are always growing and evolving. And if you don’t focus on DevOps, you will still have to manage infrastructure as a developer anyway.

When I first started programming I started with a passion for machine learning in C, then over time I started creating web sites using asp.net for my organization. Over time I felt suffocated by the limited nature of the technologies people expected me to work with day in and day out. It wasn’t that I did not like the technologies, but I felt like I was in single technology hell. And once you gain a lot of experience in a specific technology or system people just expect you to focus on that area: recruiters, managers, developers, everyone. With DevOps variety is part of the job description so if you ever feel trapped by technology and find yourself looking to the stars wondering what the hell did I get myself into, DevOps can free you from that limited mindset.

That is why I got into DevOps. I certainly don’t claim to be the best or the most experienced out there, but I am a heck of a lot happier these days. And I am constantly learning new things that I can apply to any new project, whether it is a new AI platform or a mobile application.

At the end of the day your happiness and the passion you feel for what you do is all that really matters.

There are many efficient way to measure the maturity and performance of an agile team and one of them is by measuring cycle time. In a traditional project, we have the project cycle time, which is the time between start of the project and the final delivery or release. In an agile project, the cycle time can be measured on the level of user stories.

Lean: The above reasoning is supported by two main principles of a lean mindset:

Reducing cycling time is one very important principle of a lean mindset: we get faster feedback and test results, we prevent work from clogging our development queues and we improve morale because the team sees that the work ‘disappears’ faster from the backlog.

‘Avoiding waste’ is the primary objective of a lean approach. Any wait time between the different process steps can be considered as waste. Reducing the wait time can therefore be considered as a good way to get rid of ‘waste’, and also reduces the overall cycle time.

How to measure: The following image shows a simplified view of the life cycle of a user story in an agile project, including the potential wait times:

Measuring Agile Maturity 1.gif

The cycle time is the time between ‘Ready to Start’ and ‘Release’. This time can be significantly reduced by reducing the wait times (WT1, WT2 and WT3).

Depending on how we organize our sprint backlog, we can start by noting on each user story card the date on which the story became ready for development by our agile team. This might be the start date of our sprint.

As soon as we meet all the ‘definitions of done’ for that particular user story, note the actual date again on the story card. The difference between the two dates is the user story cycle time.

How to calculate the average cycle time: At the end of each iteration, calculate the average of the cycle times and divide the result by the average complexity of the user stories of the sprint.

The result of this division gives us the average cycle time of that particular sprint. The division by the average complexity is necessary to standardize on complexity.

Over time, when our agile team matures and becomes more efficient, we should see a decrease of the average cycle time. If this is not the case, then we are doing something wrong, and there are probably some bottlenecks in our process that we want to resolve.

Advanced mode: Cycle time variance: Reducing cycle time is one thing, but what about cycle time variance?

Let’s see these two different cycle time-related metrics on a graph:

Measuring Agile Maturity 2.png

The red curve show the distribution of cycle times of team 1, whilst the green curve shows the distribution of cycle times of team 2. The green team has a higher maturity level because of two reasons:

  • The mean/average cycle time CT2 is smaller than the mean/average cycle time of team 1, CT1.
  • The standard deviation of the measured cycle times of team 2 (B) is smaller than the standard deviation of the cycle times of team 1 (A).

This means that over multiple sprints or projects, team 2 is able to produce better predictable output.

Why cycle time is the right indication for maturity:

  • It focuses on the final goal (customer value) and not on the process (the customer does not care).
  • It is very easy to collect the relevant data.
  • It is a very simple and understandable metric.
  • We do not need additional tools, only a pencil and a calendar.

Every project has a certain level of complexity in it. When we say a project is simple, we actually mean that its degree of complexity is very less or can be considered as negligible, but nevertheless, it does exist even if in a minute magnitude. Project management methods deal with project complexities depending upon how flexible they are, and what kind of provisions they offer to deal with them. Tradition management methods such as Waterfall are often rigid owing to their staged working processes, which are also often irreversible. Using pre 1990 era methods, one can try to address the complexity in a project to a lesser or greater degree of success, but typically such management methods do not make it possible to reduce or minify the actual level of difficult in executing the project and neither its complexity. This is not necessarily true in case of Agile. With Agile we can actually try to control the level of complexity in a project provided we have the correct level of experience in implementing Agile principles and techniques.

What is “Complexity” in a project?

Broadly speaking the term ‘complexity’ can be best understood as difficult or complicated conditions arising due to the availability of multiple options, or options which make us simultaneously focus upon different directions at a given time, and which result in a multi-dimensional scenario that is hard to understand and resolve. Complexities can be of different types in a project. Business level complexities arise due to uncertain market conditions, technological advancements, and other such factors which affect the business logic contained in the project. The project level complexities can be of two types: Project complexity and Requirements complexity.

Project complexity

Project related complexity can be different for different types of organizations. Several factors contribute to it; however, the most important ones are uniqueness, limiting factors, and uncertainty.

Uniqueness: Every project is unique and has its own attributes and requirements. As the project commences it gains maturity over time and benefits through the learning process. This is most significant when the organization is running a project which is the first of its kind, or if it has no prior experience dealing with projects.

Limiting factors: Projects are subjected to certain factors which can affect its execution or commencement such as budget constraints, the technical knowhow of the team, working schedule, and at times even cultural differences.

Uncertainty: Uncertainty in a project can be due to external or internal factors. External factors may include government imposed rules and regulations, uncertain market changes, and fluctuating economic climate. Internal factors may comprise of the levels of management’s participation in the project, constantly changing company policies, stakeholder’s involvement in the project, etc. All these factors affect the scope of the project.

It is a known fact that a project’s complexity affects its success. The manner in which a business anticipates, fully understands, and addresses the complexity determines whether a project is going to be successful or not.

Requirements complexity

Requirement analysis is the journey to discover the ‘unknowns’. It is an understanding of the business problem, needs and what it takes to address them. Requirements complexity is defined by two key factors: The level of ‘unknowns’ and Volatility.

The level of ‘unknowns’: At the start of the project, how much is ‘known’ about the problem statement? How much is known about the business processes? The level of ‘unknown’ must be assessed at a very granular level, particularly pertaining to business rules, systems, functions etc.

Volatility: What is the expected level of requirements volatility once the project is launched? ‘Volatility’ in requirements emerges due to frequent changes, starting from the design phase all the way through implementation. Project management methodologies often assume that, when requirements move onto the design phase, they stand ‘complete’ and are not subject to change. However, that is not always the case as there is always some level of uncertainty and unpredictability. Requirements volatility leads to significant risk and its consequent uncertainty.

Can Agile reduce complexity?

In traditional project management methods, the complexity in a project is often managed by investing a certain amount of time in the analysis phase with the sole objective of analyzing the levels of complexity and making plans to deal with it. It is based upon the assumption that the time invested in the analysis activity will help to reduce complexity and increase the chances of developing a successful project by using various methods and processes. The investment in time should be considered worthwhile since the analysis can help managements to make informed decisions.

Unlike traditional project management methods, in Agile there are no special stages to deal with project complexity. The product owner who is responsible and who oversees the entire project tries to address the complexity levels based upon his or her experience in the subject, in addition to what the team can contribute in terms of efforts and suggestion to deal with complexity. However, there is a big plus point in how the Agile process works and how the inherent product incremental model makes it possible to reduce project complexity to a great level.

In Agile we don’t work with the entire project at any given time rather we select a few important features and develop them in short bursts of activity known as sprints. The time spent in developing the features varies from team to team depending upon the team’s level of maturity and its hold over the technology used for developing the project. A project may appear to be complex when its overall complexity is considered, but in Agile since we don’t have to deal with the entire project during the sprints, the levels of complexity can be easily addressed to by estimating the levels of difficulty in the features and developing them individually. That way we only encounter a fraction of the actual complexity, which can be easily tackled by the team.

Project complexity is inevitable and should be acknowledged to enhance the team’s ability to respond and adapt to change while staying focused on the end objective. Agile practices and methodology promote the capability to drive and manage change through an understanding of the inherent complexity in projects.

It is pretty incredible how often we complain about our best employees leaving, and we really do have something to complain about — few things are as costly and disruptive as good people walking out the door.

We tend to blame our turnover problems on everything under the sun while ignoring the crux of the matter: People do not leave jobs; they leave their managers.

Here manager does not mean only the boss or the owner. For a junior developer it can be his senior one. For a senior developer it can be lead developer. For a lead developer it can be development manager. For a development manager it can be scrum master or product owner.

The sad thing is that this can easily be avoided. All that is required is a new perspective and some extra effort on both ends.

Let us discuss few things that send good people packing:

Overwork: Nothing burns good employees out quite like overworking them. It is so tempting to work your best people hard that we frequently fall into this trap. Overworking good employees is perplexing; it makes them feel as if they are being punished for great performance. Overworking employees is also counterproductive.

New research from Stanford shows that productivity per hour declines sharply when the workweek exceeds 50 hours, and productivity drops off so much after 55 hours that you do not get anything out of working more.

If we must increase how much work our talented employees are doing, we better increase their status as well. Talented employees will take on a bigger workload, but they would not stay if their job suffocates them in the process. Raises, promotions, and title-changes are all acceptable ways to increase workload.

If we simply increase workload because people are talented, without changing a thing, they will seek another job that gives them what they deserve.

Not recognize contributions and not reward good work: It is easy to underestimate the power of a pat on the back, especially with top performers who are intrinsically motivated. Everyone likes kudos, none more so than those who work hard and give their all.

We need to communicate with our people to find out what makes them feel good (for some, it is a raise; for others, it is public recognition) and then to reward them for a job well done.

Not care about employees: Study shows that more than half of people who leave their jobs do so because of their relationship with their boss. We need to balance being professional with being human.

We should celebrate an employee’s success; empathize with those going through hard times, and challenge people, even when it hurts. If we fail to really care then we will always have high turnover rates. It is impossible to work for someone nine-plus hours a day, five days a week, fifty two weeks a year when we are not personally involved with each other and do not care about anything other than your production yield.

Not let people pursue their passions: Talented employees are passionate. Providing opportunities for them to pursue their passions improves their productivity and job satisfaction. But sometimes we want people to work within a little box. Sometimes we think that productivity will decline if we let people expand their focus and pursue their passions. This fear is unfounded. Studies show that people who are able to pursue their passions at work experience flow, a euphoric state of mind that is five times more productive than the norm.

Fail to develop people’s skill: Management may have a beginning, but it certainly has no end. When we have a talented employee, it is up to us to keep finding areas in which they can improve to expand their skill set. The most talented employees want feedback — more so than the less talented ones — and it is our job to keep it coming. If we do not, our best people will grow bored and complacent.

Finally, I want to use one line to summarize the whole discussion: We want to create such a team that everyone wants to join.

Theory of the perfect team

Google is known for being obsessed with studying their own performance and then adjust accordingly. At one time Larry Page decided to get rid of all the middle management but then quickly realized that the increased amount of direct reports slowed the company down immediately. So, they brought them back. With its Aristotle project Google wanted to figure out what are great teams made of. At the beginning, they focused all on the individual skills of the team members and their performance but the results were sobering. Having all superstars on the team did not necessarily made a great team. After hundreds of teams studied, a pattern emerged. In all the high performing teams the members felt psychological safety. That means they were not scared to admit an error and talk about their ideas. Furthermore, these teams had all empathy for each other (social sensitivity) and equal voices, which mean that everybody’s opinion is heard when a decision must be made. This does not mean democracy, still somebody can make the decision but people feel that their opinion is considered. For our Scrum teams that means, it is more a cultural challenge rather than a skill challenge and the culture should incorporate those values:

  • Psychological safety
  • Social sensitivity
  • Equal voices

A popular thought for hiring a new employee regarding this: Hire for fit not for skill.

Multifunctional teams

A frequent question I get is: Does multi-functional team mean everybody on the team has to be an expert in everything? This is probably due to 2 facts:

  • We want the team to be independent with a few dependencies as possible to the outside. Therefore, we need all skills inside the team.
  • We advertise that there are no more specialists rather everybody has to be able to help out with other tasks.

Now if there is for example a data base expert on the team, it does make sense that he helps with the database issues but if he has idle time he should help out other team members. On the other hand, when there are a lot of database related tasks and he becomes the bottle neck others should be able to help him out as well.

The approach of A-teams (small engagement units) from the Special Forces is this: Each team member is cross trained. That means he is usually an expert in 1 or 2 specialties e.g. medic, radio, etc. but is also able to do general tasks in other areas. They have what is called T shaped people, a person that is expert in 1 or 2 areas but has general knowledge about all the other topics. Furthermore, every specialty exists twice (in case you lose the medic, there is still another one).

How can we apply that to our Scrum teams? First, it is important that all necessary skills are on the team to be independent. Then, we want every expertise twice, for example 2 persons with highly developed database skills. But we want also T-shaped people that can work on other parts as well (not only their expertise).

Bringing it all together

In sum, it is not so important to have superstars on the team rather than how the team is operating. The number one indicator is psychological safety followed by social sensitivity and equal voices. Secondly, culture has a huge impact on team performance. Even though, culture is probably the most difficult thing to change, moving to a commitment culture is one the most performance improving steps. Lastly, when it comes to who is on the team it is important to have T-shaped people that do have the necessary skills to work independent from the outside. However, they do not have to be superstars.

Takeaways

  • Define team ground rules that lead to psychological safety, social sensitivity and equal voices.
  • Implement a commitment culture.
  • Create a team of T-shaped people covering the necessary skills.

We always feel like our teams could be organized better. How to organize teams in an optimal way is a common question in Agile organizations. A question we should always discuss and answer together with the people in the actual teams.

Here I will try to provide an overview of 5 possibilities for organizing teams and the main factors to take into account. Our main considerations should be based on product complexity and maturity, as well responsibility, coordination efforts and sustainability. Depending on our specific context one will suit our situation better than the other, or maybe we will come to the conclusion that we are so Agile we can leave the concept of team all together.

Component based teams: Component based teams focus on specific components in the product that do not deliver customer value in it. They are also sometimes called focus teams for that reason. The focus is usually necessary because of the use of very specific or exotic techniques. Learning the skills and expertise to work in these teams takes years so it is more logical to put experts together. We can compare this kind of teams with functionally specialized teams.

Feature based teams: Feature based teams focus on a specific customer feature or feature areas within a product. For example: search optimization in a web-shop, or payments and conversion. Feature teams are necessary if the product is too large for a single team to optimize or when a single larger base product is differentiated into various market segments in retrospect. For instance, the same web-shop product can be differentiated for selling kitchen appliances, or tailored for selling TV sets.

Product based teams: Product teams carry responsibility for the entire product or product/market combination. This means all of the skills and expertise necessary to develop and run the product reside within the team. Product teams are therefore suited for smaller products with a relatively small feature set. That is why we often see these types of teams in start-up settings. When developing our new product, we do not want it to be complex and we will have a limited feature set if not a single one we want to validate first. Once the product matures, team orientation eventually changes towards feature teams to excel in feature performance and fulfilling customer needs.

Other factors to take into account: There are a number of other things to take into account when discussing team organization with the teams. These are: responsibility, coordination and sustainability.

Responsibility does not decrease or increase given we work on a component or product as a whole. The person working on the component probably wants to do a good job just as well as the person working on ordering functionality. But the thing is, it is just easier to feel responsible for an end-result. The other question is, if we only have component teams, who feels responsible for the end-result?

Coordination is also something to take into account. The need for coordination increases from product towards component teams. Just think about all the planning, hand-overs and dependencies before a group of component teams manages to deliver the final product.

The final thing is sustainability. What if technology becomes end-of -life? Are we going to fire the entire component team? Do we want to split them up over other teams? Could they adopt a totally new technology, and would not this undermine the whole reason for forming the component team in the first place? Technological advance is moving faster than ever. We know that new tech will come and go so better stay as flexible as possible.

One good conclusion so far can thus be to avoid component teams as much as possible and move from product to feature teams when product scope and/or maturity increase.

Customer journey teams: Customer journeys are becoming more and more important in differentiating our digital product or services from the competition. It is not just the core product that matters, but more and more how customers can experience added value from that digital offering; from problem recognition, to orientation, to acquiring, to aftercare, to discarding or switching to a new offering. Teams aimed towards optimizing the customer journey, can therefore be very effective. Pretty much the same goes for a customer journey teams than for product teams though; if complexity increases we can have multiple teams, but eventually we will need team specialized in certain parts of the journey, specific actors, or channels to differentiate further.

No teams: But wait, there is more! Yes, people, because real Agility would be to self-organize in shapes and forms that simply work. Should not we let go of the whole concept of “team”? Should not we trust people to form groups that just tackle the challenges at hand, no matter what form or shape? Like a swarm of bees or a flock of birds. I sure hope to see this sometime! I would expect to see it in an organization operating in a highly volatile and competitive market where constant change is necessary to keep up.

To recap, our main considerations should be based on product complexity and maturity, as well responsibility, coordination efforts and sustainability. Do we want to push the boundary towards customer journey teams or maybe even dare to let go of teams? Whatever we decide, we should decide it together.