How Engineers Can Manage Complexity in Their Organizations – Podcast interview w/ Jeff Doolittle

How Engineers Can Manage Complexity in Their Organizations – Podcast interview w/ Jeff Doolittle

“The goal is to contain complexity. You can never get rid of complexity, but you can shift it around. The irony is if we make something too simple, we are shifting the complexity to somebody else.” – Jeff Doolittle

In this episode, I interviewed Jeff Doolittle, a software architect and transformational leader with more than 20 years of experience designing and architecting software systems about the importance of complexity in engineering, and how engineers and engineering leaders can manage complexity in their organizations.

Here Are Some of the Key Points Discussed in This Episode:

  • Engineers have a better grasp of complexity than most others do because they have been exposed to mathematics and real-world constraints. If you have a system with seven components, and those components can interact with each other without constraints, you will have a system that has 5,040 possible combinations and permutations of the seven components. This is where the complexity comes in because people can only have four to seven different things in their mind at one time, and do not inherently understand things like large numbers. Most software systems today have a lot more than 14 micro-services, which gives them over 87 billion possible combinations. The challenge of complexity is not only a software problem – it’s a human challenge too.
  • Technological complexity is excessively reliant on technologies and tools, especially new and shiny ones. If we are obsessed with our tools or technologies and want them to work in vast and different ways, you will get a system that has a lot of technological complexity.
  • Integration complexity is where you have an overabundance of components that are connected together with a lack of consistency, cohesion, and constraints. It causes micro-service mayhem – many parts that are thrown together, making it unmaintainable.
  • Organization complexity results when managers do not filter the new information they receive, and the engineers who work for them therefore discuss all variations of the information with each other. It causes the organizational structure to mimic the integration complexity by having integration everywhere. It results in a system that is not consistent, cohesive, and properly constructed. Managers must help the engineers to organize in a way that can contain and manage complexity.
  • A poorly designed system (one that was not designed with quality in mind) will suffer from operational complexity. Many software teams are not responsible for the maintenance of the software they produce, which naturally leads to operational complexity. When the person who built the system is responsible for operating the system, there is a natural reduction of operational complexity.
  • Quality assurance is design integrity from the beginning. It empowers everyone in the organization to say there is a quality problem and be rewarded for it. The cheapest time to discover a defect is in the design and requirements phase – before you build anything.
  • In market complexity, it is helpful when people building the software and systems understand the nature and requirements of the people they are serving, and the nature of the complexities that those people as individuals and groups will face.
  • To contain complexity, you must identify capabilities. Understand the purpose and goal of what you are trying to build. Determine the minimum number of capabilities that can be integrated to solve the main problem. Sometimes you identify a way to have fewer capabilities to solve a problem, which can be used as an evaluation and analysis tool to help you improve existing systems.
  • To enforce constraints, you must first find the appropriate constraints for the system. Once you have the constraints, you can validate things against them, like the capabilities.
  • Encapsulation is one of the most important ways to allow a team to be most effective by defining and clarifying what communication can stay inside the team, and what communication is allowed outside the team for evaluation.
  • To commit to vigilance, everyone must be aware that quality assurance is part of everyone’s job, and that they will be rewarded when finding a problem with the quality.
  • To better manage complexity, engineers should read a couple of books on system thinking. Thinking and reading about systems thinking generally, and teaching your colleagues to think the same way, will be a great benefit when it comes to managing complexities.

You can subscribe to this podcast through any of these podcast platforms:

Land a Job You Love

I’ve created a new masterclass to help those out of a job or seeking a new job to help them get hired fast. Go watch my FREE Engineering Career Accelerator Masterclass by clicking on the button below!

Watch the FREE Masterclass Now!