According to research on Software Architecture, decision making processes were not created until the early 2000s. The topic was not covered in research papers until 2003[1]. But in system engineering, various tools have been used since the 1940s to aid the decision making process and these tools can carry over to Software Architecture. I wrote an earlier blog post was about using decision tools in architecture[2] I also worked with Axiomatic Design[3] as a student and used it on several personal software projects. Axiomatic Design tools help decouple functional requirements with design so that a change in a design parameter only affects a single requirement.
However, I have observed that the decision making comes down the the interaction of the various personalities in the process, regardless of the most detailed documentation. I’ve observed a Wizard of Oz behind a black curtain making decisions on architecture. Here’s how it works: you follow the detailed process to the letter, and for some reason that you don’t see, your architecture doesn’t get done. Someone behind a black curtain is making the architecture decisions.
Case Study 1: Just Say No
We had a feature that customers had been screaming to get for years. We had a solution that didn’t work very well. It was over ten years old, was refactored to the maximum. It was also complex to work with it. When if failed, customers could distinguish the source of failure. My boss called it a “two-headed state machine”.
When we first introduced a new architecture for this problem, there was strong push back against it. “It will take too long”, said one seasoned lead architect. Customers ended up using their own software and hardware, which was a financial loss to us.
At that same time, some development on another feature was done to make our new idea more feasible. And yet we still heard “It will take too long” from a new seasoned lead architect.
So we prepared a truck-load of documentation from our customers and detailed use cases of how it would make our customer’s life easier. It got approved and was about to start development. Then we heard “It’s a security risk”. This came from our boss’s boss’s boss. She didn’t read the documentation or history of this idea, and how little it would take to implement it.
Our understanding of the process was that this decision should have been done by a large group of engineers that meets regularly. Their decision can be overturned without a whimper.
Case Study #2 - Bullheads
Years ago, we had to make a major decision about what infrastructure to use for an API. It included a set of standards and technologies and would affect our products for years to come.
We had over fifty requirements and many options to figure out how they met those requirements. I used a decision matrix to evaluate the options with our team. This matrix laid out all the options on one axis and all the requirements on the other. We filled in how well each option met each requirement. Our team had built in bias: we had a set of team members that wanted one way, and another set of team members that was already biased another way. When we finalized the matrix, we found two options that were in just a few points of each other. The team whose favorite lost was upset.
Perhaps there was another non-functional requirement that we missed? I asked them but they couldn’t come up with one. It seemed arbitrary that their favorite way would lose by a few points out of almost 1000 points total tallied.
Amazingly a few years later, a third option which we considered, won the contest. A bull headed engineer had a set of emergency meetings, and pushed this way through. And sadly for me, it was the right decision.
So much for spending weeks doing a DSM.
Case Study #3 - Group Think
Most companies I’ve worked for have a review board to review design and architecture. In this scenario, the decision is made by a team of people. If the team doesn’t agree there is a process to raise the issue to a higher level person who makes the final call. The team then must live with that decision.
While this works well in my experience, it can be not as altruistic as it sounds. First, people can canvas members before hand to make sure things go the way they want in the review. I’ve heard “Hi Chris, I know Bill will be against my design today. Can we review and will you support my design?”. Let’s all gang up on Bill when he objects.
Recent research is that group-think can settle into teams.[4] To have a functioning group, you should get people you don’t like because they think differently. Years ago, I had two people on my team. One was a nice person, but did not contribute very much to our work. The other had a prickly personality, but questioned everything we did, and was sharp as a tack. Ms. Prickly ended up being far more valuable by disrupting a lot of things we were doing that no longer worked. With a more diverse team, we disagreed more but work through some difficult design problems and came out better for it.
Case Study #4: I’m with the boss
I had a nasty feature to design. It had a lot of moving parts and some difficult security challenges. But I was loath to overdesign it. After we had presented an initial design to the team that was going to be implementing it with us, the lead architecture burst into one of our meetings. He talked so rapidly no one else could talk. He presented a new design and announced “Oh and this has been approved by ”. Everyone went silent. Most in the room wouldn’t go against what a VP designed, and those of us that would, well, this person wasn’t in the room for us to debate.
When I tried to debate the fast talker, I was shut down. “ said that is way to do it”.
I had a fight on my hands, but this feature got dropped. I’m preparing to present my case to the Big VP when it returns.
Conclusion
Be clear about how you make decisions, and learn what really goes on behind the curtain (in other words - what do you not see about decisions). Document decisions. Understand why they are made and make sure up to your Big VP to the developers and testers - they all understand. And I don’t know why but whenever I document something well, we never have to go back and look at why (which drives me crazy but I guess it works).
[4] https://player.fm/series/worklife-with-adam-grant/how-astronauts-build-trust - listen how you need people you don’t like or trust on your team to think differently than you.
