Ch-Ch-Ch-Changes: Software Interface and Software System Dynamics

When a software system is created, there are typically interfaces to that system so that third party developers can write programs that use the functionality of that system. These external interfaces, once released, must be supported practically forever. Once a third party developer uses them, we the system designers, must make sure any changes we make to this interface be both forward and backward compatible.

Forward and backward compatibility put considerable constraints on the interface designer, especially when changes are to be made. The definition is:

Forward compatibility or upward compatibility (sometimes confused with extensibility) is the ability of a system to gracefully accept input intended for later versions of itself.

If we use the same sentence structure, we can define backward compatibility as:

Backward compatibility or downward compatibility is the ability of a system to gracefully accept input intended for earlier versions of itself.

As an example, think of some of a software library like the Boost C++ Libraries. If a change needs to be made in an interface, it must be done very carefully to ensure at least backward compatibility. If a method must be deprecated, it must not be done immediately. The method is supported but it is documented that support for this method will be removed in the future. And it is often years before that happens.

Much of my work is on existing systems so I have the difficult task of making sure my changes to external interfaces are both forward and backward compatible. This is difficult to do intellectually and I often miss things that break compatibility. I’ve built up what our team has learned into a set of standards we follow to assure that we can keep this compatibility.

But our interfaces are getting more complex and we are still missing things.
This is causing an unfortunate dynamic on our team. The first behavior is tremendous fear. Team members fear that what they release will not be forward and backward compatible so they get in the mode of carefully trying to find every problem imaginable.

Here’s a systems dynamics model of our current problem. The models were developed using Vensim and there is a free trial download of the software at their website.

The first picture is of a simple model. Note that the development rate is constant if the rate of problems found equals the rate of problems fixed.

Note: The graphed results of a simulation of this model are shown in the picture of the model (see the flat rate of software productivity and the rate of problems found and fixed).



Now we’ll add the fear factor to the model. When fear occurs, more problems, whether real or imaginary, are discovered. The software development rate goes down.



When this occurs, schedule pressure starts occurring (note that this dynamic is not shown in the model). Schedule pressure creates more fear which causes more problems to found, real and imaginary, both of which need to be analyzed. Schedule pressure might create an increase in productivity (“you must fix more bugs – please work as much as you can to do this!”). But it can’t match the fear factor and the software development rate decreases even more.



My theory is that if we create a more supportive team environment, we can counter the fear and get our development rate back up. The team will have the following operating conditions:
1. All problems raised are considered and accepted respectively
2. Problems are classified and analyzed by a cross disciplinary team (people with variety of expertise) and decision is made – do we do this change or not
a. This should decrease the churn we have from changes – we probably won’t do as many changes as we have in the past.
3. This team will also recommend the best fix for the problem so the rate of problems fixed will increase.
Voila – software productivity is back up and even increases.


I’m trying this experiment on our team and will report back the results in a few months. I feel confident that this supportive team model will change our dynamics.

Architectural Decisions – Accidental or On Purpose?

“Every interesting software-intensive system has an architecture. While some of these architectures are intentional, most appear to be accidental.” - Grady Booch (http://www.informit.com/articles/article.aspx?p=471929 )

Last week I wrote about what software architecture is and that I rely on a software architecture consisting of goals which map to structure and behavior. But in my everyday work in the field of software design, I rarely get the luxury of creating such a complete mapping.

What typically happens is that I join a project that already has had the vision established and may not have any artifacts explaining that vision. I have to do some archeology in the code to discover what the original vision was. And more often than not, there are several conflicting visions in place.
Typically where I work, we make design and architecture decisions on a daily basis. And as we iterate through our work and develop code and test our solutions, we find changes we need to make. To make sure these decisions are accidental and that they merge with our team’s vision of our architecture, we meet weekly to review changes. We often argue.

Part of my job as a software architect is to examine and analyze various points of view from team. There are two tools I use that are invaluable to getting the team to converge on a solution (and sometimes convergence is not possible, and a decision has to be made, but in the majority of cases, convergence does occur).

The first tool is a decision matrix. This is from the book “Getting Started in Project Management” by Martin and Tate (http://www.amazon.com/Getting-Started-Project-Management/dp/0471135038 ) - double click the image to get a close up:



This works well but is very quantitative and the human nuances of decision making are often lost in this format. We also sometimes break the rule above “Do not change the numbers to affect the selection of a “favored” solution.”

My favorite format is to use a Mind Map (http://en.wikipedia.org/wiki/Mind_map). And the tool I use is Mind Manager (http://www.mindjet.com/ ) but there are free tools available for download and templates available for Visio.
Here are two the templates I work with:






I just noticed that I have hundreds, yes, hundreds of these maps in place. We refer back to them when we forget why we made a certain decision and when someone new to the team questions the decision. Sometimes we re-evaluate our decision and use our map as a launch point to see why our thinking as changed.

In the article Booch says:

"Thus, having an accidental architecture is not necessarily a bad thing, as long as
• the decisions that make up that architecture are made manifest and
• the essential ones are made visible as soon as they are instituted and then are allowed to remain visible throughout the meaningful life of that system."

By using these tools you can achieve, and thrive, with an accidental architecture. Let me know if these tools are useful to you or what tools you use.

Software Architecture Definition?

When I was a student in the MIT Systems Design and Management Program, I had to take a course titled “Systems Architecture”. The course was about overall systems and not specifically on software. In that course, there was a paper I read that forever shaped my view on what software architecture should be. The paper is titled “A Taxomony of Decomposition Strategies Based on Structures, Behaviors, and Goals” by Phillip J. Koopman (DE-Vol. 83, 1995 Design Engineering Technical Conferences, Volume 2, ASME 1995). In this paper, Koopman describes architecture as containing structures, behaviors, and goals along with a variety of decomposition strategies. In my own work in software architecture, I find that I must tie my structure and behavior specifications to business goals and when I don’t have all three pieces, my architecture often doesn’t withstand the rigors of peer review from a wide audience.

How does this definition of architecture components stand up to other definitions? The standard on software architecture - IEEE 1471-2000 - says
Software architecture is the fundamental organization of a system, embodied in its components, their relationships to each other and the environment, and the principles governing its design and evolution.

The standard defines the following bold terms:

A system is a collection of components organized to accomplish a specific function or set of functions. The term system encompasses individual applications, systems in the traditional sense, subsystems, systems of systems, product lines, product families, whole enterprises, and other aggregations of interest. A system exists to fulfill one or more missions in its environment. [IEEE 1471]

The environment, or context, determines the setting and circumstances of developmental, operational, political, and other influences upon that system. [IEEE 1471]

A mission is a use or operation for which a system is intended by one or more stakeholders to meet some set of objectives. [IEEE 1471]

A stakeholder is an individual, team, or organization (or classes thereof) with interests in, or concerns relative to, a system. [IEEE 1471]

This definitions correlates to Koopmans article in the following ways: the goals of the architecture are the mission; the behavior maps to the relationships and the principles, and the structure maps to the fundamental organization.

In practice I see many designs that show mapping and decomposition with behavior and structure (think UML class diagrams and sequence diagrams) or a mapping between goals and behavior (UML Use Cases and Sequence Diagrams) but very architecture artifacts that successfully map all three.

I will be looking at other definitions of architecture in future blog posts.