MIT Systems Thinking Webinar

Please use the attached link to view a playback of the webinar I presented at MIT on Systems Architecture:

Systems Architecture Webinar at MIT

This series is open to the public and they have many intersting topics for architects. The next one is on System Interaction and Complexity Metrics.

Book Review: Universal Principles of Design

I’ve always thought of Software Design and Software Architecture as different from other design disciplines. After all, building a house is way different then constructing software. And I also read a lot of books on Software Architecture, and in these books, the authors often point out the differences, such as:

- We have only been building software for fifty years but we have been building houses for several thousands of years.

- Software is more malleable and can be modified easier than a building.

- Design patterns for software don’t look much like design patterns in other disciplines.

And yet I have discovered a book about design that applies to software in a big way. It is title Universal Principles of Design, by Lidwell, Holden, and Butler. This book contains 216 design principles, with each described on a few pages or less, with diagrams and quotes from other sources for each principle. The book is brilliant.

Here are some of my favorite principles that I use in my work:

The 80/20 Rule

“A high percentage of effects in any large system are caused by a low percentage of variables”.

There are many examples you probably heard about with this rule. Some listed in the book are:

- 80 percent of the town’s traffic is on 20 percent of its roads

- 80 percent of a company’s revenue comes from 20 percent of its products

And my favorite:

- 80 percent of a product’s usage comes from 20 percent of its features.

What if we could focus our efforts only on that 20 percent of the features? Productivity and success will follow.

Another favorite:

Mental Model

“People understand and interact with systems and environments based on mental representations developed from experience”.

A great example of this given in the book is the interaction with ABS brakes. With a conventional braking system, the drier should pump brakes on a wet slick surface, but with ABS brakes, the car will properly stop and make a turn when slamming or steering while braking. If you are a driving used to conventional brakes, you may not understand this when using ABS brakes.

The book says that designers often have accurate system models but inaccurate user interaction models. Users have accurate user interaction models but inaccurate systems models. Designers need to obtain user interaction models to create designs. They can obtain this by testing and interacting with their designs as users. One of my goals in the next year is to be a tester for all my designs.

And my favorite so far is:

Ockam’s Razor

“Given a choice between functionality equivalent designs, the simplest design should be selected.”

A great example of this is the Google search interface. Nothing could be simpler and it continues as the main interface to the search engine.

Another application is with interfaces. When I design interfaces I strive to make the fewest methods possible. It is a struggle to do this but each method is carefully evaluated.

I plan to read and reflect on a principle a day with this book and will blog on other principles in the future. I feel like I found a treasure cove of ideas for my daily work.

Less Is More

I discovered this web site completely dedicated to Agile Software Architecture. In fact, when I search the web I can’t find anyone that thinks that Software Architecture is dead or gone with the Agile Software movement. For once, the world agrees with me.

The web site outlines what we would like to achieve with Agile Software Architecture:

1. Deliver working solutions
2. Maximize stakeholder value
3. Find solutions which meet the goals of all stakeholders
4. Enable the next effort
5. Manage change and complexity

In my experience the first objective is the most important. I have always wanted to deliver a working solution as soon as possible to test out the ideas of the architecture. In fact, I often work on the implementation before the architecture is completely documented or modeled.

The web site also lays out a set of principles that align with the Agile Manifesto but are specific to architecture:

1. Value People
2. Communicate!
3. Less is More
4. Embrace Change: Plan It, Manage It!
5. Choose the Right Solution for the Enterprise
6. Deliver Quality
7. Model and Document in an Agile Fashion

My favorite of these is #3, Less is More. In my work, I consider the expense of everything. I once had a colleague who was a master requirements writer. She wrote hundreds of requirements for each feature where I would write about a dozen or two for similarly sized features. From my experience, each requirement had a high cost. A test was created for each requirement on each of our product platforms. If that requirement failed, error reports would be generated. If the requirement was truly necessary, then these things (tests, error reports) were indeed necessary. But what I found was when there were too many requirements, very few people would read them and thus their usefulness went downhill.

With documentation and requirements, I like to use advice that Robert C. Martin gave at a seminar I attended: “Rip content out until it hurts!”. After all, you can always put it back in.

Long Projects

I recently found out that a project I started years ago was just cancelled. I was devastated. How could this happen? We did everything right with this project! It even had a code name that indicated it would be a success.

This was a technology disruptive project. It would change the way we do our work and work with our business partners. It had strong backing from many management tiers. These managers were committed. They knew the perils of supporting a multi-year project like this and also meeting their commitments to deliver a steady stream of new products in the meantime.

From the beginning there were constant challenges because we could not get enough engineers on the project. I was moved off the project in a cascade of reorganizations. But as I watched from afar, many talented engineers were moved on the project.

And yet, I smelled the trouble from afar. The project communications were stellar. I was impressed with the progress. They were creating deliverable systems and getting all the key stakeholders involved. More and more of a working system was delivered. But every time I asked how much time before it would be delivered I was gold “two years”. Another year passed and I asked the questioned again and got “two years”.

That “two years” is kiss of death. In fact “two” of anything is the kiss of death. We have an engineer in another organization who we have been waiting months on for a design. Every month he tells us it will be done in “two weeks”.

After six months we gave up and now we did the design ourselves. It’s not as solid a design because he is an expert in this area but we are learning. We’ll have to build the design expertise ourselves.

Long projects notoriously fail. Here is some research from objectmentor.com:

The project I started on was about four years old.

I recalled when we made the key decision. We were going to do something that was more malleable and could be done slowly over time or something disruptive, which was ultimately chosen. At the time, it appeared the disruptive technology would be more robust and indeed that is still the case. But ultimately, the business could not sustain the investment to make this disruptive change.

What does this all mean for architecture? Architects have to make decisions that the organization can follow through on. I had my doubts seeing our organization not follow through on other disruptive changes. And yet I felt at the time, that this environment was different. In the end, however, business needs usually win. And software architecture may have to work with business demands versus the most elegant architecture.

Design Patterns in Everyday Architecture

Software Design Patterns were a sensation when this book about them was published. I became aware of this book in 2001 when a team I worked with was using design patterns. The book promoted software design patterns similar to using a cookbook in other engineering disciplines like building architecture or electronics. The authors define a Design Pattern:

A design pattern systematically names, motivates, and explains a general design that addresses a recurring design problem in object-oriented systems. It describes the problem, the solution, when to apply the solution, and its consequences. It also gives implementation hints and examples. The solution is a general arrangement of objects and classes that solve the problem. The solution is customized and implemented to solve the problem in a particular context.

It all made sense. And yet…

I have a confession to make. I haven’t used Software Design Patterns in a very long time. I think I developed an aversion to them. In the last project where we actively used them as a team, we developed a massively BDUF (Big Design Up Front) and the project was an utter failure, ultimately being canceled and many people on the team being laid off. Since then, while I am a proponent of thinking about the project up front, I also know that the team can and will get lost in oodles of documentation. Our team did and ultimately that documentation was of little value to our stakeholders.

Our team used to work like this: we would see a problem that needed to be solved and then look through the Design Patterns book for a solution.

The problem with doing that is this: often the pattern didn’t precisely solve our solution. Since we worked on embedded systems, object oriented solutions often not used to save memory. Our patterns seemed to be a lot simpler then what was presented in the book.

I found a lot of criticism of design patterns on the internet. One person felt that design patterns only existed because of language restrictions and that in languages like Perl, the patterns are built in and ready for use. (This poor guy got a ton of backlash for not believing in Design Patterns).

The biggest problem I had with design patterns is that the ones I wanted to use weren’t in the book. That said there is an effort to create a larger repository of design patterns. Since I’m working mostly with XML for my work, I plan to explore this XML repository.

My experience with Design Patterns left me running away from them. While they have some obvious detractions, it’s worth it to look at them again and see if I can find a use for them in my daily architecture work, where decisions are not usually made up front but on a daily basis.

Software Architects: The Best Job

I was very happy to see this article on CNN’s Money web site: http://money.cnn.com/magazines/moneymag/bestjobs/2010/snapshots/1.html

This article ranks the Software Architect as the #1 ranked job in America!

I couldn’t agree more with the ranking of course. Like the person profiled, I work on business strategy some days and other days I’m deep in code. And I’m everywhere in between.

There is a projected 34% job growth in this area over the next ten years.

Agile Processes – So What’s the Architecture?

About seven years ago I attended a Best Practices conference for Software Development. The seminars were on RUP (Rational Unified Process ) and CMMI (Capabilities Maturity Model ). At that time, I was working with my own company’s internal product release processes that were waterfall http://en.wikipedia.org/wiki/Waterfall_model in nature. There were a handful of seminars on agile development.

Fast forward two years: I’m attending the same Best Practices conference। And the seminars couldn’t be more different. Agile development was the subject of the vast majority of topics presented with titles like: Agile Development and XXX (you fill in the XXX). There were lively roundtables with book authors on agile. One poor fellow asked skeptical questions about agile methods and was roasted by the roundtable leaders.

And one seminar that stood out but itself because Agile was not in the title: Documenting Software Architectures with one of the authors of the same titled book। This seminar was packed and one of the most well attended of the conference.

But don’t software architecture methods and agile methods directly conflict with each other? Don’t they have different values?

Being a software architect and systems engineer at my current company, I have observed and been part of BDUF (Big Design Up Front) and features that are YAGNI (You Ain’t Gonna Need It)। I’m guessing that poor fellow that was roasted in the roundtable was also part of this effort. And yet those efforts sometimes have some incredible forethought. The foundation was laid down to allow for robust change in the future. As someone who has worked with existing architectures, I have great respect for the foresight the earlier designers had when that serendipity occurs “Wow, this is already in place and all we have to do is….”.

During a business trip with the SDM at MIT, we were visiting a software company in Dublin, Ireland that was an early user of agile methods in 2001. One of the students asked the lead software engineer how they will deal with legacy issues given the way they were using agile. The lead designer didn’t know and thought it would be a problem in the future.
Fast forward and it’s 2010। My company has rolled out agile methods and we’ve all been using them for several years now. We show improvements in productivity and reliability. And what has happened to architecture? It’s more important than ever.

Like most companies, our software systems are large and complex। An overall framework is needed at the beginning of a project and decisions that are the hardest to change during the lifecycle of the project are made.

Many architectural decisions are made during the project and the key for the architect is to communicate those changes to all stakeholders. See my previous blog on communication tools for architectural decisions.

At that conference, Agile zealots accused architects of being software dinosaurs and architects accused Agile zealots of being cowboys। I don’t really see the contradiction in the two. I’ve come from using CMMI to using Agile. Both are disciplined processes and Agile is more disciplined and constrained with the focus on the final product. Complex systems have architecture whether it is done iteravely or not. It just smart to make some of the overarching decisions before coding begins. The architect’s role continues as someone who manages key decisions and tests the code to make sure it is executing those decisions.

When those decisions are not made up front, it can be disastrous for the project (at least some that I’ve been involved in)।

I wonder there are any software developers that don’t use Agile method? Maybe not. And I also wonder if there is another better process in our future. Regardless, classes like Documenting Software Architectures will always be interesting because we all need to know “What’s the architecture?” Architecture occurs whether you have someone working on it or not. It’s better to be aware up front what it is and how it meets the business requirements of the project.