Why is discussing architecture so hard?

In my previous post I outlined a few of the ways that application architecture fits with the Agile values.  In this post, I attempt to outline some of things that make it tough to get everyone on the same page architecturally.

As an architect I've noticed a curious thing over the years.  Most of the issues I face in software have to do with the lack of sound architectural principles or the tangled mess of monolithic style applications strung together by not following SOLID object-oriented principles.  (I include much of what I've written in the past in that mess.)  However the first thing you hear when you mention architecture is a fear that the application will be "over architected."  (This is also often confused with the term "over engineered.")  I suggested creating one class this week in a meeting and was met with, "well we don't want to over architect this."  Yikes!

Why the fear of over architecting when so much software is so poorly designed in the first place?   It didn't make sense until I had a conversation with my friend Eric.  He mentioned that if you can't visualize the architectural solution in your own head, then it just becomes an impediment.   You just can't get down that path from where you are.  And it may make you feel stupid.

No one wants to feel stupid, especially not smart people and software people are almost all smart.  No one likes to feel as if they don't know how to do something.  Talking to someone who is not an architect about architecture has a tendency to make them feel stupid.  I've noticed that even software architects don't go around talking too much about architecture.

Looking back on it, most of the time when I've heard the fear of "over architect-ing" mentioned it was followed by comments like "I can do this faster without (that)" or "that's just in the way" or "I don't understand why we need that."   I could never reconcile that with the promise of design patterns (remember: a design pattern is a solution to a problem in a context).  Applying design patterns within a solution should make you go faster, not slower.  They certainly shouldn't be more "in the way" than they are helpful.  In fact, they should allow you to talk about and refactor large chunks of functionality with ease.  More importantly, they should communicate what they do and how it might be tested.

I've watched the same developer that expressed concern about too much architecture struggle with the lack of structure.  With chaos that is barely manageable.  I've watched them struggle to go fast with so many tangled dependencies and interwoven responsibilities plaguing their objects.  I've watched them struggle with where to put concepts and ideas as the requirements change.   I've watched them be forced to use debugging to get their code into submission as the immediate feedback from QA is a string of bugs they didn't expect but which were completely predictable.

While watching all this, I'm convinced that these developers don't connect their struggles with the cause.  They'll often say things like, "Well that changed," or "no one could have predicted that" or if they are talking to non-technical people they will just mention the technical reasons for the issue.   A break when reading a feed will be blamed on it's structure, and while technically correct, the rejected architectural components which would have made adjusting to the change is rarely brought up.   

I think part of the problem is that when you first introduce most architectural style changes they do get in the way.  Most architecture is meant to prevent a problem that you don't have yet.  When you are building a house you often have a stage where you put up a ridge support of some kind.  These "ridge" beams tend to run the entire length of the building and are usually extremely heavy.  Putting up the ridge beam tends to be very hard to do.  Often it's even followed by a little celebration.

After you put in the ridge beam, it looks sorta stupid.  Here you have this huge beam, basically floating in the air supporting nothing.  Even after the walls are put on, it seems a waste.  But everyone knows it's not.  They know it holds the house together and that when all the roofing, insulation and heating components are installed, a sturdy beam like that will be a benefit, a structural requirement even.

Well, the first day you decide to move a set of classes into a control layer, or a domain layer, or a persistence layer, or a UI layer, or a services layer the added infrastructure is going to look and feel like that ridge beam sitting out there for no reason.   At this point (for the first few days or a few weeks on larger projects) it is extra, you probably could go faster without adding it.   But unlike the ridge beam, not everyone can envision it's necessity down the road.   The comment above that "I don't understand why we need it" is a killer.  And it's not just the architecture that's at risk, the success of the project is at risk.  If the efforts to put structure behind the chaos are undermined, the chaos to follow will bring the software to it's knees like a building without a ridge beam.

So that's the chasm we are trying to cross.  On one hand, Agile, RAD, and solid architecture go hand in hand.  On the other hand, some developers aren't ready for design patterns, application layering, and constant refactoring.  It threatens them because before you can refactor, you have to see at least a hint of where you want to go.  Before you can layer, you need to understand the sort of things that might share a layer - you also have to understand how the layers will stitch together and why the constrains they allow are a good thing.  You have to understand that layering is one of our few lines of defense against the chaos of dependencies.   As communicators of architecture the architects need to understand that talking about it can make smart people feel stupid.   It's tough.

It may be tough, but if we want to be RAD, do Agile, while still creating quality software we have to find a way to cross the chasm.

Suggestions?