When was the last time a technology consulting company had an Initial Public Offering (IPO)?

Exactly.  It rarely happens, and when it does, it is the exception, rather than the rule.

However, back in the pre-dot-bomb era, you had these things all the time.  Remember Scient, Viant, Sapient, MarchFirst, and a plethora of other companies that either ceased to exist, or significantly scaled down during the tech downturn of 2000-2003?  Boy, how things change.

So what changed?  Well – we have to imagine why companies decide to do an initial public offering.  In theory, it should be to get a large source of capital with which to quickly expand and gain marketshare.  Of course, we know the real reason, in a lot of the cases, was simply to get the founders very, very rich… but lets play along for a bit.  Anyone investing money in a company like one of those aforementioned consultancies was based on the premise that a consulting company can leverage the additional money to hire hordes of programmers and bill them with large margins – creating truckloads of profit.  Of course, it didn’t work, and the rest is mostly history.

One of the things we learned is that there is a constraint on how fast a consulting company can grow.  Simply put, in a well run consulting firm, your growth will be more constrained by the speed at which you can acquire human capital far more than the speed at which you can acquire financial capital.  Put another way – if you had a billion dollars, you could not create ThoughtWorks out of thin air – in a year, or probably even in five years.  Why not?

There are a couple reasons.  Even if you wanted to, you could not immediately hire 1000 great software developers who work well together, have decent client skills, and are smart enough to command the rates required to make consulting work.  You also could not simply hire 50 top partners/client principals/or “rainmakers” at once (or if you did, without massively overpaying and killing your margin).  And even if you did all that, you would not build an instant cohesive culture that creates the kind of cooperation across groups that are required to compete.

This is not to say financial capital does not matter.  It clearly does, especially on the 1st and 15th of the month when you need to make payroll.  But that said, the right combination of human capital is much harder to nurture over time and create than simply writing a check.  Put another way, MarchFirst had financial capital – lots of it, and lasted barely a year.  On the other hand, companies with Human Capital will generate the financial capital to keep things going.

This, of course, has important implications, which I will explore in the next post…

A shorter version of the essay I wrote in The Nomadic Developer: Consultopia, is up on InformIT.com.

Of all the things I wrote in my book, I think this essay was the most fun for me to write.  It is not every day that I get to yell out to the world that mission statements are basically BS pieces of marketing, or call out the inefficiencies of capitalism as it relates to pursuits that are more about intellectual capital than financial capital.

Go ahead – give it a read!.

At ThoughtWorks, my role is technical, however, as do many ThoughtWorkers, I do work on things like proposals or business development opportunities from time to time, especially when between assignments.

One thing you might be surprised about with ThoughtWorks is that the value proposition isn’t that “We are The Agile Company”.  This makes sense if you think about it – as good as Agile is, it is but merely a method.  It isn’t a destination.  Value to the client is the destination.  Agile is – at best – a means to an end.

There are clients – who… gasp… don’t care how we get the results.  We could get them by hanging bats upside-down in a glowing cave where we serve coffee to leprechauns.  They don’t care how (within, of course, the bounds of ethics and law) – they just care that they have a quality solution that meets their business needs at the end of the project, while not burdening them with endless technical debt.

Results matter more than process.  The problem, historically, with waterfall, and more recently, it’s uglier twin, scrummerfall (the process of doing scrum without TDD and other quality oriented practices) … is that at best, when they work, they give you a solution that might meet the business needs as they existed at the start of the project.  The traditional constraints of software development dictate that when you have quality, scope, and time each in contention (as they always are) – the one that is hardest to measure typically loses.  In software, quality is historically hardest to measure, which means it is the first to go.  Viola, unmanaged, hidden, technical debt.

So what do you do when your client does not want to see the sausage factory?  Well – that is where you have to focus on quality.  What I want to see more of in contracts are success criteria that state a required level of quality, measured by (albeit imperfect) metrics that help us get there.  Test coverage is a good start – but there are many others.

Another success criteria is that a late change of some definable complexity should be achievable at some definable, lower than what it would be without TDD, cost.  I would love to see some work done to find a way to put something to this effect in a contract clause (maybe some friendly contract attorney can help me there).

Competition will always, at some level, come to price for a solution.  However, smart procurement folks will make sure to ask that the quality and adaptability variables be controlled, so that when prices are compared, they are comparing the price of Morton’s Steak to Ruth’s Chris, not Morton’s Steak to Sizzler.  Agile is just a means to delivering a solution that scores well not just on the cost variable, but on the quality and changeability ones as well.  The business value that comes from solutions that do not take on unmeasured and ill understood technical debt?  That is what, in my opinion, a significant part of the ThoughtWorks value proposition.

Yes, it’s true – I finally made the jump.

Magenic (my employer through the end of May) was – and is – a great place to work.  Leaving my Magenic has been one of the hardest decisions I have ever made.  However, given the opportunity to not just do agile development – but do agile development side by side with the inventors of agile development, it was just not an opportunity I felt would be wise to pass up.

June 1, 2009, my journey with ThoughtWorks commences.  Stay tuned!

In my book, The Nomadic Developer, I have a section of the book that defines various terms in a section called, appropriatley, Appendix B, A Consulting Lexicon.  One term I define is a new term I like to call “Faith Driven Development”:

Faith Driven-Development

A development practice also known as “code n’ pray” or “Waterfall”. It relies on an ever knowing business analyst write an Old Testament like book known called “The Requirements”; non-coding architect to write New Testament like book called “The Design”. Once those are done and cast in stone tablets, a developer (you) is called upon to code the system, with results in a situation that resembles the last chapter in the New Testament, called “Revelations”, in which the forces of good and evil fight on a giant battlefield, with massive bloodshed (a process known as “Testing or Quality Assurance”) after which there is a promise of a thousand years of utopia.

Sadly, it seldom ends this well.

Of course, as you might guess, when you run into the term “Waterfall”, you get redirected to “Faith Driven Development”.  I will admit up front that I am no fan of waterfall methods.   The largest reason why I am no fan of such methods has to do with what I believe is going on, in reality, when a waterfall project is sold:

A confidence trick or confidence game (also known as a bunko, con, flim flam, gaffle, grift, hustle, scam, scheme, or swindle) is an attempt to defraud a person or group by gaining their confidence.

(from Wikipedia)

In software, this game is played when someone sells a certainty of outcome when such certainty of outcome far less likely than he or she perports it to be.  The history of waterfall projects – especially waterfall projects with durations longer than a sprint or two – tells us that outcomes are highly variable as to when such a project will complete, if it is completed successfully at all (see Standish Chaos report, among other sources).  This is nothing new, of course.  The reason companies today ask us to underwrite fixed bid projects is to use it as a risk sharing mechanism to presumably reduce the chances that they will be conned, by having a company underbid the project, and then the project’s momentum to continue billing indefinitley.

There is, in effect, a race to the bottom effect in this business that means a consultancy can go out of business – or at least lose a lot of bids, because, buyers, will rush to firms that will tell them what they want to hear.  For the same reason that I might be tempted to try a snake oil that made it such that I could lose weight without effort, because there is demand for custom software with certainty of cost outcome, there will always be companies trying to sell into that market.  The toughest part of selling agile is being an honest broker in a world that accepts selling snake oil as viable business practice.

Which gets to my main point.  It is bad enough trying to be honest when many in the business world want desperatley to believe “the big lie of waterfall certainty”.  When the downturn comes though, the race to the bottom intensifies, resulting in some of the worst, riskiest projects known to man.  These projects are the waterfall projects sold during the downturn.

So what happens in a downturn that makes this risky?  First of all, competition intensifies.  You don’t just have two players telling the buyer what they want to hear, you have four or five.  Second, downturns cause short term to come into high focus.  If you are an account executive, and you are way short of quota, you might quote something at 50% of it’s true cost just to get  the deal and buy yourself some job hunting time.  If you are running an office and have a bench, you might even assume that you can cover any mistakes by throwing your bench at the problem – despite that such a move probably turns your net margin on the deal negative – even based on your most ideal assumptions.

In other words, the things that make waterfall a bad idea for consulting – the need to make promises based on hubris rather than observable facts, become even worse when consulting during a downturn.  During a downturn, the promises tend to be bigger.  The idea of forcing developers to work unpaid overtime becomes not only possible, but probable.  Quality not only goes down, it flat out gets cut from the budget (i.e. we all have seen QA become the first to go in such scenarios).

I know we all want certainty.  Nobody wants to just keep throwing money at a problem – or even at a solution, without some basis for finding out what the return on investment is going to be.  The problem, however, is waterfall is simply bad risk management.  It spends all the time mitigating change and process controls, while saying very little about how people actually work.  Waterfall and more old school PMBOK approaches don’t deal with:

• The fact that velocity picks up during a project – if you have the same team, based on the team growing in understanding of the domain
• Changes occur late in cycle even if the business didn’t change – based on increasing understanding

In fact, if you are doing steady “earned value management” type approaches with a presumed fixed set of deliverables, because average velocity starts low and picks up, you essentially start the project from behind.  Because you start from behind, you demoralize the team – and create the initial tendency to hide information about status that goes up the chain (remember – bad news tends to get hidden).

Waterfall assumes that people don’t learn anything valuable by doing.  It assumes that change is to be controlled, not harnessed.  It lies to it’s sponsor about team dynamics – creating the basis for “cover your butt” project management that stands behind dysfunctional teams and negative suprises.  In an upturn, at least when the project fails, you can get another job.  In a downturn, for you, and the sponsor, it is a far riskier problem.

When I conceived of this post, my first thought was to say “maybe you should do a risk matrix” to be transparent about the risks… but the more I thought, the more my instinct is to “just say no”.  I think I will go with that for now.

As friends of mine who became victims of the recent US economic crisis start to look for jobs, I am often getting, usually over (a few) beers, very scary stories about what it is like interviewing today.  Sadly, it is reminding me that not much has changed since last time I interviewed during hard economic times, back in the early 2000s.

Lets face it, technical people are mostly horrible at technical screening.  We choose questions from lists, like a famous one Scott Hanselman put on his blog a few years back.  And then if someone knows a sufficiently high number of trivia questions, we assume they will be a good developer, and the next phase starts, which usually involves making sure they shower, shave, etc. – and are not a total psychopath.

The real issue here, of course, is that forming good technical interview questions extremely difficult.  Ideally, most companies would hire for aptitude, since we all know that good developers learn things very quickly – that is part of what being a good developer means.  Sadly, many still do not.

Let me illustrate through a story.  Lets say that, for sake of example, someone was hiring a developer to build something useful, like the i4o indexed Linq framework that I wrote a couple years ago – and continue to maintain to this day.  My guess is that if someone else were trying to build such a framework in 2007, given that in 2007 I had zero Linq experience – other than a vague idea of what it was for, I would have never been even granted an interview, much less passed it.

I, like most developers, learn things when I need to learn them, rather than just studying trivia just for fun.  Thus, I learned Linq – because I wanted to write an indexing framework.  On a Friday night.  During which time I wrote 500 lines of code.  By the end of that architectural spike, I had the essence of i4o working.  I then spent the next Saturday – another 8 hours or so, cleaning things up and handling some corner cases.  No experience to working software that – now – actual software companies like Component ONE are copying and selling for money (see LiveLinq).

The point isn’t that I was, or am, some sort of genius.  Actually, I am a rather ordinary software developer.  The thing is though that despite all the world’s APIs, I understand computer science.  In college, we had classes for:

• Algorithms and learning the meaning of computational complexity (i.e. why a hashtable is faster than a linked list – and why the former takes up far more space)
• Software Design – where you learn about the general value of reducing coupling and limiting scope of variables (the “fewer moving parts” bit of software engineering)
• Compilers – so you understand what happens when you, well, compile from the language into native code
• Language Paradigms – Procedural, OOP, Functional (F#, Haskell, etc.), Logic, and yes, God’s Language, Lisp
• Operating Systems – so you understand the environment of most things you will write, what processes and threads are, etc.
• Databases – so you understand how things generally get persisted, normal forms, and so forth

Frankly, everything important I know about software architecture is in one of these, or a small number of other more fundamental areas.  When I operate in a programmer role, I only decide to understand what I need to understand when I need to.  Lets put it this way, if I had to design a multi-threaded system in Java in 3 days, I probably could.  Not to take anything away from Java gurus, but given I know that Java is a language that generally runs on a JVM, that it probably has a thread system, that there are probably some libraries people use to manage that and not have to operate on thread primitives most of the time, I could be reasonably competent in not very much time.

But this is really the symptom of the larger problem.  The real issue is that some companies, when they hire software developers, hire them like contractors.  I understand testing for how well someone knows a given part of an API if all you are going to do is bring them as an expert in that API for a month.  In consulting, we run into that interview all the time, and expect it.

The problem is when companies who are hiring for long term – who should care about very different things, start with questions like “what are 4 methods supported by all delegates in .NET? (know 3… not good enough… gimme 4 you idiot!)”.  I am convinced that people who do this kind of interviewing really don’t care that much.  They want an easy screen that is easy to administer, makes them feel smart (because it is their own favorite API), and gives them plausible deniability if a hire fails (well, he passed the “hardest test we had”, not my fault he came in and messed up the group).

Real interviewing is hard.  I don’t know all the answers, but a a good starting place is Joel’s “Guerrilla Guide to Interviewing“.  And for the candidate, frankly, I would tell you, strongly consider whether you would want to work for an organization that relied on quiz show, trivia question interview techniques.  While not reliable, there is a pretty good unscientific correlation between “cultures of asshole” and places that use quiz show interviews.

Now that my book is largely done, I finally have time to focus on some things that have not received the attention they deserve lately.  Thankfully, Jason Jarett was kind enough to introduce some great new ideas to i4o, which after having mulled over them for far too long, I am releasing into the wild as of this weekend.

Details of the changes are nicely documented on his blog.

The jist of the changes though are to introduce a better way, leveraging lambdas, to specify an index (the new IIndexSpecification<T> interface).  Prior to this release, you either had to use attributes – something you can only really do with code you control, or you had to use strings to specify the property.  IIndexSpecification<T> is a nice seperation of those concerns, so that your classes can truly be not concerned about how they are being indexed (as we have done with the POCO changes from the previous release), while more easily being refactored (i.e. member name change refactoring will now work with indexes).

In addition, performance improvements to reduce the number of lookups using reflection have been introduced as well.

I really owe a huge debt of gratitude to Jason, who took it upon himself to introduce these features and share them with the world.  I am very excited about how this has evolved over the past year and a half.

As for where we go next, now that I am done with the book and have more time to work on open source projects, the next round of changes will (finally) widen the number of query types that can leverage the index, similar to the work Rocky and I did with indexing in CSLA.  Look for that sometime in April/May of this year.

In IT, too often, we are viewed as a cost, when we should be considered an investment.

Are you measuring ROI? We sure do talk about it a great deal. But when asked, few can point to the actual rate of return on their historical investments. This, as it should, create skeptical investors – which in your case as a CIO or IT Director, are the people that sponsor your projects.

That all said, IT as a cost center is an idea that needs to die. This presentation digs deeper into that idea, and presents a framework for finding those places where you can start finding strategic technology investments that you can make in your own company.