The Nomadic Developer

Now that Agile has passed the 10-year mark, many people are starting to wonder what the next step should be in the evolution of Agile. As we start to think about what’s next, it doesn’t hurt to think for a moment about how we got here in the first place. As the saying goes, it’s hard to know where you’re going if you don’t know where you’ve been!

So let’s turn back the clock through the mists of time in the years leading to the Agile Manifesto in 2001. Back when this movement started, many of the “reasons for Agile” just seemed very intuitive. People over process? Sure. Responding to change versus following a predefined plan? Common sense. On the surface, few people would disagree with the assertions made in the Agile Manifesto. But is that enough? Can you following the Manifesto—do it all by the book—and be guaranteed to create software that delivers economic benefit? No. By itself, Agile doesn’t lead to business value!

How Can By-the-Book-Agile Fail?

Let’s be clear. With its increased collaboration with the customer, more frequent releases, and increased engineering and testing discipline, Agile makes delivering value more likely. It’s certainly a vast improvement over multi-year “too big to fail” Waterfall software projects. But even if you do everything right, even if you have the best practitioners in the world building your product, you can still create a product that fails to make money.

Agile—and here we mean the kind that includes the right engineering practices, such as test-driven development (TDD), pairing, SOLID principles, automated acceptance testing, and continuous integration—will ensure that you create a product that works…in the lab, at least. However, despite your best efforts, you might build a very functional eCommerce site that tries to sell something that nobody really wants. Or you might build an internal application that can’t go beyond the lab because the operations people can’t support it in production. These issues—and many other things outside direct control of the team practicing Agile—can thwart even your best efforts.

Article continues here

Many of you will remember Revenge of The Nerds, that fine classic movie where a bunch of, well, nerds take over the campus of Adams college by outsmarting and outwitting the jocks. For people who work in computers of a certain age and disposition – say, a late 30s geek from a western culture like myself who might have fit the nerd stereotype at various points in his early upbringing – the movie was somewhat influential. It told a story that, translated into economist-speak, equates to “intellectual capital may someday trump other kinds in terms of economic value creation.”

As it seems to have turned out, we are now in the Revenge of The Nerds economy. Despite 9%+ unemployment in the general economy, overall unemployment for software engineers is a tad below 5%. But this data, which is compelling enough, does not tell the entire story.

The bigger story is around what kinds of things are seeing investment. There is talk of another bubble right now in technical startups. If you are a technology based startup that has reached “Ramen Profitability”, you will almost certainly attract capital. Things are now even getting to the point where we are seeing companies that lack profits going to the IPO market (think Pandora and Groupon) – something that has been out of vogue since at least 2001. One could make the case that we will soon cross the line where it is easier to get a startup funded than it is to get a jumbo mortgage.

If you work in tech, there is a good chance you are feeling this, at least if you are currently employed and working for an employer that has some level of visibility. If you work for Google, Facebook, or some other high tech company, you likely do not have to work that hard to get a new job offer. Notwithstanding the cruel irony that people who are unemployed are often being systemically discriminated against, the job market for really good, currently employed software developers is as robust has it has been in 10 years.

So if the nerds are all right, mostly, what about the jocks? What occupations did they end up in? While some of them may have made it to the NFL as professional football players, most others end up in spilling into the general job market. While the stereotype is that jocks end up in menial jobs, construction, or manufacturing; research being hard to find, my own experience of knowing several such folk points to careers tending to be sales, low-end finance (think mortgages), real estate, and personal training. Given, a sample size of a couple dozen doesn’t really prove anything. However, if one did extrapolate, one could come to a conjecture that while “jocks” for lack of a better word, do not do worse then average now, the nerd/jock investment and employment dynamic has changed.

Why the change? Now that simply taking big leveraged risks with a big pile of money isn’t in fashion (i.e., as it was before the global financial crisis), you need advantages from superior intellectual capital in order to sustain a higher than mean return on investment. Why not in fashion? From 2003-2008, it was accepted that financial engineering was the way to riches. If you only structured your collateralized debt obligation in the correct way, you could invest at 40x leverage and still retain a AAA rating on your debt. Financial engineering gave you higher profits under such a regime than traditional “engineering engineering” could provide. So money flowed into CDS structures and away from the nerdy parts of the economy that invents things.

So now we find ourselves in an economy where capital flocks to things like Pandora, Groupon, Facebook, Linkedin, and other things that, at their core, have interesting algorithms inside them. And we have a situation where if you are a developer capable of writing an interesting and valuable algorithm – or helping a company scale out a system that leverages one of these inventions, you are in high demand.

Good thing for the nerds. What this means for everyone else remains to be seen.

Imagine you have a need to take one type, which may or may not be a discriminated union, and see if it “fits” inside of another type.  A typical case might be whether one discriminated union case would be a possible case for a different discriminated union.  That is, could the structure of type A fit into the structure of type B.  For lack of a better word, I am calling this “structural similarity”.

Lets start with some test cases:

module UnionTypeStructuralComparisonTest
open StructuralTypeSimilarity
open NUnit.Framework

type FooBar =
 | Salami of int
 | Foo of int * int
 | Bar of string

type FizzBuz =
 | Toast of int
 | Zap of int * int
 | Bang of string

type BigOption =
 | Crap of int * int
 | Bang of string
 | Kaboom of decimal

type Compound =
 | Frazzle of FizzBuz * FooBar
 | Crapola of double

[<TestFixture>]
type PersonalInsultTestCase() =

 [<Test>]
 member this.BangCanGoInFooBar() =
 let bang = Bang("I like cheese")
 Assert.IsTrue(bang =~= typeof<FizzBuz>)
 Assert.IsTrue(bang =~= typeof<FooBar>)
 Assert.IsTrue(bang =~= typeof<BigOption>)

 [<Test>]
 member this.KaboomDecimalDoesNotFitInFizzBuz() =
 let kaboom = Kaboom(45m)
 Assert.IsFalse(kaboom =~= typeof<FizzBuz>)

 [<Test>]
 member this.SomeStringCanBeFooBar() =
 let someString = "I like beer"
 Assert.IsTrue(someString =~= typeof<FooBar>)

 [<Test>]
 member this.SomeFoobarCanBeString() =
 let someFoobar = Bar("I like beer")
 Assert.IsTrue(someFoobar =~= typeof<string>)

 [<Test>]
 member this.SomeFoobarTypeCanBeString() =
 Assert.IsTrue(typeof<FooBar> =~= typeof<string>)

 [<Test>]
 member this.CompoundUnionTest() =
 let someCompound = Frazzle(Toast(4),Salami(2))
 Assert.IsTrue(someCompound =~= typeof<FooBar>)

To make this work, we are going to need to implement our =~= operator, and then do some FSharp type-fu in order to compare the structure:

module StructuralTypeSimilarity

open System
open Microsoft.FSharp.Reflection
open NLPParserCore

let isACase (testUnionType:Type) =
 testUnionType
 |> FSharpType.GetUnionCases
 |> Array.exists(fun u -> u.Name = testUnionType.Name)
let caseToTuple (case:UnionCaseInfo) =
 let fields = case.GetFields()
 if fields.Length > 1 then
 fields
 |> Array.map( fun pi -> pi.PropertyType )
 |> FSharpType.MakeTupleType
 else
 fields.[0].PropertyType 

let rec UnionTypeSourceSimilarToTargetSimpleType (testUnionType:Type) (targetType:Type) =
 if (testUnionType |> FSharpType.IsUnion)
   && (not (targetType |> FSharpType.IsUnion)) then
 if testUnionType |> isACase then
 let unionType = testUnionType
  |> FSharpType.GetUnionCases
  |> Array.find(fun u -> u.Name = testUnionType.Name)
 let myCaseType = caseToTuple unionType
 myCaseType =~= targetType
 else
 testUnionType
 |> FSharpType.GetUnionCases
 |> Array.map( fun case -> (case |> caseToTuple) =~= targetType )
 |> Array.exists( fun result -> result )
 else
 raise( new InvalidOperationException() )

and UnionTypeSourceSimilarToUnionTypeTarget (testUnionType:Type) (targetUnionType:Type) =
 if (testUnionType |> FSharpType.IsUnion)
  && (targetUnionType |> FSharpType.IsUnion) then
 if testUnionType |> isACase then
 targetUnionType
 |> FSharpType.GetUnionCases
 |> Array.map( fun u -> u |> caseToTuple )
 |> Array.map( fun targetTuple -> testUnionType =~= targetTuple )
 |> Array.exists( fun result -> result )
 else
 testUnionType
 |> FSharpType.GetUnionCases
 |> Array.map( fun case -> (case |> caseToTuple) =~= targetUnionType )
 |> Array.exists( fun result -> result )
 else
 raise( new InvalidOperationException() )

and SimpleTypeSourceSimilarToUnionTypeTarget (testSimpleType:Type) (targetUnionType:Type) =
 if (not (testSimpleType |> FSharpType.IsUnion))
  && (targetUnionType |> FSharpType.IsUnion) then
 targetUnionType
 |> FSharpType.GetUnionCases
 |> Array.map( fun u -> u |> caseToTuple )
 |> Array.map( fun targetTuple -> testSimpleType =~= targetTuple )
 |> Array.exists( fun result -> result )
 else
 raise( new InvalidOperationException() )

and SimpleTypeSourceSimilarToSimpleTypeTarget (testSimpleType:Type) (targetSimpleType:Type) =
 if (testSimpleType |> FSharpType.IsTuple) && (targetSimpleType |> FSharpType.IsTuple) then
 let testTupleTypes = testSimpleType |> FSharpType.GetTupleElements
 let targetTupleTypes = targetSimpleType |> FSharpType.GetTupleElements
 if testTupleTypes.Length = targetTupleTypes.Length then
 let matches = Array.zip testTupleTypes targetTupleTypes
 |> Array.map( fun(test,target) -> test =~= target )
 not (matches |> Array.exists( fun result -> not result ))
 else
 false
 else
 testSimpleType = targetSimpleType

and (=~=) (testObject:obj) (targetType:Type) =
 let objIsType (o:obj) =
 match o with
 | :? Type -> true
 | _ -> false

 let resolveToType (o:obj) =
 match objIsType o with
 | true -> o :?> Type
 | false -> o.GetType()
 let testObjectIsAType = testObject |> objIsType
 let testObjectTypeIsUnion =
 match testObjectIsAType with
 | true -> testObject |> resolveToType |> FSharpType.IsUnion
 | false -> false
 let targetTypeIsAUnion = targetType |> FSharpType.IsUnion 

 let resolvedType = testObject |> resolveToType

 match testObjectIsAType,testObjectTypeIsUnion,targetTypeIsAUnion with
 | false, _, _ -> resolvedType =~= targetType
 | true,true,false -> UnionTypeSourceSimilarToTargetSimpleType resolvedType targetType
 | true,false,false -> SimpleTypeSourceSimilarToSimpleTypeTarget resolvedType targetType
 | true,true,true -> UnionTypeSourceSimilarToUnionTypeTarget resolvedType targetType
 | true,false,true -> SimpleTypeSourceSimilarToUnionTypeTarget resolvedType targetType

Getting this to work seemed harder than it should.  While my tests pass, I am sure there are both cases I have not yet covered, and probably some simpler ways I could accomplish some of the same goals.

While this is a work in progress, if anyone has any thoughts for simpler ways to do something like this, I am all ears.

In 2003, Harvard Business Review published Nick Carr’s seminal essay, “IT Doesn’t Matter.” I remember that month well. The previous years of the PC boom, followed by the dotcom boom, had seen a tidal wave of money spent on technology. Much money was wasted on heavy investment in systems that either sat unused on a shelf, or ultimately were scrapped. To top it all off, the big “emergency” of the time, the millennium bug, turned out not to be as bad as everyone had feared. The time was rife for someone to declare the strategic irrelevancy of technology, and Mr. Carr stepped in and made the case. Because many executives desperately wanted to reduce technology spending at the time (in the aftermath of all that gluttony), reading Carr’s article was like receiving manna from heaven.

As a technology practitioner myself, however, it should surprise no one that I believe Mr. Carr to be wrong. Regardless of any biases I may have, the reality is that his argument contains several holes that have become ever more glaring with the passage of time. My intent with this article is to shine a light on what’s wrong with the “IT doesn’t matter” argument. I’ll even go further: I’d like to demonstrate why, now more than ever, investing—not just in technology, but in the intersection of technology and people—is the most important action that a company or organization can take.

The Company That Couldn’t Run a Second Shift

In my years as a technology consultant, I’ve certainly seen what happens when you don’t invest in technology. One client I vividly remember—we’ll call it Company X—had quite a conundrum. An amalgamation of nine different manufacturing organizations built up over time, Company X spent very conservatively on technology. When making an acquisition, Company X would do as little as possible on integration; for example, only enough to get the accounting systems and the order-entry systems sending orders to a central mainframe.

Over time, this management behavior at Company X resulted in a “Rube Goldberg” system requiring the work of hordes of COBOL programmers just to keep running. Overnight order processing required more than 200 manual steps. The maintenance cost of this Frankenstein’s monster of a codebase included at least 40 full-time COBOL programmers, a couple of dozen system operators, and who knows how much management hierarchy on top of all that. Company X was spending an estimated $15 million annually just to keep the system at its current level of functionality.

Unfortunately, that $15 million wasn’t even the most significant problem. When demand increased, Company X decided to run a second shift. The issue was that the system literally could not process orders in greater than about a 12-hour window. Moving to a 16-hour window, and eventually to a round-the-clock business, would require either radical changes to the old system or building an entirely new system—and it would have to match the quirks of the old system, in order to maintain compatibility with the nine other satellite systems!

Business analysts estimated that adding a second shift would increase gross margin in this low-margin business by around 3%, while increasing top-line revenue by over 20%. Moving to round-the-clock processing was also a precursor to being able to sell into new global markets, which would have increased top-line revenue even further.

Sadly, none of these happy improvements could come to pass for Company X.

The problems at this company were many, of which technology was merely one aspect. The deeper issue was that the operators and programmers who understood that mess of a codebase had a vested interest in keeping the current technology set running. But even if all those people were replaced, technical debt in Company X had built up over a long period of time, to the point that the company would need to invest around 2–3 times its annual profit in order to get out from under the load of technical debt.

In this culture of hubris, fear, and inertia, any new technology initiative to pay off the technical debt would almost certainly be killed. It was literally a recipe for company bankruptcy. When competitors are blowing you out of the water with 3% higher margins and 20% higher top-line revenue, taking that increased profitability and investing it into further productivity-enhancing technologies, it’s hard to see why “IT doesn’t matter.” If you’re a CEO in a company that can’t compete because of technical debt, you almost certainly will understand just how much IT does matter.

The rest of this article is available at Pearson Education’s InformIT website here.