How Startups can work with Big Companies and not get Killed

(Note: this post was originally published on the Wonolo blog under the title “Collaborate. Innovate. Top Tips for How Large Enterprises and Startups Can Have a Winning Partnership”)

Recently, I was invited by Unum, one of our FORTUNE 500 customers, to participate in a panel session about corporate innovation at Maine Startup and Create Week. At the heart of our discussion was how large companies like Unum can be more innovative and how startups and large companies can work together toward this goal.

It‚Äôs a topic that‚Äôs close to my heart: I spent the first part of my career in the wireless industry, and back in 1998, I was a part of the founding team of Symbian, one of the first operating system platforms for smartphones (although they weren‚Äôt yet called ‚Äúsmartphones‚ÄĚ at that point).

Symbian was a joint-venture between Nokia, Ericsson, Motorola, Psion, Panasonic and, later, several others. Their rationale for investing in Symbian was a desire to have a common software platform for smartphones. However, what these companies actually had in common was that they were large, bureaucratic, and they were arch-competitors.

Getting these large companies in the Symbian joint-venture to work together was somewhere between very hard and impossible. (For the full story, see David Wood‚Äôs excellent book.) The term of art at the time was term ‚Äúcoopetition,‚ÄĚ and it didn‚Äôt work. None of the participants in Symbian really had any desire to share their product plans with their competitors.

So, the irony was that, while Symbian was arguably at the spearhead of technology innovation, it was frequently stymied from actually being innovative by the inertia and culture of its participants. This left the market open to more focused, agile and independent companies like Google and Apple to dominate the smartphone market of today. In contrast, Nokia had an ignominious end – broken up and sold off, with billions of dollars in market value destroyed.

So, fast-forward to 2016 and my panel discussion at Maine Startup and Create Week…How can big companies be more innovative, and how can startups and large companies work together to the benefit of both?

Designer, Builder or Maintainer?

First, let‚Äôs take a look at the kinds of people that tend to work at startups versus larger companies: I have a simplistic but hopefully powerful model that divides people into three groups – ‚Äúdesigners,‚ÄĚ ‚Äúbuilders‚ÄĚ and ‚Äúmaintainers.‚ÄĚ

Let’s use an analogy: here in San Francisco, arguably our best-known symbol is the Golden Gate Bridge Рjust look at any tourist tchotchke.

If we think about the Golden Gate Bridge, first there were the designers. In our culture, the designers generally have the ‚Äúsexy‚ÄĚ job – they are the visionaries.

GG_Bridge_Plans.png

Next come the ‚Äúbuilders‚ÄĚ who actually constructed the Golden Gate Bridge.

GG_Bridge_Maintainers.png

Last come the ‚Äúmaintainers.‚ÄĚ ¬†These are the workers who hang on ropes off the bridge, scraping off rust and continuously repainting it in International Orange.

International_Orange.png

This is the least sexy job in most people’s eyes: which would you rather be Рthe visionary designer of the Golden Gate Bridge or someone who hangs off it on a rope, scraping rust?

Now, in a startup, what you need for success are just a few designers Рthese are typically the founders.  You can’t have too many because they tend to butt heads.

What you really need for a startup is a boat-load of builders: these are the doers – people that create and Get Shit Done (GSD). Builders are the backbone of any startup.

What you don’t need in a startup is maintainers: everything in a startup is being created anew so there isn’t anything to maintain. You’re also focused on growth rather than optimization.

Contrast that to a big, established company: there, most people are maintainers. Their job is to ensure that an already successful business continues to be more successful. They are there to grease the wheels and optimize.

So What?

What this means is that there is a cultural mismatch between a large company and a startup.

At the core of the startup mindset is a willingness to fail and an acceptance of it. In a startup, failure is the norm – as the clich√© goes, you fail your way to success. Since “failure” has negative connotations, I think it better to simply reframe it as ‚Äúlearning.‚ÄĚ

Another important aspect of building a startup is understanding the art of the ‚Äúgood enough.‚ÄĚ Because you‚Äôre bandwidth-constrained, you are forced to be very selective and very efficient in how you do things. You have to get them done quickly. You have to not let the great be the enemy of the good. You have to focus on delivering 80% of perfection for 20% of the effort.

Naively, when large companies aspire to become more innovative, they trot out clich√©s like ‚Äúwe reward risk-takers.‚ÄĚ This is a lie. The last thing you want when you have a large company generating billions in revenue it to have some cowboy risk-taker come in and break it. What you want are maintainers to keep it working and keep it generating billions of dollars.

To take it back to the Golden Gate Bridge example, would you want a maintenance worker who said, ‚ÄúLet‚Äôs see what happens if we take all the bolts out‚ÄĚ?

I think it would be better to rephrase it as, ‚ÄúWe reward people who make small, smart bets.‚ÄĚ Making a series of small, smart bets to test various hypotheses is the basis of iteration, and iteration is the how you build great products and great companies.

Recognizing these problems, many large companies have started to take a different approach Рthey have created specific initiatives intended to foster and drive innovation. Wonolo itself was created through The Coca-Cola Company’s innovation program.

Creating a Great Corporate Innovation Program

So, how can a large company create an innovation group and/or program likely to succeed? These initiatives can take various forms, but I think these are the most important elements:

  1. Set money aside Рthe budget for innovation can’t come out of the normal, operating budget for any existing business unit. If it does, it competes with the budget needed for maintenance of what’s already working.
  2. The innovation group must report directly into the CEO – this demonstrates genuine commitment to innovation and also helps unblock bureaucracy.
  3. Be clear with objectives – what specifically are you hoping that the innovation program does for your business? What are you looking to achieve? How does it positively impact your core business?
  4. Build the right team Рa good mix is designers and builders from outside of the organization, along with some inside players who can help navigate the existing organization, as long as they carry enough weight. You will also need to reassure your best maintainers that they should stick to what they do well rather than trying to join the innovation program because it’s sexy.
  5. Accept failure – as discussed above, you must accept that failure is a vital part of the process. Not all initiatives you start or companies you fund will be successful, but you will learn something important from each.

How Can a Startup Engage with a Big Company and Win?

Big companies can kill startups. I’ve seen it happen.

Big companies can lead startups on and consume lots of their time and bandwidth with no pay-day. At the end of the process, the large company has perhaps lost a few hundred thousand dollars. Meanwhile, the startup has run out of funding and is dead.

Here’s what I’ve learned (the hard way) to avoid that outcome:

  1. Find your champion. Ted Reed is our champion at Unum. Not only is he an all-round great guy, but he also understands the need to be completely transparent with us. A great champion is your guide to the large company Рits structure, how it makes decisions and the key players you’ll need to win over.
  2. Seek trust, honesty and transparency – any great relationship is built on mutual trust. Get feedback early and often (from your champion) on your likelihood to succeed.
  3. Don’t over-invest until you have clear commitment Рbe prepared to scale back or end the relationship if it’s not clear you are on a path to success. The opportunity cost of your time in a startup is huge. Don’t do anything for free Рfree means there’s no value, and it won’t be taken seriously.
  4. Ensure clarity in objectives, value and define success – if both sides are not clear on the business value that your product or service is providing to the large customer, be very cautious. Make sure both sides agree on what success means.
  5. Start with a small, well-defined trial Рrather than trying to boil the ocean, it’s wise to start with a trial that demonstrates the value your product or service provides to the large company. This has less risk, requires less investment and has a higher likelihood of success. For more tips on how to best go about setting up a pilot, check out our related blog post.

How Can a Big Company Engage with a Startup and Win?

On the other side, how can big companies successfully engage with startups and win? Here’s my personal recipe:

  1. Be honest and transparent Рdon’t lead startups on. Be honest about chances of success and what it will take.
  2. Be respectful of bandwidth and provide funding, if possible Рrealize that a startup’s most precious commodities are bandwidth and funding. Do everything you can to reduce the sales cycle. Structure the deal to provide the funding and/or revenue necessary for the startup to succeed.
  3. Have realistic expectations in terms of maturity of a startup and its processes -don’t try to apply your pre-existing vendor onboarding process when engaging with a startup. For example, a 10-person startup won’t pass your 50-page IT security audit.
  4. Respect the need for independence Рyou may be providing a startup with revenue, funding and a great customer reference. However, a startup needs to be in control of its own destiny and own its own product roadmap. Don’t treat a startup like a consulting company or development shop, unless that’s how the startup sees themselves.

Overall, the relationship between a large company and a startup can be a marriage made in heaven. I would marry Ted Reed if I could.

How to Fire Someone Humanely

I‚Äôm surprised by how often I encounter someone in a leadership role who has never had to fire anyone. I suspect it‚Äôs a combination of technology companies generally having pretty flat organizations and also the tendency to have dedicated HR functions in larger companies that insulate people from any ‚Äúunpleasant business‚ÄĚ.

Whenever I’ve had to fire someone, I’ve not slept well the night before. Regardless of the reason, you are changing someone’s life and everyone deserves to be treated humanely. It definitely gets easier after you’ve done it a few times but I hope that it never becomes routine.

Here are my 10 recommendations for how to fire someone humanely:

1.  Immediately set the tone of the conversation upfront.  
It gets super-weird if you have a friendly ‚Äúhow are you?‚ÄĚ conversation and then fire someone. ¬†I normally get straight down to business and open with ‚ÄúI’m sorry we have to have a difficult conversation today‚ÄĚ before the person¬†even sits down.

2. ¬†Do not use the word ‚Äúfire‚ÄĚ.
The word ‚Äúfire‚ÄĚ creates an emotional reaction – to‚Äúfire‚ÄĚ someone implies an active act of aggression. ¬†I generally say¬†‚Äúthis will be your last day‚ÄĚ. ¬†That way, it‚Äôs not a value judgment or a process – it‚Äôs just a fact.

3.  Keep it short.
There is no point in dragging it out. ¬†But, you should at least give the person a short and true reason as to why it’s happening, unless there is a good legal reason not to.

4. Do not get drawn into arguments.  
If someone wants to argue, be clear that this is the decision, it’s already made, that you understand they are angry but that it’s not beneficial for either side to drag it out.

5. Have your paperwork in order and be aware of the local employment law.
For example, here in CA, it’s the law that you have to give an employee their final pay in the form of a check when they leave.

6. Say you’re sorry it didn’t work out.
It helps humanize you in the process. It might seem trite or hollow to say¬†‚Äúsorry‚ÄĚ but, on balance, I believe it helps soften the blow.

7. Don’t fire someone on a Friday.
You’ll read¬†conflicting recommendations in this regard but I am strongly in the camp that says firing someone on a Friday is a Bad Idea(tm). ¬†It means they are likely to¬†just seethe about it all weekend. ¬†If you fire them during the week, they are more likely to focus on finding another job.

8. Have others ready to cut the cord.
You will need to confide in a small group of people ahead of time so they are cued up to terminate account access, etc as soon as the person leaves the room. Create a list of accounts ahead of time so it is not a rush and nothing gets missed.
You may be tempted to think something like “I’ll just leave Dave’s email on until the end of the day”. Resist this temptation at all costs. Although this may seem counter to the “being humane” approach, the potential benefits in terms of seeming more humane and feeling better are massively outweighed by the downside risk of the person doing something stupid.

9. Treat the person with respect and try to make it as comfortable as possible.
If possible, try to make sure that their team mates are not around to gawp as the person clears their desk, etc. This will again require the confidence of someone you trust.

10. It should not be a surprise.
Firing someone should be culmination of a process of clear and honest communication over weeks if not months.  If you are a good manager, the person on the receiving end should be clear on the gap between what is expected of them and their performance.
With the exception of gross misconduct,¬†if there hasn’t been such a dialogue,¬†you probably shouldn’t be firing the person.

So, you inherited a codebase… just how screwed are you?

In a perfect world, every software developer would finish school, build a product from scratch, IPO¬†the company and retire. However, that’s very, very rarely the case and the majority of software people will at one time or another in their careers inherit a codebase that they weren’t involved in creating.

The most common situations where this happens are when a company is acquired by another and when joining a company (because of staff growth or turnover).  But, regardless of the reason, the software developers, managers and executives involved are expected to take it over and continue to achieve forward progress.

The problem is that not all codebases are created equal Рthey can range from absolute disasters to dream scenarios; from the ridiculous to the sublime.  Plus, aside from the code itself, there are a number of other factors which have a huge impact on the success or failure of the transition.

So,¬†the purpose of this post is to give a framework for assessing how much of a risk the transition will be. ¬†Put another way, it’s also a framework for asking the right questions and for setting expectations at the right level.

How Screwed Are You?…a formula

Every situation is unique, but this is my rough formula to calculate the “Codebase Risk Factor” (CRF):

CRF Formula

 

If you’ve immediately started to glaze-over from the math, don’t worry, I’m going to break it down for you.

This formula factors in what I believe are the 5 main risk factors when you inherit a codebase:

  • Test Coverage (“tc“)
  • Team Availability (“ta“)
  • Team (availability) Duration (“td“)
  • Defect Find/Fix Ratio (“ffr“)
  • Age of Codebase (“ac“)

The formula then weights each of these factors to give an overall score – the higher the number, the more screwed you are.

Let’s¬†look at each of these factors in more detail.

Test Coverage

There are many other places you can go to read more about the benefits of test coverage and Test Driven Development (TDD) in general.  I am not going to make that case here.

Suffice to say that, when you inherit a codebase, Test Coverage is your friend.

For those that are new to the concept, Test Coverage is usually defined as a percentage and measures how much of the codebase comes with automated tests that validate¬†whether it’s working or not.

When you first inherit a codebase, you understand almost nothing about it. However, despite that, in¬†most cases you are going to be expected to a. keep it working and b. add new features. Good Test Coverage lets you add and change things in the codebase with a much lower risk of inadvertently breaking something else that you don’t understand yet.

So, how do you assess Test Coverage? Fortunately, for most platforms and languages (in fact all the platforms and languages I’ve come across), there are one or more automated tools to analyze Test Coverage. These are not perfect and you should always ask the developers who worked on the code originally what they believe the Test Coverage to be (more on that below).

Also, beware of getting a false sense of security from a raw Test Coverage % number. Just like any codebase, not all Test Coverage is created equal. I’ve seen many cases where tests are written but they don’t actually properly test the code in question. ¬†Again, this is why you should seek out the view of the developers who worked on the codebase originally.

Team Availability

To be blunt, if you inherit a codebase but don’t have any access to the developers that wrote it, I’d say you’re in Big Trouble ™.

Only the original developers know where the bodies are buried: they understand how much Tech Debt¬†has accumulated in the codebase that you’ll need to deal with, they know which parts of the code are solid versus¬†which kept them awake at night and they know which bits of code were written by the good developers and which by that idiot that was fired.

By “developers”, I also count devops people since they are arguably the most important in keeping what you inherit up and running.

Now, by “Team Availability”, I don’t necessarily mean that the developers have to still be working for you full-time (although that would be best) but that they are¬†available to answer questions and provide advice on an as-needed basis.

So, Team Availability is also defined as a percentage. You can either think of it as the % of people (e.g. 2 members of the original team of 10 are available, so 20%) or as a % of the people’s time (e.g. the original developers are available 1 day a week, so 20%).

Team (availability) Duration

Having access to the original team is great but often the follow-on question is, for how long?

From experience, I’d say 6 months is enough for most cases. It’s enough time to learn what’s already there and to make significant forward progress with new features. ¬†After 6 months, the original developers no longer have familiarity with the newest code so their usefulness starts to diminish.

If you’re planning on acquiring/inheriting a codebase, I’d recommend that the absolute minimum you should try to keep the original team available for is 3 months.

Defect Find/Fix Ratio

The Defect Find/Fix Ratio is simply a measure of how rapidly bugs are being found in the code versus fixed in the code. It’s a useful measure of how troubled the codebase is at the point in time you inherit it. If bugs are being found faster than they’re being fixed, that’s an indication of a problem.

In practice, the amount of time allocated to bug fixing varies based on the team’s other commitments and, therefore, the Find/Fix Ratio will tend to oscillate. ¬†So, it’s good to look at the Find/Fix Ratio over a period of time; the past month at least. ¬†The lower the ratio, the better.

Age of Codebase

While it’s true that code itself doesn’t age or rust, older codebases are, in general, more problematic.

There are a number of reasons for this: firstly, the older the codebase, the more likely that the people who wrote the code are no longer available and/or have forgotten how it works.

Secondly, the older the codebase, the older the versions of the libraries, packages, tools, etc that it depends on, unless there has been very deliberate effort to update these (hint: this rarely happens). This makes it much harder to make changes to the codebase, especially if some of the versions it depends on have reach end-of-life or are no longer compatible.  This is all Tech Debt that you will have to address before you can move forward.

An Example

Let’s imagine that ACME Corp¬†acquires Widgets Inc and the poor VP Engineering at ACME Corp inherits Widgets Inc’s codebase.

Widget Inc’s codebase is 4 years old. ¬†The team has been pretty stable at 10 people but 7 of those people have quit in anger because of the acquisition. ¬†The remaining 3 have been persuaded to stay around for 4 months in return for some ongoing stock vesting and a retention bonus.

The VP Engineering quizzes the remaining 3 and runs the test coverage tool.  He also looks at the bug tracking system to see the recent trend.  He discovers that Test Coverage is about 40% and, in the past 3 months, 280 bugs have been reported, of which 80 have been fixed.

So, here are our inputs to the formula:

  • test coverage: tc = 40%
  • team availability: ta = 30%
  • team (availability) duration: td = 4 months
  • defect find/fix ratio: ffr = 280/80 = 3.5
  • age of codebase = 4 years

Plugging them into the formula, here’s what we get:

CRFformula_example

 

So, this yields a Codebase Risk Factor of 320, compared to (as the mathematically-inclined will have noticed) a perfect score of 20.

In layman’s terms, pretty screwed.

 

Agree or disagree? Please leave a comment.

The Truth about Tech Debt

This is a topic which almost all non-technical and a lot of technical people don’t understand. ¬†This is sad and somewhat ironic because, if you are in any way involved in the software development business, you HAVE to understand it.

I think part of the problem is that, to non-technical people, Tech Debt sounds very nebulous and like precisely the kind of excuse that software engineers come up with for not getting stuff done.

Be assured that tech debt is very real and very deadly. ¬†Even those people who do understand what it is find it all too easy to kick the can down the road. ¬†Never forget the old saw “a stitch in time saves nine”.

So What is It?

Firstly, let’s explain the analogy: it’s called “debt” in part because you pay “interest” on it. ¬†That interest consists of the extra time and risk (therefore, cost) of adding or changing product features.

Furthermore, just like a credit card, the interest compounds. ¬†If you don’t address the current debt but go ahead and build new features anyway (i.e. spend more) then you are building those new features on top of your existing debt, meaning you’re paying interest on interest.

The only way to avoid paying interest is to pay back the debt. If you let the debt grow too far, you’ll end up spending more servicing the debt than building new features. Ultimately, you’ll become bankrupt.

Just like other kinds of debt, it’s all too easy and tempting to ignore but it never goes away if you do that – it just gets worse.

Err…So What is It?

Let’s try another analogy: ¬†think of your product like a building. Each feature you build is a floor you add to the top of your building. ¬†In order for a new floor to stay up, it’s dependent on the support of all the other floors below.

Imagine your building has 8 floors. Now you’ve got demand from new tenants so you want to add a 9th floor. ¬†Easy right?

The problem is that the 8th floor isn’t in great shape. ¬†It looks ok from the outside but remember you put a lot of pressure on Mikey, the contractor who built the 8th floor, to get the job done because you had tenants waiting? Mikey’s a good guy but he definitely cut some corners to make the deadline you imposed. That wall that was supposed to be load-bearing is just a facade. The wiring is not up to code. The plumbing was rushed so it’s leaking – you haven’t seen the stains on the ceiling of the floor below yet…but you will.

And…remember when you built the 7th floor? That contractor, Dave, didn’t really know what he was doing, did he? ¬†He didn’t understand architectural diagrams properly and didn’t know where the load-bearing walls were on the lower floors so he didn’t really lay out the 7th floor properly.

Oh…and don’t forget that Brian who built the 6th floor wasn’t involved in the building the original 5 floors so he didn’t understand the specialized heating system that had been put in and just added to it using standard components. The heating has never worked quite right on the 6th, 7th and 8th floors, has it?

So, ready to build that 9th floor? You could just go ahead. ¬†It’ll probably be ok, right? I mean, the building hasn’t fallen down yet…

I think you get the point.

Of course, when it comes to buildings, this is unlikely to happen, at least in a modern, industrialized country. We have professional architects, permits, building codes and inspectors.

Guess what? There ain’t no permitting or inspection agency for software development (outside of some very specialized areas). ¬†It’s the Wild West.

Causes

The causes of Tech Debt are manifold but I would say these all boil down to three root causes:

  • Time pressure. ¬†Needing to meet a deadline is just business reality. ¬†However, time pressure leads inevitably to corner-cutting. Corner-cutting creates tech debt. Beware artificial ship dates and their hidden cost (more below).
  • Pivots. Pivots are also part of business reality, especially for a startup. The only alternative to pivoting is flogging a dead horse…and then being a dead horse. But, to software engineering efforts, pivots are moving the goal posts. ¬†Pivots lead to stretching existing product implementations to fit new objectives, to shoving square pegs in round holes. Keep stretching and eventually things break.
  • Team Changes. ¬† If you inherit a codebase but not the team that wrote it, you probably don’t understand where the tech debt is or how big it is. You also don’t understand the quickest and easiest way to build on what has already come before. Similarly, when you add new people to the team, they typically have a steep learning curve to learn what has come before. In either case, that learning is never perfect.

Particularly Deadly to Startups

Tech debt is particularly deadly to startups because:

  • there is¬†enormous time pressure¬†because of funding rounds, meaning it accumulates quickly (and you never have time to address it),¬†
  • they tend to pivot frequently, meaning tech debt accumulates quickly (and you never have time to address it), and
  • startups are resource constrained so there is little to no bandwidth to address tech debt.

Beware Artificial Deadlines

Sometimes there is a real deadline to ship a product – a trade-show where the product will be launched, or running out of funding being two common examples.

In the absence of a real deadline, creating an artificial deadline definitely has its place. A deadline serves to galvanize a team. It helps stop the team endlessly tweaking and gilding the lily.

However, the hidden cost of an artificial deadline is Tech Debt. Because of the planning fallacy (beyond the scope of this post), software development projects are almost always late so corners have to be cut to make deadlines. ¬†That might be ok if you put in place a plan to address the accumulated tech debt immediately after the product launches…but, let’s be honest; that never happens. Tech debt almost always gets kicked down the road because of the pressure to fix defects and respond to user feedback and requests.

Therefore, be wary of artificial deadlines – they have their place but they also have a dangerous, hidden price.

How to deal with Tech Debt

So, now we know what it is, how should you deal with Tech Debt?

1.  Understand what it is

Firstly, accept the reality that Tech Debt is a thing. ¬†It exists. ¬†It’s real and it has to be monitored and addressed. Ignore it and it will get worse and bite you in the rear.

2.  Understand where it is

Next, understand how much Tech Debt you have and where it is in your product. If you are not part of the development team, the developers will be happy to tell you; trust me.

3.  Prioritize it

Once you know where the debt is and how big it is, it’s time to assess how critical it is to address it. ¬†There will always be some Tech Debt – it’s inevitable. Not all Tech Debt is critical to address immediately.

There are no easy answers because the future is not easy to predict.  However, here are some rough rules of thumb:

  • Tech Debt that exists in core features that differentiate your product and/or which users rely on should be addressed urgently, because you are likely to want to continue to develop those features.
  • Tech Debt that exists in any layer of your product which future development will rely on should be addressed urgently, or you’ll be continuing to build on shaky foundations.
  • Tech Debt that exists in rarely used features, features that you don’t compete on or features that you plan to deprecate is a lower priority to address.

4.  Plan to address it

This might be the hardest part. Addressing Tech Debt will take time. Unless that time is planned and built into the priorities and the schedule, it will not happen.

There is also an associated opportunity cost. You cannot change the laws of physics. Addressing Tech Debt will mean you can’t use that same time to move forward on new feature development. You have to accept that reality.

How to Minimize Tech Debt

You can’t avoid Tech Debt completely – it’s an inevitable part of shipping a product. ¬†There is unfortunately no perfect answer here – you have to strike a balance that minimizes Tech Debt but, at the same time, avoids the equal evils of premature optimization, gilding the lily and complete rewrites. ¬†Swinging the pendulum too far in another direction will be just as much of a disaster. ¬†(Hint:¬†Software is never “done” and, left to their own devices, most engineers would prefer to tinker indefinitely without the annoyance of real users using the product and doing stupid things that they shouldn’t.)

But, just because you can’t avoid Tech Debt, doesn’t mean you can’t take some actions to minimize it:

  1. Spread awareness of Tech Debt in your organization to make sure you have a common understanding that it exists, that it needs to be dealt with and the dangers of kicking it down the road.
  2. If you’re not personally involved in the details of software development effort, try to keep your finger on the pulse of the level of Tech Debt is accumulating and communicate that with other stakeholders (particularly the board and investors).
  3. Always build time into product development schedules to address Tech Debt and factor it into commitments made to customers and investors. Expect to spend a chunk of time/money after each major feature release to address the Tech Debt accumulated during the push to get the feature over the finish line.
  4. If you inherit a codebase from a prior team or company, make sure you have ongoing access to someone who worked on it originally, for an absolute minimum of 6 months.
  5. Practice Test-Driven Development (TDD). The details are beyond the scope of this post but good test coverage is invaluable in minimizing Tech Debt.
  6. Avoid “proof-of-concept” implementations of products and features, especially in a startup where you are very resource constrained. While it’s definitely critical to develop iteratively and build a Minimum Viable Product (MVP) to test if users are interested in your product before you invest too much, you should also assume that all code written will make it into production – you can’t afford to write things twice. This doesn’t mean over-engineer; it means engineer once.

In terms of striking the right balance between being aggressive about shipping products versus premature optimization, my recommendation is that you only make investments that will get you through your next funding milestone or the next 12 months, whichever is later. ¬†i.e. if it won’t pay off in 12 months or before you flame out, it’s not worth it.

Further Reading

“Shipping first time code is like going into debt. A little debt speeds development so long as it is paid back promptly with a rewrite… The danger occurs when the debt is not repaid. Every minute spent on not-quite-right code counts as¬†interest¬†on that debt. Entire engineering organizations can be brought to a stand-still under the debt load of an unconsolidated implementation…” – Ward Cunningham, 1992

I recommend you also check out “Lehman’s Laws of Software Evolution“. ¬†They started in 1974 so we’ve known about Tech Debt for a while.

How to Hire Software People

At the time of writing (Dec 2013), hiring software people in Silicon Valley is as hard as its been since the dotcom boom of the late 90s (remember getting a Porsche as a signing bonus?). While the rest of the US and the world is still staggering through an uneasy recovery, we live in this weird bubble of 100% employment and inflated salaries.

I could debate whether this situation is sustainable or healthy (hint: no and no), but it is the current reality and, if you’re trying to build a software company today, you have to deal with this reality.

Your ability to find and retain software talent is currently one of the biggest, if not the biggest, barrier to the success of your business.

Know Who You are Looking For

Write a Job Spec

This sounds obvious but this step is often skipped, especially within smaller companies. Perhaps its perceived as too “big company” for a startup but this is a classic false economy.

It’s one of those activities where the process is more important than the output: by forcing yourself to write a job description, you force yourself to think about who you are looking for: you’ll create a lens through which to view candidates and you’ll give yourself clarity on where you are willing and where you’re not willing to compromise. ¬†You’ll also have a very useful start for recruiters (more on them later).

Make sure your job spec not only covers the skills and experiences you’re looking for but also the attitude that you are looking for.

Cultural Fit

“…culture isn’t just one aspect of the game, it is the game…”Lou¬†Gerstner

Everyone says that “cultural fit is important”, but what does that really mean?

Firstly, let’s define what “company culture” really is. ¬†There are many definitions out there (Google it) but the two that I find most useful are:

a. company culture is “the way things get done around here”, and

b. company culture defines who gets respected and rewarded in your company.

Your first task is to take a long, hard look at your organization and ask yourself what your company culture truly is.  If you need help, there are various online tools and surveys to help you tease this apart.

Bottom-line: if you run a boiler-room full of fist-bumping, Izod-clad Brogrammers, own that reality.

If your company culture is not what you’d like, what do you want it to be? ¬†Armed with this insight, you can select people that fit that culture and have a chance of nudging it in the direction you want it to go.

But, don’t fool yourself that your company culture is what you wish it was, rather than what it is. Hire someone that is a misfit for your culture and they’ll pretty quickly suffer “organ rejection” and at best you’ll be back to where you started, having lost some credibility. ¬†It takes a very strong personality to come in from the outside and change the culture in a significant way so be realistic about the degree of change any individual hire can make.

3 Kinds of People

There are many tools and models to categorize people. One model I use to help understand the kinds of people you need in a startup is to divide people up into just 3 groups:

  • people who are good at designing the machine
  • people who are good at building the machine
  • people who are good at operating the machine

By “the machine”, I mean your business and your product, whatever that may be.

Very rarely you’ll find someone who is good at 2 out of 3. I have never met anyone that is truly good at all 3.

With a startup, you need a few designers and lots of builders. ¬†Only later, when you’ve probably pivoted multiple times and worked out the business you are actually in, do you need operators. ¬†At that point, you’re probably not a startup any more.

The Danger of Big Company People

The history of Silicon Valley is littered with smart and accomplished people with established careers in big companies who have tried and failed in the startup business.  The question of why this is true is probably a whole post in itself but here are my quick views:

  • specialists versus generalists – as organizations grow, people’s responsibility tends to become narrower and deeper; more specialized. In a startup, you need to wear many hats and will likely pivot multiple times. Therefore, generalists tend to fare much better.
  • discomfort with ambiguity – running a startup is all about making an endless series of low-data, high-risk decisions. Big companies have established markets and products, lots of data and a more predictable future.
  • inability to work with extremely limited resources – big company people are used to established infrastructure and an organization that has the scale to do multiple things in parallel. With a startup, attempting to do several things at once is often fatal and acute focus is required at all times.
  • managing not doing – in a larger company, people typically spend the majority of their time just managing the organization. In a startup, you need to be able to both do the work and hire and manage self-starter people who can help.
  • the romance versus the reality – last but not least, many big company people fall in love with the idea of a startup but have little understanding of the reality.

I advise extreme caution if considering hiring people who have no prior experience working at a startup, however accomplished they may be in their big company career.

Big company people are generally operators. ¬†You have to take into account the stage of your business – the “right kind of people” changes over time.

Finding Candidates

Find a Good Recruiter

In today’s hiring market in Silicon Valley, the hard reality is that the people you want to hire aren’t looking for jobs. ¬†This means that to hire the people you want you have to:

a. spend an inordinate amount of time networking, going to meetups, searching on LinkedIn, etc, or

b. get very lucky, or

c. hire a recruiter.

As I said in my post on Focus, I think that spending your time on the early stages of recruitment is not a good use of your time as a founder/exec of a company.  There are people that do that for a living and there are better things for you to be spending your time on in terms of building value in your business.

So, although they are much reviled, my strong advice is to find yourself a good recruiter.  For most non-executive level positions, you should only need a commission-only recruiter (you pay them a % of salary when they successfully hire a candidate, nothing upfront).

Whenever we’re in a boom/bubble, a huge crop of recruiters springs up and, like me, you are probably inundated with their unsolicited inbound messages. ¬†Here are a few things to watch out for in dealing with recruiters:

  • look for recruiters that have been through multiple boom/bust cycles. We’re in a boom/bubble now so it’s easy to be a recruiter. Only the good ones have the staying power through the tough times.
  • adding multiple recruiters (for a role) often doesn’t help much, due to the very limited supply of candidates. ¬†It actually makes more busy work for you since you’ll have to track which recruiter has introduced which candidate first; otherwise you’ll end up having to deal with an irate recruiter who wants his commission for introducing a candidate to you months ago who you just hired via another recruiter by accident.
  • beware of recruiters that try to hook you in with candidates that they don’t actually represent and have never actually spoken to. Some recruiters are not averse to copying resumes from LinkedIn and then presenting them to you as candidates they actually represent in order to get your business.
  • beware of recruiters who overly “coach” candidates ahead of interviews, based on feedback from prior candidates they’ve sent you. If a candidate’s answers to your questions sound too good to be true, they probably are.
  • fees are always negotiable, as are guarantee periods. ¬†The standard initial offer from a recruiter currently is usually 25% of first-year salary with a 3 month guarantee. ¬†Negotiating down to 20% and 6 months should be possible in most case, bubble or no bubble.

On your side, you should also make sure you have realistic expectations of any recruiter and understand what you will need to do to be successful:

  • provide the recruiter with a written job spec (see above)
  • recruiters are not software engineers. Although a good technical recruiter should be expected to have a basic understanding of different skill sets, they are not going to understand the nuances you do. ¬†They should be able to ask questions you provide them, though.
  • tell the recruiter which companies ¬†suitable candidates are likely to be currently working at or ¬†worked at recently.
  • be responsive and professional in the recruitment process.
  • a recruiter can bring a horse to water but it can’t make it drink – the onus is still on you to make joining your company the most attractive option of the many options that any good candidate will have.

Compromise

Given the ridiculously competitive hiring market and the level of entitlement (see below), you are highly unlikely to find the “perfect candidate” and will almost certainly have to compromise on at least one attribute.

The important thing is to be clear on what to compromise on and what not to.

Don’t compromise on cultural fit or talent – that’ll slowly erode the quality of your organization. ¬†Filling a position with someone who doesn’t cut the mustard or fit with your culture just to fill it is almost always a recipe for disaster – i.e. worse than hiring no one.

The first thing you might compromise on is specific experience. Unless you are hiring for a highly specialized skill-set or have a very time-critical need, I would always choose the more talented person over the person with the more specific skill-set or experience match. This is because the more talented person will almost always out-perform the less talented person in a matter of weeks.

To take a concrete and pertinent example for software engineering candidates, Ruby-on-Rails engineers are particularly hard to find and demand a premium in the current market. ¬†But, it’s not like a great (or “rockstar” as a recruiter would put it) engineer with no Ruby-on-Rails experience is suddenly going to become a terrible engineer just because they don’t know Rails. I will put money on the great engineer with web development experience but no Rails experience outperforming the less talented engineer with Rails experience within 4-6 weeks.

Another attribute you might compromise on is work location; whether its someone talented who lives a long way away or someone more local who just prefers to work from home. ¬†If you’re based in Silicon Valley, opening up your search to other locations massively increases the pool of available candidates.

Hiring someone who is not local and helping pay for their relocation is potentially a great deal versus having to pay a premium for someone already local.

Alternatively, while allowing remote working introduces its own challenges (such as instilling culture), it is a viable option, as long as you build your organization, culture and processes in a way that supports it.

Interviewing

Dealing with Entitlement

Don’t take it personally; the market has educated everyone to think that they’re great, even when they’re not.

I recently spoke to a candidate who received over 20 inbound calls from recruiters the day he quit his previous job. He wasn’t that great. In this kind of market, evenly distinctly mediocre people only see signs that they’re superstars.

Software people in the Valley are also currently ridiculously pampered as employers go to greater and greater lengths to attract people.

Bear in mind that, in the software business, the workforce leans young. ¬†If you’re 25, you haven’t even seen one economic cycle in your adult, working lifetime – as far as you’re concerned, it’s always been like this, and always will be.

Combine this with the Millenial generation’s trademark attitude (is that really true, or were we all just dicks when we were young?) and you have a heady mix of entitlement.

So, get used to candidates asking all the questions. Get used to entitlement. Accept that, currently at least, its an employees market and you are selling to every candidate.  Take a deep breath.

One and Done

You are going to lose candidates if you don’t move quickly. ¬†The best candidates will be gone in less than a week. Promise.

With this in mind, I highly recommend a “one and done” approach to recruitment. ¬†Schedule candidates to come in for a block of 2 to 3 hours max, have everyone you want to interview scheduled back-to-back and be prepared to make an offer that same day.

Standardized Assessments

People are not easily categorized but, in order to make your interviewing process and more scientific, agree a standardized way for each interviewer to write up their responses.

Here is the format I have used for some time:

  • Score out of 5 in terms of suitability for the role (4.5s are extremely rare, I’ve never seen a 5)
  • HIRE / NO HIRE – force people off the fence by making each interviewer decide whether the candidate is a hire or not
  • Pros – max 5 bullet points
  • Cons – max 5 bullet points

To keep it truly objective, these assessments should be sent only to the person doing the hiring so that different interviewers do not influence each other.

What to Ask

Let’s be honest; an interview is a highly artificial situation. Working with someone day-to-day is nothing like an interview. ¬†What you’re trying to do is make an assessment in 30-60 minutes of whether someone is likely to perform well and integrate with the team.

That’s hard. It’s particularly hard because software people generally are highly introverted – hell, they got into this career to start with to avoid having to talk to people.

Personally, I rarely ask any specific “technical” questions. ¬†Partly, I have the luxury of a team of people who can do that better than I can but, more importantly, I believe that you can teach skills but you can’t teach attitude.

My experience is that hiring people who you can work with, trust, rely on and communicate ‚Äúat the speed of thought‚ÄĚ is more important than someone’s specific background and skills.

I also find resumes to be nearly useless since they tell me nothing about the person’s attitude and are generally so carefully word-smithed and full of generalities that they don’t really tell me much at all.

What I want to know most of all is:

  • is this person a relatively normal human being* who I can effectively communicate with?
  • is this person accountable? Will they do what they say? Can they be relied upon and trusted?
  • is this person likely to deliver results?
  • does this person learn from their mistakes?

(* this is the software industry – “normal” is a relative term.)

My main way of answering these questions is to ask people for their stories. I want to hear about projects they’ve worked on where they were pleased with the results and I want to understand why they think the project was successful and what their role was in the success.

More importantly, I want to hear about projects where they look back and kick themselves and wish it had turned out differently. I want to hear why they think it turned out badly and what the lesson is they distilled from that experience. I strongly believe that you learn a lot more from failure than from success.

All of these stories also present many opportunities to dig into the specifics to test the candidate on what they really know and what they really learned.

What Not to Ask

Firstly, familiarize yourself with what you are not legally allowed to ask candidates – you might be surprised.

Next, remember that an interview is already an artificial scenario bearing little resemblance to actually working with someone. So, avoid making it even more artificial with ridiculous riddles and brain-teasers. ¬†I think these kinds of questions are really about making the interviewer feel smart and smug than about eliciting any useful responses from the candidate. ¬†People claim that they are good at making people think “out of the box” and testing reasoning skills but how about asking them some questions that are relevant to your business that achieve the same thing?

Remember that an interview is not your opportunity to show off how smart you are (seriously, this is a widespread problem in the Valley). ¬†So, don’t ask highly specific questions about a particular subject unless it’s absolutely critical the person you hire knows about that on their first day at work. ¬†Those questions just tell you whether the candidate happens to have dealt with that particular subject, not how talented they are.

Lastly, avoid asking obvious questions that you’ll only ever get one answer to. ¬†“Are you trustworthy?” ¬†“Are you a hardworker?” “Are you a team player?” Would anyone ever say “no”?

People to Avoid

Lastly, here is my short list of People to Avoid ™:

Talented Assholes

People who rub other people the wrong way should be avoided, even if they are incredibly talented.  The damage caused is not worth it and the aggregate results of the team will suffer as a result more than any individual talent will compromise.

Heroes Need Not Apply

A close cousin of the Talented Asshole is the Hero. The Hero will discover a big problem 2 weeks before a product is due to ship and work 18 hour days to save the day, fully confident of success and the imminent adulation of the team. The problem is that the hero will fail 90+% of the time.  The Hero should have told everyone else and asked for help. The team beats any individual 99.9% of the time.

“I’m not technical”

Even if you’re hiring for a non-technical role, this is Silicon Valley and you need to have some base level of interest in technology and in the industry. There are people who seem to wear ¬†“I’m not technical” as a badge of honor. ¬†Particularly in a startup, you need people who are generalists and adaptable. ¬†I have little time for “I’m not technical”, just like I have little time for engineers who have no sense that we’re writing software to satisfy a customer need, not for the sake of it.

That’s it. Have fun out there. ¬†Oh, any one last thing; if a penguin walked through the door right now wearing a sombrero, what would he say and why is he here?

What it means to be a Good Boss

This is of course a topic that many, many others have written about but here’s my quick take on what it means to be a Good Boss(tm).

I took the approach here of thinking first about what I like in a boss. Then I did a quick survey of the people that currently work for me at Razoo to see what they think it means.  Here we go:

1. Listen, listen, listen

Sure, sometimes things just need to get done. However, we’re all human beings so we appreciate being heard and empathized with, even if there is nothing our boss can actually do to change the situation or our boss just simply disagrees.

One of my team looks for “quality in response – provide good responses showing understanding with empathy“.

As a boss, part of your role is to be a therapist, like it or not.

2. Set context

Management often focuses too much on the “what” and not enough on the “why”. ¬†Not only is the why vital ¬†for maintaining motivation – few people enjoy working on tasks that seem pointless or futile – but the context is also a big part of enabling people to think outside the box; it enables people to come up with a different “what” in the service of the same “why”.

As one of my team put it, a Good Boss(tm) “helps the team understand and focus on what’s important“.

3. Define success

As well as telling me what my role and activities mean in the context of the bigger picture, be specific in defining what achieving the objective actually means; the more specific, the better.  That way, the danger of a misunderstanding is minimized and trust is maintained.

Couch this success both in terms of what it means for me, the team and for the company as a whole.

Quantify results – provide¬†measurable¬†metrics for all my requirements,” said one of my team.

4. Give me what I need to be successful

Once you’ve told me what success means, if you’re not giving me the tools, information and support to be successful, you’re setting me up to fail. ¬†A Good Boss(tm) realizes that his or her role is to support the team; to be in service of the team, rather than the other way around.

5. Manage me at the right level for me

We’re all different. ¬†We arrive at our jobs with differing levels of preparedness for the role. ¬†Sometimes we’ve done a very similar job at a very similar company. ¬†Other times, it’s a step up for us in responsibility and/or a new industry.

Depending on who I am and my experience, I will need different levels of hand-holding.

By default, a good starting point and principle is to trust me to do the job that I was hired to do, unless and until you have evidence warranting otherwise.

I was also told that a Good Boss(tm) is “respectful of people’s abilities and aspirations but also of their limitations and reality“. ¬†Well said.

6. Understand what I need to be motivated in the long-term

A Good Boss(tm) understands that superficial carrots and sticks don’t motivate in the long-term. Doubly so in a knowledge-worker industry like IT, with 100% employment.

Most of the research that’s been done over the past 20-30 years has pretty categorically shown that knowledge workers actually do worse when working towards a reward/bonus. ¬†For an interesting and fun example, check out this TED talk on the “marshmallow challenge”. In the software development industry, there’s also this classic from Joel Spolsky.

Feeling like I’m doing a good job, adding value to the business and that my contribution is appreciated by my peers and superiors will motivate me much more in the long-term than any threat, reading of the riot act, bonus scheme or desk ornament.

For even more on this topic, read Drive: The Surprising Truth About What Motivates Us.

7. Give me candid and timely feedback

A Good Boss(tm) has to have the courage to tell it like it is. I would much rather you tell me about concerns or negative feedback immediately, before it festers.  If you wait to say something until it has become a Big Issue but I have no idea, trust is lost and the results are almost always bad.

8. Inspire me to be better

A Good Boss(tm) pushes me to continuously improve what I do and how I do it.

As one of my team put it, a Good Boss(tm), ¬†“knows how to safely stretch people outside of their comfort zone”.

A good part of this is of course leading by example. ¬†A Good Boss(tm) needs to also be “passonate – show that you care about what you do“.

9. Build and nurture a high-performing team

The above points are more about the individual and the 1:1 relationship between boss and employee. ¬†Of course, a Good Boss(tm) is fundamentally in the business of building teams that function. ¬†This requires an understanding of many issues such as the differences in personality types, learning styles, backgrounds and experiences as well as being good at hiring and managing “up or out”.

A Good Boss(tm) “facilitates healthy discussions, defuses tense/poisonous situation” and¬†“fosters a positive work atmosphere, favors collaboration“.

Again, you’re a therapist; get used to it.

10. Always try to be a better boss

It’s a two-way street. You have to help and inspire others to get better but you also have to continuously strive to get better yourself. A Good Boss(tm), “reaches out to the team constantly to get better at being their boss“.

Thanks to Patrick, Vince, Stephen and Alex who put up with me being their boss and contributed to this post.

Meaningful Metrics: 6 Steps to Metrics Heaven

Surely, one of the benefits of technology-based businesses is that it’s much easier to make data-based¬†decisions¬†– to “drive by the numbers”.

Superficially, this seems like it should be easy – just look at the data and decide what to do, right? ¬†However, in every company I’ve seen, it’s always been painful.

Hopefully, this will help you avoid some of that pain.

Six Steps to Metrics Heaven

Six Steps to Metrics Heaven

I believe there are 6 steps to “metrics heaven”:

  1. Define
  2. Measure
  3. Present
  4. Discuss
  5. Action
  6. Iterate

Sorry, there are no short cuts; you have to get all 6 right to be successful.

Step 1 – Define

Ask the Right Questions

Your path to Metrics Heaven starts simply by asking the right questions.

Firstly, try to state the question behind each metric as unambiguously as possible. ¬†Secondly, explain in plain English what this question means and why it’s important to your business – the context.

Example:
Question: ¬†“Of the users that have signed up in the past 12 months, what percentage have logged in in the past 1 month?”

Context: ¬†“This metric measures our ability to keep users coming back to our site. ¬†This is vital to our core business model as we pay $40 on average to acquire each new user and can only make a profit on them if they keep coming back.”

Clear definition of the question is particularly important because the person responsible for actually answering the question Рwriting the database query, wrangling Google Analytics, etc Р is often not the person framing the question.  So, any ambiguity has the potential to result in an answer that is misleading or just plain wrong.

Fight Data Overload

If you try to measure, track, discuss and action too many metrics, you’ll lose track of what’s important. ¬†You’ll lose sight of the forest for the trees.

Data overload is an insidious problem. ¬†The addition of each new metric seems innocuous so it’s hard to say no. ¬†Plus, there’s a tendency for people to want to get their group’s name “up in lights”. ¬†But it’s vital that you do say no. ¬†Otherwise, here’s what I’ve seen happen every time:

  • pulling and calculating all the metrics becomes a chore and takes forever
  • you end up with a big and unwieldy dashboard
  • the dashboard takes more than 30 minutes just to run through with everyone during the metrics meeting
  • people fall asleep or get distracted by their laptops/iPads/iPhones
  • you run out of time to discuss actions
  • people start to view the metrics meeting as a pain and find excuses to avoid it

Top 10 Metrics

To avoid overload, you need to identify a maximum of 10 metrics. ¬†This “Top 10” constitutes your top-level dashboard.

By “top-level dashboard”, I mean the dashboard that is shared with the whole company, which the executive team uses to drive the business and the first – and maybe only – set of metrics you review when you meet.

Each group or department may have its own dashboard too. ¬†There may be¬†multiple¬†levels of dashboards that dive deeper and deeper into the internals but don’t let those pollute your Top 10.

Your Top 10 metrics dashboard will probably be something you’ll want to work to automate so that it’s available live, displayed on a big flat-screen in your office, etc (more on that later).

To help ensure that you don’t grow beyond 10 metrics, use his rule: ¬†if someone wants to add a metric to the top-level dashboard, they have to choose one to remove also and justify why the one they want to add is more important than the one they want to remove.

Now, let’s look at how to identify which metrics should be in your Top 10…

MAIN Metrics

I have come up with a handy-dandy acronym to help identify the metrics that you should choose, particularly your Top 10 metrics; “MAIN”:

  • Meaningful
  • Actionable
  • Isolatable
  • Not misleading

Let’s look at these 4 in more detail…

Meaningful

Is the metric a true and valid indicator of success in your core business?

A good way to ask this is to pose the following question:

“if this metric changes in the right direction, and nothing else changes, will my business be doing better?”

If the answer to this question is no, then you may have identified a useful metric to measure and try to impact, but it’s unlikely that you’ve identified a Top 10 metric.

Actionable

There’s not really much use measuring something if you can’t do anything about it. ¬†You must be able to “move the needle”.

Metrics where you can’t move the needle are potentially interesting in terms of giving you a better understanding of what drives your business but almost certainly are not Top 10 metrics.

Isolatable

It’s vital that you choose and define metrics in such a way that they do not vary based on other, independent variables, whether those variables are in your control or not.

You need to be confident in measuring the effect of the actions you take to move the needle.  You need to avoid the age-old problem of confusing correlation with causation.

A classic example in web businesses is not insulating your metrics from the impact of fluctuations in traffic caused by other factors.  For example, you may have been measuring the number of user adds on a weekly basis and be working aggressively to drive up that number by reducing friction in your sign-up funnel.  The week after you launch a slew of sign-up funnel optimizations, you get 25% more users.  Yay for you Рit worked Рgold star.

Not so fast.  It turns out that what actually happened was that an article in a local style magazine that your PR agency setup some weeks back was finally published which drove 25% more traffic to your homepage.  Your conversion rate actually stayed the same.

To isolate a metric as far as possible, first frame the question in such a way that it’s not impacted by other variables. ¬†The easiest approach to this is to always define metrics in terms of rates, not in terms of absolute numbers – e.g. “what % of users that started down the signup funnel completed it last week?” Not, “how many users signed up last week?”

Lastly, always be suspicious if it seems too good to be true.  Dig deeper to check whether your action really was the cause of the change.

Not Misleading

Lastly, even if a metric is isolatable, it may also be misleading in other ways.  It may give a false negative or false positive, or otherwise cause you to react in the wrong way.

It’s important to define metrics in such a way that they are hard to misinterpret. ¬†Even if that’s not completely possible, it’s important to highlight the ways the metric could be misinterpreted when it is discussed.

Another classic example from the web is measuring the cost effectiveness of various CPC (cost-per-click) advertising campaigns. ¬†You may find that certain keywords or certain channels allow you to buy traffic at a lower cost. ¬†So, you decide to spend more money on those channels and reduce your spend on other channels. ¬†However, not all traffic is created equally. ¬†You may find that the users/customers brought in by the cheaper CPC rates actually generate proportionally less revenue for you so it’s worth paying more for higher quality traffic.

The danger of misleading metrics is one of the main reasons why stating the context of each metric (as discussed above) is so critical.

Step 2 – Measure

So, you’ve defined your Top-10 MAIN metrics in a precise and unambiguous way. ¬†Now you just need to measure them.

Easy, right?

Um.  Mostly not.  Measuring is usually hard for a variety of reasons:

  • the relevant data is spread across multiple systems (your database, Google Analytics, MixPanel, AddThis, etc)
  • data is buried in the depths of your system and extracting it requires in-depth knowledge of your systems (and probably wicked SQL skills too)
  • the people with the know-how to extract the data are probably engineers who are busy with other things and consider extracting metrics beneath them
  • metrics were not really considered when the original system was built so the “instrumentation” has to be added retrospectively; this work contending with other product feature work
  • lots of manual steps are required, making it a chore

Tools

The good news is that there is a growing range of tools available to help.  The bad news is that any tool, if badly used, can be worse than no tool at all.

Let’s look at a few…

Google Analytics (aka “GA”)

I hate Google Analytics. ¬†However, I can’t argue with the price. ¬†And it’s incredibly powerful…if you can work out how to use it. ¬†It’s unfortunately incredibly unintuitive. ¬†It’s a camel (horse designed by committee).

The good news is that it’s fairly ubiquitous so you probably already have people on your team who have experience with it.

Mixpanel

I like Mixpanel. ¬†I have no relationship with them other than as a happy customer. ¬†Mixpanel does the 10% subset of Google Analytics that you actually need and does it well – the charts make sense, things are consistently referred to by terms that make sense. ¬†It’s not free but it’s not expensive compared to the value it provides, in my opinion.

KissMetrics

Personally, I had a bad experience with KissMetrics but that’s just one data point. ¬†They are probably Mixpanel’s main competitor.

SQL Queries

The data in your app is probably in a relational database.  That means you need at least one SQL ninja to write queries on that database.  SQL is a technology that is older than I am and therefore is definitely not cool but it does what it does better than anything else, if you know how to use it.

Unfortunately, modern web frameworks mostly insulate developers from having to use SQL directly. ¬†Whilst this may be a good thing from the perspective of developer productivity and “separation of concerns”, it means that the number of engineers that can write complex SQL queries is reducing.

Microsoft Excel

There ain’t no shame in Excel, in my opinion. ¬†Likewise, Google Sheets, although the graphs are much better in Excel. ¬†Both have plugins to pull in data from various sources, including Google Analytics.

Hire Someone

You might want to consider hiring a “data analyst” – i.e. someone who runs the metrics process as their main job, rather than it being done badly as a part-time job by others. ¬†I use¬†parentheses¬†around the job title because I’ve found that people who self-identify as Data Analysts have a tendency to be quite academic and unable to roll their sleeves up and actually grapple with Google Analytics to extract the numbers.

I would suggest you’re better off starting with a technically-minded digital marketing person. ¬†They’re like unicorn poop in the current job market so good luck.

Step 3 – Present

Once you’ve got the actual data, you now have to present it in a clear way that can be digested by others.

Build a Dashboard

The first step in presenting metrics is to build a dashboard that quickly and clearly communicates your “Top 10” metrics. ¬†Don’t worry about finessing the dashboard for metrics outside your Top 10; ¬†it’s better to get the Top 10 dashboard as good as possible before you spend time elsewhere.

Excel is the most common tool used for this purpose and, to repeat myself, I believe there is no shame in Excel.  However, as discussed above, there are an increasing range of more automated options.

Use Charts

Some people are able to “see the matrix” – they can see patterns and trends in raw numbers. ¬†But, most people can’t, including a lot of very technical people.

So, show charts in your dashboard, not numbers – they really help highlight trends over time, progress versus budget and other patterns.

“One Number” Nirvana

I already talked about limiting yourself to a maximum of 10 top-level metrics. ¬†I’m going to go a stage further and say that every business has just ONE number that represents how successfully the business is performing. ¬†This may seem¬†impossible at the outset but but I bet you it’s not.

So, set yourself the stretch goal of identifying that One Number. ¬†You’re unlikely to get to it immediately so start with your Top 10 and see if you can iterate towards your One Number. ¬†It may be one of your Top 10 or it may be a calculation based on a number of metrics.

Simplistically, you might say that revenue or profit is the One Number for any business but absolute revenue or profit is driven as much by scale as anything else. ¬†Before you can scale to maximize revenue or profit, you need to have a business model and product that is worth scaling and you’ll also likely need to persuade investors to give you the cash required to scale. ¬†That’s where your One Number comes in. ¬†So, your One Number is likely to be something like customer acquisition cost (CAC/CPGA) or profit per user add.

Build a Live Dashboard

Data junkies will argue that the ultimate nirvana is to have a live dashboard that shows your One Number and Top-10 metrics, with graphs, on a web page that anyone in the company can view at any time to see a real-time view of how the product and business is performing.

But, however easy it is to get view the metrics, that does not remove the need to meet to discuss them and to action them. ¬†So, let’s talk about that.

Step 4 – Discuss

Having your metrics beautifully presented is no use if you don’t actually talk about them as a team.

Meeting to discuss the metrics regularly keeps them in the front of everyone’s minds, makes sure that everyone understands what actually drives your business and, most importantly, is the venue to come up with your action plan for pushing them in the right direction.

Meet Weekly

The only cadence that I have seen work when it comes to reviewing the metrics in an online business is weekly.  A month is an aeon in Internet time.  Daily is good for quota driven sales team but too much for a startup team that is trying to build and rapidly iterate a product.

So, set a fixed time to meet every week.

The most important people to have at the meeting to review metrics is the people in the company that can actually impact those metrics. ¬†Others can attend but you don’t want too many cooks.

Manage the Time

It’s very easy for the weekly metrics meeting to turn into a mechanical run through of all the metrics. ¬†This is especially true if you’ve let your dashboard grow beyond 10 metrics or if you spend time diving into lots of very detailed metrics below your Top 10.

Most important is that you spend at least as much time on actions as you do on reviewing the numbers.  People who attend the weekly metrics meeting should be expected to have reviewed the metrics ahead of the meeting and come to the meeting armed with questions and suggestions for actions.

Step 5 – Action

Having perfectly accurate and beautifully-presented metrics that you discuss weekly as a team is useless if you don’t actually do anything about them.

What’s surprising is how often this happens. ¬†People come to the weekly data meeting, review the metrics, ask questions and then go back to their jobs.

If this is what’s happening, you’re wasting time preparing the metrics and might as well get a job rearranging deckchairs on a sinking cruise ship.

Set Targets

It’s way easier to move a metric in the right direction if you specify a target of where you’re trying to get to.

Frankly, any target is better than none but it’s of course better to have a target that is achievable. ¬†It’s better to start with a small,¬†achievable¬†increment and then move the target as you get better and more sophisticated in impacting your metrics through action.

To learn more about this, read a good dieting book. ¬†Trying to either “stop being fat” or “lose 40lbs” is much, much harder and more demotivating than setting an achievable target for each week.

Actions for every metric

Every metric in your Top 10 should have a corresponding list of actions aimed at moving the needle in the right direction, along with an owner for each metric.  These can be marketing efforts, new features, A-B tests, whatever.

If you repeatedly don’t come up with actions for a Top 10 metric, ask why. ¬†It probably means you should remove that metric from your Top 10 and replace it with something you feel is more important and which you can actually impact.

Likewise, if agreed actions don’t get performed by their owners, you can keep people honest by asking whether people still think the metric is important in terms of making the business successful. ¬†If it is, then ask what other tasks they are doing that are more important. ¬†If it’s not, remove the metric.

Step 6 – Iterate

I would say that the chance that you’ll choose the right set of metrics, be able to extract all the data you need, present them clearly and ¬†build an action plan to move the needle first time around is approaching zero.

Expect it to be painful initially. ¬†Expect it to take time. ¬†Expect that you’ll need to iterate.

Summary

If there are just 4 things I suggest you take away from this, they would be:

  • Ask the right questions
  • Fight data overload
  • Focus on action
  • Iterate and expect it to take time to get right

Agree/disagree?  Please leave a comment (link is top-right of the page).