• Focusing on architectural debt

    My last few posts introduced technical debt and then explained what you should do about it.  This recent theme hasn’t completely satisfied my desire write about the topic because I think there is a lot more to this ‘debt metaphor’ than most realize.  So I’d like to dig into it a little more here.

    The commonly understood definition of ‘technical debt’ does not actually describe all aspects of what you might experience in the world of product development and engineering.  Technical debt is classically defined as writing sub-optimal code with the goal of paying it back (refactoring) at some point in the future.  And depending on how far along you are with your project/application/business, this may be all you ever get exposed to.  But as time goes on, and we increase the altitude of our perspective, we find that this classic definition actually doesn’t tell the whole story.

    Technical debt doesn’t tell the whole story

    As your organization matures, you are (hopefully) going to have very different needs for your technology.  Let’s take the example of a rapidly growing social networking platform:

    On day 1, all you cared about as a founder was building neat features that peopled cared about.  The whole goal was ‘stickiness,’ because if people didn’t like the experience enough to come back, you’d be out of business in no time.  Let’s say, in this example, that you were very fortunate and your social network really took off, spreading across college campuses around the country.  Before you knew it, your focus changed really quickly.  You had to figure out how to keep the site up.  Scaling the application became the bane of your existence.

    Eventually, you solved that problem, and new challenges arose.  It became about keeping people hooked on the experience even after they were veteran users.  Then peoples preferences evolved, and they wanted continuous streams of information, so you had to deliver that also.  Then you realized you had a platform on your hands, and it made sense to build in targetted ad-serving technology.  People were already standing in line to advertise, and they had quite a lot of feature requests, so your initial implementation of this system had to evolve many times over.  Somewhere in there, you realized that you could open up the platform to the world and enable them to build social-powered applications on top of your platform.   Then you realized that people actually freak out about privacy, so you had to make everything customizable…

    Needless to say, even in the over-simplified view of the world above, we see that the evolution of a business brings with it many different stages where you must change your focus and evolve your product vision.

    The evolution of a business creates architectural debt

    If we zoom out and look at the whole picture in a scenario like this, we see that the reality is much deeper and more complex than ‘simply writing sub-optimal code for the sake of velocity.’  We have technology needs that change with time, whether they are new features or entirely new systems/sub-systems of the core application.  Over time, features can drastically change their manifestation.  As the user group grows, inevitably, everything is more customizable than it ever had to be before.  That’s not to mention the architectural changes that need to be made in order to scale.

    You see, it’s all about the altitude of our perspective.  Even if we attacked the new features and technical needs in the most optimal way as each of them came along, we’d still be in a situation where the architecture of the application would be sub-optimal after some amount of time.  Even if each new component worked perfectly (in the most elegant and optimal way), the way they fit together is likely to be imperfect because not all of them could have been foreseen.  It all depends on how far up we go in terms of altitude-of-perspective.  Eventually, if we look at a large enough series of time, there is no way to avoid the architecture being lacking in some significant way.   So we see here that the simple term ‘technical debt’ won’t do anymore.  So I’m coining a new one: architectural debt.

    The simplest acid test I can think of for architectural debt is asking yourself this question: If I was to rebuild my application/platform from scratch today, would the architecture be exactly the same as it is now?  If your answer is no, that probably means that your business and its needs have evolved over the last few years, and you are in a much better place to think about it holistically today than you were when you started.  Welcome to having architectural debt.

    Refinance your architectural debt

    So, you have architectural debt.  Now what?  Well, that’s easy, right?  Just pay it down like you would any other kind of debt.  Well… not quite.  The world gets a little more complex when you are dealing with architectural debt.

    Architectural debt is much more challenging to deal with because it involves a lot more ‘capital’ in order to pay it down.  It takes a much bigger block of effort because it’s not as easily chunked into smaller blocks that you can tackle within your normal workflow.  Worse yet, you are likely to have architectural debt in the first place because your business has been growing rapidly and its needs have been changing just as quickly.  So you have pressure to maintain velocity in developing new features.  Obviously, in an ideal world, you could just stop all new development and work on paying down your architectural debt.  Alas, this is not likely to be so in your case.

    So what you have to do pay down technical debt is akin to refinancing your loan.  With a refi, you end up paying down all of your debt at once by taking out a loan at a lower interest rate.  You still have debt, but it is cheaper to pay it off in the long-run.  The technical equivalent of this is to re-architect the core of your application, which generally includes the data model and the most frequently used/accessed components.  You will probably have to leave most of the other components alone due to lack of resources that can be committed to the project.  So you’ll just have to write legacy adapters into the core (re-architected) part of the application so it can still effectively talk to the other components.  But all components written from this point forward should be using the new architecture.

    With time, you’ll be able to pay down the rest of the debt by refactoring all of the other components of the application (and systematically removing the legacy adapters as you go.)  You’ll begin by refinancing your debt, and ideally, refactoring the other components of the application is like making ‘lower payments’ because of a lower interest rate.  It should be easier to do and a lot more pleasurable for the engineering team.  And eventually, you’ll end up being debt-free instead of bankrupt!

  • What you can do about Technical Debt

    In my last post, I introduced the ‘technical debt’ metaphor and talked a bit about why it makes a lot of sense.  In this post, I’d like to dive into why technical debt is so important and what you should be doing about it.

    Why is technical debt important?

    Technical debt is often overlooked because it generally has symptoms that lay ‘beneath the surface’ to the average user or manager.  By the time the symptoms become visible to the real world, it’s generally too late.  Those who ignore the importance of technical debt do so at their own peril.  Here’s why it should be top of mind:

    • You are likely to face challenges with technical debt at some point in the future.  That is, of course, unless you are already have accumulated some and don’t know it yet.
    • It is a problem that you sort of want to have.  That’s because it is a problem that scales with success and growth of your organization.  The faster the outside world makes you go, the more likely you are to rack up debt.
    • If you don’t deal with it, it will catch up to you.  And when it does, it’s likely to be a big problem, such as downtime, slow velocity, or low engineering morale.

    However, if you are smart enough to actually take the concept into account and work diligently to ensure you don’t go bankrupt, you’ll be a much better company.   You’ll be more agile and lean in the future because you’ll have a codebase that is ready for change and quick iteration.  If your code and/or infrastructure become stale and brittle, then you are simply asking to eventually become irrelevant.

    In my opinion, velocity is the key competitive advantage in any business: young or new.  If you don’t have the foundation laid to go fast when you need to, your competitors will be able to out-innovate you.

    What should you be doing about technical debt?

    Step 1: Acknowledge it

    Now that we know how important technical debt is, we can look into figuring out how to stay ahead of it.  The first thing to do is acknowledge that it exists.  While this sounds sort of simple, it actually can be more challenging than you think.  Most organizations have cultural elements that impede progress in recognizing debt.  In many organizations, sweeping things under the rug is a great way to keep management off of your back.  If you admit that something out there is sub-optimal, you’ll often get hounded and reprimanded by superiors.  In many organizations, the working environment doesn’t exactly value honesty and openness.  Mistakes are embarrassing, so you are better off not bringing them up.

    You cannot let this be your organization!!  Work to create a culture of honesty and open communication.  People shouldn’t be embarrassed of finding sub-optimal technical issues.  Instead, they should feel a sense of ownership and empowerment.  The best way to get issues like this on the table is to explicitly set aside time for digging into them.  I’ve had a lot of success with carving out time to talk about technical debt at the sprint/release planner level as well as bringing it up in team retrospectives.  If you don’t explicitly ask for people to think critically about areas for improvement, they may never bring them to light because they are so focused on driving velocity with the upcoming features.

    Step 2: Pay it back

    Now that your organization knows that technical debt exists, you need to start allocating time to pay it back.  The best way to allocate time to paying down technical debt is to build it into the velocity calculations of your existing iteration planners.  That approach works whether you are using story points for estimation or hours for estimation.  Let’s say that your team has an average velocity of 10 story points for each iteration.  But you’ve come to find that there are is some sub-optimal code sprinkled throughout the  codebase.  You can start by allocating 20% of your time to paying down the debt.  So the team can take 8 points of normal feature work and 2 points of technical debt work off the backlog.

    The key here lies in staying consistent.  Whatever you do, try to keep it up and make it part of your normal team process.  If you decide to maintain a backlog for technical debt, or you want to build technical debt tasks into your main backlog, just be sure to continue revisiting it and re-prioritizing it (like you should be doing with all your other tasks).  And be sure to consistently spend some velocity, even if its a very tiny amount, paying down the debt.  Just like with any amount of debt, you don’t want to continually be running a deficit.

    Occasionally, you may find yourself in a place where you can do a ‘big payback’ of debt.  That’s sort of like overpaying on your mortgage to pay down the principal — it can’t hurt.  Sometimes the urgency of feature velocity is lower than others.  Also, sometimes there are bottlenecks in other parts of the organization that give you some time to focus.  At ModCloth, we engage in a ‘feature-lock’ around Holiday time, which gives us the ability to spend a little more time on debt than usual.

    Some takeaways and other insights

    So we’re done exploring some of why technical debt is so important and what you can be doing to stay ahead of it.  Some things to keep in mind going forward are:

    • This is one of the more advanced concepts of engineering and product development.  Don’t beat yourself up if you aren’t effectively managing it yet.  If you’re thinking about it, you are ahead of most of the game.
    • Effectively managing your technical debt really falls under the bigger umbrella of staying relevant and agile.  Agility allows you to be ready for change.  Anyone that has been in product development for long enough knows that your product is likely to become more of a liability than an asset at some point in the future.  The degree to which you can keep your technology relevant is the critical factor in how long your product remains an asset.
  • Technical debt

    Oh, the power of metaphor.  I have been thinking a lot about how powerful metaphors go a long way in creating effective communications and laying the foundation for solid teamwork.  One of my favorite metaphors currently is the term ‘technical debt.’  It draws an excellent parallel between the worlds of technology and finance.  Also, it is a very important concept that I think anyone in the world of technology should be aware of.

    What is technical debt mean?

    Technical debt is a metaphor that was originally coined by Ward Cunningham in 1992.  It likens the ‘cost of going fast’ in the world of technology to the ‘cost of buying something you can’t afford today’ in the world of finance.  Technical debt often refers to writing sub-optimal code for the purpose of shipping a product on time (or within the current iteration).  Like financial debt, the goal in taking out these ‘technical loans’ is to eventually pay them back.

    I sketched up a quick visual in the form of a box that explains the effort needed to do a technical task.:

    The area of the box is the total amount of effort needed to complete the activity.   The total effort of a task can be broken into two main parts: a) the part that makes it work and b) the part that makes it work really well (under the hood).  The issue here is that folks in the business care only about the blue box.  But, of course, technologists care about the green box.  Focusing on the blue box can drive a lot of business value in the short-term, but foregoing that ‘green effort’ does not come without a cost.

    Why is technical debt such an excellent metaphor?

    The power of any metaphor is driven directly by its ability to create a picture in your mind that quickly and efficiently gives you a model of how to think about something new.  The ‘technical debt’ metaphor is such a good one because it really does give us a clean, simple, and familiar framework to think about something that most of us don’t currently understand.  Let’s explore some of the similarities between technical and financial debt:

    • They both trade now for later.  When people go into debt, they are borrowing money to purchase something now as opposed to ‘saving up’ to buy it later.  That’s concept aligns perfectly with aiming to get some code out the door quickly.  You could ‘save up’ and do it elegantly and nicely, but that would take longer.  Might as well ‘borrow’ and get it out the door now.
    • They both build up over time.  People don’t get into money trouble overnight; and they often don’t get into technical trouble that way either.  It’s a slow and steady progression of taking some liberties that you should eventually rectify that really gets you.
    • Neither is necessarily a bad thing.  Debt is simply a financial instrument.  Many people use it quite effectively.  In fact, for very large purchases, most Americans would not be able to survive without debt.  (Imagine cutting a check for your house!)  Technical debt is the same.  The goal is to drive user value, and ‘borrowing time’ allows you do that more effectively.  Debt only becomes a problem when it spirals out of control.
    • Both can be paid down.  Just like you can make monthly payments to slowly chip away at your financial debt, you can incrementally pay back your technical debt.  The secret for both is the same: budgeting for it.
    • You CAN go bankrupt.  In your financial life, a series of small gaps can turn into a really big one.  If that happens, you might find yourself in a situation where you simply can’t get there from here.  Technology isn’t much different — at some point, the gap between where you are and where you need to be becomes too great.

    It’s importance and what you can do about it…

    I have lots more to say on this subject, but I don’t have the time to write about it just yet.  Stay tuned though!

  • Agile Design: continued thinking

    In my last post, we just began exploring the idea of agile design.  The quick conclusion was that there are significant similiarties between design and development as creative processes.  Agile thinking can be effectively applied to both.  The big question is about specific implementation.  How would it work?

    There are two ways that I have seen agility applied to design.  For lack of better terminilogy, I’ll call them the:

    • two-wheel approach
    • one-wheel approach

    Let’s start with a quick explanation of what each wheel means.  The two-wheel approach treats design and engineering as two separate teams (or even two separate organizations).  In the two wheel approach, the both teams are working iteratively to push ‘shippable deliverables.’  For designers, it would be shippable designs.  And for coders, it would be shippable code.

    The natural progression of work tends to be that the design is done before the development.  As such, the the design iterations generally drive the development iterations.  Once a design product has been marked as solid enough for implementation, it is ‘completed’ in the agile design cycle and it marks the beginning of an agile development cycle.  The lengths of the iterations don’t have to be the same across design and development.  In fact, I’ve seen shorter design iteration length have a better chance of success.

    This is one way agile design and development can be done.

    The one-wheel approach is primarily different in that it treats design and development as one ‘product development team.’  This is a cross-functional agile approach for which there is very little precedent.  In this approach, the team is all pushing towards one shippable deliverable together, and each person is doing their part side-by-side with folks of other disciplines.

    In the one-wheel approach, the work is not as cleanly sequential.  What ends up happening is that some work is sequential, other work is parallel, and a lot of work ends up happening in mini-iterations within the iteration.  For example, a designer may produce a mock and pass it to a developer, but can be involved in ensuring that it is implemented in a pixel perfect way by the developer on the team.  As the developer has real-time feedback, the designer could very quickly re-mock and pass the mock over again.  As the UI of the app increases in fidelity, the designer can start producing unforeseen assets like icons and other images that are implemented downstream.

    Another way to accomplish agile design and development.

    Of course, the million-dollar question is: which approach is better?  The honest truth is that I don’t know yet.  I have used both and have seen pros and cons to each.

    The advantages of the two-wheel approach are:

    • Much easier to do when teams are distributed.  (Even moreso if they are distributed by function.)
    • Enables cleaner divides in the organizational structure.  This is particularly useful if your organization is already entrenched this way.
    • Easier to implement from a complexity and organizational change standpoint.
    • Potentially more efficient in terms of people working on the thing they are skilled at all the time.  (Little to no downtime.. and designers are always designing and coders are always coding.)

    The advantages of the one-wheel approach are:

    • You get more happiness and buy-in because everyone is involved in the final implementation decision.
    • When velocity really matters (you are under the gun), the micro-feedback cycles of a cross-functional team are significantly faster.
    • Potentially better outcomes from having diverse viewpoints at the table.
    • Potentially more scalable.  Having many cross-functional teams scales more cleanly than bigger and bigger functional teams.

    What I am trying to figure out now is: are these approaches more or less appropriate for different circumstances or phases of growth?   I, unfortunately, don’t know the answer yet.  Of course, if you are not in a startup or not in a position to mold the structure of your organization, then the constraint of organizational structure can drive your decision.  (Most of the time, that will drive you to the two -wheel approach.)

    The one-wheel approach has some potentially large benefits (like better product outcomes and more buy-in!), but it takes a lot more organizational buy-in.  Cross-functional teaming can be very complex to implement, particularly in organizations where different leaders of different organizations have strong visions for ‘how things should be done’ in their world.

    I’ll try to report back after I know more!

  • Agile Design: Myth or Reality?

    So, I have been fortunate enough to be part of a lot of conversations with a lot of smart people recently.  A good number of them have been centered around one topic: agile design.  For me, it’s sort of funny that this topic has come more and more into the spotlight over the last year.  The first time I started talking about ‘Agile Design’ was in 2008, when ModCloth was just starting to build out it’s design and development competencies.  Back then, I think people mostly thought I was crazy.  The feedback was plain and simple: “agile doesn’t make any sense in design.  Developers don’t understand our world.  Our process has to be ‘creative.'”

    Today, I think folks are slowly but surely coming around.  If I had to guess why, I would say that it has to do with the environmental factors mostly.  Agile methodologies have grown in popularity, considerably so in the world of software development.  A lot of folks have been evangelizing their specific brand of agile over the years, and I feel like it’s really hitting a tipping point.  (Or more likely, it has already hit the tipping point.)   So people are realizing that agile is really here to stay, and that means that it’s really hard to ignore in the design world.  The most compelling reason, of course, is that one of the core principles of agility is empowering the team.  So if the software developers are empowered to start envisioning the software — when do the designers get involved?  Well, in some companies I know, the designers are mostly charged with making stuff ‘look pretty.’  How fulfilling is that?

    Ok, so agile is here to stay.  And it creates a weird situation for designers (and others) in organizations where only the software developers are practicing agile methodologies.  And now days, people are starting to throw around this phrase ‘agile design,’ even though what it means is is not clear to anyone.   So, let’s figure it out:

    The core principles of agility are simple, in my words:

    • Chaos: Agility is all about unpredictable processes.  The primary goal is to be able to respond to change as opposed to following a plan explicitly.
    • Bravery: Unless you are brave enough to make mistakes or get feedback before things are ‘perfect’ in your mind, you are always going to lean towards waterfall.
    • Get real: The idea is to build stuff instead of creating comprehensive documentation that tells you what to built.
    • People over process: It’s about getting smart people together and figuring out the solutions as opposed to being too prescriptive about following rigid processes.
    • Collaboration:  If everyone is on the same team, everything gets done faster.
    • Iteration: Don’t try to build towards a ‘grand unveiling’ — that never works.  Instead, build iteratively and get feedback along the way.

    Because I can’t live with myself without sketching out a quick visual of how these principals drive one another:

    So, before we do anything else, let’s ask ourselves this: do the principles above apply to design?  My gut tells me the answer is yes.  Chaos is a reality of nearly any organization.  You cannot clearly predict the future, so you have to be ready for change, and you have to be ready for your vision to evolve.   And the same bravery that would enable a developer to show someone their code or their half-complete feature is the very same bravery needed by a designer to show someone a design before they think it’s 100% ready, no?   And the principles of iterative work and collaboration also still apply to design, no?

    So can you engage in agile design?  The answer appears to be yes.  Is it exactly the same as agile software development?  Of course not.  It’s safe to say that both software development and design are creative processes.  But they are not exactly the same in terms of how they are accomplished (or how long they take to get accomplished.)

    But there is more to this topic than meets the eye.  If your organization is going to buy-into the idea of agile in the realms of both design and development, how are you going to enable both of them to get together?  That seems to be the core question.  I’ll followup with some thoughts on that soon!

  • Agile office space continued

    Ok, so I have had the opportunity to sink my teeth into workplace design a bit more in recent days.  My recap of requirements is as follows:

    • We need collaborative space — so that people who are working on laptops/workstations can work together seamlessly.
    • We also need space that allows people to focus and get work done on their own.
    • The space has to look good and feel good for the employees – because we want to attract the best talent and keep them happy.
    • The space has to align with our culture.  We’re a creative company to whom style matters.
    • We are on a budget, and we’re still a scrappy young company, so we don’t want to be ridiculous.
    • The workplace must be agile and allow us to adapt to change quickly and easily.

    So far, I’ve had the opportunity to research what other interesting companies have been doing with their workspaces.  I’ve also done some research on what some of the leading workspace design companies have come up with.  So I feel a little smarter on the subject.

    Here are some things that are really interesting to me:

    Pair-Programming Stations

    The more I think about it, the more I think that having workstations like the one shown below would be really solid for pairing:

    If you want to insert best practices into engineering, you have be willing to walk the walk.  You can’t just preach it and then ask two people to cram together around a 17-inch MacBookPro.  It’s not realistic, and it doesn’t jive with human nature.  If people aren’t comfortable in the pairing environment they are put in, they will just stop pairing.  (Duh.)

    Collaboration for Bigger Teams

    I think the above is relatively doable.  But what about for larger collaborative teams?  How do we get a workspace that suits the needs of 3-4 people working together on a creative project?

    I was checking out the local Steelcase showroom and I saw a few things that really piqued my interest.  One was a collaborative space that allowed people to share what they were looking at in real-time.  It’s called “media:scape.”  Unfortunately, they don’t have great pictures up online, but these will give you a sense:

    The idea here is that anyone can just walk up to a space like this and sit down with their laptop.  They can then work collaboratively as a collaborative team, easily tapping the ‘puck’ on the table to have the larger screen show what they are working on.  Does that sound productive and effective to you?  It sure does to me!

    I am just thinking about how much time and effort (and thereby, money) that we’ve spent in the past just trying to get setup to show other people in the room what we are working on.  That problem is exacerbated significantly when there are multiple folks in the room trying to show people what they are up to at the same time.

    Finally, and potentially just as interestingly, this kind of setup is totally useful from a user testing standpoint.  Being able to easily sit around a table with a testing participant and have a small group in the room observing their interactions would be fantastic, and apparently not that hard to set up!

  • Agile office space

    I had a remarkably interesting week.  First of all, I realized that we needed an office space last month here in San Francisco.  (At ModCloth speed, we need everything last month!)  With the rate at which we are planning to hire, it’s critical that we find a solid space that suits our needs today and gives us a tiny bit of room to grow.

    But the more exciting part, of course, is thinking about how to design the space.  I think that workplace design is easy to overlook.  By that, I mean that people won’t give it the attention it deserves.  Of course, they’ll buy furniture, they’ll go through the motions.. but they won’t REALLY THINK ABOUT IT.

    That’s what I am trying to avoid when I think about building out our San Francisco office.  In the end, a lot of life boils down to ‘user experience.’  Whether you are shopping on the web, playing golf at a well-designed course, or sitting down at your desk to get some work done, someone SHOULD have thought through your experience and worked to to make it pleasurable and easier.

    I guess that someone is me.  Given the urgent timeframe, it’s up to me to get smart about this really quickly.  So here’s what I’m thinking:

    • Collaboration is of critical importance to any creative organization, ours included.
    • Most workspaces don’t REALLY center around collaboration.
    • People especially don’t think about the realities of human nature.
    • Furthermore people don’t think about the ‘little things’ that drive usability, or lack thereof.
    • Form is important, but function is more important in a space that people work for 8 or more hours each day.
    • We want to promote an atmosphere of collaboration as well as distraction-free work, somehow at the same time.

    We’ll have a variety of roles at work at our San Francisco office:

    • Marketing
    • UX Design
    • Graphic Design
    • Product Management
    • Engineering
    • Administrative

    It feels to me like most of the people in the workplace will be collaborating in others in cross-functional teams.  If not all the time, then people will be working in teams for at least some part of their day.

    How do we effectively promote that kind of work?  What we really want in these collaborative workspaces is for people to be able to work in close proximity and to share what they are working on with other people when that’s useful.  At some point, their teammates and colleagues will want to share back.  But the whole day won’t be sharing.  Just the times that need that collaboration.  (Unless, of course, they are engineers doing full-time pairing — but we’ll come back to that.)

    In an ideal world, people would even be able to seamlessly collaborate on the same workstation from time-to-time — using different sets of mice and keyboards to drive the same machine.  This way, they can all collaborate in real-time on something.

    But once a shared understanding has been built within the cross-functional team, it’s likely that people will need to do some heads-down (distraction free) work on their own.  Working in a group-setting all day would make it nearly impossible to actually get anything done.  So they’d need the ability to get into the zone and focus somehow.   Does that happen somewhere other than the collaborative space?  In other words, can people move from a private place to a more collaborative space throughout the workday?  Possibly.

    So what does that workspace look like?   Well… that’s what I am trying to figure out.  Would love insight from others.  I’ll continue this thread in another post soon…

  • Why no one thinks about usability

    I’ve been in the software development business for a while now. And the lesson that I have learned, and re-learned, more than any other is simple: Usability is WAY MORE important than everyone thinks.