What does a ScrumMaster do?

Here are three different ways I will try to concisely summarize the ScrumMaster role:

  1. As a ScrumMaster, your job is not to control the team or stakeholders or anybody, for that matter. Your job is to ask questions and make observations that will help lead people to their teachable moment. Whether they choose to learn or not is up to them but you must keep leading them there, every day
  2. Being a ScrumMaster is rather like being a mirror. Every day you are saying “Here is what I am seeing. What do you see?”
  3. Flow is the psychological state when working of being completely immersed in what you are doing. Flow is the most productive (and pleasurable) state in which to perform work. But many Scrum teams start life at the opposite end of the spectrum in a state called thrash. Thrash is that frustrating feeling that it takes 10 units of effort to get one unit of results. It is not a ScrumMaster’s job to manage the team or “make” them perform. Rather, it is his/her job to do everything in their power to move her team towards flow and away from thrash.

What’s missing from the Agile process?

Architecture.

I know. For agile folks, that can be a dirty word. Yet XBOX Live was developed entirely using Scrum with well over 100 developers, in one-week sprints, with continuous integration, without breaking into silos.

And the resulting platform is 100 times more secure than the monolithic PlayStation online platform.

How was it done? Architecture.

Software architecture creates an API that reflects underlying information objects, captures the needs of scalability for different classes of services, and allows the front end to vary completely independently of the back end.

Software architecture allows the partitioning of functionality without losing system cohesiveness and without creating silos in your team structure.

Good architecture is the missing ingredient with most agile teams and without it, you can never be truly agile.

Hire some decent architects.

How do Project Managers break down requirements into Stories?

A good product manager doesn’t “break down” requirements into user stories, they start with good user stories and work from there.

The whole point of using user stories is that they are how you communicate “requirements” to your development teams — by stating the user persona that you’re targeting, the task that user is trying to perform, and the goal that the user expects to achieve.

If you’ve already got a list of “requirements” then you’re not really using user stories for the purposes for which they were intended.

That said, you may start with a very large user story, such as “A new customer wants to set up an account so that they can access the product.” That story is way too big for a team to do in an iteration, if at all. We generally call these “epics” which can be broken down into “themes” and then further broken down into “stories” until we get to something that a single development team can fully complete in a single iteration. For this story, that might look something like this:

  • Epic: “A new customer wants to set up an account so that they can access the product.”
  • Theme: “A new customer wants to confirm their email address so that they know we have the correct account information for them.”
  • Story: “A new customer wants to enter an email address so that we can contact them if we have information that is important to them.”

Communicating Agile “requirements” using user stories requires a completely different perspective and starting point than traditional product management approaches.

Why developers at Google consider Agile to be nonsense?

Responding to a junior developer who asked “Why developers at Google consider Agile to be nonsense?”

I said,  because in practice, “agile” is code for sloppy thinking & lack of design. Code first, think later. It’s also great for giving managers metrics they can report against that sound good (e.g, “tickets closed”), while not having to show real results – like sales, or lawsuits avoided.

Ok, let’s start with the four principles from the Agile Manifesto. The mindset of a hacker, not an engineer. You wouldn’t even think of applying them to a construction project – buildings would be falling down all over the place (if you could get approval to start construction, without detailed plans to approve), and all involved would be spending most of their time in court.

  • Individuals and interactions over processes and tools: For any complex piece of software, one needs to be very disciplined about pinning down design details (e.g., interfaces), about configuration management, about testing, etc. All of those are process intensive activities. The work can sometimes be reduced by good tools (e.g., a good version control system, automated build/test).
  • Working software over comprehensive documentation: A rather poor dichotomy. What does “working software” mean, without documented specs? How does one install, use, interface to, or maintain software without comprehensive documentation? Personally, I’m a big proponent of “write the documentation first” – otherwise, you don’t know what you’re building, or buying.
  • Customer collaboration over contract negotiation: Again, a poor dichotomy. In a lot of cases, contract negotiation IS collaboration, and/or defines the framework for collaboration. If you’re in business, particularly if you’re building custom software, the absolute last thing you want to do is spend a lot of time coding, only to have a customer tell you “that’s not what I meant.” Collaborate over use cases, storyboards, test & acceptance criteria, transition plans – but be damn sure to get to an agreement over what goes into a release, and how it’s going to be accepted, before starting to write code – otherwise you’re going to be out of business very, very, soon. (Might I add, that almost all companies try to insulate developers from end users. It’s kind of hard to actually get anything done if you’re spending all your time responding to support calls, or “can you just change this one thing?”)
  • Responding to change over following a plan: Yes, things change. But by and large, this says “code first, think later.” That’s just sloppy. If you’re fighting a war, agility is critical (“no plan survives first contact with the enemy”), or if you’re playing basketball. But if you’re writing software, you’re a lot better advised to “plan the work, then work the plan” – and if things change, change the plan. (One might note that most of what pass for “agile methodologies” – such as scrum – are really planning processes, just ones with very short time horizons.)

By and large, “agile” is code for laziness in up-front design. Just fix the bugs & add features in future releases. Sometimes, that works – Microsoft has been very successful in shipping quickly, then fixing things later. It also works fairly well for periodic releases of mature software (fix some bugs, patch some security holes, add a few features). And, it’s absolutely necessary if you’re responding to a security attack, at the moment.

It doesn’t work as well if you’re building buildings, or mission-critical software (no, we can’t recall the spacecraft after it goes off course), or laying rail tracks from the coasts, that have to meet in the middle of the country. Or if you’re a large team, building 100s of modules that all have to fit together – you’d better get the architecture, interfaces, and databases all pinned down well before writing a single line of code. Otherwise, you’re just going to have to do lots of re-work down the line, and God help you if you want to add new features, without having well-defined hooks built in from the start.

And then there’s: Dilbert Comic Strip on 1993-05-04

Why do programmers wear headphones while working?

There is one critically important reason to wear headphones while programming. When you understand the impact of this, it will change how you think about what it means to make things and what that process really requires.

Programmers wear headphones so that they won’t be bothered. That much is pretty obvious. The reason why being bothered is an issue is something I’ll get to in a moment.

You see, the modern workplace is designed to ruin programmer productivity. If I were to design an office space to maximize programmer output, it would look like the opposite of nearly every modern office space I’ve ever worked out of.

Actually, I now work in an office that basically maximizes my potential output as a programmer and it really does look like the opposite of most modern corporate offices.

The big gimmick right now in office space design is “open office spaces”. This is designed to improve collaboration (as if that is the great problem of modern work).

Open office spaces really exist so that it looks like everybody is busy working. It looks cool to see so many people busily walking around, talking, and in general looking busy. There is a sound component to it as well.

Open office spaces are loud!

But again, it looks and feels like an improvement over the previous decades’ bad idea – cubicle farms.

Also, open office spaces are cheaper than cube farms, so companies are saving money while making everybody look busy (even at the cost of huge productivity decreases).

In general, all of this is terrible for the actual act of programming.

The art of programming tends to rely on the same thing most art relies on – flow.

Getting into the “flow state” where you are focused on a problem deep enough to make connections and leaps and do your best work without “thinking” is where the real magic of programming happens. This is very similar to playing music, writing, and painting.

Imagine if Leonardo da Vinci was expected to paint the Mona Lisa in an open office space while his coworkers and manager would stop him mid stroke every 15 minutes to ask a question. Would he have ever painted it?

No, I don’t think so.

And yet, programming is a creative process that requires a similarly creative environment as most other art does.

Take authors as a similar example. Stephen King doesn’t “collaborate” to write his books. He sits down at the typewriter or keyboard and starts writing. Imagine if he had to do a daily standup about his latest few pages each day with a committee of his peers.

His work would suffer.

It turns out that to code well we rely on that same kind of process of getting into the flow and doing creative work with our minds. It’s how our brains seem to be wired for creative work.

People like Paul Graham have written about Makers Schedule vs. Managers Schedule before with a similar understanding.

It takes something like 5–30 minutes to get into a flow state. If uninterrupted, time distortion kicks in and a couple hours fly by and a bunch of code is created.

Every interruption resets the clock on that process.

So, what programmers need to maximize flow (and thus output) is blocks of 2–4 hours without interruption. Ideally, a whole day without interruption would allow for as many as 3 or 4 blocks of “flow”.

As near as I can tell, the average programmer tends to only get maybe one flow block a day. Sometimes a really diligent programmer will manage two or three flow blocks a day, but that doesn’t happen very often from what I can tell.

Headphones, especially noise canceling headphones (like the ones I’m wearing right now), are an effort by programmers to block out interruptions and stay in a flow state.

With open offices, email, slack notifications, coworkers tapping you on the shoulder, and so on…

It’s almost impossible to get to a flow state as a programmer.

So, my advice to you is simple. If you are a programmer trying to maximize the total productivity and flow blocks each day, wear headphones, avoid meetings, don’t check email, turn off slack, and basically go into a cave and write code.

And if you don’t have the option to do that, accept the fact that you might only be able to get one flow block a day and plan accordingly.

Agile Scrum Estimations Fail

The whole driver behind Agile was that we can’t estimate very well. Sure, we can make vague claims like ‘adding a connection pool will take half a day’.

But we cannot account for the fact that somebody else already added a conflicting one in a module we’ve never heard of. The library we plan to use has a bug in it that is only triggered by our specific usage. The precise handling of timeouts breaks our circuit breaker behavior two services up and none of us yet know there even was a circuit breaker there.

Software development is full of uncertainty. It has a multiplier:

uncertainty actual uncertainty of core problem
X team unfamiliarity
X size of the team
X number of stakeholders
X quantity of new team members
X lines of code
X amount of unclean code
X amount of duplication
X network and connection issues
X outstanding technical debt
X number of separate teams
X inexperience of BA and PM
X distance of new requirements from old ones

This all gets washed away by the pretense that story point estimation and a velocity chart “makes that all go away”.

It doesn’t.

As the amount of existing work increases, the number of unknown ‘gotchas’ increases team-wide.

I have yet to see a formal, accurate forecasting method for software delivery schedules that a PM would actually bind themselves to.