Dysfunctional In a Chill Way

I've had many jobs over 20+ years in the software industry, and every one of them has been dysfunctional to some degree. Of course, that's normal. No person is without flaws, and the same is true of jobs. Even my best jobs had dysfunction.

There are many ways for a job to be dysfunctional...

• Micromanagement
• Unreasonable deadlines
• Poor communication
• Etc.

But something I've learned over time is that jobs can be dysfunctional in a miserable way or a chill way.

A company doesn't need to be as dysfunctional as Kruger Industrial Smoothing, but a little dysfunction can be a positive thing for a job...if it's the right kind of dysfunction.

Some of the dysfunctions of a job (e.g., micromanagement) can make one's life stressful. But not always. The chill dysfunctional flip side of micromanagement, little-to-no supervision, can be awesome depending on your situation:

• Time to work on your own priorities
• Build according to your own standard of quality
• Decide what you think is important
• Relaxed pace

Just like every person, every job has dysfunction. If they're flawed in a chill way, you're doing pretty well.

Who Wrote This Crap?

I remember many years ago interviewing at a software consultancy where one of the consultants in the meeting was talking about a project they had going on. He was marveling at how fast the team of junior developers on the project was "cranking...out...code." I'll never forget the way he said it, and the look of awe on his face as he slowly shook his head back and forth.

This of course happened well before the advent of LLM assistants that can produce code at an astonishing speed. I wonder if that consultant's head would have literally exploded if I had showed him GitHub Copilot or Claude Code at the time.

Hell, I remember being gobsmacked as a junior engineer myself by CodeSmith, which was an early code generator product for C#. It could automatically spit out huge amounts of boilerplate code that an engineer would typically write by hand. I heard about it from a coworker at my very first job in the software industry, and I remember how unnerved I felt, thinking about a program that can automatically write code (isn't that why I'm here?).

The next code generator that came into my life was Ruby on Rails, which I first tried a couple years later. By that time, I had experienced what it was like to write the most tedious code in most applications, which was code wiring up database tables to web forms, and all the plumbing to pipe the data through each layer. It was a mind-numbing yet essential task, and almost every application needed it. Rails came at me like a breath of fresh air. At this point, I was like, oh yeah, this rules. I hated writing all that data access CRUD, and Rails made it largely disappear. Code generation clicked for me.

In the .NET world, where I've mostly worked for my career, we had a series of object-relational mappers that could generate classes off of your database schema, and resulted in engineers having to write way less code that shuttles data from a web application to a database and back. To name a few, we had NHibernate, SubSonic, LINQ to SQL, and then Entity Framework. I welcomed these tools with open arms, but I do remember there being pushback at the time from more senior people at my companies who were accustomed to writing this kind of code by hand and didn't trust what the tools were doing under the covers.

The common denominator in my early experience with code generation was eliminating the manual work of writing a lot of extremely predictable, relatively dumb, utterly tedious code that was also essential. And this usually took the form of data access code that moved data between layers of a web application, from the front-end to the database, where you had SQL tables that corresponded to C# classes, that corresponded to web forms. Classic CRUD. It's usually highly predictable stuff, and ripe for code generation. Yes, there were times where the tooling would break down, and an engineer would have to get under the covers and debug an edge case that the tool couldn't handle automatically, but, in my experience this was relatively rare.

What I'm seeing in the industry now with AI coding assistants like Copilot feels fundamentally different to me. I've witnessed fellow engineers in recent years generating code more akin to "business logic". This is the kind of code that's specific to the domain of the company and not transferable from codebase-to-codebase. I've also seen fellow engineers letting AI write the code for areas of the codebase that they don't understand well. For example, they may not know how to do a certain thing with React, so they describe what they're trying to do to Copilot, which then generates code that the engineer accepts without understanding, as long as it seems to work.

What's fundamentally different to me about the scenarios I just described and the code generation scenarios of yore, was that the pre-AI code generators were deterministic, and they were applied only to non-domain-specific logic.

What happens when a codebase is peppered with business logic that no human working at the company wrote, and hence cannot definitively explain? I have already seen first-hand times when bugs in important processes were not discovered until the AI-generated code had been in production for weeks. The engineer committing the code did not know that what Copilot generated did not match the logic they intended. Maybe I can write another whole blog post about this topic, but I'll say briefly here that in many scenarios, increasing the speed at which the code is produced is less important than a human understanding what it does at a granular level. In other words, speed of coding is not a bottleneck.

Another consideration is that AI code generators like Copilot are non-deterministic. As in, you can run them multiple times with the same input and they will produce different results. Going back to my examples before of pre-AI tools, the code generation features of CodeSmith and Entity Framework are deterministic. You can run them multiple times with the same input, and they will give you the same output every time. This is because a human software engineer wrote the code behind those tools, and the rules are directly and unambiguously traceable back to the lines of code a human wrote while designing them.

I can't help but wonder if, as an industry, we're hurtling toward a future where many production codebases will be littered with code that no human at the company understands or could definitively explain, not years later, but even weeks later. My personal relationship to AI-generated code as a working software engineer is that I will not commit code that I cannot explain. And when doing code reviews, I cannot accept the explanation that code included in the pull request was AI-generated and hence the submitter does not know what it does.

I also have to wonder if the AI slop era we're all in at the moment says something about the illusory nature of quality. Maybe quality was just an unintended side effect of manual coding that business leaders never really cared much about in the first place. In my multi-decade career in the software industry, the emergence of Copilot represents the first time I've ever experienced non-technical people mandating the use of a particular tool to software engineers. It seems that the idea of faster code production was so mouthwatering that quality flew out the window within seconds.

Who wrote this crap? Maybe the answer never mattered.

Don't Mess With Delivery

If your team has consistent, reliable delivery of working software to production, you’re crushing. Don’t mess with it. It's astonishing how many teams in real world software development can't do this. I've witnessed it over and over and over in my career.

If you have a backlog, sprint planning, a dependable QA function (even if manual and/or slow), and scripted deployments on-demand, you are very fortunate.

Of course, we don't want our engineers working in a feature factory, so we make sure they have input into what they're building and they know why they're building it. That's the motivation to keep delivering consistently. 

So how do teams "mess with" delivery? For teams that already have consistent delivery, the way they tend to mess it up is by prioritizing speed over consistency.

I think that continuous deployment is an incredible technical capability to possess. It's amazing to have a fully automated pipeline capable of taking a code commit and moving it fully out to customers without manual intervention. This is huge for production bugs and emergency fixes.

My belief is that feature work should prioritize ease of communication over raw speed of deployment. Just because it's technically possible to deploy new features every day doesn't mean it's actually beneficial to users. I've seen firsthand the chaos that results when new features appear in front of users without people beyond the immediate team knowing about them.

Every organization wants to "go faster", but I believe once you're touching production, communication matters more than speed. If you're consistently getting new features in front of users every two weeks, you're already doing so well. Optimize something else.

If you've got a reliable, dependable, consistent pipeline of new features to real users on a reasonable sprint duration, it's not worth the communication overhead of going out-of-band. Please don't mess with delivery.

Above All Else, Sustainability

From the principles of the Agile Manifesto:

Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.

Extreme measures are always doomed beyond a limited time horizon. Whether we're talking about diets or software development practices, without sustainability you have nothing.

Initiatives to increase productivity, improve quality, lower costs, or any other "good thing" simply don't matter if they're not sustainable.

A common topic in Agile circles is "sustainable pace", which usually focuses on the futility of working overtime, typically over 40 hours a week. I would argue that sustainable pace is about more than just the number of hours clocked in a work day or work week. 

Is the work emotionally sustainable? Do people hate working here? Do people end the work week feeling a sense of accomplishment? Do people feel like their leaders have reasonable expectations? Are they constantly battling reality?

Sometimes the degree to which an enforcer must continuously apply pressure to get people to follow a process indicates how sustainable the process is. Powering through is not a sign of discipline, it’s a sign of delusion.

Ignorance of reality is unsustainable. Coercion is unsustainable. Surveillance is unsustainable.

Can I and the people around me keep this up forever? If not, it's time to pause and reflect. Above all else, sustainability.

Legibility

I recently came across a blog post called Seeing Like a Software Company by Sean Goedecke on Hacker News. The post hooked me right away by introducing to my vocabulary the term "legibility": 

By “legible”, I mean work that is predictable, well-estimated, has a paper trail, and doesn’t depend on any contingent factors (like the availability of specific people). Quarterly planning, OKRs, and Jira all exist to make work legible. Illegible work is everything else: asking for and giving favors, using tacit knowledge that isn’t or can’t be written down, fitting in unscheduled changes, and drawing on interpersonal relationships.

One of the advantages that small companies have is that they can achieve greater speed compared to a large company by eschewing legibility. When you know everyone else in the company by name, or maybe you all work in the same room together, you don't need standardized processes to go about your work, in fact, they just slow you down. Work happens through what the author calls illegible backchannels:

An engineer on team A reaches out to an engineer on team B asking “hey, can you make this one-line change for me”. That engineer on team B then does it immediately, maybe creating a ticket, maybe not. Then it’s done! This works great, but it’s illegible because the company can’t expect it or plan for it - it relies on the interpersonal relationships between engineers on different teams, which are very difficult to quantify.

One of the struggles that small companies face as they grow into larger companies is that they can't get by anymore without legibility. Maybe they've grown to hundreds of employees or they're working with people distributed across multiple timezones.

The fact of growing up as a company is that the loss of speed in individual tasks is outweighed by the predictability of process. As the author writes:

The processes that slow engineers down are the same processes that make their work legible to the rest of the company. And that legibility (in dollar terms) is more valuable than being able to produce software more efficiently.

I feel like this is a hard concept to sell people on who are used to working in small companies. In the long run, going slower is actually better for the company. Writing things down and organizing the work in a more structured way reduces the chaos left in the wake of localized, marginal speed-ups.

The author concedes that illegible work is necessary in small doses even in large companies. In cases of show-stopping production bugs, for example, large companies create temporary sanctioned zones of illegibility in which they gather together a strike team of experienced people to swarm on a critical issue until it's resolved. They then return to legibility.

Legibility is a massively useful concept that I will carry with me. It explains so much about how companies function internally, in often counterintuitive ways.

Slow Is Smooth, Smooth Is Fast

If you want to move fast, don’t make speed your goal. Make smoothness your goal. Fast naturally follows smooth.

Every software organization wants to be fast. And fast has a certain look to it. Buzzing Slack channels, video calls, people huddled around a screen or whiteboard. It's very easy to conflate "high touch" work practices with speed.

As I mentioned in my previous post, one of the huge benefits of sprint-based development is that sprints act as a delivery protector. A bulwark against constant churn.

The principles of the Agile Manifesto talk repeatedly about change, continuous X, face-to-face conversation. Some teams and leaders really latch onto these concepts. Let's change the requirements every day; let's discuss every decision face-to-face.

One of my favorite principles is:

Working software is the primary measure of progress.

All the churn that looks like "Agility" is canceled out if you don't deliver on a regular enough cadence that your customers notice. They don't see anything else, they don't measure you by anything else.

The passé methodology of Waterfall became passé because it didn't respond to change well enough. People were wed to process instead of reality. Why spend a year building something that no one actually wants, right?

But I firmly believe there is such a thing as being too responsive to change. There is such a thing as too much communication. Remember, your customers only see delivery. They only see what pops out the other end.

There is a benefit to following a well-defined process from idea to delivery. Process is perhaps a dirty word in Agile circles, but I think in some ways being anti-process is being anti-delivery. Every time we circumvent a process, we create communication overhead. Each team member impacted must be notified and followed up with to make sure they know about the circumvention. 

Any change is only good if the ripple effects created by the change are worth suffering. I reject the interpretation that Agile implies that more change is better. We have to always balance change with delivery. In fact, we have to balance anything that takes time and effort for a software team with delivery.

Sprints Need a Cooldown

Teams vary in the amount of handwringing their leaders do about sprints that aren't perfectly rightsized. What if we bit off too much work at the start of the sprint and can't finish it all by the end? What if we overestimated the work and didn't fit in as many tickets as we could have?

I've worked with teams where having work incomplete on the last day of the sprint was no big deal, it happened regularly, and no one was too concerned about it. And I've worked with teams where the team committing to a certain scope of work was sacred, and tickets carrying over was a grave error that we fretted over and vowed not to repeat.

The teams that prefer the overstuffed sprint are trying to maximize velocity and reduce downtime of the team members. Teams that treat the sprint timebox as sacred are trying to maximize predictability and often striving for a kind of "standardization" across teams. Some teams will even wonder why they're bothering with these stupid sprint things and opt out for a methodology like Kanban where people just grab work items when they finish the previous one, and toss out the timebox.

I have always believed strongly that whether you're using Scrum or Kanban, and working in discrete timeboxes or not, that team members need a regular cooldown. Just as an athlete cannot sprint indefinitely and must rest, I believe the same is true of software engineering teams.

As you might imagine, this puts me in the camp of "fill the sprint conservatively so that we don't go over." This, of course, means that many sprints will end without the theoretical maximum amount of trackable work items packed into them. And to that I say: Good. That's the point.

I believe that implicit in the social contract of teams is this: If you want people to sprint, then they must be allowed to cool down. That's the deal. Continuous sprinting means burnout. 

Quoting a previous post of mine, here are a few things engineers need to do that are not represented by tickets in a sprint backlog, and are perfect for filling the cooldown period at the end of a sprint, at their discretion:

• Training
• Preparing for a presentation
• Proof of concept / demo of an intriguing tool, framework, technology
• Updating documentation that's been bugging you
• Spikes into performance improvements
• Cleaning out your inbox
• For companies that do some sort of periodic "goal setting" for each employee, let people work on goals from their list

If we set a strong sprint goal, bite off a chunk of work that the team is confident of completing, and we get it all done without rushing a day or two before the end of our sprint, that's a good thing. The team feels a sense of accomplishment, the business gets a valuable increment of working software that fulfills a real need, and the individuals get some time to cool down and shift their attention to some low-intensity stuff that nevertheless needs doing before the next sprint starts again.

For me, the cooldown at the end of the sprint is what sustainable pace is all about. We go hard toward an important goal, and if our planning and teamwork are on point, then we know we'll have that down-shift at the end.

Are You a Stable or Volatile?

I've written before on this blog about the different personality traits of software engineers, and how different traits balance each other on a team.

Some of the spectra I came up with were:

• Dreamers ↔ Pragmatists
• Big Picture ↔ Detail-Oriented
• Move Fast & Break Things ↔ Slow & Methodical
• Optimists ↔ Pessimists
• Answerers ↔ Questioners

Recently I read a comment on Hacker News (which I can't find now) that mentioned a blog post from 2012 on Rands In Repose, a blog I've been familiar with for years, but somehow this post had escaped me.

The author, Michael Lopp, introduces the Stables and Volatiles divide, which feels to me like a meta-category that encompasses the ones I listed above.

You can probably guess just from the names, but Stables tend to like having a well-defined plan, are more risk-sensitive, predictable, and reliable. Volatiles are disruptive, like to blaze their own trail, dislike process, and have a strong bias for shipping something (even if it's ugly and unstable).

Lopp says:

Stables will feel like they’re endlessly babysitting and cleaning up Volatiles’ messes, while Volatiles will feel like the Stables’ lack of creativity and risk acceptance is holding back the company and innovation as a whole. Their perspectives, while divergent, are essential to a healthy business.

I believe a healthy company that wants to continue to grow and invent needs to equally invest in both their Stables and their Volatiles.

I think the Stable / Volatile concept feels very true to my experience, and it's one I'll keep with me. Assuming that everyone on a team is equally intelligent, competent, and well-intentioned, you need people of both the Stable and Volatile persuasions. Even though they annoy each other, a project staffed by only one or the other type is doomed.

Who Writes the Backlog?

The Product Owner has the final say in backlog prioritization. But who writes the backlog? Is it always the product owner?

My experience with Agile has been that, in the real world, the backlog contains two types of items:

1. The "user stories" that discuss functionality from the perspective of an end user
2. Technical stories--including things like technical debt, maintenance, package upgrades, etc. Things that don't represent user goals that the product owner would be focusing on.

Both types of items are important and need to be worked on. 

You can get into trouble in a couple ways:

1. Every item in the backlog is forced into a "user story" template ("As a...", "I want to...", "So that...").
2. The product owner is writing backlog items that are of the "technical story" variety since they think they need to write every story in the backlog.

If you have a product owner who has consistent time to write Jira tickets (or whatever issue tracker you have), you're pretty lucky. My experience with actually-existing-Agile is that the Product Owner is typically someone in middle management who is juggling multiple jobs, "product owner" of this product being one more job, and certainly does not have the time to dedicate to backlog management.

Occasionally you get an overactive product owner who tries to capture every task the development team is working on as backlog items, including purely technical tasks that are better written up by engineers or engineering managers/leads.

My opinion is that anyone on the team (engineers, QA, managers, product owner) should be empowered to write backlog items. When the nature of the backlog item is technical, let the technical people write it. When the backlog item is a "user story", the product owner should write the item, or at least delegate the requirements to whoever writes it (probably not an engineer).

Backlogs are just another tool in the tool chest of software engineering. As long as the people doing the work are clear on what the goal of the item is, and the acceptance criteria for its completion, then anyone can write them in whatever format they want.

Scrum Tensions: Code Review

There are tensions in Scrum anywhere you have intra-sprint cycles that must resolve by the end of the sprint. In my last post I wrote about manual quality assurance and the tension that causes in a Scrum setting. Another one of these intra-sprint cycles that most Scrums teams include in their process is code review.

You could, in a way, consider code review to be another form of "QA". In both cases we're initiating a cycle-within-a-sprint with a loop of approval, rejection, re-work, and acceptance.

In the sense that Scrum teams strive to get every sprint backlog item to "done" by the end of the sprint, back-and-forth cycles kill sprints. But they're also essential. Therein lies the tension.

Most Scrum teams conduct estimation sessions prior to starting a sprint. The estimates that a team assigns to product backlog items are typically based on a gut feel of how long it will take or how complicated it will be to "complete" a backlog item. In most teams I've worked with, that estimate is implied to mean how long it will take a software engineer on the team to submit a code review (via a pull request or something similar), i.e., when the engineer thinks their part is "done". Some teams also build into the estimate a duration or complexity estimate based on the work necessary by the QA people.

What estimates don't ever capture, in my experience, is how much effort/time/complexity is required to complete the code review or QA cycles on a backlog item. How can we know ahead of time how many pieces of feedback are going to be left on a pull request? How many of those will necessitate re-work on the item before the end of the sprint? How long will the re-work take? What if the re-work is still not up to snuff? Etc., etc., etc. Each code review starts a cycle of unknown duration.

And all the while everyone who cares about the completion of the sprint is tap-tap-tapping their fingers, nervously wondering if these intra-sprint cycles are going to complete on time.

Even though people mean well, what happens in practice is that thorough code review is tacitly disincentivized. Most product owners are not engineers, and although they understand intuitively that code review, like any form of quality assurance, is necessary, it also adds delay to the immediate "done-ness" of work.

Engineers also know that code reviews are important, but there is always pressure to get to them faster. When bugs crop up later, or the codebase slowly accumulates technical debt, no one in management is going to track down the specific pull request that introduced the problem, read the list of names in the approvers list and assign blame to those individuals. A swift click on the "Approve" button satisfies everyone in the moment.

What do we do about this tension? We can't have code review without introducing an unpredictable amount of delay to each sprint backlog item. You can never fully resolve this tension, in my opinion. But there are some ways to ease the tension.

Be Your Own Reviewer

I'm going to put this one on the engineers. And, no, I'm not suggesting that an engineer who submits a pull request should be clicking "Approve" on their own pull request. What I am suggesting is that before an engineer submits a pull request, it should only be after they have examined their own code to the degree that a reviewer would.

I can't tell you how many times over my career that I've reviewed a pull request where it was clear that the submitter had not even looked at what they were submitting for review. You're not ready to submit a pull request until you've looked line-by-line at the diff you're about to submit and pre-corrected every issue you can find. We're not talking about architectural judgment calls here, we're talking about misspelling method names, pushing blank files, including huge sections of commented-out code from various abandoned local experiments.

It should be rare that a reviewer needs to point out an obvious mistake in a review. I personally feel embarrassed when a reviewer points out something that I know I could have found myself. And I feel doubly embarrassed if it's a mistake they've pointed out multiple times.

Automate, Automate, Automate

Take advantage of every opportunity to automate the tasks associated with code review. We have linters, pre-commit hooks in version control systems, IDE integrations. There are ways to automate basically any tedious, repetitive aspects of code review. If reviewers are repeatedly finding the same kinds of mistakes, it's time to automate the checks for those mistakes, so submitters can't submit them in the first place.

Back-and-forth intra-sprint cycles are what kill sprints. Code review is a cycle, but what can we do to get down to one iteration per cycle as often as possible?

The tension between any kind of quality assurance (including code review) and sprint-based methodologies is impossible to erase, but we have techniques to make it less tense.

Scrum Tensions: Manual QA

After working for many years in the software industry, almost always using a methodology resembling Scrum, there are certain tensions that I've come to believe have no good solution--at least--I've never seen them solved elegantly.

One of those tensions comes from manual quality assurance. When sprint backlog items need to be manually tested by dedicated QA people, there is simply no clean way to do it in my experience.

Here are some of the issues that I've seen repeatedly:

• QA does not have time to test items where the development work is completed late in the sprint
• QA people are sitting idle for the first few days of the sprint with no completed development work to test yet
• If a bug is found, there is no time in the current sprint to fix it

Here are some attempted remedies that I've seen:

• Let's build some slack into the sprint for the developers so they can complete their development work with X days to spare at the end to allow time for QA
• Let's have the developers "work ahead" of the QA people, as in, the development work that we say is "done" within one sprint is actually tested in the next sprint

And here's where those remedies fall down:

• Inevitably, there is pressure to work more items into successive sprints, and development work starts creeping over the false "deadline"
• We end each sprint not knowing what is actually "done", and any bugs that are found have probably already been merged into whatever common branch the developers work from

QA people are in a really tough position as they are always sort of passively pressured to not "hold up" the sprint. They know that everyone wants to see a clean sprint where every item is signed off on by QA by the last day. And the double whammy is that they get heat when bugs are found in production.

This is one of those essential tensions in Scrum-based development that I've come to accept is not resolvable. I have never seen manual QA fit neatly into Scrum. 

As I see it, there are two options to break the tension:

• Do manual QA, and abandon sprint-based methodologies (try Kanban, for example)
• Use a sprint-based methodology, and avoid manual QA (some teams fully automate testing)

In my experience, organizations do neither of the above, and the tension continues.

Continuity of Leadership

There's a Kafkaesque situation that develops in companies that cannot retain senior employees. A team feels a sense of urgency about shipping software, but no one can tell them what to build.

Engineers are responsible for making the products that sustain the company, but there's no one around who can say what the products should do exactly.

It's like a movie where the protagonist wakes up every morning with no knowledge of what happened the day before, and tries to piece together his identity from artifacts scattered about.

Big organizational initiatives fail repeatedly because there's no one around from the beginning to the end of the initiative. People working on the initiative find that they can't even remember why the initiative was necessary. The leader who championed the initiative isn't around to take credit for its completion, so there's no one working on the initiative at any given time who cares about its completion.

Cross-team relationships barely exist because whoever is leading one team at the moment doesn't know the leaders of the other teams and has no history with them.

Leaders are continually "backfilling" for other leaders that left, being asked questions that they can't answer.

Morale stays at a low simmer. People show up to work each Monday morning knowing they won’t be productive. Another week where they won’t know if they did good work or not, because no one can define the goal.

An organization without continuity of leadership is an organization with amnesia.

Vision

Vision is so important in software development. Without the engineers understanding the overall vision, they can't resolve ambiguity in their daily work without consulting someone who holds the vision.

If the engineers don't understand why they're doing any of these things, then they can't fill in the gaps logically. They can't suggest improvements, improvise, or have confidence that they're moving the organization closer to the vision. Teammates talk past each other. One person has more of the vision than another, but doesn't know that. Misunderstandings are common. The track being laid from each end doesn't meet up in the middle.

Every little bit of vision transmission compounds in value. The decisions we make today form the foundation for work that comes later. A misunderstanding in vision today requires re-work tomorrow, a week from now, a month from now.

One of the things that can get left behind in the just-in-time fashion of Agile sprints is that the team can get lost in the weeds. We have to remember that we're building toward a significant milestone of some kind for the business, not just a random sequence of tasks.

It's difficult when backlog items are being entered by one person or a small group of people separate from the engineers and QAs who will be actually building and testing the stuff. They get queued up and drip-fed every two weeks to the broader team. But often there's no shared context transmitted to the whole team about what broad goal we're doing all these tasks for.

Sprint goals are tricky to set. But if a team can't ever seem to define a clear sprint goal, that's almost certainly a sign that the team is lacking a vision.

People like to make fun of heavyweight methodologies like SAFe, but doing a Program Increment Planning event--although tedious--sure does get everyone on the same page with a shared vision for the next few months of work.

Most of us are not working on the Manhattan Project--the end goal should not be obscured from the individuals making the parts. In fact, the end goal should be so clear to everyone that any person on the team could describe it clearly in their own words.

A Standup Free of Should, Probably, and Hopefully

It's interesting to listen to the word choices that people use in the daily standup.

"I should be done with that today."

"I'll probably be done with that today."

"Hopefully I'll be done today."

"I'll try to wrap that up today."

I like to take a mental note of the "shoulds", "probablies", and "hopefullies", and see if the next day that work was truly finished.

There are serial offenders on every team. If a should one morning comes back with another should the next morning, you're really worried.

Why do people feel the need to give these hopeful yet indefinite pseudo-pronouncements about their progress? Is it natural optimism, a people-pleaser temperament, willful deceit?

More importantly, what is the temperature in the room where people feel more inclined to give optimistic projections over more realistic ones? Hopeful wishes over definitive statements?

It could be...

• The person does not have a good understanding of the goal of their assigned work, so they don't have a good idea of what it will look like to be "done" with it.
• They're operating within an environment where people routinely make weak promises and exaggerations of progress, so that seems like a normal thing to do.
• They're operating in a chaotic environment, where it's hard to predict how much focused time they'll get on any given task on a given day.
• They know they're not being given enough time or resources to complete work in a politically acceptable timeframe, but it's also not politically acceptable to just say that, so their best option is a hopeful statement about the timeframe in which they intend to deliver it.

But, hey, sometimes people are just inexperienced in the kind of task they've been assigned, and so they can't see the road ahead of them and the milestones along the way. That will absolutely make it hard for them to predict when they'll reach the finish line. Fair enough!

But...I think the "should", "probably", and "hopefully" indicate something else than inexperience. They indicate a foresight about the task ahead and a suspicion that they don't feel comfortable stating.

What is making it hard for people to tell the truth in this environment? We need optimists in software, but too many unchallenged "shoulds", "probablies", and "hopefullies" in the room is a sign of trouble.

How Many Spikes Is Too Many?

Spike is fun to say. Spike! For the unfamiliar, the spike is a concept from Agile methodologies that means a time-boxed backlog item where the end result is learning, rather than delivered software.

Mike Cohn from Mountain Goat Software offers this example:

As an example of a spike, suppose a team is trying to decide between competing design approaches. The product owner may decide to use a spike to invest another 40 (or 4 or 400) hours into the investigation. Or the development team may be making a build vs. buy decision involving a new component. Their Scrum Master might suggest that a good first step toward making that decision would be a spike into the different options available for purchase, their features, and their costs.

Because spikes are time-boxed, the investment is fixed. After the predetermined number of hours, a decision is made. But that decision may be to invest more hours in gaining more knowledge.

I've worked on teams where the process was spike-heavy. We'd commonly have backlog items within most sprints that were dedicated to learning about a topic that we knew would be important for future work. We had features we wanted to get into the software, but we didn't have a good idea of how we were going to accomplish that work on a technical level. For teams that put a big emphasis on accurate estimation and minimal to no carry-over of items at the end of sprints, they want to know that a technical foundation for work is understood before its implementation is "promised" within a particular sprint.

I've also worked on a team where spikes were basically not part of the process at all. Backlog items were oriented around features the product owner wanted in the software, but they wouldn't allot an item into a sprint without a "technical approach" filled out on the item first. The "technical approaches" were usually written ahead of time by team leads or architects that did not have "on the board" responsibilities within sprints and would work on these things ahead of the rest of the team, as time allowed. Sometimes senior engineers would also work on technical approaches for future sprints if they finished their assigned items for a sprint with time to spare.

One of the downsides of a spike-heavy process is that you're reducing the amount of "business value" delivered at the end of a sprint. If your entire sprint was consumed by spikes, I don't think the business would be very happy. On the flipside, learning has to happen somewhere. Whether you call it a spike, refinement, technical approaches, or anything else, the learning must happen.

I wouldn't necessarily say that the spike is an anti-pattern, but if it feels they're being leaned on too heavily, it might be time to stop and ask why they're necessary. Is it because we're shifting decision making to the engineers about requirements that a business analyst or product owner should be making? Are we not dedicating enough time to refinement? Is the product owner spread too thin? Is the development team stacked with junior engineers or lacking in engineers that are experienced with the technology at hand?

Learning has to happen somewhere—that's the nature of software engineering. But an over-reliance on spikes for decision making can indicate deeper organizational issues.

Escaping the Bikeshed

I wrote in 2017 a post called Don't Trap Your Clients in the Bikeshed. That post was about avoiding the trap of seeking feedback on trivial decisions from clients before they're necessary.

Sometimes in software development a group of stakeholders will become very suddenly interested in a particular aspect of the software, and intense debate will ensue with a flurry of changes discussed in an area that everyone has an opinion about. In these occasions, as an engineer, your clients have put you in the bikeshed.

What do you do as an engineer when you find yourself in the bikeshed?

Slow Down

The pressure to immediately address every opinion can be daunting. Try not to get caught up in the whirlwind. Let the group debate while you maintain a healthy distance.

Pause for Documentation

Gently remind people that in order to get features into software, they need to write down clearly what they want. They have a responsibility to be clear about what they're asking for. That's the bargain.

Invoke the Process

QAs need to know what to test. A larger audience needs to know what changes are making it into the software. Customers need to know what's on the horizon. Releases need to be organized. Changes in one part of the codebase impact other areas.

Your engineering team has an established process for making and releasing your software. In the fervor to change the bikeshed, interested parties get excited to see their opinions realized. How long does it take to paint the thing blue? It's clearly not blue right now, and I want it to be blue.

Remind people why the process exists, and the drawbacks of making changes to a software system that don't follow the same process as other changes.

Expose the Bones

Sometimes it's really an issue of transparency. Non-engineers get frustrated that they don't understand why a feature is behaving the way it is. It can help to "expose the bones" as it were. Make a report that anyone can see that shows key metrics (how many requests, how long is it taking, who's using it, etc.). It can help to surface the internals of a feature. Make a page that shows diagnostic information about the input to a feature, how the feature calculated a result, and what the "raw" result is.

Remove Engineering From the Loop

Ultimately the goal is to remove engineering from the debate. If there's an intense debate about the color of the bikeshed, engineering can build a configuration option into the software where non-engineers can change the color at will. Try to distill the points of contention down to self-service features that don't require engineers to make code changes to the system.

Tell Us Why We’re Doing This

I’m constantly shocked that business people don’t make much of an effort to communicate to engineers the impact of their work. It’s par for the course that the engineers don’t know how many users their product has, how many clients they have, how much revenue the product makes, etc. It’s so common in fact that I wonder sometimes if it’s intentional.

There is an Agile concept known as the information radiator. Some companies will put dedicated big screen TVs throughout their offices that show key metrics at a glance. If you have a remote-first culture, a television isn't going to do much good, but a web-based dashboard placed somewhere the whole team goes every day (like your issue tracker) is a great substitute. 

These ideas are more about passive awareness, but I think the next step up is active discussion. If you're doing some kind of regular all-hands meetings, like a retrospective, that's a perfect time to pull up that dashboard as a team and discuss it together.

How do we know we did a good job over the last sprint? Yeah, we marked all of our tickets as "Done", but so what? How do we know that our customers are happy with our work? Company leaders always want to know how to get their employees more "engaged" in their work. Show your team how their day-to-day tasks impact real end users. Working a backlog of items sprint after sprint is so far removed from the impact of the work. Why are we doing these things?

How many users did we add over the last two weeks? How many new customers came on board? How many usages of [NEW FEATURE X] did we have? 

I feel like I'll be banging this drum for the rest of my career: Engineers are not robots. We want to know the high level goals of our work and how our work impacts real people.

Please, tell us why we're doing this.

The Experience to Say "I Don't Know"

One of the difficulties with managing a software project is simply getting people to be honest about the progress they're making and the difficulties they're facing.

I think one of the signs of experience in engineers is that they're not afraid to admit when they don't know something or are having a hard time with a bit of work. I know that when I was a junior engineer it was hard for me to feel okay about not doing something perfectly, and I didn't want anyone to see me struggle with anything.

There's a sense of security that comes with experience. You know enough to know that you're never going to know everything. Just surviving in the industry as a working software engineer for several years, experiencing the inevitable ups and downs, and then simply continuing on. You realize that feeling out of your depth is a common feeling in a line of work that is constantly changing.

So I think it's important to encourage engineers and other technical folks, especially more junior folks, to raise their hand high early when they're bogging down. For most teams that are using an Agile methodology, any kind of daily stand-up is a great time to check in on progress. But of course, we can't help each other out if we aren't honest with each other about our progress.

Say these things as early as possible:

• I have too much work assigned to me this sprint, and I don't feel confident I can get it all done on time.
• I know less about this area of the project than Person X, and I think it will take me longer to do it than they would take.
• I've run into some unforeseen difficulty with Feature X, so I think it will be necessary to wrap up the part now that I understand, and tackle X again in a future sprint.

It's okay to say, "I don't know." It's okay to say, "I'm struggling." We should all say them more often! In fact, the health of a software project depends on it.

Contextualizing the 1-on-1

It's pretty common for engineering managers to have regularly scheduled 1-on-1 meetings with their engineers. Topics often include things like goal setting, problems the engineer needs help with, or a chance to air grievances that the engineer doesn't feel comfortable bringing up at other times.

Personally, I don't find myself coming to 1-on-1s with topics that I was holding onto specifically for that meeting. If I have an issue that needs attention, I'm not going to wait for a monthly meeting to bring it up.

My favorite topic for a 1-on-1 is everything that is happening at the scope that is beyond our day-to-day interactions. How is hiring going? So-and-so left 3 months ago--are we hiring a replacement?

What did you think about that announcement from the town hall? How is the work of our team being perceived by the broader organization? How is customer X liking feature Y that we launched recently?

How do you feel the team is executing toward our strategic goals for this quarter?

I want to "take the temperature" on a thermometer that I don't get to see often--get the context that I'm not getting on a day-to-day basis.

1-on-1s can be anything. They don't have to be a one-way venting session. They can just as easily go entirely the opposite way where we talk mainly about higher level things.

Let Unsustainable Things Fail

I read a post by Max Countryman recently titled Let It Fail (Hacker News discussion), where he talks about the importance of failure in software engineering organizations. He recounts an experience he had where his boss gave him surprising advice to let a legacy system that was not getting enough attention fail visibly rather than paper over the cracks.

I've written before on this blog about how heroics cover up systemic issues. Organizations become dependent on heroics. Heroics beget more heroics. Heroics are normalized. Eventually the heroes burn out.

That one long-serving, loyal employee who had become a silo of critical knowledge up and quit. That manual, error-prone process we had been using to deploy our software for way too long finally caused an outage. We've been rushing code changes right before big releases, and one finally bit us hard.

It's important to talk openly about unsustainable engineering practices and the failure they portend. But issues of technical debt are hard to understand when placed alongside new feature development and other business priorities. The value of under-the-covers work is hard to understand. Sometimes you need to pull the covers away.