Tech's COVID Hangover

So many layoffs! I can't open Hacker News or LinkedIn without seeing constant stories about the next tech company to lay off hundreds or thousands of employees. For a couple years there, the tech industry was hotter than I had ever experienced in my career.

It never quite made sense to me, but I was riding the wave. Well, that wave has now crashed, and I'm left wondering what happened.

Ben Thompson of the Stratechery blog, a frequent guest on the front page of Hacker News, wrote recently about The Four Horsemen of the Tech Recession, and I found the analysis pretty interesting.

In particular, his discussion of the "COVID hangover" caught my attention. Essentially, demand for digital transformation in companies around the world suddenly sped up when the pandemic hit. Planned investments to modernize were all made at once, rather than the slow rollout that they envisioned before the world suddenly changed. Growth in the tech sector is now slowing as the demand was "pulled forward." Thompson explains…

When corporations the world over were forced literally overnight to transition to an entirely new way of working they needed to scale their server capabilities immediately: that was only realistically possible using cloud computing. This in turn likely accelerated investments that companies were planning on making in cloud computing at some point in the future. Now, some aspect of this investment was certainly inefficient, which aligns with both Amazon and Microsoft attributing their cloud slowdowns to companies optimizing their spend; it’s fair to wonder, though, how much of the slowdown in growth is a function of pulling forward demand.

…the COVID pull-forward was massive, but underneath the inevitable hangover there was a meaningful long-term shift to digital broadly.

The layoffs and slowdown in hiring we're seeing now after the massive boom in hiring during the pandemic is…

precisely because these [only-possible-on-the-Internet] jobs — and similarly, many of the COVID-specific workloads like work-from-home and e-commerce — were digital that it is tech that is in a mini-recession even as the so-called “real” economy is doing better than ever.

The level of growth in the tech industry that kicked off during the pandemic was not sustainable, but if it represented a pulling forward of pent-up demand, then it would make sense that there would be a slowdown in spending to follow. Hopefully after the hangover we get back to a healthy level of demand, that while not at the highest of highs, will be sustainable.

The AI Can't Destroy Anything Worth Preserving

The front page of Hacker News has been flooded lately with articles about ChatGPT, a seemingly magical AI tool that can generate realistic paragraphs of text that can seem indistinguishable from human written text.

My favorite article on the topic is ChatGPT Can't Kill Anything Worth Preserving by John Warner, a former English teacher. In that article, Warner is talking specifically about the prospect of students using AI to cheat on writing assignments, as it seems that ChatGPT can often fool the people grading them. He points out that if writing assignments are so tedious, boring, and formulaic that students aren't interested in doing the exercise themselves, and if an AI can write a passible solution, then we have to conclude that the exercise itself is absurd and not worth asking students to do. The AI is just a more convenient and efficient form of cheating than has been available to bored students for years.

Before ChatGPT was the hot topic on Hacker News, there were daily articles about GitHub Copilot and DALL-E 2. 

I've written before on this blog about the disturbing effect of automation in areas of work that one holds dear. In my post Be the Automator, I described the experience of discovering as a junior software engineer that there were tools called "code generators" that could automatically write code for you. I felt an existential threat at the time, but learned over the following years that not having to write tedious and formulaic code myself is a blessing rather than a curse.

My general feeling for many years has been that, fundamentally, the task of automation is about human liberation. It's demeaning to make a human being do something that they don't want to do if a machine can do the job just as well (or better). If the code I'm writing is so trivial that an AI assistant like Copilot can write it for me, then by all means let the machine do it! If English students are writing prose so trivial that ChatGPT can do it for them, then let the machine do it, and give them something better to do.

To paraphrase the title of Warner's article, the AI can't destroy anything worth preserving. If a machine can do it, a machine should do it. We might learn some hard lessons along the way that we weren't as creative as we thought.

Prototypes Live Forever

Code always lives longer than you want it to.

So often as developers, we're writing code under the gun. Just get it working. We have a deadline. We'll come back and fix this later.

Something I've learned through experience is that prototypes go into production. No one wants them to, but they do, 99% of the time.

That thing that you wrote that you rushed, that you're embarrassed to put out into the world—that thing will be in production, probably for a long time. Future developers, maybe long after you've moved on, will be reading it over and over, wanting to change it but being too afraid because it works, and it's been like that for a long time.

Code ossifies, the temporary becomes permanent. That next sprint where you're going to come back and clean up that code never comes. 

What does that mean for the daily practices of the software engineer? Do the little things now that can make your code better. Before the end of the sprint, before you've marked the ticket as "Ready for QA", before you've told anyone it's "done", before the pull request is merged. Give that variable a better name. Write one more (or just one) unit test that covers the basics. Split this class into two. Write a comment. Leave one more bit of feedback on that pull request. 

There are generations of developers that are counting on you. They'll be living with the shortcuts of today for years to come. I promise.

RIP Fred Brooks

Fred Brooks passed away this month. I first wrote about him on this blog in April 2007, right after I had finished reading his seminal book on software engineering, The Mythical Man-Month. I was about 2 years into my career as a software engineer at that point. How time flies!

I thought it might be interesting to pull out my dog-eared copy of the book and reflect on a few of my favorite concepts and how they've borne out in my career 15 years on.

The Mythical Man-Month

First off is the titular concept from the book, in which Brooks's Law is coined.

Adding manpower to a late software project makes it later.

I will say that in my career, I've only seen one project where bodies were hurled at a late project to hasten its completion. Interestingly, that was also the latest project I've ever seen in my career. 

Thankfully, my experience has been that Brooks's Law has permeated amongst the managers I've worked with—if not in name, they at least intuitively understood that they couldn't just add more people to a project to get it done faster.

The Second-System Effect

I've seen this play out several times in my career. The idea that we'll replace the hacky system we developed before we knew better with one where we take our time and "get it right" this time.

As he designs the first work, frill after frill and embellishment after embellishment occur to him. These get stored away to be used "next time." Sooner or later the first system is finished, and the architect, with firm confidence and a demonstrated mastery of that class of systems, is ready to build a second system.

This second is the most dangerous system a man ever designs.

The general tendency is to over-design the second system, using all the ideas and frills that were cautiously sidetracked on the first one.

It's easy to forget that just because we have a lot of ideas for improvements that not all of them are worth doing.

No Silver Bullet

I would dare say that most software engineers who have heard of Fred Brooks, found their way to him via his essay No Silver Bullet.

There is no single development, in either technology or management technique, which by itself promises even one order of magnitude improvement in productivity, in reliability, in simplicity.

Not only are there no silver bullets now in view, the very nature of software makes it unlikely that there will be any.

Personally, the message of No Silver Bullet is so ingrained in me as a software engineer, that I don't consciously think about where it came from very often, although it colors my world view constantly.

Every time I see a headline on Hacker News about Framework X vs. Framework Y, or Language A vs. Language B, I understand intuitively how little they can possibly impact the success of a software project.

When I wrote my post Stack Chasers vs. Evergreens in 2018, I don't think I was even aware that I was channeling No Silver Bullet that I had read over a decade earlier. It was in my bones at this point that technological solutions for improving the art of software engineering are inherently, severely limited.

Brooks's treatment of essential complexity vs. accidental complexity had such an impact on me, that I reference it when approaching challenges in my own life that have nothing to do with software engineering or my career at all.

The Mythical Man-Month is holding up amazingly well in 2022. I can't think of any other nearly 50-year-old book written about computer software that is still so relevant and insightful.

Thank you, Fred! R.I.P. And get a copy of The Mythical Man-Month if you haven't already read it. 

What Is a Staff Engineer?

What's next for someone who has been at the Senior Engineer level for several years or more? It used to be that you had to go into people management to level up. But nowadays, more and more companies are offering a level above Senior called Staff, that is not inherently a people management role. So what is Staff? 

There's an interesting post over on the GitLab blog where the author talked to several Staff Engineers in the company and identified the common threads in their roles:

• Proactive problem identification
• Cultural stewardship
• Knowledge sharing
• Technical empowerment
• Broader impact beyond the team

I would summarize this list as doing technical work that crosses multiple team boundaries. A Staff Engineer would be doing whatever it takes to make cross-team initiatives that need an expert software engineer successful.

It's hard not to come across Will Larson's work when searching for definitions of Staff Engineer. He literally wrote a book called Staff Engineer that is all about this very topic. Here is Will's answer to the question, "What do Staff Engineers actually do?"…

The most straightforward answer is that Staff engineers keep doing much of what made them successful as Senior engineers: building relationships, writing software, coordinating projects. However, that's a misleading answer. Staff engineers do those same tasks, but whereas previously they were the core of their work, now they're auxiliary tasks. Their daily schedule varies a bit by archetype, but there's a shared foundation across all archetypes: setting and editing technical direction, providing sponsorship and mentorship, injecting engineering context into organizational decisions, exploration, and what Tanya Reilly calls being glue.

Will talks more about archetypes of the Staff Engineer in another article which is what introduced me to his work (Hacker News discussion here). He's identified four different kinds of Staff Engineers that are commonly found in the wild:

• The Tech Lead guides the approach and execution of a particular team. They partner closely with a single manager, but sometimes they partner with two or three managers within a focused area. Some companies also have a Tech Lead Manager role, which is similar to the Tech Lead archetype but exists on the engineering manager ladder and includes people management responsibilities.
• The Architect is responsible for the direction, quality, and approach within a critical area. They combine in-depth knowledge of technical constraints, user needs, and organization level leadership.
• The Solver digs deep into arbitrarily complex problems and finds an appropriate path forward. Some focus on a given area for long periods. Others bounce from hotspot to hotspot as guided by organizational leadership.
• The Right Hand extends an executive's attention, borrowing their scope and authority to operate particularly complex organizations. They provide additional leadership bandwidth to leaders of large-scale organizations.

Read Will's article for deep dives into all the archetypes, complete with examples of what their schedules are like and interviews with actual Staff Engineers who he sees as fitting these different archetypes. It's very enlightening, and I can't really imagine a better way of getting insight into the real-world nuts and bolts of that title.

My takeaway is that a Senior Engineer sort of maxes out as an expert software engineer. The Staff Engineer is about taking engineering expertise and expanding the scope of one's influence, which could mean overseeing a group of people, facilitating communication between groups of people, or solving problems involving multiple codebases. And maybe this goes without saying, but you're not likely to see the Staff Engineer title existing outside of large technical organizations, where the scope of engineering activity requires this sort of role.

Remote Hiring and the Paradox of Choice

Oh, how I love remote work. And oh, boy, I've written a lot about the topic on this blog. We all know the reasons why remote work is amazing; there's no need to rehash at this point.

But I haven't written much about the flipside, particularly the process of being hired for a remote job. If you consider how much more awesome working from your home is compared to a typical open plan nightmare, then a similar multiplier exists in the other direction for how much more frustrating a job search is for a remote job compared to a local in-person job.

Just as the abundance of remote jobs is a dream from a job seeker's perspective, the abundance of the talent pool from a hiring company's perspective leads to some annoying behavior.

There are clear comparisons to be made to the shift to online dating from "traditional" dating. As the pool of eligible participants increases, the value of each individual participant drops. And just as the modern phenomenon of "ghosting" is common in the world of online dating, it's also common in the world of remote job hunting.

It's way more common in a remote job search to go through multiple rounds of interviews with a company and then never hear from them again, for example. Or for recruiters to hound you on LinkedIn until you agree to a phone call, gush about the perfect fit for a role, and then never contact you again about that next meeting with the hiring manager. Follow-up emails go into a black hole. If you get feedback at all, you find out you were rejected for some seemingly minor and arbitrary shortcoming. That's just how it is when the potential talent pool for a job opening expands across a country or across the globe.

Just as I mentioned earlier a multiplier for how much better a remote job is to a traditional job, you will experience general flakiness from hiring companies at a similar multiplier. There's just something about the abstraction and lack of physical proximity between the participants that amplifies this behavior.

And believe me, it's not just employers who are making this difficult, there's no end to the shenanigans that occur on the candidate's side. Both sides are enjoying the benefits while cursing the drawbacks.

So let's file this post under "Advice for a Younger Me." I write this post in the same spirit as posts I wrote years ago like The Many-Worlds Interpretation of Developer Interviews and People Hire Clones of Themselves. The hiring process for software engineers seems so bizarre and arbitrary a lot of the time that it helps to remember that there are reasons for the weirdness that have nothing to do with you, and it's like this for everybody.

It's a brave new world out there. Long live remote work. :)

Swamped to Sustainable

I've written a lot about the concept of sustainability on this blog, and it's something I'm constantly thinking about with regards to software development.

That's why I enjoyed this recent post by Greg Kogan about being "swamped" all the time:

I used to think being swamped was a good sign. I’m doing stuff! I’m making progress! I’m important! I have an excuse to make others wait! …

Now, I’m impressed by people who are not swamped. They prioritize ruthlessly to separate what’s most important from everything else, think deeply about those most-important things, execute them well to make a big impact, do that consistently, and get others around them to do the same. Damn, that’s impressive!

Being swamped isn’t a badge of honor, it’s something to work on.

There were good comments in the discussion of the post on Hacker News, including a link to a manifesto of sorts I wasn't previously familiar with called Sustainable Development.

Sustainable Development is a set of principles for software teams to use to operate in healthy and productive ways, and for the long term.

Software teams practicing Sustainable Development follow guidelines that benefit them in three areas:

Physical: They work in ways that allows them to maintain good physical health.

Emotional: They work in an environment that supports their emotional health.

Cognitive: They work in ways that encourage creativity and support the intellectual nature of software engineering.

I could see this being used as a framing device for discussions in a sprint retrospective.

Development teams wanting to adopt Sustainable Development simply write down the practices they wish to embrace, and then commit to following them. Each practice should benefit the team in at least one of the three areas of sustainability (physical, emotional, or cognitive).

It's often hard getting people to speak productively in retros, and I feel like a good place to start is asking the question to the group: What did we do in this last sprint that doesn't seem sustainable? What can we do to increase the sustainability of our sprints? The suggested sample practices could give the group some ideas on action items to work toward greater sustainability.

I hope every team can agree that swamped is not sustainable.