Tag Archives: software engineering

Some of the most high performance and potential people I've ever worked with as part of the PayPal TLP Program

5 Ways to Avoid Getting Laid Off: Reverse Engineer the McKinsey 9 box Model

A quick guide to layoff calculus

First, not many people are familiar with how layoffs are performed – now I don’t have all of the answers for every company here, but generally speaking – here’s what happens in the larger companies that have a dedicated HR team…

  1. Managers have “a list” that is essentially stacked ranked
  2. They give that list to their manager
  3. That manager adjust the lists and combines it with the lists from other managers
  4. This process continues as the lists are merged and sent up the chain of command

HR or “People teams”, as a non-biased third party comes in with their formula that leverages some data they have from finance. HR uses a formula, because that removes bias from the layoff equation (as long as the formula is unbiased, but that’s a topic for another day). A layoff formula might have these variables and weights:

VariableWeight
Location

Is the person colocated with their team? Some companies believe this is important for performance and productivity.

4 = Remote, 3 timezones or more away
3 = Remote, 2 timezones or more away
2 = Sometimes visits office
1 = In office 3+ times a week		10%
Performance

What have been the last two performance ratings for this employee? Companies have a bias for performance (even if performance reviews are biased and inaccurate). This is part of the McKinsey 9-box model since the company uses this to judge/rank performance.

4 = Last two ratings have not met expectations
3 = One of last two ratings have not met expectations
2 = Both of the last ratings have met expectations
1 = One of the last ratings have exceeded expectations
20%
Productivity

Some companies track overall productivity for employees either by tracking statistics on their work on “spyware” on their computer.

4 = Employee is not productive during the workday
3 = Employee is productive 50% of the workday
2 = Employee is productive 75% of the workday
1 = Employee is productive 90%+ of the workday		20%
Salary

If the company is looking to cut budget, you can expect this to be part of the calculus. This is where senior people who have been with the company will be analyzed. This is also where employees based in higher cost areas might be targeted more than others.

4 = High Salary (over $350,000 total compensation)
3 = Medium Salary (over $250,000)
2 = Average Salary (over $150,000)
1 = Below Average (under $150,000) 		20%
Progression

Has the employee been stagnate in their level for a long time? If someone is at the same level for 5 years, this could indicate a productivity problem and a lack of performance and potential (from the McKinsey 9-box).

4 = No promotion in 10+ years
3 = No promo in 8 years
2 = No promo in 4 years
1 = No promo in < 4 years
10%
Cost Center or Profit Center

Is your team making the company money (Profit Center) or part of Operational Cost (Cost Center). Companies looking to save money will likely cut more teams that are a Cost Center instead of a Profit Center.

4 = Cost center
1 = Profit center		10%
Team/Product/Org Investment

Does the company plan to invest in this team or area in the next year? If not, it’s likely they will look to make cuts here.

4 = Product/Org/Team needs to shrink
3 = Some investment likely, need to keep product running
2 = Investment needed
1 = Must invest		10%
How a company might calculate layoff targets – note that this is only an example

⭐️ Now is a good time to read this other article on the the failure of using a 9-box for software engineers since we’ll be using the table, and the bias that it creates, for the next section. ⭐️

The 9-box plays into career progression and performance assessments

The McKinsey 9-box model (taken from RapidBI)
The McKinsey 9-Box Model

From what I’ve seen, most companies will use a 3 point scale on each axes for Potential and Performance where Low is 1 and High is 3. To avoid any ties and to clearly articulate value, you might see a prioritized 9-box like this:

7 Enigma8 Growth Employee9 Future Leader
4 Dilemma5 Core Employee6 High Impact Performer
1 Under Performer2 Effective3 Trusted Professional
A prioritized 9-box where 9 is the most prized/rewarded employee (think: raises, bonus, RSUs, etc.) with high performance and potential

This might play a factor as part of stack ranking, bonus assessments, and overall performance management so it’s important for you to understand how leadership will use this mental model.

Example 9-box Scenarios

So now that we have the weighting and the 9-box, let’s try some examples using IDEs as our employee names.

employeeformulascorepossible story
Eric Clipse4 Location: Lives 3 timezones away from core team or manager
4 Performance: Last two reviews have not met expectations
2 Productivity: 75% productive – clickity-clacking
4 Salary: High salary over $350,000
2 Progression: No promo in 2 years
4 Cost/Profit Center: In a cost center
2 Org Future Investment: Some investment needed in the future		 3.2Story 1: Senior engineer that has been around for 10 years, and relocated during COVID, since being promoted to very senior level, they have struggled to be productive (see Peter Principle) because they may have been promoted without appropriate considerations or their managers have been biased in previous years to rapidly promote.

Story 2: Even though Eric went remote, there was a re-org and now the team is 3 timezones away.
Xavier CodeEverything as above, but change from 4 to 2 salary

2 Salary: Average salary over $150,000
 2.8Story 1: Average software engineer position as they came over from another company. Could have been a bad hire through the interview process since they have had 2 reviews that have not met expectations.

Story 2: Joined from another company and thought this company and team was great, but the manager and work is not what they thought it would be so they have underperformed.
Ned BeansEverything as Eric Clipse, but change:
Location 4->2 since they sometimes visit the office.

Performance has met expectations the last 2 reviews, so 4->2 when compared to Eric.

Also similar salary to Xavier.


2 Location
2 Performance
2 Salary: Average salary over $150,000		 2.2Came from another company with some experience, average performance and salary.
Vince S. Code2 Location
2 Performance
4 Salary: High salary over $350,000
4 Progression: No promo in 10 years
 2.8 (Same as Xavier)Story 1: Has been with this company for a very long time, has been promoted, pretty average reviews, goes into the office sometimes, and has a high salary due to living in a high cost location.

Story 2: At one of the most senior levels, they are not interested in being promoted past this role, they love their job and have a good work/life balance.
Example scenarios with fun names

To recap, we have the following list where the higher the score, the most likely to be laid off:

  1. Eric: 3.2
  2. Xavier: 2.8
  3. Vince: 2.8
  4. Ned: 2.2

For the sake of simplicity, let’s say this team is 10 people, and we’ll duplicate these four people assuming that there are similar cases.

  1. Eric: 3.2
  2. Xavier 1: 2.8
  3. Xavier 2: 2.8
  4. Vince 1: 2.8
  5. Vince 2: 2.8
  6. Ned: 2.2
  7. Ned: 2.2
  8. Ned: 2.2
  9. Ned: 2.2
  10. Ned: 2.2

Now, let’s say a layoff comes – typically this occurs behind closed doors away from front-line managers and their teams to limit knowledge and the amount of damage any employee could do knowing they were going to be terminated – it covers the company from any potential issues, but now leaves front-line managers helpless. HR comes up with a target, 10% – so in our case, Eric having a score of 3.2 would be the one person to leave this team, but let’s add another team to this mix:

team 1team 2
1. Eric: 3.2
2. Xavier 1: 2.8
3. Xavier 2: 2.8
4. Vince 1: 2.8
5. Vince 2: 2.8
6. Ned 1: 2.2
7. Ned 2: 2.2
8. Ned 3: 2.2
9. Ned 4: 2.2
10. Ned 5: 2.2		1. Xavier 1: 2.8
2. Xavier 2: 2.8
3. Vince 1: 2.8
4. Vince 2: 2.8
5. Vince 3: 2.8
6. Ned 1: 2.2
7. Ned 2: 2.2
8. Ned 3: 2.2
9. Ned 4: 2.2
10. Ned 5: 2.2
Another Vince-like person added here – maybe for one of these reasons:

1. Manager is less critical during performance reviews, so this team has fewer people not meeting expectations.

2. Similar to (1): Manager has been managing this team for awhile and is now friends with everyone so performance reviews are less critical.

3. Manager is unaware of what the team is doing so doesn’t provide critical assessment of team and thinks everything is awesome.

4. Team is more colocated than Team 1
Example scenario

Now the same layoff hits both teams with a 10% target, with 20 total people, HR would have to select 2 people to be impacted, from before we know Eric would be selected, but now we have 9 people with the same 2.8 score:

  • Xavier-like people may not be performing, but are paid average: 2.8 score.
  • Vince-like people have been performing at expectations, but are more senior and have been with the company for a long time: 2.8 score.

When faced with a tie, likely one of two things happen:

  1. HR uses a stacked rank to determine the next person to be impacted
  2. Since the scores are the same, one person is selected at random

Both of these sound pretty terrible – a stack ranked list of people comes with a ton of bias since this is usually done by managers, then merged, then ranked, then merged as it goes up the chain.

Random selection isn’t any better either – no one wants their career to be determined by Math.random() when their performance, passion, and drive hasn’t been random. You can see that in either approach, a faceless algorithm will likely decide fate.

So what do you do? Let’s dive in.

1. Hack the 9-box

You want to be in the right && top of the 9-box

7 Enigma8 Growth Employee9 Future Leader
4 Dilemma5 Core Employee6 High Impact Performer
1 Under Performer2 Effective3 Trusted Professional
You want to be 3,6,8,9

There are too many variables at play here. In the past I’ve seen:

  • Someone who was happy in their role and not interested in being promoted perceived as having less potential
  • A team who was doing more mundane work perceived as having higher performance
  • Teams who create problems for them to continue to solve appear to be heroes and have higher performance

How can you avoid most of this? Keep your head down and focus on delivering results and value. This will put you at either 3 Trusted Professional or 6 High Impact Performer. Although it might be appealing to target that middle (5) box, if you are consistently seen as someone who has both average performance and average potential, you won’t be rewarded as much and will be average in a stacked ranked list (which we’ve learned above: middle or average 2.8 for Vince and Xavier could create a situation where you’re somewhere in a layoff list).

Here are some other options:

Boost your performance

  • Find low hanging fruit that’s being ignored and fix it.
  • Look at how your team talks about your victories – are they aligned with business priorities?
  • Is the work you’re doing attributed to those victories?
  • How can you deliver results above expectations? Can you do something faster? Deliver earlier? Reduce the complexity?

Be consistent with performance. Someone who is productive only when asked, or has sine-wave ebs of productivity is not going to be doing themselves any favors when compared to consistent performers.

Boost your potential

  • Come up with interesting and innovative ideas that make sense, are low return on investment and drive the company’s bottom line, even if you don’t think your team can work on it right now. Write it down and publicize it.
  • Educate colleagues on what you know, about the company’s products or infrastructure/hardware that it’s built on.
  • Educate yourself – take classes, read books about new topics, stay informed of new technologies. I love Tech Radar.
  • Find gaps in the work that others do or enhance existing ideas.
  • Sign up to do things that no one else wants to do.
  • Be a giant supporter of the company, post a lot of positive things on social media, recruit people to join you, give external talks.

Since the 9-box is used during performance management (for companies that have been McKinsey’d), where there’s likely a bell-curve forced ranking applied, you want to understand where the people on your team might fall and ensure you’re doing something to get you uniquely into one of these boxes.

7 Enigma8 Growth Employee
1 person max		9 Future Leader
1 person max
4 Dilemma
5 Core Employee
3 people max		6 High Impact Performer
1 person max
1 Under Performer
1 person min		2 Effective
3 Trusted Professional
An example forced-rank curve applied to the 9-box – this is for a sample 10-person team where 7 are shown above and 3 more (not shown) need to fall in other positions [1,2,3,4,7]

2. Be closer to your team

It sounds weird, but if HR is using location as one of the weights (likely because your company has some sort of return to office policy), you should try to be closer to your team. The farther away you are – the more you will be scrutinized and the more you’ll need to focus on being present and performant.

If you’re a remote employee, you should make sure you over-emphasize:

  1. Constant communication – use chat and documents constantly. Make it a point to communicate something valuable every day to be visible.
  2. Purposeful meeting time – setup time with people, you won’t run into them in the halls of the office, so you need to think more about who you need to spend synchronous time with.
  3. Use async communication to your advantage – record demo videos, videos that ask questions and gather input, educational content. You don’t need a meeting to contribute value. Be concise in your messages – don’t create noise and don’t create work for others.
  4. Volunteer for things – if your commute is from your bedroom to your office, you are saving hours of time that your other colleagues are spending driving into the office. Use that extra time you have by effectively by helping the team.
  5. Be a meeting superstar – if you’re remote and others are in the office, you have an advantage of taking notes and paying attention to meetings – you have no in-person distractions. Take meeting notes, summarize key points, speak up during meetings to assert your opinion or ask questions. Don’t let in-person people not hear you – speak up, keep speaking up if you aren’t heard.

3. Make sure your product/org has a future

Be selective for the team you join. If your product has a known end-of-life or is consistently losing revenue or costing the company more money than it’s worth – and you might be a “4” in other areas of the layoff calculations – try to find something else. Yes, you may not be as stressed on this team because you’re just keeping the lights on, but that runway is limited (you could also join this team and come up with an amazing idea that turns the ship around).

HR might look at positions that can be eliminated to save cost. If you do jump into a team with a limited future, make sure you’re also reading the room to know when to jump to another team. If your company is publicly traded and the stock continues to decline, that may trigger a layoff, so position yourself well before that happens.

4. Be part of a profit center or a cost center that significantly avoids financial pain

A cost center that provides revenue safety is almost as important as a profit center. Most companies don’t want to leak money if it can be avoided. Here are some example teams that would be cost centers:

  • Quality Assurance (QA) Team: Focuses on testing and ensuring the quality of software, incurring costs without directly generating revenue. Any bugs caught before hitting production would avoid a revenue loss. Some bugs can be hilarious though, like the time PayPal accidentally credits man $92 quadrillion.
  • DevOps/Site Reliability Engineering/Production Engineering Team: Manages the infrastructure, deployment, and operations of software applications, primarily a cost to ensure smooth operations. Any downtime would incur revenue loss and/or brand impact to the company (see PayPal’s Outage in 2004 or this PayPal Outage from 2015).
  • Security Engineering Team: Works on securing software applications and systems, which is a necessary cost but does not directly generate profit. Any public security issue might negatively impact brand (see Roku’s Data Breach) or create a fine for the company (see Equifax Data Breach Settlement). This also includes security bugs like this PayPal SQL Injection Vulnerability.
  • Technical Documentation Team: Produces documentation for software products, a necessary support function that incurs costs. This could be a public facing team that works on developer API documentation (see Square’s API docs), for instance or it could be a team that focusing on education of internal employees.
  • Internal Tools Development Team: Develops and maintains tools for internal use by other engineering teams. Anything that would be built or maintained by a team inside the company. For example Splunk or other monitoring tools deployed on-prem would likely require a team to maintain it.

On the flip side, here are some teams that would be profit centers:

  • Feature Development Team: Develops new features and enhancements for a software product that can directly drive sales and revenue. This is a pretty common team for most companies.
  • Custom Software Development Team: Creates custom software solutions for clients, generating revenue through bespoke projects. This might a team that drives sales using the companies product to show how it can be used.
  • Product Engineering Team: Builds and maintains the core product(s) of the company, directly contributing to sales and profit. Likely a core feature that is known to make money with small enhancements.
  • Platform Engineering Team: Develops and maintains a platform used by external developers or companies, generating revenue through platform usage fees or subscriptions. This is likely a Software-as-a-Service (SaaS) that might drive recurring subscription revenue or a product that charges based on usage (like AWS, Google Cloud, etc.).

5. Be productive

Quite simply: get work done. Use tools and systems effectively, don’t be a hindrance and always add value.

Try to be responsive to messages, don’t forget about commitments – don’t drop the ball too many times.

Wrap Up

As I’ve discussed before, the 9-box model is inherently flawed. Ask around to understand if this model is being used at your company, then ask questions to validate that there are enough checks and balances (discussed in the linked article) to attempt to make the field more fair.

If your organization is using some variant of this system, be sure to check yourself on where you’re at – since the 9-box can be used for many things, and could be foundational when it comes to algorithmic layoff determinations, I hope this empowers you to take hold of your career and make daily choices that help you.

Silent firing is Engineering Managers who don’t give feedback

Good managers will always encourage you and tell you that you’re doing a good job. Great managers will challenge you to mature as a software engineer. In my career I’ve observed EMs who “silent fire” someone by not spending enough time as a coach. The goal of this article is to remove this veil for software engineers so they can improve their manager’s effectiveness. Managers: your team is your responsibility. One rower in a boat can slow down the team.

Having good optics as an engineer can help you navigate through ineffective managers. Photo taken at Museo Galileo in Florence, Italy.

A great engineering manager maximizes these key traits:
1. Problem solving
2. Project management
3. Effective communication
4. Emotional intelligence
5. Great attention to detail
6. Technical skills
7. Delegation skills
8. Time management
9. Maintain a good work/life balance
10. Provides effective feedback
11. Fosters trust
12. Motivates

All of these twelve are important, but an engineering manager’s ability to be a multiplier as a coach and mentor relies on effective communication and feedback.

Have you ever received feedback like this:

1. The feedback I received from others is that you aren’t meeting the bar (but the specifics of that feedback are left out)
2. You could be doing better (but better is left up to chance)
3. We need to see an improvement in the next few weeks (without providing specifics)
4. You are doing great! Do you have any plans for the weekend? (Where a discussion about your performance is shunted to weekend chat)
5. I don’t think you’re ready for this project, so I’m going to put this other person on it

The abstract things that bad EMs say

Indirect and abstract feedback creates a negative flywheel, downward trend, 📉 that looks something like this:

1. An engineer has a clear gap in ability where coaching could help
2. During several 1:1s an engineering manager lacks radical candor (caring about someone enough to give direct feedback) to rumble and have a direct conversation about it
3. The engineer continues to fail because this gap isn’t addressed
4. Performance reviews come and a poor rating is given with abstract and unhelpful feedback
5. The engineer doesn’t feel valued
6. The manager is asked for a ranked list of talent on their team
7. The company decides to make budget cuts and sets a 10% layoff target for low performers (likely with other qualifiers like: in a high cost location, not near an office, working on a product that isn’t returning on investment or hasn’t been delivered fast enough).
8. The engineer is caught in a layoff

The keystone here is the manager. A manager who doesn’t dare to lead will eventually find the truth, but it takes years for that truth to play out while engineers who might have contributed more to the company are let out the door.

Without any additional mentorship force, an engineer’s career will continue to fall along the same trajectory. Photo taken at Museo Galileo in Florence, Italy.

One of the clearest patterns I’ve witnessed is the “nice manager”. Someone who is extremely personable, makes friends with everyone, and is mostly optimistic about the company and people. This creates a halo effect: engineers see them as an ally, the manager is well-liked. During upward feedback surveys, direct reports tend to give glistening feedback. During calibrations this manager believes that every single person is performing above their level. Since they have a halo effect, people tend to take them at their word (another reason for checks and balances in a calibration system). Outside of this team, people question why milestones are missed and quality is poor – on their merits, the team may not be producing, but it’s challenging for that feedback loop to have any impact. This manager is “silent firing” because they never provide valuable feedback for fear of their kind image. Left undetected, correction can take years.

The second pattern is the “lack of trust” manager. This manager doesn’t believe in people from the start and limits their opportunities, but also doesn’t give them feedback to improve. For engineers that become managers, it’s hard for them to risk slowing down their project deliveries by migrating away from a known-good workhorse on the team who might have more experience in a domain. These managers “silent fire” by not coaching people who have low task relevant maturity. In a 9-box system, this expediency bias (and these others) limits the potential the manager sees in the employee.

As a manager, how can you avoid this?

  1. Practice giving direct feedback. You can still be well-liked and give good feedback.
  2. Read Dare to Lead and Radical Candor. Check out this DevInterrupted podcast episode. Engineers: suggest that you’ve read these books and recommend them to your manager.
  3. Listen to others around you. Model your behavior after leaders who you see giving clear feedback to others. If you can’t figure this out on your own, ask another engineer: what was the last impactful feedback you received? You rely on tools to debug memory leaks, why not rely on others as a self-improvement tool?
  4. Fix your scaling problem. If your team is too large (more than 12 people) and doesn’t allow you to go deep in 1:1s with each person, look to hire another manager or instead of load balancing over everyone, only focus on specific people for a given week. Take notes in your 1:1s and follow-up – write-to-disk, trying to keep everything in-memory will not scale.
  5. Debug external feedback. Did you receive feedback about someone on your team? Ask for specifics. Try to understand if it’s a pattern or a single event. Take notes for each person on your team like you’re preparing a research paper without a due date. During your next 1:1, say “hey, I received some feedback about ______ and wanted to discuss this more with you.”
  6. Ask others for 360 feedback for each person on your team. Put it through your own filter: not all feedback is valuable or actionable. Over time, identify patterns and create an action plan to address them.
  7. Give people on your team a chance. Start newcomers with smaller projects, test the water with longer tenured people by giving them projects outside of their comfort zone. If everyone on the team is extremely specialized, you cannot effectively pivot to new asks, and at some point not providing opportunity will undermine your potential as a manager. You ramp features in production, why not ramp your trust in people?
  8. Check your peers. If you see that teams aren’t consistently performing because of a “nice manager” or “lack of trust” manager, give that feedback directly to that manager. Not providing this feedback prioritizes short-term gains, look to influence over the long-term to keep that team engaged and people performing well. Ignoring deprecation warnings will eventually bite you.

Engineers, what can you do?

  1. Check yourself. Are you checked out and not contributing to the team? According to this HBR article, by checking out, you might be creating a self-fulfilling prophecy where your manager is likely to “silent fire” you. Turn on your own linter.
  2. Apply BFS to your work network. Ask for feedback outside of your team, find a mentor where you observe admirable traits.
  3. Improve your survey responses. In the upward feedback company survey, make it clear by selecting the lowest options available for questions that indicate you’re not receiving quality coaching. If you can, add text comments that validate this. Most company surveys are anonymous, as long as you don’t add specifics. If your manager keeps dangling carrots for “just this one more thing” to justify a promotion, you may bias towards giving a great upward review – check your own biases each time you’re asked to review your management chain.
  4. Feedback horizon. If you’ve been on a team for about 4 months, you should have received some kind of feedback about the work you’ve been doing. At a minimum, after 6 months, a manager should be giving you feedback on your strengths and opportunities. You wouldn’t push code without a logging statement to validate your feature is working, why would you go months without instrumenting logging in your job?
  5. ⭐️ Ask for specific feedback. That presentation you gave, that document you wrote, that product you pushed to production – ask for some direct feedback on an artifact that you created. This is a good unit test – it should force your manager to give you feedback and can be a warning sign if this doesn’t occur. I added a star to this one because I think it’s the most important. If your manager can’t give you specific feedback for growth, ask them for more challenging projects or stretch goals. Think of this as pen testing – keep testing until you find a way in.
  6. Humble brag about yourself. In public docs (like a Wiki), maintain a table of contents that link to everything you’ve worked on. Make it easy for leaders to see that canonical document so they can jump to various artifacts and give direct feedback.

The failure of using a 9-box for engineers

The 9 Box Matrix is a tool developed by McKinsey in the 1970s to help GE prioritize investments across its business units. It evaluates the units on industry attractiveness and competitive strength. In recent years, Human Resources teams have adopted the model as a talent management tool, replacing the two industry axes with performance and potential to categorize employees and determine which to promote, retain, and invest in, and which to reallocate. The HR version of the model is not standardized and there are variations in circulation.

It can take years to find the goldilocks zone for performance

I only figured out these secrets as early as 2015, where I’m not familiar of how calibrations were performed in those orgs at the time. So while I’ll share what worked for me, this is not a checklist of things that are guaranteed to make you successful.

If your company has been in contact with a consulting company like McKinsey or Deloitte in order to reduce costs (read “improve efficiency”), you can sure bet that this model is being rolled out at your company. For some companies, there may be a theme of efficiency that started around 2018 with brainstorms of how to drive down cost mostly by improving data center efficiency. In the years that followed a Chief of Staff might have came in from one of these consulting companies and changed the way you looked at cost savings, at some point the 9-box model was likely introduced.

The 9-box model (taken from RapidBI)

When we began to use this for calibrations, it also came with a suggestion that only 10% of the team should be in the “Future Leader” or Box 9. The general logic is that bonus should pay people for high performance and RSUs (e.g. with a 3-year vest) should compensate potential. The “Under Performer” or Box 1 also had a similar 10% guidance that aligns with the Vitality Curve created by Jack Welsh and misused since then.

This 9-box may have worked with GE as they prioritized their investments, but doesn’t fit to calibrate engineers, specifically because potential measurement in people is different than with capital expenditure investments. This adds noise to the process.

For example, these could be used to determine potential:
– “Next level” (e.g. at the levels higher than their current role) abilities and motivation
– Skills and mastery
– Ability to determine future target state
– Thinking beyond themself
– Automation
– Team player
– Learning
– Drive

Performance likely looks at:
– What they have worked on
– How they have done it

You may have heard the saying “pay for performance”, but that was more like “bonus and merit increase for performance, RSU for potential”. In some organizations, merit increases were essentially taken away from managers in favor of an HR algorithm that looked at role, level, and location and gave managers a value that was typically less than 3% unless someone was being promoted, then it was less than 9%.

The baseball card process

Summarize an entire year on a PowerPoint slide. Created with stable diffusion with prompt: davinci style front of sports card modern with graphs and laptop with pcb circuitboard background

Every organization is different, but some teams may be asked to pre-fill this 9 Box before calibrations.

During calibration meetings, managers presented a PowerPoint Slide that looks similar to a baseball card.


It might have sections like:
1. Box Number (e.g. Box 3 “Trusted Professional”)
2. Outcomes – things they worked on (notice I didn’t say completed work)
3. Skill dimension attributes like customer centricity, technical skill (design, quality, requirements, expertise, proficiency, efficiency), collaboration and an icon to indicate which level they performed at: developing, proficient, accomplished, and expert.
4. Basic details, like time in role (e.g. 5 months), level (e.g. E4, L5, MTS1), and date of last promotion

Shoehorn

Now we have two things: a 9-Box rubric and a PowerPoint slide for each person.

This step of shoehorning now attempts to assign each person a Box as engineering managers meet. Each manager, who doesn’t know much about other software engineers in other teams now needs to read an extreme summary of mutually exclusive information like “They worked on a database migration from Teradata to BigQuery” or “They are doing a great job as a scrum master”.

Given twelve different skill dimensions, and a baseball card summary, managers then decide how that related to being a “Core Employee” versus “Effective”. This is where much of the final decision rested on the manager’s ability to:
1. Present their case
2. Defend their case
3. Know what engineers worked on
4. Draw comparisons between other engineers

If someone is strong in their collaboration skills, but weak in their efficiency where does that put them? There are too many variables at play to be able to translate them into a single box. This is different than how GE may have used this by reviewing their portfolio of products to align on a box by compare revenue and growth estimates.

What can do you do as an employee? Humble brag about yourself, all the time, and keep a record of it. Record how significant your achievements are and make sure your manager, other managers, your skip-level, and other people know about them. Ask for a copy of the template your manager is using: fill it out for them. Use specific wording and highlight key achievements. You know the most about what you did, the goal of a calibration is for those achievements to be compared against others. Personally, I would want to control my own destiny as much as possible and over the years I’ve filled out my own promotion slides and calibration documents several times.

Rating all employees on one dimension at a time (in this example, safety) exemplifies a noise-reduction strategy we will discuss in more detail in the next chapter: structuring a complex judgment into several dimensions. Structuring is an attempt to limit the halo effect, which usually keeps the ratings of one individual on different dimensions within a small range. (Structuring, of course, works only if the ranking is done on each dimension separately, as in this example: ranking employees on an ill-defined, aggregate judgment of “work quality” would not reduce the halo effect.)

Kahneman, Daniel; Sibony, Olivier; Sunstein, Cass R.. Noise (pp. 293-294). Little, Brown and Company. Kindle Edition.

Ranking reduces both pattern noise and level noise. You are less likely to be inconsistent (and to create pattern noise) when you compare the performance of two members of your team than when you separately give each one a grade.

Kahneman, Daniel; Sibony, Olivier; Sunstein, Cass R.. Noise (p. 294). Little, Brown and Company. Kindle Edition.

How should it be done? Engineering Managers, here is my advice to you.


1. Reduce subjective skills that are assessed. Twelve different axes is too noisy. If this can be reduced to 5, then managers can provide an assessment of [Does Not Meet Expectations, Meets Expectations, Exceeds Some Expectations, Exceeds Most Expectations, Exceeds All Expectations and Beyond]. These scores should be transparent and specific for all of the 5 attributes and should be calibrated among all engineers.
2. Transparently weigh performance more heavily than potential, or remove it all together since most potential assessments in their current form are subjective. Use promotions as the time to look at potential at the next level using historical evidence, but use annual calibrations for salary/bonus/RSUs to consider performance.
3. Be more strict representing people. Move from PowerPoints to Documents with strict templates for sections, headers, length, and content. PowerPoint allows people to change font sizes, and change the sizes of sections: reduce noise by tightening the restrictions on managers. Add a total word requirement per section: Outcomes should be larger, subjective sections should be smaller. Give everyone a fixed amount of time to read the content for each person and also provide a fixed time for discussion for each person to keep calibrations efficient.
4. Look at some software engineering stats as directional, but not the only metric. Things like number of PRs opened, PR closure time, PR comments given may draw out outliers that will force a longer discussion if they are also in one of the top talent boxes. It may also force them down from the middle box if the justification is light. I have seen a few engineers where the number of commits dropped, their deploys dropped, and it triggered a discussion on productivity – as a manager, you need to be proactive here: don’t let this wait 6 months.
5. Have someone designated to check bias. “She was a rockstar and knocked everything out of the park” would likely bias others if a manager said that – and this kind of comment is subjective. A person who checks bias would stop this discussion from continuing. A less biased way to say that would be “She delivered all major milestones ahead of time with performance unlike anyone else within our team and at her level”.
6. Take notes. We would spend hours calibrating, but some managers would not keep their own notes. This creates a negative flywheel when they go back to the software engineer to discuss strength and growth areas. Feedback should be normalized during this process so that everyone receives it irrespective of their manager. This is different than promotions, where that process should, by default, collect decent feedback to folks seeking promotion.
7. Get rid of the 9-box. Come up with an anchor case, and rank engineers above or below that anchor case. Refer to rankings for each attribute as part of the calibration and determine how five different attributes will be averaged to create an overall rating. If your organization didn’t plan their budget effectively, and can’t manage the budget, and can’t be creative with solutions (okay, maybe not even creative, but humane) to avoid a layoff – use this ranked list, and a set of specific algorithms determined ahead of time, to determine where to eliminate roles. Key here: the 9-box system is noisy, especially for engineers: using this as your method for layoffs is a failure.
8. Don’t keep your performance management practices a secret. Everyone should know what specific actions are taken to determine top/mid/low performance. If you’re rating someone’s potential (don’t do this unless this can be objective), they should be getting consistent feedback on how they can increase that potential.
9. Practice, especially with new managers. Don’t do a single calibration. Create anonymized versions of each level and rating, then remove the rating and have managers rate them independently until you see a consistent final result. This has the side-effect of a more performant process as people get more task relevant maturity.

Goal Posts

Regular goal posts are immobile, so should your criteria for calibrations: Created with stable diffusion with prompt: painting of moving goal posts that are blurry

This section deserves it’s own discussion as I’ve seen this happen numerous times over my 7 years as an EM.

Research suggests that a combination of improved rating formats and training of the raters can help achieve more consistency between raters in their use of the scale. At a minimum, performance rating scales must be anchored on descriptors that are sufficiently specific to be interpreted consistently.

Kahneman, Daniel; Sibony, Olivier; Sunstein, Cass R.. Noise (p. 297). Little, Brown and Company. Kindle Edition.

Here are some examples that have come up in calibration meetings:

  • Not aspirational enough: Should a senior engineer who is delivering the most challenging projects, before deadlines, and helping raise everyone around them be penalized if they aren’t aspiring to be an even more senior software engineer?
  • Critical project: Should someone who delivered a critical project be rewarded more than someone who delivered multiple projects during the same timeframe?
  • Program Manager Software Engineering: Should a software engineer be rewarded for scheduling meetings with the team, and managing the backlog, but isn’t delivering any significant code?
  • Responsiveness: Should someone who is responsive at all hours of the day and night be rewarded when they haven’t been able to deliver a project?
  • Design over delivery: Should a senior engineer be rewarded for designing multiple systems and features when delivery of them has faltered?
  • Delayed impact: Should someone be rewarded for designing a roadmap if that is for next year and unable to be proven out until then?

All of these questions should be answered with a sufficiently specific anchor, as Kahneman points out, for each level of performance.

For example, for a Senior Software Engineer, emphasis should be on both design and delivery of their projects. Equally weighted: where if the team is not delivering quality solutions fast enough, no amount of brilliant design will improve the rating. For Junior Software Engineers, design may not be as important, but consistent and quality delivery should be.

The key point here is that these discussions should not happen when you’re already in calibration sessions. Get together as a team and make sure you talk through these cases.

If there is an HR guideline (read: forced ranking) for who can be a top/mid/low performer, each person should be ranked against each other so that logical lines can be drawn at the respective cut-off points. What you don’t want to see are last minute attempts to figure out who to move into different buckets where goal posts get added just to justify a rating change.

Rewarding

One of the fundamental questions you should ask if you leadership team is: How are bonuses/RSUs/salary increases distributed? Does the VP make the decision or delegate budget to Sr. Director, Director, or Managers? If budget is distributed all the way down to each manager (ideally in a completely fair way, split equally), you could be missing out if you’re going the extra mile.

If budget is split among managers (instead of Directors or Senior Directors), this restricts the overall bonus that top talent can receive.

$1000 split between 5 managers allows each manager to give $200 as they see fit.

Taking that same model, but allowing for the $1000 to be pooled where 80% of that is given to 5 managers ($160 to distribute instead of $200), then leaves $200 to be distributed to top talent.

As a manager, I would prefer for the money to be pooled at the Senior Director level, with a fair calibration system, so that top talent can receive a larger bonus for going above and beyond. Unfortunately, at some companies, this is left up to the organization to determine on their own, so people are rewarded differently.

9-box for people or factories

“We can’t choose our fate, but we can choose others. Be careful in knowing that.” Photo taken at Harry Potter Studio Tour in UK.

Factories, like the ones General Electric used their 9-box for have physical machines and capital. Small changes to production lines, machinery, and formulas can improve performance. New equipment or future tech can improve the potential of that factory. Engineers aren’t the same as factories.

Engineers are curious learners, they can change their focus, improve systems, and deliver results. Communication is what helps engineers thrive. A failure of communication doesn’t mean engineers don’t have potential. A poor engineering manager may not create an environment where people are curious. Senior engineers may be overworked, stressed, and don’t give junior engineers something to aspire to: that doesn’t mean that junior engineer doesn’t have potential. Withholding opportunities for engineers to thrive is the fault on the potential of the leadership, not the engineer.

A manager who cannot accurately determine performance or potential introduces noise to the 9-box system. An industrial engineer assessing the performance and potential of a factory leverages a rubric with measurable criterium: you can’t say the same for performance reviews of software engineers. Don’t force-fit processes from one system to another because a consultant says it works. Don’t outsource your own brains to find the way to measure performance – if it doesn’t work at first, iterate – that’s what great engineers do.

FAQ

Why only engineers? I have the best knowledge of day-to-day engineering expectations, but I imagine a similar case could be made for Product Managers (PM), Program Managers (PgM), User Experience (UX), and Business Operations (BusOps).

If you remove the 9-box, what should be used? Only leverage your existing career ladder (for example, Square). Having two different frameworks creates overhead. As discussed in this article, slim down the amount of skills within that ladder to five or less.


Enterprise Engineering Teams: Why You Need a Strong One

How do you know if you have an enterprise engineering team? If you do, how do you know if you have a strong product, team, and happy customers?

Most companies aren’t doing enough with their enterprise tools.

They usually have more junior engineers, creating buggy products running old platforms and technology. It takes months to create a tool, and when it’s released it’s not user friendly and wasn’t built with the customer in mind.

They may rely on contractors that have limited company perspective and lack long-term vision to build these applications. What you really want is a solid engineering team delighting employees with the tools they use: it’s a retention tool at minimum and a productivity driver on average.

I wanted to be extremely pragmatic with this article, forcing you to retrospect and introspect about your company, noting that the grass can be greener if you look outside.

A quick test

You have an enterprise engineering team if:
1. You have a CIO organization: typically there is a software engineering team for HR functions, an employee phonebook system, and other internal tools to support employees.
2. You have an internal phonebook system that is built (e.g. Amazon Phonetool) and not purchased (e.g. WorkDay, Gusto, Friday, etc.)

Enterprise tools should stand the test of time, but not be built with ancient technology

General features might include

1. Employee directory: Allow employees to search for organizations, people, teams
2. News: Learn about recent events
3. Events: See a calendar and subscribe to upcoming events like a brainstorm, or a ERG meeting
4. Policies: HR, Security, etc
5. Thank you system: Applaud someone for helping out
6. Search capabilities: Search documents, news, people, etc.
7. Site-specific details: (e.g. Austin office details and perks) – how to get a badge, main contains, coffee shop hours, etc.

There are some more uncommon features to look for as well

1. Internal job portal: Allow employees to find jobs that are open
2. Employee transfer system: Allow employees to move into other roles easily
3. Badge system: Reward employees with internal currency and digital badges of honor for various things
4. Active Incident display: All tools used by employees display the current state of the production, sandbox, and developer environments along with any current Incident that may impact Customers or their work
5. Integrated financials and system health dashboards: At a glance, give people a general feeling for how the company is doing without triggering any deeper insider training restrictions
6. Ideas and patent portal: Keep track of awesome ideas that can turn into the next great product feature, allow employees to engage with others to build a business plan, go-to-market, and write code – then submit it for legal review to check for patentability to secure that intellectual property!
7. Promotion and rating systems for managers: If you are fortunate to have a standard promotion and calibration process across the company, this tool should allow for common practices to be implemented like anonymization, timed discussions of each candidate to remove bias, notes, assigned reviewers, voting, feedback back to the candidate, and decisions/ratings.

How do you know if my company is doing a good job with enterprise tooling?

Employees move from team to team easily+5: As an employee, when I want to move teams, I can mark in the system that I’d like to be considered by other managers for their teams
+5: I can move an employee to another manager in 10 minutes
+5: It’s easy to copy/paste usernames and first/last names because managers have to do this several times a day
-5: I can move an employee in 1 day
-10: I can move an employee in 5 days
-10: As an employee, I have to hunt for internal jobs on my own alone in the forest and have to wait for weeks for managers to respond to me making me not feel valued
As a manager, I love hiring because it’s easy+5: I can see how many positions, and at what level I need to hire for
+5: Within a few seconds, I can review the candidates resume, take notes so that my entire hiring group, including peer managers and senior engineers who help with hiring, can see it
-5: I can’t easily filter and mark candidates for the next phase of the interview pipeline
-10: I have had my positions taken away from me due to reorganizations, and hiring freezes within 2 months of each other wiping out any hiring pipelines and making candidates upset tarnishing your company brand
-20: A manager can hire someone without ensuring a significant cohort of people have been considered to ensure proper diversity, inclusion, equity, and belonging practices were followed see How to Actually Hire for Diversity
I can easily recruit people who have been referred by others+10: When you mark someone as a referral, you can easily follow them through the interview process and you’re consulted by the hiring committees for what you know about the candidate
-5: When you refer someone it goes into a pile where you can never tell if the hiring manager has ever seen it, there is no follow-up as the hiring process can take months to close a position even after someone is hired for that role
-5: As a hiring manager, you believe that referrals are the same value as an organic applicant applying online because referrals are sent by employees who don’t even know the person they are referring
We have a system that handles promotion panels and performance reviews+10: Across the company, performance reviews are conducted in similar ways for engineering teams
-5: Employees know they can get a better performance review in another organization
-5: Employees are promoted faster in other organization due to a lacking standard performance and promotion process
-10: Engineers who work harder, not smarter, working long hours due to poor planning and unrealistic deadlines are rewarded more and promoted due to fear of attrition
I can tell if there is a current Incident with our customer-facing products across all platforms I use+5: Our culture is such that people can easily find an active Incident for a product and can easily join in to see how things are going no matter if they are call center teammates, managers, directors, engineers, or project managers
+5: We hold the bar on post-mortems where we ask folks to leave if they have not prepared properly for their Incident review
+10: The same Incident doesn’t happen again, along with similar ones because proper testing, canary rollouts, and documentation is put in place
We have a centralized place to keep track of ideas, including patentable ideas, we can innovate on our products easily+5: When I have an idea for a new feature, product, or internal process I can collaborate with others to push it into production and/or receive a patent for it
+5: Our code base is built so that teams can easily collaborate on other products and features, even if they aren’t on the same team because we all have the same production push processes across the entire company and a culture of keeping good documentation.
Examples: Facebook Live and Timeline
Standard things like 1:1s and performance conversations are consistent across the entire company+10: Notes from 1:1s, performance reviews, calibrations, are all kept in the same place and travel as people change teams. Managers have consistent impactful 1:1s with their teams that don’t require threats from their VP because they are unable to hold a high bar for their leadership team
-10: PowerPoint templates are sent for various initiatives without any go to market plan or follow-up to ensure consistency
A rubric to use to score your company: Enterprise Tools

Engineering teams need even more from enterprise tooling

Aside from enterprise tools for the entire company, engineers also need high-quality, robust, easy to use tools to make their life easier. Engineering teams can make more than 30% of large tech companies.

Craftsmanship in enterprise tools with scale in mind cannot be underestimated
  1. Data exploration with Alation and Data Explorer
  2. Data querying with Trino
  3. Data analysis
  4. Configuration like Remote Config
  5. Keys and encryption like with Google Cloud Key Management
  6. Alerting and monitoring with DataDog or Splunk
  7. Build, test, deployment, verification of code like Semaphore CI
  8. Experimentation systems allow for ease of A/B testing and multi-arm-bandit like Optimizely
  9. A rollout system allows for different rollout strategies like Argo

How do I know if my company is doing a good job with tooling for engineers?

I don’t need to wait for a license to be able to analyze data+5: I can analyze data in under 10 minutes
+10: We are empowered no matter the role of TPM, PM, EM, SWE to query data on my own
-5: I need to request a license to get access to data
-10: I have to ask someone else to access data for me because it’s too complicated
I can figure out what column names mean along with the data inside of them+5: The columns and values inside of the database are well commented so I can determine what they mean
-5: I have to query another system to figure out what columns and values mean
-10: I have to ping on Slack to ask people how to interpret tables and data
Our Incident management system is tied into all of our tools, so I know when there is an issue with the tool I’m using+5: I don’t have to go anywhere to realize there is currently an issue with an enterprise tool
-5: I have to ping a help channel on Slack to see if there is an Incident going on
Our key management system for storing sensitive data is stable, reliable, and easy to use+5: They key management system just works and I understand how to use it
+5: I can easily write code to access passcodes, PGP keys, etc. There are mock frameworks that allow us to unit and functional test them.
-5: The key management system I use is confusing
-10: Our enterprise tools use a different key management system than production (or none at all because you copy/paste configuration files in production for enterprise tools)
The configuration system is state of the art+5: I can target a subset of users (e.g. employees) easily with my configurations
+5: Employees can opt-in easily to dog food new features
+5: A feature flag can be disabled if there is an alert triggered to disable a feature that may have a production problem
+5: Our internal tools use the same configuration platform as our production system to allow for dog-fooding and internal testing of new configuration features
Observability is fast and insightful+5: I can setup an alert to page my oncall team in under 5 minutes
+5: It takes a few seconds to get metrics an log data
-5: It takes a few minutes to get log data
+5: I can comment and mention other people on a dashboard to collaborate directly within our tools
Oncall teams are linked to data, products, code+5: It’s obvious who owns a specific piece of code
+5: It’s obvious who owns a database table or data pipeline
+5: It’s obvious which oncall teams on a specific product feature
Code rollouts are slick+5: I only need to click one button to roll out my code to production after it has been reviewed
+5: When it gets late in the day, or on Friday when you should never push code, I’m cautioned against being stupid
+5: I can easily test my feature within an employee base before rolling to production
+5: As code passes different gates, our team is notified via Slack chat and mobile pings
+5: Functional tests with 90% coverage run on production as code is rolling out
+5: It’s expected of engineering teams to not only write unit and functional tests, but these same tests are used to test production rollouts
-5: I have to click multiple buttons and get multiple approvals to push code that takes over 5 minutes to complete after my code review already gets a LGTM! 🙌 comment
I can do most everything from my IDE+10: I can save the code in my IDE and immediate test it on a staging environment
+10: I can save code in my IDE and another developer on their laptop can use what I’ve done on my staging environment
-5: I have to constantly create a test environment to test my code
-5: My test environment is broken most of the time
-5: I can’t reproduce something from production on my test environment because the environments are not even close to parity
-5: Functional testing is not standard across the company, I can’t easily look at another code base and write a functional test for it
A rubric to use to score your company: Engineering Enterprise Tools

So…how did your company score?

Max points: 220
Min points: -180

Add up all of the pluses and minus and consider the best score is 220, the worst is -180. Where did you land?

What actionable steps can you take to help repair these problems upstream, as a system? As a next step, you might try using an Ishikawa or Five Whys to get to the root cause.

How much does operational cost, employee moral factor in to these focus areas? Use this to create a business plan and speak with your leadership about your recommendations for change: even something small can go a long way.

If you need more inspiration, check out these podcasts:

All of the content from this blog is my personal opinion and does not reflect the opinion of any company I have worked for.

Flowers in France

The other side: 4 reasons why the grass is greener

I recently left my job of 10 years at PayPal (5 as an engineer, 5 as an engineering manager) to join Meta. One month into my new role seems like a great time to check-in and reflect on the past.

Greener, cuter, grass

I want to take this time to update my fellow confidants, friends, and the world, on why this job is 100x better (with a caveat that I’m still in a honeymoon phase and have recency bias). I’m going to only talk about four key areas that are most salient for me right now: engineering tools, management expectations and accountability, human-first relationships, and an early emphasis on scaling.

Continue reading →