The Joel Test for Programming: An Essential Guide for Software Teams in 2021
If you lead a software development team, you likely believe your programmers are highly skilled and your development process is top-notch. But can you actually prove it? It‘s easy for leaders to assume everything is running smoothly, but much harder to measure and verify the true quality of their teams.
Back in 2000, software developer Joel Spolsky created a simple 12-question test to evaluate the quality of a software team. Known as The Joel Test, it became a widely-adopted standard in the industry. Over 20 years later, is The Joel Test still relevant? Let‘s break it down question by question and find out.
Who Is Joel Spolsky?
First, some quick background. Joel Spolsky is a New York City-based software engineer, writer, and entrepreneur. In 2000, he started a popular blog called Joel on Software where he shared his thoughts on software development, business, management, and more.
Joel went on to co-found Stack Overflow, the go-to Q&A site for programmers around the world. He also co-founded Fog Creek Software, a successful software company and startup incubator. To say Joel is an expert in what makes great software teams tick is an understatement.
The Joel Test Explained
The Joel Test is deceptively simple – it‘s just 12 yes-or-no questions to assess a software team:
- Do you use source control?
- Can you make a build in one step?
- Do you make daily builds?
- Do you have a bug database?
- Do you fix bugs before writing new code?
- Do you have an up-to-date schedule?
- Do you have a spec?
- Do programmers have quiet working conditions?
- Do you use the best tools money can buy?
- Do you have testers?
- Do new candidates write code during their interview?
- Do you do hallway usability testing?
The scoring is simple too. Give your team one point for each "yes". 12 out of 12 is perfect. 11 is acceptable. 10 or lower is concerning. Let‘s look at each factor in more depth.
1. Do you use source control?
Source control, also known as version control, is essential for tracking code changes, coordinating work between multiple programmers, and maintaining a record of the project. Without source control, chaos and lost work will ensue.
Every programmer should be using a source control system like Git, Mercurial, or Subversion and committing their changes frequently. Code that‘s not checked into source control essentially doesn‘t exist. Make it a hard requirement.
2. Can you make a build in one step?
Investing time to automate your build process will pay huge dividends. With a single command, you should be able to pull the latest code from source control, compile it, run automated tests, and package it for release. If it takes more than one step, automate it.
Having a scriptable, repeatable build process saves time, reduces errors, and ensures consistency between different programmers‘ environments. It‘s a prerequisite for continuous integration and deployment. Don‘t make builds any harder than they need to be.
3. Do you make daily builds?
Speaking of builds, they should be happening every day, ideally multiple times per day. On a multi-person project, daily builds are important to catch integration issues early before they snowball into bigger problems. Even on a solo project, frequent builds keep you honest and prevent you from drifting too far off course.
If a build breaks, fixing it should be the top priority. Some teams have a penalty for the programmer who broke the build, like putting a dollar in a jar. The important thing is that the whole team stops what they‘re doing and rallies to fix the build quickly.
4. Do you have a bug database?
Bugs are inevitable in any substantial program. How you track them makes a big difference. Trying to manage bugs through emails, sticky notes, or spreadsheets just doesn‘t cut it.
Every software team should use a real issue tracking system like Jira, Bugzilla, or GitHub Issues. It should be the central repository for reporting, prioritizing, assigning, and fixing bugs. Without a bug database, bugs are guaranteed to slip through the cracks.
5. Do you fix bugs before writing new code?
It‘s tempting to plow ahead and write new features while putting off bug fixes. Resist that temptation. The longer a bug lingers, the harder it becomes to fix as the code evolves.
Make it a policy to fix bugs as soon as they‘re discovered, before starting work on anything new. Fixing bugs early keeps your code clean, reduces risk, and leads to more stable releases. Save new features for when the known bugs are squashed.
6. Do you have an up-to-date schedule?
Schedules are notoriously difficult to get right in software. Unforeseen issues crop up, requirements change, and estimates prove too optimistic. But that doesn‘t mean you shouldn‘t try.
A schedule is essential for planning purposes, managing stakeholder expectations, and keeping the team accountable. Break the work down into milestones, track your velocity, and update your schedule frequently as things change. An out-of-date schedule is useless.
7. Do you have a spec?
A spec, short for specification, is a description of how a feature or program should work from a user‘s perspective. Trying to build software without a clear spec is like setting out on a trip without a map. You might get there eventually but you‘ll burn a lot of extra time and fuel.
Specs can take different forms – a document, a prototype, wireframes, etc. Whatever format you use, the key is to define the requirements clearly up front and get all stakeholders aligned before coding begins. You‘ll still discover things during implementation, but a spec provides a shared reference point.
8. Do programmers have quiet working conditions?
Programming requires deep focus and concentration. Even brief interruptions can be very costly, taking 15-30 minutes to recover from. Programmers need a calm, quiet environment to do their best work.
Avoid open office plans. Provide private or semi-private offices. Establish a culture that respects focus time and discourages unnecessary interruptions. Your code quality depends on it.
9. Do you use the best tools money can buy?
With the wide availability of open source, programmers have access to incredibly powerful tools for little to no money. But the old adage "you get what you pay for" still applies. Using the best tools makes programmers more productive and results in better output.
Within reason, don‘t skimp on tools. Outfit your programmers with fast machines, large monitors, comfortable chairs, and standing desks. Buy them any software tools that will make them more effective. The cost is minor compared to the benefits.
10. Do you have testers?
As good as your programmers may be, they‘re not perfect. It‘s a mistake to rely solely on programmers to test their own code. They‘re too close to it and likely to make assumptions that a user wouldn‘t.
Dedicated testers are essential for finding issues before release. They approach the software with fresh eyes and try scenarios a programmer may not think of. Generally you want about one tester for every 2-3 programmers. Don‘t treat testing as an afterthought.
11. Do new candidates write code during their interview?
Resumes and interviews are notoriously poor at predicting programming ability. The only reliable way to assess a candidate is to watch them solve actual coding problems.
Every programming interview should include a coding test where the candidate writes real code and explains their thinking. It‘s the best way to separate those who can talk a good game from those who can actually deliver. If you‘re not doing coding interviews, you‘re hiring blindly.
12. Do you do hallway usability testing?
You can‘t rely solely on your own team to judge usability. You need to see how people outside the team interact with your software, especially non-technical users.
Hallway usability testing simply means grabbing people who pass by in the hallway and asking them to try your software while you observe. You‘d be amazed how many obvious usability issues you‘ll discover within the first few minutes. Make it a regular practice to reality check your work with users.
Is The Joel Test Still Relevant Today?
The short answer is yes. The 12 questions touch on timeless fundamentals of software development that are as true now as they were 20 years ago.
That said, passing The Joel Test is really just a baseline. With the rapid advancement of development practices, tools, and platforms, there are many other important factors not covered by the test, such as:
- Automated testing and continuous integration
- User experience and design
- Cross-platform/device functionality
- Performance and scalability
- Security and data privacy
- Accessibility
- Agile planning and estimation
- Collaboration with non-technical stakeholders
- Remote and distributed teams
The Joel Test is still a great quick gauge of software team quality. But it‘s not an exhaustive evaluation by any means. Think of it as a starting point to be supplemented with more team-specific assessments.
Building World-Class Software Teams
Attracting and retaining top programming talent is incredibly challenging. Highly skilled programmers have their pick of jobs. What can you do to make your team stand out? Create an environment where they can do their best work.
Following the criteria laid out in The Joel Test will get you well on your way. Provide the right tools, minimize distractions, and build a culture of quality. Let programmers spend their days focused on solving interesting problems, not fighting broken processes. Do that and you‘ll have a major advantage in the battle for talent.
Ultimately, your team‘s output depends on the caliber of your people and how well you enable them. Technical skills are critical but team chemistry and environment eat raw talent for breakfast. The best software teams bring out the best in each other.
The Joel Test remains a valuable, if imperfect, measuring stick for software teams. But it‘s just a start. Building an exceptional team takes continual investment and commitment at all levels of the organization. Pass the test, then keep leveling up from there. Your software quality and business results will reflect that dedication.
