I’m shocked that more startups, including their boards and investors, don’t thoroughly check references. I hope by the end of this post you will agree that to not check references is both irresponsible and dangerous.

When I started Coremetrics in 1999, Accel Partners wanted to invest in our Series A alongside Highland Capital Partners. We had already chosen Highland as our lead. We were impressed with former general partner Keith Benjamin in particular, and he was joining our board of directors. Unfortunately, Keith passed away in a tragic accident in 2008, as I wrote about in this Bazaarvoice blog post; I think about him often—he was an incredible friend and online commerce and Wall Street visionary.

Accel put forward Arthur Patterson, the co-founder of Accel and a venture capitalist since 1973, to join our board alongside me, Keith and Bong Suh, our independent director and a terrific mentor. As I had done with Keith, I insisted on checking Patterson’s references. Most people in the Accel-backed portfolio were surprised, and I think they thought I was naive at the time—I was a 26-year old CEO and they probably chalked it up to inexperience. But those references turned out to be very helpful to me, specifically how to best work with him as a business partner. I believe Patterson had more respect for me as a result of being one of the first entrepreneurs to check his references. I couldn’t see any other alternative—I deeply loved Coremetrics, and I wanted to make sure that we fielded the best team possible, and that included our investors and our board of directors.

Fast forward to Bazaarvoice. We initially were obsessive about both our hiring process (which I detailed in this Lucky7 post) as well as checking references. I would probe and probe on reference checks until I got something useful to work with. Usually I was able to dig into some of the “areas for improvement” by saying something like, “Please level with me. I most likely am going to hire them and I could use your advice about how to best coach them to perform at their best from day one.” That type of personal plea was hard to ignore. If that didn’t work, I would ask the reference to walk me through areas for improvement noted on their last performance evaluation. We all have them—no one is perfect.

But I learned later that I wasn’t as thorough as Scott Cook, the co-founder and initial CEO of Intuit. First, let me take a quick diversion to tell you how I got to know that.

I was fortunate enough to speak at First Round Capital’s CEO Summit earlier this year. First Round, and Managing Director Josh Kopelman, W’93, in particular, did a lot for Bazaarvoice as one of our first-round investors. I spoke on the lessons learned on my journey from founder to CEO, and when First Round launched The Review, which I highly recommend as a regular periodical for entrepreneurs, I was happy to see them post a summary of the talk. A more in-depth version of my talk is in this Lucky7 post.

But the highlight of the summit for me was seeing Cook present his lessons learned as the initial CEO of Intuit. He touched on the subject of references:

Keeping these four key attributes in mind, Cook gets on the phone with the reference and begins by letting them talk about working with the candidate. However, the reality is people generally want to be nice and don’t have much to gain by being fully transparent and honest, so they generally start by saying how great the potential hire was and how valuable they were to the team. Cook has found that it’s most effective to completely ignore this opening feedback and throw it out. Once they’re done rambling on, Cook asks, “Among all of the people you’ve seen in this position, on a zero to ten scale, where would this person rank?” They go, “Seven.” Cook says, “Why isn’t this person a nine or a ten?” And then you’ll finally start learning about what this person really thinks.

Cook concludes the call by asking this person for other people who can give a reference on the candidate and then begins the process again. Getting as far away as possible from the candidate’s suggested references often leads to the most valuable data.

(The full summary of his talk, which is well worth the read, can be found here on the First Round Review.)

The brilliance of this approach is the comparative aspect. You are forcing the reference to stack-rank the candidate, and the reference’s integrity will most likely force them to be candid in response to such a direct and comparative question.

Editor’s note: Stay tuned for Part 2 of Brett’s blog—about hiring stories that will make you reconsider how you check references.