You're Only As Good As Your Data

The Last Industry To Be Automated By AI Will Be...

Well, it seems like we’ve moved on from the age of web3 to the age of AI. I can’t log onto Twitter without seeing a founder or VC tweet about a new product built on top of OpenAI or coming across applications that will put humans out of business. Unlike the last 30 years, i’d agree that based on current market dynamics and the technical evolution of companies like OpenAI, I think AI is here to stay and it’s only getting stronger.

The first applications of this modern technology, like OpenAI’s DALL·E 2, are creative…impacting words and visuals, where data is plentiful and commoditization potential is high. I’m sure visuals and words are not the last domains AI will automate. I could see writing code (See Github Copilot), managing projects, and maybe even running large companies could be in the wheelhouse of a great AI. With that said, I do think there is one industry that is essentially immune from AI, for the time being at least, and that industry is talent identification.

If you told an AI to predict, out of 1,000 people which, 10 would have the highest likelihood to become the most influential person in the world in their lifetime, how would it decide? It would probably look for the same filters humans look for when identifying talent. Impressive background? Can they speak well? Are they well liked by important people? Are they already on an impressive career track? You may agree with some of these yourself.

The questions above may sound familiar, as these are the filters venture capitalists look for when making investments in the founders of startups. Notably, VCs are wrong about their decisions over 90% of the time. Note that to a VC, this doesn’t matter, because as long as one company becomes at least a $10B company in each fund, it makes up for all their losses. With that said, if an AI wants to become the best talent identifier in the world, it needs to model a way to make decisions and needs to know the right questions to ask. And clearly, VC’s questions may not be the best data to train an AI on. So what else?

We could go earlier in the stack. We could train the AI on the attributes of students who get into the top 10 colleges in the world. The qualities? Great writer, spends time doing extracurriculars, gets great grades, stays out of trouble, wonderful soft skills, etc. But if we trained the AI on this data, it would go 100% against Malcom Gladwell’s outlier theory, which is that the largest impacts (financial or not) in the world come from outliers. Paul Graham, founder of famed Y Combinator agrees, noting that school trains people to “pass the test” of life, which is not a quality of people that eventually change the world.

Forget VC and college admissions, let’s look at where out current talent identification process led us as a society. Most CEOs of public companies are men. A fraction are ran by women. And the number of black or latino business leaders are in the low single digit percentiles. A disproportionate number of VCs AND VC backed founders attended Stanford, Harvard, or another Ivy League school, while that makes up under 1% of people vying for professional opportunities. And for some reason, these numbers don’t change much over time. It seems like we got something wrong along the way…

What point am I making here? There are 26 letters in the alphabet, and that’s very easy to feed to an AI and have it do whatever it wants with that dataset. There are a finite amount of colors that can be created through the hex code system, which gives AI the data it needs to create any sort of art it desires to. But as a society, I think we have completely butchered the concept of talent identification and and have focused on almost all the wrong data points and questions to be asking, which renders AI in this industry completely useless. By introducing AI into human evaluation processes, we are allowing it to run with the flawed system that humans have stitched together, which does a complete disservice to the potential of the world.

As we’ve seen by now, humans know no limits. And where there is potential to make money, humans will follow. And surely there have already been thousands of AI apps built to help evaluate aptitude in another person. But if you want to make the real money and impact, slow down and rethink everything. Ask yourself, “Is the current way we evaluate humans for opportunities the correct one?” You may think it’s good enough, and start to metaphorically dig for gold in this new age of AI. Just remember, the data we have on talent identification is pretty bad, and you’re only as good as your data. Best of luck out there, trailblazers.