A few days ago, in one of the chats I follow, a heated discussion broke out: can everything in marketing be defined by funnel metrics?
As I was writing my own answer, I came up with a useful way to distinguish between knowledge, skill, and intuition.
Example of knowledge: the t-value of 1.96 corresponds to 95% statistical significance. Knowledge is easy to acquire — in fact, you don’t even have to keep it in your head, as long as you know where to find it.
Example of skill: being able to tell, roughly and without calculation, that if your control group has a conversion of 275 out of 1000, then 295 out of 1000 isn’t statistically significant growth, while 320 out of 1000 probably is. Skill can’t be “bought” or copied — though it can be replaced, to some extent, with a mental checklist of rules.
Example of intuition: knowing which directions to explore in the first place. A funnel might show that at some step — say, when filling out a form — you’re losing XY users. You might hypothesize: the form is too long; it’s not adapted for people with disabilities; it’s not available in Uyghur. Then you have to decide which explanation you believe the most. If you’re unsure, you can answer it with an A/B test — or a series of them.
But even Amazon doesn’t have enough traffic to answer every dumb question without intuition. Which means intuition — as a synthesis of skills and knowledge — is something you can only develop systematically.