The best we can hope for is people with a good win-loss record. But how do you measure win-loss record? Lots of people worked on this (especially Philip Tetlock) and we ended up with the kind of probabilistic predictions a lot of people use now.
He points to a paywalled substack post by Matt Yglesias which demonstrates thinking in bets. If that post had gone up during the Fantasy Intellectual Teams season, Yglesias would have scored a whole bunch of B’s for his owner.
NOP and Yglesias need to start talking to actual market participants before they go around designing these systems. I have some modest experience in making money from conditional probabilities and even structured games like poker don’t work the way their models think they do. Mistakes fit into different categories, there are mistakes that are binary- call this bet or don’t call this bet when this is the last possible bet to be made but those are the rarest forms of decisions. Then there are endpoint mistakes, folding when you should call or raise means your mistake is done, but calling when you should fold but betting isn’t concluded yet means you are set up for additional potential mistakes.
The problem with punditry and the views expressed in these pieces is that mistakes are treated as individual actions and not cumulative. In tournament poker one very common mistake for otherwise good players is to make a large bet with a small edge because being knocked out prevents you from exploiting other larger edges that come along. Attempting to evaluate such actions in a vacuum will lead you to incorrectly identify poor decisions as good decisions.
Does each instance of B’s count? Even within the same article?
if the bet is on a new topic, it is a new B. So they could come in bunches from one blog post.