Adult marshmallow-test winners do better

William H. Hampton1\, Nima Asadi and Ingrid R. Olson write.

Participants engaged in a delay discounting task adapted from O’Brien et al. (2011). In the task, participants were asked to make choices between a smaller sum of money offered now versus a larger sum of money (always $1,000) offered at five different delays.

They then use this variable along with other variables to predict the person’s income.

The results of each model were quite consistent, with occupation and education paramount in each case. On average, the next most important factors were zip code group and gender. While zip code group was highly associated with income, it is worth noting that our data do not adjudicate directionality. Logically, a person’s income is more likely a determinant of where they live than vice versa. Nonetheless, zip codes are a useful proxy for socioeconomic status, which is also related to income (Winkleby et al., 1992). As our zip codes were binned by average income, the association between zip code and income is not surprising, but does suggest that the individuals in our sample had incomes roughly representative of the incomes from their respective zip code group. Regarding gender, we found that males earned more money than females, a result consistent with a corpus of research on the gender wage gap (Nadler et al., 2016). The fifth most important variable was delay discounting, a factor closely related, but distinct from impulsivity. Although previous research had indicated that discounting was related to income (Green et al., 1996), it was unclear to what extent, relative to other factors, this variable mattered. Interestingly, delay discounting was more predictive than age, race, ethnicity, and height

Pointer from Tyler Cowen.

Oy. It would be nice to be able to cite their comment that “delay discounting was more predictive than age, race, ethnicity, and height.” But the flaws I perceive in the study are just too fatal to allow me to do that.

1. Most of the variables that they use to “predict” income are not plausibly exogenous to income. For that matter, it is possible that your level of income helps determine your willingness to delay receiving money, so even their key delay-discounting variable is plausibly endogenous.

2. When you compare the strength of different predictors (hardly ever a valid exercise), measurement error is everything. A variable that is measured unambiguously will do much better than a variable that is measured subject to errors, even if the latter variable has more influence in reality. So gender has the advantage of being unambiguous*, while self-reported ethnicity can be ambiguous.

*all right, some people insist that gender is ambiguous, but I don’t think those people find their way to this blog.

6 thoughts on “Adult marshmallow-test winners do better

  1. It does seem like you could do this study properly with a large enough sample and pre-registered analysis (even better, pre-reviewed) with control variables. If you don’t find much variation in delay discounting, controlling for the other factors, or if such variation does not explain much of the remaining variation in target, you have a negative result.

    A positive result obviously doesn’t overcome the problem of establishing the causal direction, of course. But that’s when you bring in the long term prospective study. Marshmallow test people every 4 years starting at 18 and track their income.

    But then you have the problem of subjects learning from the marshmallow test itself and _that_ affecting their income, so you’d need an untested control group matched on the other factors. Then we’d really be getting somewhere on this question.

  2. Still trying to wrap my head around what it means to recruit a sample via Amazon Mechanical Turk.

    And hypothetical rewards are a valid proxy for real rewards?

    “In this case, all rewards were hypothetical, but participants were asked to answer as if they were real. Use of hypothetical choices in a delay discounting task has been shown to yield no systematic difference in discount rate compared to real choices, suggesting that hypothetical rewards are valid proxy for real rewards.”

    I’d be curious to see how transferable those finding have been actually demonstrated to be transferable. I’d imagine lower income people might be a little more likely to be disgusted with the little game and that might transfer to the responses.

    And its curious the phrase “educational attainment ranged from pre-high school to doctorate degrees” appears at least 3 times that I noticed, as if that meant something. Did you get 2 people with online doctorates in theology to participate? I tend to be suspicious when it looks like someone is trying to sell me a bill of goods.

    Studies like these might be enhanced if they included some personal interviews with participants to get them to explain their reasoning for responding the way that they did. Yet again I am left thinking that people may not be as fungible as researchers would like to be able to treat them as.

    • > all rewards were hypothetical, but participants were asked to answer as if they were real.

      Economics Joke #1: Two economists walked past a Porche dealership.
      Ec 1: I really wish I had one of those convertibles.
      Ec 2: Obviously not!

      Any study, theory, simulation etc. that is based on the equivalence of asserted preferences and demonstrated preferences is a joke.
      Regards,
      Bill Drissel
      Frisco, TX

  3. Unless I missed something, the reverse causality story here is pretty clear: All else being equal, high income reduces discounting rates for a fixed amount of cash. If I’m already adding $1,000/month to my savings, than I’m in no particular hurry to get another $1,000, and I’ll choose to wait as long as I need to to get the extra money, as long as it’s not so long that I can expect to do better by taking the money now and investing it.

    I do think that it’s probably true that discounting plays an important role in life outcomes, but this isn’t particularly good evidence for it.

    • ” it is possible that your level of income helps determine your willingness to delay receiving money”

      Absolutely. A better test would have been to vary the amounts only by the current interest rate.

Comments are closed.