Grit Research is Alive and Well

<a href="http://psycnet.apa.org/index.cfm?fa=search.displayrecord&uid=2016-06824-001">"True Grit..." by Rimfeld et al (2016) is a current and well received paper. It also gives academic credibility to a rumor I've heard around the <a href="https://www2.gmu.edu/">GMU campus. The rumor is that the <a href="https://en.wikipedia.org/w/index.php?title=Grit_(personality_trait)&oldid=761415235#Scientific_findings">personality trait called grit which was <a href="http://psycnet.apa.org/journals/psp/92/6/1087/">pioneered by Duckworth has reached a dead end as a research program. The present article disagrees strongly.

Rimfeld et al argue that grit is essentially the same trait as <a href="https://en.wikipedia.org/w/index.php?title=Conscientiousness&oldid=760419198">conscientiousness, and therefore adds nothing to the prior literature which establishes <a href="https://en.wikipedia.org/w/index.php?title=Big_Five_personality_traits&oldid=765739426">Big 5 personality traits as predictive of success.

First, let me state that I think "True Grit" was well done. At the same time, I think the study provides great evidence in the direction opposite its own conclusion.

Section I: 8 Cheers for Grit and Some Sub-points

1. By its own admission, grit outperforms 4 of the Big 5 personality traits.
   1. In significance, coefficient size, and explanatory power.
2. By its own admission, grit adds 10% to the explanatory power of the Big 5 model alone.
   1. The Big 5 model explains 5.5% of the variation, while the addition of grit adds another statistically significant .5%. See this also noted in an interview with Rimfeld.
   2. Any market firm would consider such a gain to be a huge competitive advantage. Any investor would consider it a solid ROI. Why is 10% not economically important?
   3. Here's a plausible pro-unimportance argument: Government policies to fund a grit training would cost more than the benefits they would generate.
   4. I agree with #3, but who says research which doesn't entail a policy change is unimportant? General success factors are valuable to the typical teacher, student, firm, and person. It need not lead to a grit training program.
3. By its own admission, grit and conscientiousness have a .53 phenotypical correlation. One can hardly claim equivalence on this ground.
4. The study used low-precision instruments and yet still found important and significant effects.
   1. The grit scale used was Grit-S, the shorter and less precise survey instrument.
   2. The abbreviated FFMRF was used, which is the low resolution implementation of Big 5.
   3. The dependent variable was GCSE grades. These are letter grades. Letter grades are notoriously lumpy/imprecise measures.
   4. I think a study which predicted income, graduation rate, or SAT score based on the high resolution measures would find more importance and significance.
5. The study presents clearly weak arguments against the malleability of grit.
   1. Other very fresh literature strongly suggests grit may be trainable. Refer to Houser et al (2016), which shows based on observational data that impulse control is acquired.
   2. \"True Grit\" argues against grit's malleability in 3 ways:
      1. They argue that all traits including grit are hereditary. Point 5.1 disputes this and I also discuss this more after the list concludes.
      2. They argue that non-shared environmental factors matter, not shared environmental factors. First, this does not dispute that grit is not malleable. Training can be a non-shared environmental factor. Second, I dispute this finding after the list concludes.
      3. They argue \"we are not aware of studies that have shown the effects of training Grit.\" This is a classic argument from ignorance, which is a terribly weak argument. I think the lack of such studies is best explained by the infancy of the grit literature rather than the absence of any such effect. Moreover, I claim the Houser study linked in point 5.1 provides plausibility for the effects of training grit.
6. Grit may suffer a masking effect from aggregation of two or more subfactors. If so, decomposition could find larger explanatory power and effect size.
   1. I suggest a dichotomy between the \"cold grit\" of an unenthusiastic, high-work ethic soldier, and the passion of a generally low work ethic artist.
   2. The artist might appear to have as much or more grit than the soldier when considering a task of interest, but without interest the artist-type would suffer from low grit.
   3. Bryan Caplan suggested an interesting dichotomy might be grit vs ambition (even while assenting to the view of grit as another word for conscientiousness).
   4. One could easily include all three subfactors because it amounts to a few additional survey questions.
7. The fact that new and important research is being actively published and discussed in late 2016 shows the literature is extremely current.
8. The entire grit literature depends on survey measurement. What about observational studies? Grit can easily be operationalized for an observational study.
   1. Suppose participating in extracurricular activities through the varsity level counts as grit. Consider American football in particular. There is good evidence these activities improve educational outcomes.
   2. Consider that survey includes an overconfidence effect. First, overconfidence may be endogenous to grit. Second, overconfidence shrinks the variance available in the data. In the \"True Grit\" study conscientiousness had a mean of 3.7 out of 5. This makes awfully little upward variance available for explanation, and it highlights the value of an observational study.
   3. Rimfeld et al criticize the extant literature on grit as largely an analysis of specialized populations, such as spelling competition finalists. Looking at anyone who played sports in high school and went on to college (as an ordinary student, not a student athlete) is not a terribly special population in the US. It's rather common, and may rebuff this criticism.

In addition to the points outlined above, I will state that I find the genetic argumentation in the paper entirely unconvincing. This sort of analysis permeates the related literature, however, and it may reflect my own ignorance rather than the widespread methodological problem it seems in my view. Why do I find the genetic argumentation unconvincing?

1. The genetic differences are claimed on a differential between paternal and maternal twins, but these twins were not separated at birth. So I do not see how the environmental effects can be cleanly delineated.
2. The claim that shared environmental factors have no explanatory power supports the intuition behind criticism #1, which is that there is a faulty attribution of shared environmental effects to genetic differences.
3. The genetic differences themselves are not clean: The analysis shows the difference between paternal and maternal twins. This does not genuinely signal genetic differences: It signals differences attributable to the Y chromosome. The X chromosome obtains no variation.
   1. It follows that if the X chromosome obtains high explanatory power then genetic correlation will be overstated.
   2. This is plausible considering the decomposition of grit into \"cold grit\" and passion, where passion or emotional effects are intuitively linked to gender.
   3. Or, perhaps my gender intuition is wrong. I'm far from a gender expert.
   4. Here's a cherry picked article claiming a link between gender and emotion. Here's an article against gender and passion. I haven't done a real literature review.

Given the above points, and the further content of that study and the broader literature, I find it entirely implausible to claim either of the following:

1. Grit has an unimportant, insignificant, limited-application, or non-robust effect.
2. Grit as a research project is a dead end. The important questions around grit have already been answered and it turns out that it is essentially the same thing as conscientiousness.