4 Factors of Question Quality

This article discusses some of the properties of a good question or research project.

Consider a group of research activities. They are expected to have a variety of costs and benefits.

From economics we can generally assert that the quantity of research effort should be increased until the marginal benefit equals the marginal cost.

In practice, I propose that there are a few qualitative ways to assess the likely benefit of research beforehand. I think good questions are important, interesting, big, valuable.

1. Important questions are ones which cause change.
   1. Particularly those which cause behavioral change in an individual whom receives the answer
   2. I propose that a statistical finding which is significant but unimportant is of a lower quality than a finding which is statistically insignificant but practically important.
2. Interesting questions result in academic advancement or satisfy utility.
   1. These are two very different employments of the term interesting, but they are both common.
   2. As an academic, a topic is interesting if it will result in publishing, financial awards, or reputation improvement. It is interesting because an editor said so.
   3. As not an academic, a question is interesting because the pursuit or attainment of the answer directly provides me utility and substitutes for income. I just want to know!
3. Big questions have high quantitative and qualitative explanatory power.
   1. Information theory has a robust way of measuring the quantity of information implicit in a statement. In theory, the whole universe may be described by a certain number of bits. Let's call that number Total Information. As the information content of a statement approach Total Information, it is in some sense bigger and more explanatory of everything.
   2. Questions are also qualitative in nature, but I haven't heard much good theory around this.
      1. We can ask who, what, where, why, when, or how. These question forms exhaust all questions in English, but they overlap a bit.
      2. I think they are reducible to what, why, and how.
      3. Total Information would perfectly explain what. That is, it would perfectly describe the universe, but it would not explain why. It would explain much and/or all of how as well.
      4. We can consider what to be the fundamental quality of a question of science. Why is more like philosophy or religion.
      5. I'm unsure whether how is a category other than what. How is an action or verb. It can also be considered an intermediate process, mechanism, or means. A burger and a planet are whats. Eating is a how, and with my bare hands is another how. A person could list everything in the universe, but this would not make for a useful or meaningful description without including the process by which these things arose and interact. However, perhaps the processes and verbs in nature are also whats, or perhaps the set of actions a thing can take are implicit to the description of that thing. Whether these are viewed as fundamentally separate determines whether questions come in 2 or 3 fundamental qualities.
      6. I think whys are of a higher quality than whats. Whys bring meaning, purpose, morality, and other features which I consider superior to an explanation or existing lacking them.
      7. But the low-level nature of whats sometimes makes them more productively beneficial. Whys are more abstract and generalizable, which is cool for theory, but whats are more low-level, localized, and specific, so they may be better for application. I could use tools for a reason, which is a why, but a specific tool is a what. Being able to use a specific tool is often more technically beneficial than understanding the purpose of the tool, although obviously there is some relation. Actual whats vs whys are a big deal in regression analysis, where it often comes up regarding causal analysis.
4. Valuable questions aid production and technological progress.
   1. Self driving cars are valuable because they can displace more costly forms of transportation, reduce collisions, etc. So it would be valuable to research how to construct a self driving car.