From the comments on statistical malpractice

1.

Frankly there are just too many PhDs. The talent pool is too diluted.

Imagine if the NBA had 1,000 teams instead of 30. Those extra 10,000+ players will be playing basketball by its official rules but it will mostly be of very poor quality compared to what we see from LeBron James. There’s nothing you can do about that if you’ve committed to having 1,000 teams. There simply aren’t 15,000 basketball players who are as talented as an average NBAer.

2.

Basically, every clever idea that requires human beings to do their duty without reliable detection and penalty for violation, has already been thought of, implemented, and failed entirely. Not just failed entirely, which is bad enough, but made it around two orders of magnitude more burdensome to get papers done. Not good papers, just any paper, which are still mostly bad papers. “Huge additional costs, zero apparent benefit” is the worst of all possible worlds, and such a bad world, that one just needs to move to a totally different planet.

So, the only solution is a completely different mechanism and institution of accountability.

I agree with both of these points.

16 thoughts on “From the comments on statistical malpractice

  1. Both points refer to the consequences of massification. For any social group, there is a size after which the marginal benefit of increasing its size declines (first point), and another size after which the marginal cost of increasing its size increases (second point).

    You can change both “optimal” sizes by searching but the marginal cost of searching soon increases sharply. Thus, the relevant scientific question is how some large political jurisdictions (today, nation-states) have been able to survive for centuries. And a relevant engineering question is how can keep the U.S. together in 2020.

  2. I do not believe that the erosion of quality in science is due to a shortage of talent. Nor do I believe that there has been a sudden decrease in the ethical standards of scientists.

    The fundamental problem with science, in my view, has been created by the excessive power of university administrators and the perverse incentives this situation creates. These administrators use various “metrics” to evaluate their faculty, but the metrics nearly always fail to capture the most important thing: the quality of a faculty member’s scholarship. The worst metric is “research expenditures,” in which the amount of money the faculty member spends is used as a proxy for scholarly merit. You can imagine the damage done to science when the faculty are incentivised by their “betters” to focus on winning grants rather than scholarly rigor.

    • That’s ultimately another consequence of too many PhDs. Grant funding hasn’t increased at nearly the same rate as the number of PhDs and the competition for funding has, predictably, reached destructive levels.

      Full disclosure: I have a PhD in chemistry. I’m much happier as an accountant.

  3. Re (1), the problem is too much similarity in PhDs. If we had 10,000 PhDs who created a well-diversified portfolio of research styles it wouldn’t be so bad. I mean, we need more people like Yitang Zhang who take brave risks (who ex ante was a “marginal” PhD in the statement of (1)), and fewer PhDs who just produce epicycles of epicycles of epicycles… And yes, admin bloat and perverse incentives of research grant money…

    • Very true. But keep in mind that what is driving this, at least in part, is that faculty search committees try to hire faculty that are “fundable.” In practice this means hiring people who are working in a popular, mainstream area rather than someone, however brilliant, working in an unpopular area.

  4. The presence of a large number of bad papers is not a big deal in itself. If the cost of publishing a good paper is publishing 10 bad ones, it’s worth it

  5. Frankly there are just too many PhDs. The talent pool is too diluted.

    Kling is endorsing the classic anti-libertarian argument.

    Usually the story is told with dentists, or hair stylists, or taxi drivers, but here it is with PhD academics: the existing workers want to raise the bar of entry to reduce the quantity of incoming workers that compete with the existing workers. They claim this is to preserve important quality and safety guarantees, but ultimately they want to selfishly minimize competition and raise their own status and market leverage at the expense of blocking people who want to enter a particular field.

    The more productive avenue is to build a better filter for quality rather than simply check whether someone has a PhD diploma or not.

    And how many PhDs should we have? How many hair stylists and taxi drivers or uber drivers and dentists should we have? Let market forces and the preferences of the masses determine that.

    • While I appreciate the sentiment, I think it is hard to argue that market forces are at work in the first place in higher education.

      • The status quo is not a free market nirvana, but I’d at least advocate reforms in a free market direction, rather than going in the other direction with more heavy-handed top-down licensing + credentialing caps.

  6. I’m going to go out on a limb and suggest that sociologists and many others who use statistics in their work are probably just using statistical software packages as “black boxes” with only the fuzziest understanding of how the underlying math works. Unfortunately, this problem is only going to get worse as machine learning software proliferates.

  7. Although these two observations may be true of the USA, we might not want to generalize to the globe. According to the OECD, Luxembourg and Switzerland have more PhD’s per capita than the USA, yet are largely free of the dystopian Venezuala-like intellectual environment that dominates elite life in the latter. And both countries out perform the USA on GDP per capita (PPP) with Luxembourg at $121,293, Switzerland at $70,989, and the USA at $65,281 (World Bank, 2019), although to be fair both countries have an advantage over the USA with average IQ at 101 and 100 compared to 98 for the USA.

    Switzerland, of course, as the world’s most populist country, was ranked first in 2019 as the most innovative country in the world by the World Intellectual Property Organization and is a world leader in productive, meaningful areas of research, such as machinery, chemicals, and metals, with about 28% of the workforce in manufacturing. In relation to its population size, Switzerland is the country with the highest number of universities among the 100 top universities on the Academic Ranking of World Universities.

    But Luxembourg’s system of education and research is perhaps less-well known and much more under-appreciated. Luxembourg is of course known for its leadership in the space industry (https://www2.deloitte.com/lu/en/pages/technology/articles/luxembourg-space-industry-companies.html ) but it also exports iron and steel, machinery, nuclear reactors, boilers, and plastics. The national innovation agency, Luxinnovation, works in concert with the University of Luxembourg to promote the flourishing of the national economy and the welfare of its citizens. The school system focuses on literacy with most graduates achieving tri-lingual proficiency. The University of Luxembourg, founded in 2003, is ranked 12th in the Times Higher Education Rankings for universities less than 50 years old. Young citizens have a choice too in pursuing relevant education credentials, with lycees techniques offering practical three-year Brevet de techicien superiur credentials that are respected and recognized. The discipline of a parliamentary multi-party system of proportional representation with compulsory voting and relatively liberal standards for national referenda assures that national investment in human capital is productive and conducive to human flourishing in stark contrast to the Laputan universities and Balnibarbian bureaucracy in the USA.

    Patching with band-aids is pointless. The systemic production of a third-rate elite class is the fundamental problem.

    • In what sense is Switzerland the world’s most populist country?

      My favorite Switzerland quote–Orson Welles as Harry Lime in The Third Man:

      Harry Lime: In Italy for thirty years under the Borgias, they had warfare, terror, murder, bloodshed. They produced Michaelangelo, da Vinci, and the Renaissance. In Switzerland, they had brotherly love, five hundred years of democracy and peace, and what did they produce? The cuckoo clock.

      • Switzerland is the most populist country in that it has has a tradition of direct democracy. For any change in the constitution, a referendum is mandatory (mandatory referendum); for any change in a law, a referendum can be requested (optional referendum). In addition, the people may present a constitutional popular initiative to introduce amendments to the federal constitution. The people also assume a role similar to the constitutional court, which does not exist, and thus act as the guardian of the rule of law. People in Switzerland therefore are largely equal and the non-elite enjoy freedom and opportunity and can demand high institutional performance. https://freedomhouse.org/country/switzerland/freedom-world/2020

        This freedom has enabled them to produce top performance in economic competitiveness, personal freedom, political pluralism, and rule of law contributing to being rated as the overall best country in the world by US News and World Report.

  8. An old problem that expanded starting in the 1980s. Here is a question from 1923, and a failed projection that the “degree-hunters” seeking a PhD for a faculty sinecure would fade in favor of scholars. Nor has the mixed university institution separated it graduate programs from the less scholarly undergraduate college.

    =================
    “The university part of our mixed institutions consists of a graduate school, devoted to teaching and to research, certain professional schools in law, medicine, engineering, teaching, and, in some institutions, to theology. The graduate schools, apart from the professional schools, have suffered in considerable measure from the fact that they have been attended by a large body of students who are not primarily scholars or investigators. For the last twenty or thirty years every ambitious American college has felt that it could not maintain fair academic dignity unless its teachers were able to write after their names Ph.D. The graduate schools have been invaded, therefore, during the comparatively short period of their existence by an army of degree-hunters who desired the degree of Doctor of Philosophy as a preliminary to obtaining positions as teachers.

    “The mingling of college and university has its disadvantages for the undergraduate college no less than for the graduate university to which it is bound. The most serious is the weakening of the college sense of responsibility for good teaching. A false notion of research in the conglomerate institution has gone far to discredit the good teacher and to weaken the appreciation of the fact that the chief duty of the college is to teach.

    “Notwithstanding these drawbacks the graduate schools and the professional schools that constitute the true university part of our American institutions have steadily grown in scholarly qualities, in facilities for study, and in the application of the fundamental qualities that make for sound scholars and sound investigators. In the orderly process of development, the time will come when the degree-hunters will lessen in numbers and when the graduate and professional schools will represent essentially what the university represents in Europe-a school whose students have already had their undergraduate experience of sports and of class rivalries, as well as their grounding in fundamental subjects, and have now entered upon a life with the primary purpose to bend themselves intelligently and energetically toward study, toward research, toward professional attainment.

    “Whether the universities can ever become universities in this sense, as long as they are mingled with, and in the public mind overshadowed by, their highly populated undergraduate colleges, with their spectacular contests in athletics and with the manifold activities in which students compete, is a question which cannot be answered at this moment. It is impossible for the wisest man to say to-day whether the conglomerate institution which we call a university will develop gradually into a form under which its undergraduate activities are separated from its scholarly and professional work, or whether the popular interest in the by-products of undergraduate life will be so great as to keep the university side of the institution in that twilight zone of public interest in which it has hitherto lived. Can a true university, devoted to scholarship, to investigation, to high professional training, be developed out of a conglomerate institution whose undergraduate activities are mainly athletic, social, and competitive? ”

    –Are Our Universities Overpopulated?
    BY HENRY S. PRITCHETT
    President of the Carnegie Foundation for the Advancement of Teaching
    Scribner’s Magazine Vol. 73, 1923, p556

    ==============
    And in this same period, we can find the lament of the decline of respect for the teacher in favor of “research men”.

    “We are informed by many that education is failing us. And well it may he so, if producing books is eulogized and repaid by advancement, while the efforts to produce men are scoffed at. It has been dinned in our ears that education must save us at the present juncture. To which, if true, I reply that, unless we regain the love and art of teaching, we are lost.

    “The truth is that at present the teacher exists by sufferance only, and stands against the current in the scholarly fraternity-a fact recognized by students as well as by faculty. For the educational field has been preempted by the so-called “research men.” Their standards of scholarship have been set up as the only norms.”

    –The Ban on Teaching by AN Instructor, Scribner’s Magazine, Vol 73, 1923

  9. Perhaps not 1,000 teams but we do have 350 more pro-like basketball teams – D1 colleges. In the right setting – March Madness – these far less skilled players are quite fun to watch.

    If “…The only solution is a completely different mechanism and institution of accountability” then I agree with Niko…do it in a free market direction.

    Maybe “Genius Tank” — a March Madness meets Shark Tank tournament for Phd students. Present their original research to “rebel experts” – Arnold for example in econ – who look for great scholarship.

    Viewers are taught that competitors take a big hit back at the university, where department heads despise the breach of Phd protocol, so they have skin in the game, and perhaps get rebel status just for entering.

Comments are closed.