Audit college courses for intellectual rigor

Toby Young proposes an intellectual trade union to protect people from the progressive mob.

If a member is targeted for defenestration by an outrage mob, it will be the union that comes to their defense—the organization, not the other members. I don’t mean it will provide the person in the dock with legal representation. To offer legal insurance of that kind would make the membership dues prohibitively high and trade unions that do offer that service usually rely on internal officers to provide support to members involved in legal disputes—not the type of support that would be much help in a complicated case. Rather, the union will provide them with access to an approved list of defamation and employment lawyers, expert guidance on how to crowdfund their legal costs, access to lists of potential donors, PR advice on how to generate favorable media coverage—most importantly—access to a network of sympathetic colleagues, many of whom will have been through a similar ordeal.

I have a more aggressive idea. Set up a private organization to audit college courses for intellectual rigor. Call the organization the Collegiate Humanities and Social Sciences Rigor Audit Bureau, or the Audit Bureau for short. The scope would only include courses in the humanities and social sciences. Foreign languages, art, and music would be out of scope. “____ studies” courses would be in scope.

I am basing this idea on three assumptions and one comparable example.

The first assumption is that many of these courses are being taught to undergraduates, including those at “name-brand” colleges and universities, with extreme ideological bias and wooly-headed thinking.

The second assumption is that this is a serious enough problem for society that it is worth putting in some effort to try to fix it.

The third assumption is that conservative intellectuals are misguided if they frame this as largely a political problem. A better framing would be to focus on intellectual rigor.

The comparable example is the College Board audits of AP courses. You may not know this, but about five years ago the College Board required every high school AP teacher to submit a syllabus and some other materials to demonstrate that the course in fact deserved the “advanced placement” designation. I assume that the College Board did this because they noted that the AP “brand” on a course had become a powerful quality indicator, and they wanted to ensure that the brand’s reputation for quality was deserved and maintained.

The vision is that having a course labeled as rigorous by the Audit Bureau would come to be so desirable that students are better off with degrees from mid-range schools with those courses on their transcripts than with degrees from name-brand schools with courses on their transcripts that are not certified as rigorous. As a result, college professors at all institutions, including name-brand schools, feel pressured to offer courses that meet the standards of rigor.

Institutions could either establish their own standards for rigor or default to model standards created by the Audit Bureau. The standards need not be extremely detailed, but they should be clear enough that an auditor can test whether or not a course meets the standards. For example, Minerva’s standards are certainly clear enough to be audited against. But standards that are far less detailed would suffice.

I would want to see standards that include two elements.

1. Students clearly are exposed to differing points of view. For example, an economics course that is oriented in a very free-market direction also should include on the syllabus readings from Dani Rodrik or some equivalent critic.

2. In order to receive a grade of B or higher, the student has to provide written work (essay questions on exams and/or papers) that demonstrate substantive knowledge, the ability to communicate ideas, and the ability to both formulate arguments and state possible objections.

For comments, I would prefer for now that you spare me your opinion of this idea. Instead, imagine that someone were out to implement it. Which prominent intellectuals should they try to get on board, and why?

23 thoughts on “Audit college courses for intellectual rigor

  1. This sounds like something the Haidt / Heterodox Academy crowd would be interested in, but, (a) they’ve got enough on their plates, (b) I don’t get the impression they’ve achieved anything, and (c) what you really need are some high status committed (i.e., unquestionably anti-Trump) progressives who a combination of Conquest’s First Law experience with the downsides of low-rigor education and who are able to provide some kind of progressive rationale and “good guys vs. bad guys” narrative (one might have to hold one’s nose and bite one’s tongue about that) for why standards of rigor must be raised and verified.

    So, for example, I would like to see Arne Duncan, David Whitman, and John King leverage their anti-for-profit-colleges bona-fides to argue that low-rigor courses from a few bad apple professors and institutions are performing a similar disservice to unsuspecting members of disproportionately under-represented groups, and that those students are leaving some courses unaware that they are unprepared to become genuinely effective advocates and activists for their cause because they won’t know how to help by countering bad conservative arguments whenever they come across them, for instance, on Facebook or Twitter or Thanksgiving dinner.

    Get someone from the Harvard or Yale faculty of X-studies to do a random evaluation of the way their subject is taught at some low-ranking institution, and thus, in shock at the low standards of rigor, invite them to become part of a prestigious standards-setting committee that plugs into the audit program.

    David Coleman and Bill Gates could explain why standards of rigor and audits are important for the College Board, also with some progressive rationalization story about giving everyone an equal chance to reach their potential and achieve excellence in personal development.

    This is not a criticism of your idea, but of my own idea above, which is that Conquest’s Second Law holds, and while it’s probably essential for both credibility and effectiveness, putting a bunch of leftists / progressives in my hypothetical audit organization is likely to abuse the dis-accreditation power and turn into an inquisition for progressive orthodoxy eventually.

    There might be some token presentation of mildly conservative points of view, but everything else dumped outside the Overton Window in the ‘flat earth theory’ category of definitively disproven things not worthy of including for the sake of ‘balance’. The only way to deal with that problem is as in the adversarial trial process, with equal time for a genuine ‘Devil’s Advocate’.

    Consider, the “urban legend debunking” website Snopes, (which is kind of like auditing in a way) could at least theoretically be seen as a neutral enterprise, but lately has stared targeting the conservative-leaning satire site the Babylon Bee as if it were part of a deliberate misinformation campaign (probably Russian Collusion, or something.) If College-Audit-Board goes that route because of recruiting too many progressives as I suggested above, then the danger is that it just becomes a sword in the hands of the enemy.

  2. Arnold, Please forgive me if I stray from your instructions.

    Independent exams are more effective than course audits.

    Highly selective colleges count AP courses for credit towards the undergraduate degree only if the student achieves an excellent score (5/5 or at most 4/5) on the independent AP subject matter test upon completion of the course.

    Accountants, attorneys, actuaries and various other professionals must pass independent exams to certify threshold competence, after completion of their studies.

    Colleges fear independent exit exams and objective competence exams. Show me an elite college that says, ‘Our students take the GRE when they elect a Major, and then again as an exit exam for the Major. The change in score is an indicator of what we have accomplished together in the Department.’

    The Swarthmore College Honors Program model, in which Faculty from other institutions evaluate senior theses, is vulnerable to two problems: (a) The rot is widespread. (b) Selection of external evaluators is readily captured by Faculty networking across institutions in specific fields.

    Why is there strong demand for independent tests to assess academic readiness at the moment of admission, but not at the moment of college graduation, especially as the signal gets noisier and fuzzier?

    Is there a hundred dollar bill on the sidewalk?

  3. I agree with John Alcorn that auditing may not fix the problem. I teach AP Calculus and AP Statistics, and I believe I passed the “audit” by just basically emailing a previous teacher’s syllabus to the College Board.

    My AP classes are quite good, in my opinion, and I work extremely hard to make sure all of the relevant content is covered, but the audit did almost nothing to ensure that this was would be the case.

    I could easily pass the “audit” and teach basically nothing to the kids, which I bet does in fact happen in some AP classes.

  4. I think the best people to get on board would be the current rating organisations that the university administration already tries to appease. I think US News in the states? If they demanded a high score on this rigor audit to rank highly, the administrations would attempt to comply.

  5. > For comments, I would prefer for now that you spare me your opinion of this idea.
    > Instead, imagine that someone were out to implement it. Which prominent
    > intellectuals should they try to get on board, and why?

    Off the top of my head:

    Steven Pinker–he’s very suavely and subtly anti-woke. He may be too suave to join up–but maybe not. He seems to believe that students should know something in particular.

    David Gelernter–he’s cranky, funny, and hates PC. and he’s a rebel. He’d be in the back of the class reading a novel of his choosing–but he’s on your side.

    Richard Vedder–he has long railed against bloat and the emptying of content from college curriculum.

    Mark Bauerlein–he sometimes hits the same notes that Prof. Vedder hits. his view is from the English Department, where he thinks nobody reads what used to be the works of the English Department curriculum.

    Sir Roger Scruton–he is a conservative in the best sense of the world–he wants to conserve what we have.

    Peter Thiel–he thinks college is so bad that he’s paying people to skip it entirely and just work in software startups instead.

    Charles Murray–in _Real Education_ he noted that he can address an audience of college graduates but it doesn’t mean they know anything about statistics–just to take one example.

    E. D. Hirsch–he believes in content rich curricula

    Murray Sperber–in _Beer and Circus_ he described the “mutual non-aggression pact” in which professors leave the students alone so they can get more research done. And he’s not a died in the wool intellectual–he may have wandered into academia partly by accident.

    Richard Arum and Josipa Roksa–they discussed the problem in _Academically adrift_.

    Kevin Carey–but he will probably favor exams. Give lots of exams and make them tournaments.

    Warren Treadgold–he just wrote about the problem, or a similar one.

    Victor Davis Hanson–he’d probably support it.

    Frederick M Hess–he writes about the problem.

    • Many of these came up on my list. However, Pinker may be the only one who would make it harder for the “woke” to dismiss the whole exercise without consideration. In any event, whoever accepts the position(a) will need to join Toby Young’s “union” right away…

    • I’m just going to list more prominent intellectuals. Prof. Arnold said he wanted us to name some.

      Deirdre McCloskey. I’ve usually felt like I was in the presence of a great mind when listening to Don (later Deirdre). I heard directly or indirectly that Don had tired of teaching undergraduates at Iowa because they weren’t motivated enough on average. The rational thing was to give up.

      Charles Sykes. The guy who wrote Profscam. Charles J. Sykes.

      I would list Robert Conquest except he died a few years ago. So, sticking to people who are alive…

      Niall Ferguson

    • One challenge in answering Prof. Arnold’s question is identifying intellectuals. I’m not in academia at the moment, though I still do a bit of serious reading.

      There seem to be different classes of thinkers and writers and speakers.

      * Intellectuals, Career Academics and Scholars in Academe

      * Journalists

      * Pundits and Columnists

      * Politician and Politicos

      * Businessmen / Businesswomen / those in industry.

      Some of the best scholars in the country are quietly going about their work and most of us have never heard of them. They may be the true intellectuals. Other people are noisy but have few useful ideas. The public might name them as intellectuals.

      The older one gets, I would say, the more scholars and academics lean toward rigor. Either that, or they give up.

      Young scholars may be enthusiastic about teaching. Personally, it took me a while to notice that most of my students wouldn’t commit anything to memory unless provided with incentives–they weren’t stupid or lazy, but they were just conserving their energy and had their minds on other things.

      The problem was that they didn’t know anything and couldn’t think hard about issues. The incentive structure permitted that to continue. Often they were interested in learning to do something useful–but mundane specifics were not something they had an interest in, though they might have opinions that far outweighed their knowledge.

      I’m trying to address Prof. Arnold’s question. Sorry for any deficiencies in tone, or typos.

  6. The point raised by Arnold seems reminiscent of the conditions that gave rise to the proliferation of “Think Tanks” beginning in the early ’70’s; especially those formed to offset the predominance of a singular class of academics in fields of “Public Policy.”

    Today there are many kinds of “Tanks,” with all kinds of “intellectual” fish, many blowing nothing but word bubbles. So that may follow this project as well.

    For personnel: stay clear of the “education” field (industry).
    Probably go to those of our military classes who have had the specialized “institutional” exposures, but have beaten out the teaching by personal learning – people like George Petraeus. Surely there are a number of others; and they could gather about them similar mind-sets from their exposures to individuals with other backgrounds, but sufficient LEARNING attainments (probably of highly individuated learning skills).

    Next, some retired jurists might be a “pool.” Further down the scale, some lawyers (even a group out of the Federalist Society, but not as part of its activities).

    Lastly, “older” (real -not just writers) historians but NO social studies, economists, physical scientists or their like.

  7. Funny thing, you can objectively test language better than any non-math discipline. Either you can order a meal, talk about the weather, and describe your job, or you’re not fluent.

    I’ll give an example. 20 years ago, I took four years of mandarin (no as an Econ major), including a semester in Beijing. Ten semesters, effectively.

    Shortly before graduation, I was invited to sit a two-hour exam by some annex of the DoD, and we were told if we did well enough we’d get some official certificate of proficiency. I didn’t expect that, but I was still surprised by the difficulty of the test. I knew my education wasn’t a sham because I’d spent four months navigating China without a phrase book (and before English was widely spoken there), but it reinforced that my level was “Tourist Proficient” rather than “Professional.”

  8. I once suggested to the National Association for Scholars a similar idea, that they should propose standards for what college graduates majoring in various disciplines in the humanities and social sciences should have read and should know how to do. (For example, no matter what the discipline, in my view a college graduate should know what sources to consult to find out what the leading thinkers on a particular question within the discipline say about it. At present, few graduates have this skill.) They never replied.

    The Intercollegiate Studies Institute publishes books with the title A Student’s Guide to [subject name] that partly do what I have in mind.

    I would prefer for such a standard-setting, or auditing, venture to be unaffilated with the professional societies in particular disciplines. The American Chemical Society can recommend a core curriculum for chemistry majors without debates (so far, at least) over whether students are reading too many textbooks or papers by white men, whether inorganic chemistry is colonialist, etc. The same does not apply in, for instance, English, where it’s possible at a number of supposedly top universities to major without ever having studied Shakespeare. So, rather than the Modern Language Association setting a standard curriculum that includes I, Rigoberta Menchu and excludes Shakespeare, I would prefer to leave the way open for multiple standard-setting bodies.

    Finally, I agree with John Alcorn that independent exams are more effective than course audits. Here again, I prefer leaving the way open for multiple standard- setting bodies. What the College Board did several years ago with revising standards for AP courses in American history and modern European history shows the danger of one organization being so dominant. Compared to the old standards, the College Board’s new standards are both unncessarily detailed and they neglect key people and themes.

    • I think that is the consensus here, and I agree.

      A big problem is that audits measure inputs, while the exit exams measure outputs. We are interested in the outputs, the outcomes.

      Measuring outputs is more sensible, but harder. Why it’s harder is a topic for another post. My sense is that generally what is measured in education is inputs. That’s one cause of sub-optimal performance. Adding more inputs to a system with a lot of slack is “like pushing on a string.”

      A useful tactic would be to identify interest groups (such as employers) that have an interest in assessing the quality of outputs. There could be more focus on things such as CPA exams or the exams that one takes to pass boards, licensing exams, etc.

      • The USNWR rankings for national liberal-arts colleges give roughly equal weights to inputs and outputs, but the output indicators are blind precisely where Arnold Kling wants to shed light: academic rigor and philosophical openness.

        The input indicators amount to 45% of the score:
        • Faculty resources = 20%
        • Student selectivity = 10%
        • Expenditures per student = 10%
        • Alumni giving = 5%

        The output indicators amount to 55% of the score:

        Graduation & retention rates = 35%
        Reputation (surveys of college administrators and high-school counselors) = 20%

        (Note: Alumni giving really is both an input and an output.)

        Graduation & retention rates might be negatively correlated with academic rigor! (Students like to say, ‘It’s hard to get into Harvard, but easy to graduate.’) However that may be, ability bias (selection for talent in admissions) naturally increases graduation rates.

        The USNWR inclusion of reputation scores from peer surveys as output indicators perversely might reinforce the entrenched, lopsided ideological equilibrium at elite colleges.

        • I think those output measures are highly correlated with selectivity and the student attributes before they are omitted.

          In that sense we are back to the James S. Coleman problem. We are measuring outcomes that are driven by pre-existing variables including household attributes. The admission cohort to Texas or Michigan is highly non-random.

          We don’t have enough colleges that are like a military boot camp, where they try to bring everyone up to a certain standard, with the belief that almost anyone can reach that level given adequate training, practice, supervision, exercise, endless drill, regular inspection, etc.

          That’s where we see value added and good output. Don’t ask me, I never went through boot camp. But maybe it’s a valid point anyway.

          • Correction:

            I think those output measures are highly correlated with selectivity and the student attributes before they are *admitted* to the institution in the first place.

  9. Who might support it: STEM faculty who are tired of competing for students and therefore funding with departments that are glorified seminaries, and would like to see the fat trimmed (as a great many liberal art professors are probably incapable or unwilling to teach a class that would pass an audit).

    I’m pessimistic that such an idea would get much support in practice though. If seriously proposed at an institution, it’d likely quickly get labelled by the obvious opponents as an affront to intellectual freedom or a professor’s sovereignty over their class, and even most academics that might favor it in a vacuum could find themselves peer pressured into to opposing it.

  10. I think you are inflating two separate issues here: grade inflation and political bias. Grade inflation has impacted all disciplines since the 1990s, though you could argue it has had a greater impact on liberal arts. I would be all for a more rigorous curriculum for liberal arts majors, but that is an entirely separate issue.

    What i’ve come to find is that because you don’t share the ideological views with the majority of university faculty, you want to throw everything and the kitchen sink at the university system until more people think like you do. Fine, that is your prerogative.

    What is much less talked about on this blog is why there is this tendency for the university system to lean progressive. There is no serious engagement of the historical progressive movement on campus and its modern day manifestations. The only thing talked about is “students are snowflakes” and “conservative speakers banned from campus” without regard to the fact that liberal speakers are too banned in equal numbers.

    Universities are progressive because they are centers of moral and cultural arbitration. They are tackling issues like technology, privacy, economic inequality, mental health, and racial equality. Students and faculty alike are engaging these topics in serious ways, they just aren’t coming to the same conclusion as conservative thinkers would like them to. Here is my suggestion: for issues which both sides agree are problems, come up with a better solution. For issues which conservatives don’t believe are problems but liberals do, engage with serious academics on the issue to try and understand why.

    • I can’t speak for Dr. Kling, but I think the ‘why’ is straightforward enough: virtually everyone of campus is on the left, and institutions recapitulate their own biases; e.g. will tend to preferentially hire faculty that share the institution’s worldview, to promote events and groups sympathetic to said worldview; and this deters people who don’t share the institution’s worldview from joining it, or from being active should they join it. There’s a fairly intractable positive feedback loop. I would argue that once an institution becomes even moderately disproportionately comprised of a worldview, it will tend to slide into this sort of self reinforcing bias pattern without a great deal of self-discipline.

      As for why academia started leaning left in the first place so this cycle could take place, obviously progressives will say “because we’re just smarter, and so of course we tend to be over-represented in academia.” I’d say rather it’s because either academics’ material incentives align well with progressivism (it makes sense that employees at state funded enterprises look more favorably on state intervention) or because teaching (and certain academic disciplines) appeal more to the progressive psyche than the conservative one (referring to the evidence that one’s innate psychological traits are highly predictive of one’s politics).

      I would also point out that the non-engagement problem is clearly a two-way street. I was a humanities/social sciences major and, outside of Econ classes, I never had a single professor honestly engage with what one would call a conservative position (Econ professes of course were more ‘conservative’/libertarian, but they taught Keynesian models or market failures as proponents of these concepts would recognize them). Every time politics came up, what came next could’ve been invariably prefaced with the statement “first let’s assume I’m right…”

      And I’m honestly more perturbed by the academic left’s non-engagement, if only because everyone doesn’t have to read right of center blogs, attend socially conservative evangelical church sermons for four years, or even take any traditional economics classes to get a job; but even an engineer, in order to graduate, now generally has to take at least one or two liberal arts classes, almost without exception, taught in a thoroughly left wing biased manner.

      • I always assumed the primary reason why academia is more Progressive is a huge self-selection problem that potential conservative professors graduating college at 22 look towards the private sector as they pay more. When graduating with Economics degree at 22, I remember that Econ Phd earned less than Econ Bachelors, and Econ Master degrees which was Econ Phd went to academia more.

        Then the we people like us might take place more over the decades.

        • I think this may fit in with the category of psychological reasons: people with conservative disposition tend to value monetary rewards more than those with progressive dispositions. Of course there may be ‘social’ reasons as well; conservatives may be more likely to have larger families that require more financial support.

  11. Accountants, attorneys, actuaries and various other professionals must pass independent exams to certify threshold competence, after completion of their studies.

    Most people outside the business don’t realize but just about everyone who intends to take the bar exam takes an intensive “bar review [ahem] course” between the end of law school and the exam. Basically, it is a giant cram course. As with all cram courses, you forget most of it quickly. So the bar exam doesn’t tell much about what a person actually will know when he or she starts practicing.

    On the other hand, Arnold only says that the exams “certify threshold competence”. A low bar but an important one.

Comments are closed.