The reliability hierarchy of expert opinion

There’s been a mild amount of angst on the internet in recent days over a couple of well known science communicators espousing nonsense on matters outside of their field.  The first was Bill Nye, who made comments about philosophy that most charitably could be described as uninformed.  The second was Neil deGrasse Tyson’s tweets about biology expressing problematic views about sex and celibacy.

(See the conversation responses to these tweets if you’re not sure why they’re wrong.)

I’m actually a big fan of both Nye and Tyson, but they’re both human, and so will occasionally have incorrect views.  I’ve noted before that when a scientist (or in Nye’s case, an engineer) starts giving their opinions on things, it pays to remember that when they’re speaking on subjects outside of their field, they’re basically just well educated lay people.

But thinking about this recent couple of kerfuffles reminds me of the many cases where scientists, or other experts, do put forth ideas, often in a much more serious manner than the offhand remarks here from Nye or Tyson.   This led me to ponder the hierarchy I use for evaluating such claims.  This is admittedly my own personal views as a skeptic, but I’ve found it to be a pretty reliable guide to evaluating claims.

  1. When an expert is discussing the consensus in their field, we should pay attention and heed what they say.  They’re most likely right.
  2. When an expert is putting forth a radical new theory in their field, they might be right, but they could also be wrong.  Indeed, the history of radical new theories is that most eventually turn out to be wrong.  Still, the chance of them being right is high enough to warrant listening to them.  (It helps to get information from multiple experts in these cases.)
  3. If, several years ago, an expert put forth a radical new theory that the majority of experts in their field have reviewed and rejected, but the expert continues to push it, they’re not only most likely wrong, they may be a quack.
  4. When an expert is describing the consensus in other fields, unless there are issues with their research or honesty, there’s no particular reason to think they’re wrong.
  5. When an expert is putting forth a radical new theory in another field that they are not an expert in, while it’s conceivable that they might be right, the probability is so low that it’s usually safe to assume they’re wrong.
  6. If, several years ago, an expert put forth a radical new theory in a field that they are not an expert in, which the majority of experts in that field don’t take seriously, but the expert continues to push it, they’re virtually certain to be wrong, and may well be a quack.

Are there examples that violate this hierarchy?  I don’t doubt it, although frankly, I’m hard pressed to come up with one.  Most of the examples that people reach for are typically in the early history of science (Galileo) or experts in the relevant field (Einstein).  Still, the history of science is vast and I’m sure there are examples.

But when someone with scientific or other credentials is in front of us espousing their views, it pays to remember that, except in cases 1 and 4, the statistics are against them.

Unless I’m missing something?  Are there any prominent counter-examples I’m overlooking?

This entry was posted in Society and tagged , , , , . Bookmark the permalink.

19 Responses to The reliability hierarchy of expert opinion

  1. john zande says:

    I tend to agree with Nye and Tyson and Hawking on philosophy. If you’ve ever been in a conversation with a Christian philosopher you quickly see that it is all pure nonsense. It has produced not a line of verifiable truth. Not. A. Line. It’s nonsense, and the nonsense is celebrated. Dr. Maarten Boudry at Ghent University’s Department of Philosopy & Moral Sciences set out to prove just this. In 2011 (I think?) he penned an utterly nonsensical, anti-Darwin lecture abstract full of theological gibberish and submitted it to two Christian philosophy conferences, including the Reformational Philosophy Association’s “The Future of Creation Order.” Astonishingly, the abstract was accepted without a moment’s hesitation, and Boudry’s alter ego, Robert A. Maundy of the fictitious College of the Holy Cross, was slotted in as a speaker at both conferences… invitations he (of course) declined after announcing it all a hoax.

    This is the abstract:

    The Paradoxes of Darwinian Disorder.

    Towards an Ontological Reaffirmation of Order and Transcendence.

    Robert A. Maundy, College of the Holy Cross, Reno, Nevada

    In the Darwinian perspective, order is not immanent in reality, but it is a self-affirming aspect of reality in so far as it is experienced by situated subjects. However, it is not so much reality that is self-affirming, but the creative order structuring reality which manifests itself to us. Being-whole, as opposed to being-one, underwrites our fundamental sense of locatedness and particularity in the universe. The valuation of order qua meaningful order, rather than order-in-itself, has been thoroughly objectified in the Darwinian worldview. This process of de-contextualization and reification of meaning has ultimately led to the establishment of ‘dis-order’ rather than ‘this-order’. As a result, Darwinian materialism confronts us with an eradication of meaning from the phenomenological experience of reality. Negative theology however suggests a revaluation of disorder as a necessary precondition of order, as that without which order could not be thought of in an orderly fashion. In that sense, dis-order dissolves into the manifestations of order transcending the materialist realm. Indeed, order becomes only transparent qua order in so far as it is situated against a background of chaos and meaninglessness. This binary opposition between order and dis-order, or between order and that which disrupts order, embodies a central paradox of Darwinian thinking. As Whitehead suggests, reality is not composed of disordered material substances, but as serially-ordered events that are experienced in a subjectively meaningful way. The question is not what structures order, but what structure is imposed on our transcendent conception of order. By narrowly focusing on the disorderly state of present-being, or the “incoherence of a primordial multiplicity”, as John Haught put it, Darwinian materialists lose sense of the ultimate order unfolding in the not-yet-being. Contrary to what Dawkins asserts, if we reframe our sense of locatedness of existence within a the space of radical contingency of spiritual destiny, then absolute order reemerges as an ontological possibility. The discourse of dis-order always already incorporates a creative moment that allows the self to transcend the context in which it finds itself, but also to find solace and responsiveness in an absolute Order which both engenders and withholds meaning. Creation is the condition of possibility of discourse which, in turn, evokes itself as presenting creation itself. Darwinian discourse is therefore just an emanation of the absolute discourse of dis-order, and not the other way around, as crude materialists such as Dawkins suggest.

    Liked by 1 person

    • Steve Ruis says:

      Lovely reminder, John. It is also interesting that “learned journals” are defending the money they charge for copies of published scientific works (typically $35 per article, with the authors getting zilch) with how much work is done vetting and editing said articles. Others are saying that research funded by the people (aka government) should be freely available by the public (maybe with a nominal fee to support the posting. We can see what the vetting process involves in this instance. Going through the motions, anyone?

      Liked by 2 people

    • Certainly there is bad philosophy out there. It’s probably fair to say that most of it is bad, with the caveat that there is little agreement on which portion is good.

      More specifically, what you’re describing seems more like theology than mainline philosophy. (Although when you think about it, what is theology but philosophy premised on God’s existence?) It’s worth noting that Boudry is himself a philosopher, albeit of a more skeptical nature. I think we’re being too blunt if we dismiss people like him and Daniel Dennett when dismissing the philosophy you’re talking about.

      Liked by 2 people

  2. Steve Ruis says:

    How can one be wrong about philosophy? (It is all opinion, no?) One could be wrong about which philosopher said what or had such and such an idea, but. . . .

    The rest seems spot on. There used to be fabulous polymaths who were experts in many fields, but the effort required today to learn enough to be an expert in more than one or two fields is a barrier fairly insurmountable, I think. Whereas a newcomer to a field with a great deal of mental horsepower used to be able to make a dent in many fields that weren’t at all well-fleshed out, this can no longer be done as most fields have attracted substantial thinkers who have already picked most of the low-hanging fruit, making easy progress much harder.

    Liked by 2 people

    • I agree that philosophy is basically all opinion, but not all opinion is of equal quality. Opinion grounded in rigorous open logic means more to me than opinion grounded in intuition or emotion. Of course, without the reality check of science, philosophy will always operate at a disadvantage in epistemic authority.

      Totally agree on polymaths. Being a polymath after the 18th century, for the reasons you lay out, became very difficult.

      Liked by 2 people

  3. James Pailly says:

    Looks like a good hierarchy to me. I’ve encountered a few new theories where the author sort of discredits him or herself by whining about how the scientific community is conspiring against them. I know such things are possible, but they’re extremely rare.

    Liked by 1 person

    • I agree. Indeed, it’s so rare that, for me, anyone making that kind of claim has put themselves into a position of having to prove that they’re not a crackpot.

      Liked by 1 person

    • Tom W. says:

      Somewhere out there in the Internets sits a most illuminating (if a little brash) guide of quackpottery with a scoring system for each faux-pas: 10 points against you for saying that Einstein was wrong, 5 points for whinging about the conspiracy to ignore your ‘life’s work’, etc. Funny but oh-so-true, sadly.

      Liked by 2 people

  4. Steve Morris says:

    I tend to be uncomfortable about the idea of an “expert”, having been an “expert” in several fields during my life. In my experience, people are expert not in one clearly identifiable subject, but in a collection of extremely specific topics, both within their own “field” and within other “fields”.

    If you plotted the zones of expertise of a typical “expert”, it wouldn’t be a clearly located blob, but a fractal structure, stretching out in many directions, and full of holes. Two academics working in adjacent offices would typically have a small intersection in their common areas of expertise. Does this make any sense?

    So even the idea that Nye is an engineer, or Tyson an astrophysicist, are just very crude approximations to the truth.

    My point? Question everyone.

    Like

    • Steve Morris says:

      Do you think I’m starting to sound paranoid?

      Like

    • Oh, I totally agree. It’s always good to get multiple perspectives to ensure the one person you’re reading / listening to isn’t an outlier.

      Given that we’re all human, and have limited time and resources, we’re dependent on others for a lot of information. Given that stark reality, I think giving more weight to experts in a particular area for information on that specific area is rational. But always get the views of multiple experts, or keep your views tentative if you’ve only heard from one.

      That said, I’m more likely to trust the collective judgment of physicists on, say, the standard model, than I am the judgment of neuroscientists. But I’m much more likely to trust the collective judgment of neuroscientists on neural circuitry than I am physicists.

      Liked by 1 person

  5. Tom W. says:

    By the way, didn’t anyone think to not take Tyson’s tweets seriously? They’re jokes, and pretty funny if you think about it. They’re like empty little witticisms one might interject at a cocktail party. They’re what Dennet might call “deepities”. Didn’t read Nye’s remarks yet though, so I can’t say anything about those.

    Like

    • Just before the celibacy tweet, Tyson did a tweet that implied he was just teasing the biologists. Maybe it was all in jest, but the first tweet didn’t really give any signal that it was. I’ve been following Tyson for a while and I don’t recall him doing jokes with dubious scientific assertions before. That said, I’m not inclined to join bandwagons of people who freak about the odd clueless remark on Twitter.

      Nye’s comments were a bit more problematic. He made them in a Big Think video responding to a philosophy student asking, if I recall correctly, for career advice. Now, the student probably submitted that question to find out Nye’s views rather than for any real need for advice, but while Nye worked humor into his response, it overall came out as an accurate representation of his views.

      Liked by 1 person

    • hopdavid says:

      It isn’t just the occasional off hand comment on Twitter.

      Bush’s post 9-11 speech was incorporated into Tyson’s naming rights routine. Tyson would repeatedly performed this shtick. In Tyson’s account, just a week after 9-11 Bush gave a speech trying to elevate Christians above Muslims. Problem is, Bush never delivered the xenophobic rant Tyson remembers.

      Tyson’s column for Natural History weren’t off hand tweets. If memory serves Natural History is a monthly magazine, so Neil had some opportunity to review and proofread his submissions. These columns contained numerous errors.

      I am compiling a list of Tyson errors.

      Like

Your thoughts?

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s