Data Lit Exam 1:

Occam’s Razor

Amongst competing explanations for the same data, choose the simplest (but it might be wrong): 

• A linear increase with a bit of noise

• Domain knowledge can be important

Fermi Estimation

• “Back of the envelope” problems:  Use some existing knowledge to provide a rough estimate of a value, usually in order to provide a sanity check.

  • Ex: how many ping pong balls can fit in a 747?

• Unusual problems that require real world knowledge and mathematical reasoning 

• Often, knowing an exact answer is impossible.

Known Known

We are aware that… we know these thinngs

Known Unknowns

We are aware that… we don’t know these things

Unknown Known

We aren’t aware that… we know these things

• can introduce bias

Unknown unknown

We aren’t aware that… we don’t know these things

• create risks that can’t be accounted for

“Keynesian Beauty contest"

• Named after economist John Maynard Keynes. Functions as a description of forces at play in the stock market.

• Cognitive hierarchy: 

  • Level 0: you randomly pick a number 

  • Level 1: you think that others will randomly pick a number so then ⅖ the average will be…?

  • Level 2: you think that others will be level 1, so then ⅖ the average will be…?

  • Level infinity: Nash Equilibrium (0)

Features of a system that make it hard to understand: 

• Missing or incorrect data 

• uncertainty/probability

• High dimensionality (i.e. many moving parts) 

• Layers of recursion and the influence of people

This is further complicated by: 

• Cognitive biases 

• Societal influences 

• Motivation to misunderstand

What Is Science?

A knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method.

• Difficult 

• Evolving 

• Argue over 

• Steeped in uncertainty 

• Based on empirical tests driven by theories 

• Grounded in a shared argument on truth 

Demarcation Problem

“The demarcation problem in philosophy refers to the question of how to meaningfully and reliably separate science from pseudoscience. Both the terms ‘science’ and ‘pseudoscience’ are notoriously difficult to define precisely. 

• The demarcation problem has a long history.

Necessary & Sufficient Conditions (Gold Standard of Definitions)

Necessary: everything that is X meets conditions Y

Sufficient: Everything that meets condition Y is X

You have a clean definition of things that are jointly necessary and sufficient

• Hard to come by for science

Falsifiability

Karl Popper (1934)

A theory or hypothesis is falsifiable (or refutable) if it can be logically contradicted by an empirical test. 

• Better than verifiability: 

  • To verify “All swans are white” you need to observe all swans. 

  • Falsifiability only needs the anomalous instance, such as observing a single black swan. (astrology example in class) 

Science has evolved to use methods that are: 

• Rigorous 

• Quantitative 

• Replicable 

• Peer reviewed 

• Controlled and nuanced

• According to Hansson (2013), a statement is pseudoscientific if it satisfies the following (doctrines):

• It pertains to an issue with the domains of science in the broad sense (the criterion of scientific domain)

• It suffers from such a severe lack of reliability that it cannot at all be trusted (the criterion of unreliability). 

• It is part of a doctrine whose major proponents try to create the impression that it represents the most reliable knowledge on its subject matter (the criterion of deviant doctrine)

  • These criteria inherently rely on a definition of science and the ability to judge scientific evidence.

The problem with the consensus view (psuedoscience)

• what topics are considered (pseudo)science has changed over time… think about phrenology (disproved) on the other hand evolution was treated as pseudoscience initially but we know it is not

Reconsidering the Demarcation Problem, published in 2013 by the University of Chicago Press

 marked a new approach to the demarcation problem.

“Science is best considered as a family of related activities, with no fundamental essence to define it. Indeed, the same goes for pseudoscience as, for instance, vaccine denialism is very different from astrology”

Gordin’s types of pseudoscience:

Vestigial sciences: “…systems of thought that used to be considered sciences but that professional scientists have, over time, either gradually moved away from or actively excluded such as alchemy. 

Mentalist fringe. Defined as focusing on “allegedly unrecognized or under-appreciated powers of mind,” this group includes a wide range of pseudoscience, from ESP to spiritualism. May stem from vestigial

Ideological fringes. These are the ideas that are “distortions of rational thinking in the service of a political ideology.” 

Controversy fringe. Categorized by “cases in which potentially path‐breaking work is published within the bounds of a science and is greeted with intense skepticism and debate, typically aired across the pages of professional journals,” items in this group can either become accepted, as was the case for quantum mechanics a century ago, or rejected and sent back to the fringe 

Rather than strict criteria, concepts like pseudoscience are best served by

cluster/multifactorial definitions

This approach acknowledges:

• Definitions come as a weighted list of criteria

• No single criteria is either necessary or sufficient for membership

• Category membership is therefore somewhat subjective and on a spectrum

About 2000 years ago, Cicero identified features you’d want to check to decide to believe a claim or not: 

• internal logical consistency of whatever notion is under scrutiny; 

• degree of empirical confirmation of the predictions made by a given hypothesis; 

• degree of specificity of the proposed mechanisms underlying a certain phenomenon; 

• degree of arbitrariness in the application of an idea; 

• and degree of selectivity of the data presented by the practitioners of a particular approach.

Science seeks to establish

evidence for a phenomenon and determine the underlying mechanism that causes it.

Evidence: robust proof that the phenomenon occurs under specified conditions in a reproducible way

Mechanism: a describable model of the causal steps responsible for the phenomenon

Sub-standard evidence used in pseudoscience: 

• Based on personal accounts, fiction, tradition, or superstition (e.g. astrology, ESP, alien abductions)

  • Being around for a long time doesn’t make a theory right.

  • Many people believing it doesn’t make it right

• Based on discredited/fraudulent science (e.g. vaccines-autism, memory of water, ESP-revival)

• Evidence and Mechanism: 

Lack of a possible mechanism may also be pointed to as a reason something is pseudoscience.

Substandard mechanisms offered in pseudoscience :

• Pseudo or untestable mechanism (‘energy’)

• Acknowledged mystery or difficulty

Types of false or ill-supported beliefs: 

Pseudoscience: science-y statements or theories that don’t have rigorous support.

Conspiracy theories: claims about how the concerted effort of certain individuals is secretly responsible for events or phenomena.

Science Denialism: rejection of  scientific theories that have strong evidence and support

Forer Effect: 

• individuals give high accuracy ratings to descriptions of their personality that supposedly are tailored specifically to them, that are in fact vague and general enough to apply to a wide range of people

• Also called the Barnum effect

• Might explain why people strongly believe in astrology/horoscopes

Jeane Dixon Effect

a cognitive bias where people remember a psychic's rare correct predictions while completely forgetting their many failed forecasts.

Deficit Model of Science Communication:

The idea that:

Misconceptions and skepticism towards modern science and technology is caused primarily by a lack of adequate knowledge about science. 

And therefore: 

the public’s perceptions and beliefs will change by providing them sufficient information about modern science and technology to overcome this lack of knowledge — or 'knowledge deficit' 

Why do people believe in conspiracy theories: 

• Epistemic motivations: based on a desire to understand one’s environment.

  • Good motivation that cam go awry, especially if satisfying….

• Existential motivations: desire to be safe and in control of one’s environment. 

  • Knowledge of how things work and can help someone feel safe and in control. Particularly, knowledge of “cheaters” or untrustworthy individuals.

  • Alternative (false) beliefs can provide a sense of ownership.

  • Studies show that people turn to conspiracy theories when they are anxious or powerless and if they have a lack of sociopolitical control or psychological empowerment.

  • Yet they may not actually help.

• Social motivations: desire to maintain a positive image of the self and the social group. 

  • Social status in inherently important.

  • Proclaiming to hold certain (false) beliefs shows alignment with a social group. 

  • Conspiracy theories in particular also provide social cohesion by identifying a malevolent out-group.

Conspiracy belief is correlated with:

• lower levels of education 

• lower levels of analytic thinking

• the tendency to overestimate the likelihood of co-occurring events

• the tendency to perceive agency and intentionality where it does not exist.
  
  

Many factors drive false beliefs

• Cognitive Drivers

  • intuitive thinking- lack of analytical thinking and/or deliberation

  • cognitive failures- neglect source cues and/or knowledge; forget source and/or counter-evidence

  • illusory truth- familiarity, fluency, cohesion

• Socio-affective drivers

  • source cues- elites, in-group, attractive

  • emotion- emotive information, emotional state

  • worldview- personal views, partisanship

Key principles driving anti-science attitudes

When scientific information activates one's social identity as a member of a group 1) that holds antiscience attitudes or 2) that has been underrepresented in science or exploited in scientific work, it triggers ingroup favoritism and outgroup antipathy.

When scientific information contradicts what people 1) believe to be true, 2) evaluate as favorable, or 3) moralize, they experience cognitive dissonance, which is more easily resolved by rejecting the scientific information than by changing existing beliefs, attitudes, or values.

When scientific information is delivered in ways that mismatch one's 1)construal level, 2) regulatory focus, 3) need for closure, or 4) need for cognition, it tends to be reiected.

Counteractive strategies for addressing the key principles of antiscience attitudes

1, i) Improving perceived and actual validity of scientists' work

1, il Legitimizing substantive scientific debate

2) Conveying warmth and prosocial goals in science communication and using accessible language

3) Conveying that the source is not antagonistic to the recipient, such as by providing two-sided messages that clearly state the side for which there is stronger evidence

1. Activation of shared or superordinate identity

2. Engaging and collaborating with marginalized communities

Updating the deficit model of science communication

Providing sufficient information about modern science and technology to a broad audience is probably necessary but not sufficient to fight false beliefs.

Logical challenges of correcting false beliefs

Disproving pseudoscience frequently involves “proving a negative”, e.g. claiming there is absolutely no relationship between birth year and personality. 

This is hard because “absence of evidence is not evidence of absence”

Conspiracy theories are particularly difficult to debunk because evidence against the theory can be interpreted as evidence for the conspiracy to hide the truth.

Fighting science denial requires having perfect and unanimously agreed upon evidence, which is impossible. 

What is media?:

The main means of mass communication (broadcasting, publishing, and the internet) regarded collectively” 

• Generally thought of as comprising journalists, news publications, magazines, etc. 

• Usually non-fiction content

• Can be specialized, e.g. “sports media”

Content of Media:

• News: newly received or noteworthy information, especially about recent or important events. (not supposed to express personal opinion)

• Commentary: an expression of opinions or offering of explanations about an event or situation. (meant to express someone's opinion, typically about/alongside factual info like news and studies)

• Edu-(or info-)tainment: broadcast material which is intended both to entertain and to inform.

Propaganda and mis- or dis-information: 

Some media content is intentionally designed to confuse or mislead: 

• Disinformation: false information which is intended to mislead, especially propaganda issued by a government organization to rival power in the media.

• Propaganda: information, especially of a biased or misleading nature, used to promote or publicize a particular political cause or point of view.

• Misinformation:  inaccurate/ill-supported information that spreads for many reasons.

• Fake news: information that is clearly and demonstrably fabricated and that has been packaged and distributed to appear as legitimate news.

Media Literacy: 

The ability to critically analyze stories presented in the mass media and too determine their accuracy or credibility.

• Higher media literacy is associated with higher levels of knowledge, resilience, and empowerment, lower levels of unhelpful or unhealthy behaviors (e.g., higher levels of literacy around the topic of smoking is associated with lower levels of smoking).

 8 principles of media literacy. (Patricia Aufderheide) 1-4

• All media are constructions. Media do not simply reflect reality. They present productions, which have specific purposes. The success of these productions lies in their apparent naturalness. They don’t look like constructions. But they are, and many different constraints and decisions have gone into why they look the way they do.

• The media constructs reality. While they themselves are constructions, media productions also construct within each of our heads a notion of the real. We each carry within us a model of reality, based on our observations and experiences. Using that model, we believe that we’re capable of distinguishing truth from lies, and are confident that we won;t let “them” pull the wool over our eyes. But much of our model of reality comes from the media we’ve seen, or that other people whom we take as models (our parents, our teachers) have seen. So it’s not as easy as it might seem to draw the line between personal lived experience and the world of “the media”. In fact, the media are constructing our sense of reality each day.

• Audiences negotiate the meaning in media. Even though medi carry messages, they aren’t received by everybody the same way. When you like a movie your friend hated, that’s pretty clear. Each of us ‘filters’ meaning through our different experience: pur socio-economic status, our cultural background, gender, whether we’re tired, whether we know somebody involved in the story. But some meanings end up being more widely accepted than others, a fact that reflects the relative clout, or social power, of the filters which affect pur different readings.

• Media have commercial implications. Most media production in this country is a business, and must make a profit. Even the so-called “public” media-public television, public radio- have to raise money to survive. When you secode the media, you need to ask yourself: who paid for this? What’s the economic structure underpinning this piece of work? When the producer or writer or director chose the subject and began production, how did financial pressures affect his or her choices? 

  • Mass media do not speak to individuals, but to groups of people - in fact, to demographic markets. You are part of several demographic markets– young people, men or women, people in your region, people with your particular hobby, etc. The more money you have to spend within any particular demographic, the more valuable you are to mass media’s marketers.

  • Mass media's commercial implications also involve ownership in another way. If the same company owns a record company, a movie studio, a cable service, network television, videocassette recording and book and magazine publications (as does Time Warner), it has a powerful ability to control what is produced, distributed and therefore, seen.

 8 principles of media literacy. (Patricia Aufderheide) 4-8

• Media contain ideological and  value messages. A media literate person is always aware that media texts carry values and have ideological implications. (Ideology in this sense means the set of assumptions for what we think is normal.) A media literate person does not complain that something is biased; he or she searches out the bias, the assumptions, the values in everything that’s made. It’s all made by people after all, who interpret the world according to their own values and assumptions. Most often, the media affirm the world as it is, the status quo, the received wisdom, whatever is thought of by the media makers as the consensus. And they become reinforcers of that status quo as a result.

  • Because media mostly reinforce the status quo, the fact that they carry values seems almost invisible, or ordinary, or not worth noting. It becomes clearer that they carry those values when you disagree with them.

• Media have social and political implications. Because media construct reality, under economic terms that shaped their messages, and powerfully transmit values, they have important social and political effects on our lives together in society and as members of the public.

• Form and content are closely related in media. Each medium has its own distinctive characteristics. You will get a very different experience of a major event by reading the newspapers, watching TV, and listening to the radio. A media literate person asks: What about the form of this medium influences the content? Is that formal capacity being exploited well, or is it being wasted? What about the form limits the content? 

• Each medium has a unique aesthetic form. Understanding how to “read” the media also means understanding that they are each art forms as well as information transmitters. We pay attention, in writing, to the well-crafted phrase, the vivid quote, the tightly structured argument. We appreciate editing that sharpens contrasts and makes our heart skip a beat in audio, video, and film. We understand the power of a camera to shape our own point of view on entering a scene. When we see how media are constructed, we are able to judge their aesthetic value. We ask two sets of related questions: Did it entertain me, keep my attention, involve me– and how did it do that? Did it tell me more about the world, human affairs, and my part in it–and how did it do that?

Traditional media vs social media

• Traditional media is more like news sources which are put together by a small number of people, which allows every consumer to get the same or at least consistent messages. It’s more uniform than social media. 

• Social media is dense and the role between consumer and producer is completely broken down. People serve as both. And it’s not that there are individuals that we all consume, so it’s far less centralized or uniform. Even with virality, something may be viral on your network, but not someone else’s.

  • the principles oof media literacy still apply tooo social media

It is possible that we need additional principles when dealing with social media. Here’s why:

• Information travels through networks. That favors extreme and emotional posts. Already liked things will be trusted and shared more.

• People self-select content, possibly leading to “bubbles” or “echo chambers”

• Algorithms are opaque and constantly changing. (social media companies build them for their own purposes)

  • “Algorithmic radicalization” is the concept that algorithms drive users toward progressively more extreme content over time, leading to extremist political views.” 

  • “Algorithms on various social media platforms function through the process of recommending content that is similar to what users engage with, but can quickly lead users down rabbit-holes.”

• Sources of images/video can be hard to pin down.

  • Photoshop and generative AI make this problem much worse. It is now very easy to flood the internet with realistic, fake news. Must have AI literacy as well.

What is generative AI (‘genAI’): 

A class of artificial intelligence that can generate text, audio, videos, etc.

• The main task ChatGpt and other language models are trained to do is predict the next word in a sentence, based on all the text on the internet.

• Such models are only as good as the data they’re trained on

  • Translating non-gendered languages into English revealed existing gender patterns.

  • They only learn statistical patterns of language. They are not grounded in truth. 

    • This leads to what is known as hallucinations or confabulations.

• Different AI systems can work differently

  • Some actually do something like search the internet for you

  • But importantly, we usually don’t know how these systems are built which makes it hard to know if we should trust them.

The role of Fact Checking: 

• Important to provide reliable information that counters popular lies but it’s hard!

  •  “Bullshit Asymmetry Principle,” also known as “Brandolini’s Law”: “The amount oof energy needed to refute BS is an order of magnitude bigger than to produce it”

  • Mark Twain: “a lie can travel halfway around the world while the truth is still putting onn its shoes”

How do we combat science misinformation? 

• Teach (social) media literacy and encourage a balanced “information diet”