Strategies for overcoming hypothetical bias

What are the two categories of methods used to minimise/elimiate hypothetical bias?

• Ex-ante: before the event, based on forecasts rather than actual results SO modify survey designs to minimise bias up front

• Ex-post: after the event, based on outcomes/results rather than forecasts SO adjusting data based on follow-up info gathered after initial data collection

How does hypothetical bias occur?

Occurs when individuals report unrealistic behaviours or values to researchers in surveys or experimental studies

When are stated preference/revealed preference methods used?

To value non-market goods or services --> valuation for policy analysis, environmental statements and resource management plans

• Actual good/resources does not exist in particular location --> Can be difficult for economists to rely on revealed preferences since the actual behaviour has not occurred (revealed preferences)

What 2 methods of stated preference does this article focus on?

• Contingent Valuation Method: involves directly asking people, in a survey, how much they would be willing to pay for specific environmental services. In some cases, people are asked for the amount of compensation they would be willing to accept to give up specific environmental services. It is called ¡¦contingent¡¦ valuation, because people are asked to state their willingness to pay, contingent on a specific hypothetical scenario and description of the environmental service.

• Choice Experiments:
  A choice experiment is a survey method that involves asking people to state their preference for hypothetical alternative scenarios, goods, or services, which are combinations of attribute levels generated by the experimental design. Each alternative "good" is described by several attributes in terms of different attribute levels. One of the attributes is the price of the alternative. We used the discrete choice model to analyze how people make choices. Most environmental goods are composites, made up of a variety of attributes that can be provided at various levels. This allows for the estimation of the relative importance of multiple environmental attributes and their levels, unlike contingent valuation, which cannot be used to distinguish the value of each attribute in multi-attribute environmental goods.

How to conduct contingent valuation?

1. Define the valuation problem: what services are being valued, who the relevant population is (e.g. federally owned land means relevant population would be all US citizens)

2. Make preliminary decisions: conduct survey via mail/phone/person?, who will be surveyed, sample size, related questions, size of the budget

• In-person interviews are most effective for complex questions, often easier to explain the required background information to respondents in person, and people are more likely to complete a long survey when they are interviewed in person.
• Visual aids may be presented so respondents understand what they are being asked to value.

1. Survey design can take around 6 months. Initial interviews with types of people who will receive final survey. First ask general questions, understanding of the issues related to site, whether hey are familiar with the site and its wildlife, whether and how they value this site and the habitat services it provides.

• Later focus groups, questions would get more detailed and specific, to help develop specific questions for the survey, as well as decide what kind of background information is needed and how to present it. --> people may need info on location of site, uniqueness of species, whether there are substitute sites

• Researchers would also want to learn about peoples¡¦ knowledge of mining and its impacts

• After a number of focus groups have been conducted, and the researchers have reached a point where they have an idea of how to provide background information, describe the hypothetical scenario, and ask the valuation question, they will start pre-testing the survey.

• Researchers continue this process until they've developed a survey that people seem to understand and answer in a way that makes sense and reveals their values for the services of the site.

How to conduct contingent valuation? (PART 2)

4. Implementing survey: select the survey sample (randomly selected in relevant population). May use opportunistic sampling by asking people in public places to fill out the survey. If over-the-phone, may call a certain number of times in order to get the greatest possible response rate for the survey.

5. Compile, analyse and report the results. Data must be entered and analysed using appropriate statistical test.

• Deal with non-response bias, most conservative way is to assume that those who did not respond have zero value.

What is hypothetical bias?

Difference between what a person indicates they would pay in the survey or interview and what a person would actually pay --> like the old saying 'there is a difference between saying and doing'

Why is WTP overstated for public goods?

• People don't have an incentive to reveal their true WTP and hope to free ride off others donations
• Overstate saying they will pay more because they wish to see the fund established so the firm will provide that public good

Why is WTP understated for private goods?

• Truncated by assessment of the price for which similar good sells in the market
• With private excludable goods respondent's incentive to understate to prevent or minimise an increase in price rises (e.g. deer hunting licenses)

What are the 4 ex-ante methods to reduce hypothetical bias?

1. Consequential designs
2. Honesty and Realism approaches
3. Cheap talk
4. Reducing social desirability bias and cognitive dissonance

Ex-Ante: Consequential Design

• Developed in 2007 by Carson and Groves --> economists known for they work on contingent valuation and stated perforce methods --> research focuses on the incentive properties of severe questions used to estimate the economic value of environmental goods and services

Suggestion of 3 features:
1) must be consequential to the respondents (must have an effect on their future utility such as higher taxes or increased probability that the public good will be supplied)

2) A binary or dichotomous choice question format that is demand revealing --> CE will not meet this condition but a yes/no vote would

3) payment mechanism to be compulsory (such as a tax paid by all) if the referendum passes

Results on the performance of consequential CE surveys are encouraging --> study found no statistical difference between the proprituon of people who voted yes to a water supply project in a small town in Massachusetts and teh subsequent actual vote 4 months later

BECAUSE…

• Survey was consequential to respondents and they knew this was an actual water supply project being considered --> results would influence whether the water supply project will go ahead depending on if people will pay the cost

Why would consequential design work?

Some believe hypothetical bias arises because of respondents uncertainty about the degree of consequentiality, particularly regarding the likelihood of paying

SO

if respondents are told that the good will be provided on the results of the survey and the same probability that they will have to pay, than theoretically respondents should reveal their true value

Why do I think consequential design is the best approach?

• Forces people
• Making survey responses influence real outcomes, giving participants an incentive to respond truthfully rather than hypothetically.

Ex-Ante: Honesty and realism approaches

Encouraged to report what they would honestly pay not what they think the good would sell for in a market.

• Formalised by administering an oath to respondents who were asked to swear to tell the truth --> sign an oath
• Appears to be more powerful when combined with 'cheap talk'

Ex-Ante: Cheap Talk

Confronts the problem of hypothetical bias by telling respondents that participating I past surveys have been shown to overstate their WTP. Respondents are instructed not to do this and answer what they would actually d if this were a real situation with their own money.

• Very mixed results, inconsistent at reducing hypoethtcial bias
  -Because the design is still hypothetical, cheap talk cannot fully align incentives—respondents still face no actual cost for misreporting.

Ex-Ante: Reducing social desirability bias and cognitive dissonance

Minimise the tendency a respondents might have to give socially acceptable answer or once the interviewer wants to hear

• Tendency stems from th epotenial for the respondents to feel some emotional discomfort --> psychologists call 'cognitive dissonance'

Reduce this by…

• Inferred valuation as it asks respondents what they think others would pay for this good rather than themselves --> respondents gain utility that are consistent with social norms than their own personal values --> respondent gains utility by pleasing the interviewer or maintaining a positive self-image BUT respondents tend to have a more negative view of what others would pay
  OR
• Trichotomous choice instead of dichotomous choice --> so instead of facing cognitive dissipate between two objectives especially at high bid amounts where if they say no it can create a worry that they are signalling that they don't value the good at all or yea-saying THEY CAN THEN have a third choice voting at a different bid signalling lower than the original bid amount

What are the 3 ex-post methods to reduce hypothetical bias?

1. Data Screening
2. Related market calibration
3. Uncertainty recoding

When are ex-post screening of survey response particularly useful

If the survey uses open-ended WTP questions as there is no upper bound on the monetary amount a respondent can pay in these case

Even with payment cards to circle monetary amount willing to pay --> int can be implausibly high fractions of income

Calibrating stated WTP based on 'actual behaviour; responses in market or lab experiment

Ex-Post: Data Screening

One approach to minimise questionable observations is to report the median WTP --> alpha trimmed mean by finding outliers 3 standard deviations from the mean and removing them

Ex-Post: Related Market Calibration

• CVM-X --> researcher conducts a CVM survey then a subset of those respondents are asked to participate in a lab validity experiment with the same good as used in the survey
• Based on the lab validity test, the analyst calculates the ration of actual WTP to hypothetical WTP elicited from respondents in the same experiment
• BUT the CVM-X cannot generate universal calibration factors because the authors warn that calibration facts developed for deliverable goods typically used in the lab may not be transferable for some public goods needing to be valued

Ex-Post: Uncertainty recoding

Some think hypothetical bias can originate in respondents uncertainty about various dimensions of CVM survey (e.g. what the good is worth to them). They argue that hypothetical bias in a dichotomous choice WTP response can be reduced by recoding responses that vote 'yes' 'no' based of if the respondent expresses a high degree of uncertainty bay their affirmative WTP response.

Particularly relevant approach in valuing public goods respondent ay not have through about

• A respondents answers the dichotomous choice valuatopm question and indicates how certain they are of their previous answer often on a scale of 1-10
• Uncertain responses that give Yes are recoded to be 'no' --> helpful but cash respondents may understate a respondents maximum WTP
• More likely to over-correct hypothetical bias tho