Policy Analysis Notes (Australian Policy Handbook)

The Nature and Purpose of Policy Analysis

• Good decision making about complex issues requires analysis; policy analysis is a fundamental stage in the policy cycle because research and logic underpin developing options and decisions.
• Aaron Wildavsky (1987) frames policy analysis as an art: public problems must be understood tentatively to be solved; structuring a problem through analysis gives it shape and meaning and suggests possible solutions even as issues are being defined.
• The paradox: analysis seeks solutions while deepening understanding of the problem; by the time issues are defined, possible solutions become more visible.
• Policy analysis is rigorous but not purely mechanical; judgement must precede the application of analytic tools. Decisions about scope, depth, and approach depend on the problem’s importance, time, and resources.
• Policy learning and transfer have become standard practices (global diffusion of ideas and benchmarking). Practices range from imitative approaches to smarter forms of transfer.
• Smart practice (Bardach) emphasizes adapting ideas from elsewhere to the local context rather than mere imitation; emerging, promising, and wise practices reflect varying levels of evidence and local fit.
• The evidence base in policy analysis is expanding to include qualitative, case-specific understanding and culturally informed interpretation, not just quantitative data.
• The role of the analyst includes acknowledging limits and ensuring options are defendable on their own terms, not simply reflecting a particular department’s view.

The Rational Comprehensive Model and Its Limitations

• The rational comprehensive model posits a rational, value-free, comprehensive process that canvasses all options and conducts costs and benefits analysis.
• Six obligatory steps (as commonly depicted in the literature):
  1. Problem identified and defined
  2. Objective set
  3. Values and goals determined and ranked
  4. All options identified
  5. Each option tested for costs and benefits
  6. Cost-benefit ratios compared
• This model assumes problems are clear, stable, and easily quantified, and that decision makers share objectives and resources are sufficient.
• In practice, decisions are rarely fully rational or comprehensive due to:
  • complexity and ambiguity of problems
  • contested values and multiple, conflicting goals
  • limited time, resources, and political constraints
  • uncertainty about outcomes and costs
• Although criticised, the rational model remains useful for providing an orderly sequence and justification for choices, even if real decision making is messier.

A Practical Sequence for Policy Analysis (the Analyst’s Cycle)

• Policy analysts typically work through problems in an orderly, iterative way, with phases such as shown in the policy analysis iteration:
  • Formulate the problem
  • Set out objectives and goals
  • Identify decision parameters
  • Search for alternatives
  • Propose a solution
• This sequence is not a one-off: subsequent steps (instruments, consultation, political decision making) can alter analysis; problems may re-emerge or shift with new realities.
• Analysis is invariably iterative because information is often incomplete, objectives differ, and parameters shift.

Formulating the Problem

• External processes (lobbying, media campaigns, persistent program failures) draw attention to an issue and prompt problem formulation.
• Ministers typically request advice from the bureaucracy about the problem’s nature, scale, and characteristics; reviews may reveal the need for change.
• There is no single method for formulating problems; each problem is unique and framed by the initial discussion and research questions.
• Ownership of problem definitions is crucial: multiple agencies and levels of government may have different perspectives and concerns (e.g., Olympic Dam uranium mine involved federal departments, state departments, and local councils). The analysis can become marginal in usefulness if ownership is too dispersed or conflicting; in the case of Olympic Dam, expansions were shelved despite extensive agency involvement.
• Institutional routines (consultation prior to cabinet decisions, central evaluation) restrain overly narrow problem specification and promote options that can be defended on their own terms.
• Analysts should offer options that reflect broader public interest, not simply departmental values.

Set Out Objectives and Goals

• Defining objectives is fundamental because they determine priorities, guide program implementation, and shape evaluation criteria.
• Objectives often require balancing feasibility with desirability; what is achievable depends on resources and constraints.
• Problems can present ambiguous or conflicting objectives; analysts should provide a range of options, each with a different configuration of problem definition and proposed solutions.
• Shaping options is an ethical task to treat alternatives fairly and avoid steering toward a biased outcome.
• Wildavsky noted that objectives are a function of both desirability and feasibility; practical constraints influence what is attempted.

Identify Decision Parameters

• Decision parameters frame the options in light of:
  • likely ministerial/government objectives
  • available resources and financing possibilities
  • required timeframes for consideration and action
  • relative priority of the problem
• Departments often prepare briefing notes to test the minister’s concern; if interest is present, more options or a policy proposal are developed.
• If a minister does not engage, departments may rearrange priorities or let the problem wait; agenda crowding often pushes toward incremental solutions to minimize search and analysis time.
• When problems lack viable options, are poorly understood, or lack reliable information about causes, policy analysis may reveal a wicked or intractable problem requiring more research or a reframing of the underlying issue.

Searching for Alternatives

• The search phase involves rigorous research to gather relevant information and identify possible responses; given multi-faceted problems, a team approach can be valuable.
• Sources for ideas include:
  • current policies locally and in other jurisdictions
  • international findings on best practice
  • recent reviews and reports
  • academic journals
  • discussions with internal and external experts
  • consultations with clients and stakeholders
  • consultations with service delivery and operational staff
• Leigh (2009) advocates more rigorous policy research (meta-analysis, randomised trials, natural experiments), but practical constraints (time, money) often limit formal studies.
• The search phase narrows possibilities to those most likely to meet objectives, while leaving room for new information to alter the analysis.
• Modelling and explicit causation: the phase may involve modelling to compare options, though modelling is stronger for financial/technical aspects and less precise for social/political consequences.

Modeling and Evidence in Policy Analysis

• In many policy areas, teams may model the consequences of actions to compare options; models help articulate assumptions and provide structured reasoning about causation.
• The quality of modelling rests on explicit assumptions, clear causal links, and transparent reliability.
• Models are most robust for monetary/engineering-type analysis and less reliable for complex social outcomes.

Measuring the Quality of Policy Advice (Nine Standards)

• Di Francesco (1999) identifies nine quality standards for policy advice:
  1. purpose
  2. logic
  3. accuracy
  4. options
  5. responsiveness
  6. consultation
  7. presentation and conciseness
  8. practicality and relevance
  9. timeliness
• Quality depends on the producer and the client; political context shapes how advice is judged. High-quality advice should be evaluated within the political setting in which it was provided.

Propose Solutions and Evidence-Based Policy

• The final output is a briefing or report that formulates the problem, sets objectives, identifies parameters, presents alternatives, and makes recommendations.
• Recommendations should be supported by a comparative analysis and, if appropriate, an on-balance finding across options.
• Ministers may demand further information or adjustments if the proposed options are unacceptable; policy analysis must accommodate iteration and reworking to reach workable choices.
• The rise of evidence-based policy has promoted stronger links between researchers and decision makers, but faces criticisms about how evidence is used, interpreted, and integrated with political judgment.

Evidence-Based Policy: Strengths, Weaknesses, and Debates

• The evidence movement emphasizes informing policy with research evidence and practice-based learning (e.g., in human services and medicine).
• Trends driving evidence-based policy include:
  • competition for funding and rational allocation of scarce resources
  • recognition of policy problem complexity and interdependence
  • increasing use of specialists and experts
  • greater influence of international organisations; knowledge economies that trade knowledge
  • policy entrepreneurs or brokers; policy learning within government; capacity building when supply and demand of evidence mismatch
• Despite its appeal, evidence-based policy faces criticisms:
  • evidence-informed and intelligent policy making argue that context, performance measurement, and learning shape evidence use
  • evidence-based policy remains influential in Australia and elsewhere, backed by political support
• Strengths of evidence-based policy:
  • helps ask the right questions and frame problems clearly
  • can improve efficiency, effectiveness, transparency, inclusiveness, and accountability
  • supports risk minimisation and informed decision-making
  • promotes best practices and innovation
  • provides defensible, coherent information grounded in disciplinary knowledge
• Weaknesses of evidence-based policy:
  • time pressures and electoral cycles limit thorough research before decisions
  • temporal disjuncture: difficult to obtain long-term robust data for sweeping issues
  • contesting evidence: conflicting methodologies and interpretations
  • subject-matter limitations: some policy questions are not easily testable; causality may be hard to establish
  • public sector requirements: due process, ethics, and accountability constrain what evidence can be used
  • politicisation: evidence can be used rhetorically; what counts as evidence is contested
• Head’s three-lens model (Head, 2008) suggests integrating (i) scientific evidence, (ii) professional/practical knowledge, and (iii) political judgment to inform rather than determine policy outcomes.
• Final point: the policy process requires a dynamic, organic approach, balancing different types of evidence with political realities and time constraints.

The Analyst’s Toolkit: Frameworks for Policy Analysis

• The major frameworks emphasize different angles for understanding and solving policy problems:
  • Substantive content matters: policy analysis is grounded in domain-specific expertise (education, health, economy, law); consider transaction costs and political feasibility.
  • Economic framework: market analysis, incentives, and cost-benefit considerations; used to examine demand/supply, providers/purchasers, and incentives; illustrates why private sector involvement can reduce costs but is not a panacea.
  • Social framework: addresses social justice, inclusion, and distributional effects; considers rights, equity, participation, and access; highlights potential unintended consequences for vulnerable groups; uses social impact studies.
  • Environmental framework: environmental impact statements (EIS) assess ecological consequences; covers biodiversity, habitat, ecological sustainability, and intergenerational impacts; climate change adds complexity about cumulative and global effects.
  • Legal framework: policy options operate within constitutional and statutory law; requires alignment with fundamental legislative principles, accountability, and human rights protections; policy jurisprudence guides how law and policy interact.
  • Political analysis: politics shapes policy viability; advisors should align with government priorities and anticipate public messaging; the question "What would tomorrow’s headline say?" helps test political sensitivity.
  • Agreement as a tool (Lindblom’s incrementalism): in practice, policy often progresses through small, incremental adjustments rather than radical departures; consensus-building and experimentation guide policy evolution; “agreement” can be a more realistic test of policy viability than exhaustive search.

Economic Framework: Example and Cautions

• The economic framing is dominant in public policy discourse, testing policies by market incentives and fiscal implications.
• Example: private prisons and market-based service delivery show both potential efficiencies and risks; private providers can offer cost savings, innovation, and competition, but outcomes are not universally superior to public provision, and savings may be offset by other costs or social concerns.
• Job Network example (Howard government): purchasing employment outcomes from private providers; allowed client choice and competition while shifting from a government monopoly to a market for employment services.
• National Disability Insurance Scheme (NDIS) example: aims to create a competitive, self-sustaining market for service providers.
• Caution: marketization is a tool, not an end; over-reliance on market mechanisms can erode core policy aims if not carefully bounded by public interest considerations.

Social Framework: Equity, Rights, and Inclusion

• Policy analysis should consider social dimensions that economics alone cannot capture, such as:
  • inclusion of marginalized groups (women, Indigenous peoples, remote communities, people with mobility limitations, CALD populations)
  • social justice principles: rights, equity, participation, and access
• Four key social objectives often highlighted:
  • universal services (health, education)
  • targeted services for special needs groups
  • facilitating inclusion and reducing barriers to participation
  • ensuring meaningful access and responsiveness to diverse communities

Environmental Framework: Ecological and Sustainability Considerations

• Analysis must assess environmental impacts of policy options, including:
  • land, air, water quality; pollution and noise; biodiversity and habitat preservation
  • ecological sustainability: ensuring activities allow for regeneration and long-term ecological health
  • environmental impact statements (EIS) and related analyses
• Climate change adds complexity: local vs global impacts, cumulative effects, intergenerational equity, and timing considerations

Legal Framework: Law as Constraint and Enabler

• Policies operate within a range of laws: constitutions, general laws, auditing and accountability requirements, employment and safety regulations, and legislative principles.
• Questions typically asked in legal policy analysis:
  • What legal framework and institutional structures are necessary?
  • Who is accountable to whom; what are the mechanics of accountability?
  • What are the decision-making frameworks and how are they stated?
  • How does the proposal relate to fundamental legislative principles; how are human rights affected?
  • What is the administrative and constitutional efficacy and propriety of the proposal and instruments?
• Policy jurisprudence emphasizes democracy, justice, reform, and service as guiding principles for policy development.

Political Analysis: Bridging Policy and Politics

• Politics is essential to policy analysis; analysts should incorporate political input at appropriate times and align with governing party principles and public concerns.
• A practical test: consider how a policy would read in tomorrow’s headlines to assess political viability.
• The analyst’s role includes ensuring political considerations are integrated without compromising analytical integrity.

The Incremental Path to Good Policy: Lindblom on Agreement

• Lindblom argued that ends and means are rarely clear; successful policy often emerges through a bank of knowledge and incremental changes rather than radical departures.
• Incrementalism emphasizes testing small changes, learning from the outcomes, and adjusting accordingly; it acknowledges practical constraints and the value of bureaucratic expertise.
• Critics argue incrementalism can hinder bold solutions for large problems; nonetheless, agreement-based incrementalism is a realistic description of many policy processes.

Case Study: Uber and the Sharing Economy

• Uber connects passengers with nearby drivers via a mobile app, taking a fee; it operates within the sharing economy model, leveraging spare capacity (driver time and vehicle) to match demand in real time.
• Why it became a policy issue:
  • Uber competed with traditional taxis but operated outside existing regulations, highlighting regulatory gaps and the power of digital platforms to bypass traditional licensing regimes.
  • Taxis have long been regulated, with barriers to entry (licences for cars, drivers, and companies) providing revenue and protection to incumbent players.
  • Environmental, congestion, pricing, and safety concerns arise with new mobility models; regulators debated whether barriers to entry were justifiable public-interest tools.
• Policy response across Australian jurisdictions:
  • Inquiries opened the regulatory net to Uber and other ride-sharing services; recommendations generally supported bringing ride-sharing into the regulatory fold.
  • Taxis needed to adapt and modernise; ride-sharing firms and drivers were brought within regulatory parameters.
  • The result was a hybrid regime: some taxi regulations relaxed, but taxi firms retained certain privileges like rank-and-hail operations; overall disruption reshaped industry structure.
• Policy lesson: open, evidence-informed regulatory reform can accommodate innovation while maintaining public-interest safeguards.

Case Study: Olympic Dam Mine (Ownership and Problem Framing)

• An extended problem-framing exercise across multiple agencies and government levels discussed potential impacts of uranium mining (Olympic Dam) and transport routes.
• Agencies involved included federal departments (e.g., Sustainability, Environment, Water, Population and Communities; Geoscience Australia; Australian Radiation Protection and Nuclear Safety Agency) and state/local actors.
• Despite extensive interagency work, the analysis proved of marginal use; plans were shelved by the mine’s owner, illustrating how even thorough problem framing can be overwhelmed by private decisions or external factors.
• Key takeaway: problem ownership, interagency coordination, and the political economy of a project shape the usefulness and outcomes of policy analysis.

Practical Hints for Presenting Policy Advice

• Consider the timing and political context (e.g., electoral cycles, agenda priorities).
• Understand ministerial and senior management requirements; clarify whether the minister’s office should be involved.
• When preparing key components (objectives, scope, costing, implications), ensure administrative logics account for political sensitivities; avoid purely partisan advice.
• Anticipate timing issues and inform senior management if timing might affect decisions.
• Briefs to ministers may become cabinet sub-submissions; align with cabinet requirements.
• Written and oral advice should be succinct, coherent, and clearly linked to well-reasoned recommendations.
• Assess the policy’s life: is it a closing decision, a policy with a limited life, or an open-ended proposal?
• Anticipate future needs and requirements; plan for potential updates or changes as problems evolve.
• Expect iteration: early drafts may be revised as new information emerges or as stakeholders respond.

Discussion Questions (5.1–5.4)

• 5.1 How do you understand the concept of analysis as it pertains to policy making? How would you explain it to a new policy analyst?
• 5.2 Is evidence-based policy a realistic goal for the modern policy process? What factors constrain and promote the use of evidence in policy making?
• 5.3 How would you assess the quality of analysis in any given policy? Do you think your assessment of quality is shared by other policy actors? If not, why, and what are the implications of divergent quality assessments?
• 5.4 Why are developments like Uber important for policy makers to understand?

Key References and Concepts Mentioned

• Wildavsky, Aaron. 1987. Policy analysis as an art; problem structuring shapes possible solutions.
• Bardach (1994, 1998, 2006): smart practice; emerging, promising, and wise practices; move beyond imitation to context-sensitive transfer.
• Dolowitz, Dolowitz & Marsh; Gilardi; Marsh & Sharman; Meseguer; Rose. discussions on learning, diffusion, and transfer.
• Leigh (2009): call for more rigorous policy research methods (meta-analysis, randomised trials, natural experiments).
• Di Francesco (1999): nine quality standards for policy advice.
• Lindblom (1959, 1965): incrementalism; agreement as the test of policy viability.
• Head (2008): three-lens model of evidence (scientific, professional, political) and the idea that analysis informs rather than determines policy outcomes.
• Uber case study (policy analysis of sharing economy): regulatory net, openness to innovation, and the balancing of public-interest safeguards with market disruption.
• Olympic Dam mine case: problem ownership and interagency coordination as practical limits of policy analysis.

Summary of Practical Takeaways

• Policy analysis is about shaping problems, not just proposing solutions; it requires judgment to decide how much science to apply.
• Real-world policy making blends rational analysis with incremental, consensus-driven change; perfect rationality is rare.
• Analysts should frame issues clearly, set feasible objectives, identify decision parameters, and search for viable alternatives using diverse sources.
• Modelling and evidence are tools among many; the quality of advice depends on purpose, logic, accuracy, options, responsiveness, and practicality, all within a political context.
• A robust toolkit combines content knowledge with economic, social, environmental, legal, and political perspectives; the realist use of these frameworks depends on the problem and context.
• Case studies like Uber show how policy analysis guides regulatory reform to accommodate innovation while maintaining public safety and fair competition.
• Ongoing iteration, ethical judgement, and transparent justification are essential to credible policy advice.