Lesson 2: Bayesian Reasoning

Uncertainty

can be defined as the lack of the exact knowledge that would enable us to reach a perfectly reliable conclusion (Stephanou and Sage, 1987).

Classical logic

permits only exact reasoning. It assumes that perfect knowledge always exists and the law of the excluded middle can always be applied:

imperfection

One of the common characteristics of the information available to human experts is its ____.

Information

____ can be incomplete, inconsistent, uncertain, or all three.

Information

is often unsuitable for solving a problem. However, an expert can cope with these defects and can usually make correct judgements and right decisions. Expert systems also have to be able to handle uncertainty and draw valid conclusions.

Uncertainty

real-world problems

Unfortunately, most ____ where expert systems could be used do not provide us with such clear-cut knowledge. The available information often contains inexact, incomplete or even unmeasurable data.

Uncertainty

In general, we can identify four main sources: weak implications, imprecise language, unknown data, and the difficulty of combining the views of different experts (Bonissone and Tong, 1985).

• Weak Implications

• Imprecise language

• Unknown data

• Combining the views of different experts

Weak Implications

Rule-based expert systems often suffer from weak implications and vague associations. Domain experts and knowledge engineers have the painful, and rather hopeless, task of establishing concrete correlations between IF (condition) and THEN (action) parts of the rules.

Imprecise Language

Our natural language is inherently ambiguous and imprecise. We describe facts with such terms as often and sometimes, frequently and hardly ever. As a result, it can be difficult to express knowledge in the precise IF-THEN form of production rules. However, if the meaning of the facts is quantified, it can be used in expert systems.

Unknown data

When the data is incomplete or missing, the only solution is to accept the value ‘unknown’ and proceed to an approximate reasoning with this value.

Combining the views of different experts

Large expert systems usually combine the knowledge and expertise of a number of experts. Usually, experts have contradictory opinions and produce conflicting rules. To resolve the conflict, the knowledge engineer has to attach a weight to each expert and then calculate the composite conclusion.

Basic Probability Theory

The basic concept of probability plays a significant role in our everyday life. We try to determine the probability of rain and the prospects of our promotion, the odds that the Australian cricket team will win the next test match, and the likelihood of winning a million dollars in Tattslotto.

probability

The ____ of an event is the proportion of cases in which the event occurs (Good, 1959).

Probability

can also be define as a scientific measure of chance.

Probability

can be expressed mathematically as a numerical index with a range between zero (an absolute impossibility) to unity (an absolute certainty). Most events have a probability index strictly between 0 and 1, which means that each event has at least two possible outcomes: favourable outcome or success, and unfavorable outcome or failure.

Basic Probability Theory

Basic Probability Theory

Basic Probability Theory

Example

Conditional Probability

denoted mathematically as P(A|B) in which the vertical bar represents GIVEN and the complete probability expression is interpreted as ‘Conditional probability of event A occurring given that event B has occurred.’

Conditional Probability

• P(A|B) - is the conditional probability that event A occurs given

  that event B has occurred;

• P(B|A) - is the conditional probability of event B occurring given that event A has occurred;

• P(A) = is the probability of event A occurring;

• P(B) = is the probability of event B occurring.

P(A|B)

is the conditional probability that event A occurs given that event B has occurred

P(B|A)

is the conditional probability of event B occurring given that event A has occurred;

P(A)

is the probability of event A occurring

P(B)

is the probability of event B occurring

Conditional Probability

Example 1

Conditional Probability

Example 1. Given

Condition Probability

Example 1. Solution

Conditional Probability

Example 2

Conditional Probability

Example 2. Given

Conditional Probability

Example 2. Solution

Bias of Bayesian Method

The framework for Bayesian reasoning requires probability values as primary inputs. The assessment of these values usually involves human judgement. However, psychological research shows that humans either cannot elicit probability values consistent with the Bayesian rules or do it badly (Burns and Pearl, 1981; Tversky and Kahneman, 1982). This suggests that the conditional probabilities may be inconsistent with the prior probabilities given by the expert.

Bias of Bayesian Method

Example 3 (pt. 1)

Bias of Bayesian Method

Example 3 (pt. 2)

Bias of Bayesian Method

Example 3 (pt. 3)

Bias of Bayesian Method

Example 3 (pt. 4)