ACIS 4654 - Final Exam

What information will be covered on this exam?

The exam covers Modules 4, 5, 6, and 7

• MC = 32 Pts (16 @ 2pts each)

• Matching = 27 Pts (2 matching questions)

  • SOC Reports

  • Matching Analytic (Clustering / Classification / etc)

• Free Response = 41 Pts (12 Questions)

Classical Machine Learning is split into two groups, ______ Learning (_____ driven) and _______ learning (_____ drive).

Classical Machine Learning is split into two groups, SUPERVISED Learning (TASK driven) and UNSUPERVISED learning (DATA drive).

Supervised learning

Trains a model on labeled data, meaning every example comes with the correct “answer”. The model learns how the inputs map to the output so it can predict that output for new data it hasn’t seen. (Ex - You give the model historical cases along with their outcomes. For example, past loan applications marked “defaulted” or “repaid.” The model finds the patterns that link the features to the outcome.)

Main types of Supervised learning

Classification and Regression

Unsupervised learning

Works with unlabeled data, so there’s no target variable and no “right answer.” The model looks for hidden structure, groupings or relationships on its own. (Ex - You give the model data with no outcomes attached. It sorts the records based on how similar they are, or on which items tend to appear together.)

Main types of Unsupervised learning

Clustering, Association, and Dimensionality Reduction

Data reduction (filtering)

Used to reduce the amount of observations to focus on relevant items (i.e., highest cost, highest risk, largest impact, etc.).

Steps to Data reduction

1. Identify the attributes

2. Filter the results

3. Interpret the results

4. Follow up on the results

Four Categories of Data Analytics

Descriptive, Diagnostic, Predictive, Prescriptive

Descriptive analytics

What happened?

Diagnostic analytics

Why did it happen?

Predictive analytics

What will happen?

Prescriptive analytics

How can we make it happen?

Profiling

Profiling identifies the “typical” behavior of an individual, group, or population by compiling summary statistics about the data (including mean, standard deviations, etc.) and comparing individuals to the population. ​

Benford’s Law

Example of profiling. Its an evaluation of the leading digit and its overall distribution in a set of data (ex - 1 occurs 31% of the time, 9 occurs 5% of the time). If there is a discrepancy between the data set and Benford’s law, there might be something wrong with the data.

Clustering

Example of Diagnostic Analytics. Involves finding undiscovered natural groups in the data.

Hierarchical Clustering methods

Agglomerative methods begin with n clusters and sequentially merge similar clusters until a single cluster is obtained.​ Divisive methods work in the opposite direction, starting with one cluster that includes all observations. ​

Non-hierarchical Clustering methods

K-means. Using a predetermined number of clusters, the method assigns observations to each cluster. These methods are generally less computationally intensive and are therefore preferred with very large datasets.​

K-Means Clustering​

1. Start with k initial clustering (k is chosen by the user)

2. At every step, each observation is reassigned to the cluster with the “closest” centroid.

3. Recompute the centroids of clusters that lost or gained an observation, and repeat step #2.

4. Stop when moving any more observations between clusters increases cluster dispersion.

What type of clustering does Tableau use?

Tableau uses K-Means Clustering.

Similarity matching

Type of diagnostic analytic. Similarity matching tries to recognize similar individuals based on the information known about them. ​

Co-occurrence Grouping ​

Attempts to find associations between entities based on transactions involving them. (Ex - buying hotdogs and hotdog buns)

Link Prediction

Type of predictive analytics. Predicts a relationship between two data items, such as members of a social media platform.

Regression Analysis

Type of predictive analytics. predicts a specific value based on its relationship with other values (we worked on this a bunch in class).

Logistics Regression​

A statistical method for analyzing a dataset in which there are one or more independent variables that determine an outcome. ​Basically if the result is anything binary (Yes/No, Pass/Fail, etc) its logistics regression.

Classification

predict whether data belongs to one class or another (In this graph, the blue circles are on-time vendors, and the green squares are delayed vendors and the star is a new vendor with no history)​

Decision trees

Used to divide data into smaller groups.

Decision boundaries

Mark the split between one class and another. ​

Under-fitting vs Over-fitting

Under-fitting means that there isn’t enough “adjustment” in the model to properly group all data points. Over-fitting involves making too many adjustments in terms of grouping the data points. Under-fitting doesn’t do enough and makes mistakes . Over-fitting does TOO much and introduces unnecessary variables that makes mistakes.

Naïve Bayes (Idiot’s Bayes)

Uses a bag of words model that treats documents as a collection of unordered words. It adds a probability of occurrence to each work and uses them to classify documents based on a specific topic.

False Positive

Alarm but no problem

True Positive

Alarm and a problem

True Negative

No Alarm and No Problem

False Negative

No Alarm and Yes Problem (the worst scenario)

Steps to applying Audit Data Analytics

1. Plan

2. Perform ETL process

3. Review & analyze relevance and reliability of data

4. Perform ADA

5. Evaluate and address the outcomes

SOC Report Types

SOC 1 - Financial Reporting Controls

SOC 2 - Service Controls (Security, Processes, Confidentiality, etc)

Type I - Assurance as of a specific date

Type II - Assurance throughout a specific period

Ways to increase the reliability of data (Audit evidence is more reliable if…..)

• Auditor sources data directly instead of through a middleman.

• Auditor uses a source independent of the audited entity.

• Controls involving the data are deemed effective.

• Original documents instead of copies.

• Evidence is physically provided instead of orally provided.

Key Performance Indicator questions

• What is being measured?

• What is the benchmark for that metric?

• What is the goal of the metric?

Defining an organization's strategy involves:​

1. Defining the market (local, national, global, etc)

2. Defining the customers (broad or narrow? age group? Income level?)

3. Identifying the critical internal processes

4. Determining the individual and organizational capabilities

The balance scorecard tries to establish a balance between FOUR types of measurements. Those include:

1. Short-term vs Long-term

2. Internal vs External

3. Leading indicators vs Lagging indicators

4. Objective measures vs Subjective measures

The Balanced Scorecard has four components. ​Those are:

1. Financial – how the company generates value​

2. Customer – how the company interacts with customers​

3. Internal process – how efficiently the company is operating​

4. Organizational capacity – how the company is training employees​

XBRL stands for:​

eXtensible Business Reporting Language

What is XBRL lanauge?

Business language that allows data to tagged and transmitted. Its used by financial analysts, auditors, or regulators. (Think of the annoying 10-Ks on EDGAR)

Analysis of XBRL data can be difficult because

different companies may use different tags.

Discriminant Function Algorithm

Algorithm used by the IRS to predict whether or not an individual correctly reported their taxable income. Uses things like historic tax returns, social media feeds, and personal financial data to profile taxpayers.

Information Matching

If certain numbers don't match with what the IRS knows then then the IRS will request clarification.

Related Examinations

If your tax return is related to another that they audited, you may be audited too. For example, if you are business partners with someone and their return is examined, it may affect yours since some of their numbers may change. This will prompt a look at your return.

Potential participating in an abusive tax avoidance scheme

If, for example, your CPA is examined for an abusive tax avoidance scheme, your return may be reviewed since the CPA prepared your return as well.

Four Types of AI​

Reactive, Limited Memory, Theory of Mind, and Self-aware.

Reactive AI

AI that is extremely limited as it can only react to established (predefined) rules and stimuli and does not have the ability to learn (Automatic thermostat, motion sensors, a calculator).

Limited Memory AI

AI that can access and use memory to learn and improve but is constrained by the amount of data they can retain and use in the decision-making process (Machine learning, image recognition, real-time sentiment analysis)​.

Theory of Mind AI

AI that considers, understands, and predicts the beliefs, emotions, intentions, and mental state of the human it is interacting with (Autonomous vehicles, virtual therapists and avatars).

Self-Aware AI

The terminator

AI Capabilities and Uses

• Machine Learning:​

  • Supervised Learning (Predictive Analytics)​

  • Unsupervised Learning​

• Reinforcement Learning​

• Neural Networks and Deep Learning​

• Natural Language Processing and Generation​

• Computer/Machine Vision​

• Automation and Robotics​

You get notified that there was an issue with the three-way match for a recent cash disbursement. When you go to check it, there wasn't actually an issue. This is an example of a (an)

False Positive

This type of Machine Learning using rewards and punishments could help me train an audit model to find fraud.

Reinforcement Learning

There are four broad types of AI. An audit could include this type that could utilize machine learning to assess risk levels.

Limited Memory AI

This type of regression can be used to help me categorize a transaction I'm auditing as "Fraudulent" or "Not Fraudulent"

Logistics Regression

Running a cluster analysis to find outliers is considered this "type" of machine learning

Unsupervised

You want a report to see if financial reporting controls are designed and operating effectively for the fiscal year 2021....what kind of report do you need!

SOC 1 - Type 2 Report

You want to see if there are controls around confidentiality that are designed effectively as of June 30th, 2021

SOC 2 - Type 1

You want to see if there are privacy controls that have been designed and are operating effectively for January 1st through March 30th, 2021.

SOC 2 - Type 2

You want to see if the controls around financial reporting are designed effectively as of October 31st, 2021.

SOC 1 - Type 1

You want to see if processing integrity controls have been designed effectively as of December 31st, 2021.

SOC 2 - Type 1

Wait a second...when I look at leading digits of all my expenses, 1 shows up the most, then 2, then 3...what is this magic?

Benford's Law

Name two red flags that may trigger an audit!

1. Unreported Income

2. Information mismatch

3. related exams

4. part of a tax scam

I am looking at tweets about Arrested Development. I want to look at post related to the show not the musical group. What classification method could help me?

Naive Bayes

I'm the algorithm that the IRS uses to determine who gets audited....aka, your worst nightmare!

Discriminant Function Algorithm

Optical Character Recognition utilizes Computer Vision as well as this AI Capability to translate paper invoices into machine readable text

Natural Language Processing

I want to figure out if a new customer will be categorized as "Good", "Bad", "Neutral". I should use this Analytic Type.

Classification

Are there any natural groupings in my data? This analytic type will tell me.

Clustering

I need to forecast salary expense. My boss says that headcount, revenue, and utilities expense will be good predictors. I can use this type of analytic to predict salary expense.

Regression

I want to look at just the expenses incurred for my company that exceeded $15,000.

Data Reduction

I have developed a profile for my ideal vendor. They have less than 100 employees, are local (within 50 miles), and have been in business for at least 10 years. I want to take this profile and find the vendors who fit it. I can use this analytic approach to do this.

Similarity Matching

If I want to evaluate a company's performanc year over year, I'll use this type of (directionally named) approach.

Horizontal Analysis

This is the section of the balanced scorecard where I'd list KPIs related to consumer complaints

Customer

Name two of the ways a balanced scorecard has to be balanced.

Short term vs. Long term

Subjective vs Objective

Strengths vs Weaknesses

External vs Internal