Determine Learning Analytics Strategy

learning analytics

collection, analysis, interpretation, and communication of data about learners and their learning that provides theoretically relevant and actionable insights to enhance learning and teaching

learning analytics methodologies

descriptive, diagnostic, predictive, and prescriptive

descriptive analytics

aggregate data to reveal trends from past performance—-simply present data

diagnostic analytics

investigate data to uncover why trends occurred

predictive analytics

utilize models to forecast future trends and outcomes

prescriptive analytics

recommend actionable strategies to optimize learning

purpose of analytics in higher ed

determine student success and institutional effectiveness

purpose of analytics in workforce development

track and analyze employee feedback and time spent training

identify patters and signs of knowledge retention

track patterns in employees’ responses to simulations

analytics before training

create collection system for training requests to identify knowledge gaps

get to know employees to better understand their specific needs

analytics during training

track participation and completion to determine whether to promote or modify training

analytics after training

assessments when combined with in course tracking and learner profiles can help identify which course elements contribute to learning outcomes and for whom

kirkpatrick model

model for training evaluation:

• learner’s reaction

• how much they’re learning

• how much the training has changed the employee’s behavior

• results of the program

learning analytics techniques

quantitative analysis, qualitative analysis, social network analysis

quantitative analysis

objective, numerical data, and stats

used in descriptive analytics

comprehension type assessment like mc

qualitative analysis

based on non-numerical info like observations, reflections, and interviews

used in diagnostic analytics

reflection-focused assessments

social media analysis

study of patterns or trends in relationships among groups of learners or between learners and instructors to determine engagement

used in predictive analytics

works well with discussion board

elements of learning analytics

data: information gathered

analysis: aggregated date used to measure training effectiveness on organization’s productivity and progress

action: decisions taken and changes made based on data analysis

Data measures

LMS metrics, social media stats, website analytics, surveys and interviews, and business reports and workplace evaluations

LMS metrics

analyze every aspect of eLearning course:

• performance

• feedback

• assessment results

Social media stats

monitor engagement and reach:

• preferences

• personal opinions

• habits

website analytics

reveal where online learners originate, how long they’re on the site, and what device they use:

• track site traffic

• engagement

• conversion rates

surveys, focus groups, and interviews

uncover honest opinions about eLearning course design; gives learners a voice

Business reports and evaluations

identifies areas for improvement:

• profit reports

• customer satisfaction ratings

• observations to note employee strengths and skill gaps

Data types/usage

data for improvement, data for research, and data for accountable

data for research

used to gather new data and test new theories

collects more data

testing strategy is to focus on one big test

data for improvement

used to observe student performance to answer questions about the effectiveness of instruction

collects some data

tests small changes to see what’s working and what isn’t

data for accountability

used to evaluate, rate, or rank performance

collects all recent and relevant data available

is interested in performance at a given point in time, but not testing anything

activity measures

related to learners’ participation in a course

performance measures

tell you how well you have trained

nominal data

categorical data with no numbers (race and ethnicity)

ordinal data

specifies an order to the info, but space or distance between data points are not fixed or known (strongly agree, agree)

interval data

specifies an order with equal, fixed, and measurable distances between data points; has no absolute zero (temperature, scores on test [90-95 and 95-100])

ratio data

specifies an order and fixed interval between data points, but this data type has an absolute zero, which indicates a complete lack of whatever is measured (heigh, length, time)

differentiation steps

1. adjust whole group instruction and pacing

2. strategically group learners

3. create individual pathways and identify interventions

Ethical data issues in k-12

collection and analysis may reinforce existing biases or overlook unique needs of individual students

ethical data issues in higher ed

overemphasis on data may result in standardized approaches that fail to accommodate the diverse backgrounds and experience of students, potentially perpetuating inequalities

ethical data issues in workforce

lack of careful consideration can inadvertently lead to discriminatory outcomes