PR2 | Data Collection Procedures

Qualitative Data Collection

• open-ended

• includes interviews, observations, documents, AV materials

Qualitative Data Analysis

• information in narrative form

• organize text into themes or categories, then summarize by percentages

• to understand more complex issues related to how and why

• use text and image data to relate to themes

Quantitative Data Collection

• closed-ended

• includes survey instruments, behavioral checklists, secondary data, datasets

Quantitative Data Analysis

• numerical information

• perform descriptive and inferential statistics

• to answer questions related to magnitude

• uses numeric data to relate variables and compare groups

Data Gathering Approaches

• experiments

• simulation

• observation

• questionnaire

• interview

• survey

• devices (logs, files, signals)

• datasets (internet data)

• focus group discussion

• document review

• diary study

• secondary data sources (records, test scores, reports)

Factors in Selecting Data Collection Method

• available time

• accessibility of available resources

• amount of details needed

• quantity over quality(volume vs richness/ extensiveness of the responses)

• safety of all stakeholders - proponents, data collectors, participants

Survey

• systematically collects data from a small number of people to be representative of a large number of people to be studied to get data about their attributes, behaviors, preferences, attitudes, opinions

• helps describe demographic information, draw patterns from the population studied, explain trends out of a phenomenon

• correct design is vital

Interviews

• collect a richer set of data from a small number people about their attributes, behaviors, preferences, attitudes, opinions, knowledge

• elicits people’s views and perspectives in a detailed and comprehensive manner

• uses direct and open-ended questions to go beyond surface level understanding

• helps you explain, better understand, and explore research subjects’ opinions, behavior, experiences, phenomenon

Designing Interview Questions

• Consider the target respondents, the kind of information needed, and what in-depth information do you want to explore.

• Questions should be clear and easy to understand.

• Avoid leading and closed-ended questions.

Developing an Interview Guide

• introduce yourself

• explain the purpose of the interview

• explain the interview’s length and the type of recording

• explain ethical procedures

• devise questions

• have a sequence to questions and topics by grouping them into themes

• bringing notetaking instruments

Participant Observation

• collects data by observing the sample in their natural environment

• keeps detailed records of what occurs, including those thins characteristically taken for granted

Types of Participant Observation

• external

• passive

• balanced

• active

• total

External Participation

observation from video recordings or off-site camera

Passive Participation

researcher is present at the scene but is merely a spectator

Balanced Participation

researcher is an insider and outsider and may participate in some activities

Active Participation

researcher does what others do

Total Participation

researcher is a natural participant

Computing Resources/Dataset Types

Used in more technological based research, includes:

• system output

• online data (from the web)

• sensors and devices

• online data (financial records)

System Output

• log files (activity and chat logs)

• communication apps (emails and messages)

Online Data (Web-based)

• social media posts

• photos with geolocation

• check-ins

• waze and maps

Sensor and Devices

• mobile phones

• wearables

• sensors and tags

• beep cards

Online Data (Financial)

• purchases (credit and loyalty cards)

• banking and e-payments

• government services

Ethical Consideration

• privacy and confidentiality of participants’ data

• participation is volunatry

• withdrawal is allowed

• ethical recording during data collecion

• ethical practices must be observed at all times

Sample Research Forms

• Informed Consent

• Pre-Assessment

• Daily Check-In

• Post-Assessment/Evaluation

Data Collection Instrument Sources

• Books

• Professional Journals

• Other Researchers

• Combine/Adapt multiple tools

• Develop your own research

Data Collection Instrument Qualities

• valid and reliable

• devised tool should provide comparable data every time

• cheating should be minimized

• appropriate data should be gathered to test hypothesis

• should be based on research objectives

Questionnaire/Survey Characteristics

• filled by all applicants in the sample

• can be oral or written

• uses clear language

• has a single objective

• one-to-one correspondence

• observes correct grammar, spelling, and word choice

• has a natural and logical flow

Important Things to Remember Regarding a Survey

• high responses does not guarantee high completion

• include a good invitation which describes the purpose or goal, mentions survey length, and provides contact information in case participants have concerns

• survey must be pre-tested by reserchers

Types of Questions

• open-ended

• closed-ended

Open-Ended Questions

respondents respond in their own words, for insights

Closed-Ended Questions

subject chooses between the alternatives or to rate using a Likert scale, for structured data

Questionnaire Examples

• respondents can pick one of two choices

• respondent can pick from multiple choices

Cafeteria Questions

respondents are asked for their point of view

Rank-Order Questions

respondents are asked to rank choices

Checklist

respondents can respond to multiple items any time

Structured or Direct Interviews

• questions and acceptable responses are specified in advance

• responses are rated for appropriateness of content

• also called standardized interviews as they are pre-planned

• research has prior knowledge about behavior and event of interest

Unstructured Interview

• not directed by questions or comments as to what the candidate should be asked

• no set format is followed

• interviewers look for traits of character and nature for his aspirations, strengths, weakness, potential, etc.

• more conversational and allows flexibility in questioning the subject

Quantitative Variables

• categorical

• continuous

Categorical

describe people and things in discrete categories

Continuous

numbers on a continuous scale that can be measured with different levels of precision

Types of Categorical Variables

• frequencies

• crosstab

Frequency

• counts from survey responses

• usually translated into percentages

• used to know how many people checked each answer

Crosstab

• calculation of a set of frequencies using two or more categorical variables to describe a group

• contingency table is used to show how the members of the group are sorted into subgroups based on their categories

• used for comparing one group to another

Types of Continuous Variables

• measures of central tendency

  • median

  • mean

  • mode

• variability

  • range

  • standard deviation

Mean

• used for getting the average for a set of continuous numerical values

• used for calculate a “typical” number to describe a whole group of numbers or the people they represent and compare it to another typical value

Median

• describes a set of values with extremes or outliers

• used for obtaining the number in the exact middle in a range of values

Mode

• the number or category that appears most frequently

Range

• difference between highest and lowest scores

Standard Deviation

• used to show the degree of which responses differ from each other

Change score

• used to report changes from pre to post test

Approaches for Qualitative Analysis

• content analysis

• grounded analysis

• social network analysis

• discourse analysis

• narrative analysis

• conversation analysis

Steps in Qualitative Analysis

1. Level 1 Coding - Organize data into broad themes/categories.

2. Level 2 Coding - Organize the data in each theme into related sub-categories/themes.

3. Summarize results in a table showing percentages of responses in each category in both levels.

FIRE-O (Types of Data)

• Focus Group Discussion

• Interview

• Records

• Experiments

• Observation

Document Analysis

using existing records

Focus Groups

small group discussions; the best method for facilitating group interaction to explore collective experiences and diverse viewpoints

Surveys | Strengths and Limitations

They are fast and easy analyze, however they can lack honesty if done anonymously.

Interviews | Strengths and Limitations

They provide detailed and personal insights, however are time-consuming and are prone to researcher bias.

Observations | Strengths and Limitations

They can help analyze real life behavior, however the researcher’s interpretation can be biased.

Records | Strengths and Limitations

They are reliable and objective, however, they may not be updated or are missing context.

Triangulation

combining surveys, interviews, and observations for better accuracy

DART (Triangulation Value)

• diverse viewpoints

• accuracy

• reduced bias

• trustworthiness

Quasi-experimental

pre-test/post-test; comparing two groups

Standardized Questionnaire

a large-scale data collection that reaches many and is scalable

Digital Trace Analysis

An unobtrusive form of data collection that can collect log-in times, post frequencies, and user activity from a platform

SOAP (Ethical Research)

• study explained

• opt-in (voluntary)

• anonymity

• purpose clear