Data Science Flashcards

Qualitative vs quantitative data

Qualitative data describes qualities or categories (e.g. colour, opinions). Quantitative data is numerical and measurable (e.g. height, test scores).

Levels of measurement

Nominal: categories with no order. Ordinal: ordered categories with uneven gaps. Interval: ordered with equal gaps but no true zero. Ratio: equal gaps with a true zero.

Active vs passive data collection

Active: people knowingly provide data (e.g. surveys). Passive: data is gathered in the background (e.g. sensors, website tracking). Both can be manual or computerised.

Data quality factors

Quality is judged by validity (measures what it should), accuracy (correct), reliability (consistent results) and relevance (suits the purpose). Applies to primary and secondary data.

Informatics

The practice of using technology to organise, analyse and store data to support decision-making and problem solving.

Ways to present data

Graphs and charts, infographics, dashboards, reports, network diagrams and maps. Each suits different types of data and audiences.

Structured vs unstructured data

Structured data fits predefined formats such as spreadsheets and databases. Unstructured data does not (e.g. emails, images, posts) and is often more qualitative.

Alternative data (likes, emoticons, memes)

Can show the overall feeling about a topic and act as feedback. Context and implications must be considered carefully to avoid misinterpretation.

Raw vs processed data

Raw data may contain incomplete or erroneous entries. Processed data may overlook data points and contain interpretive bias.

Data bias

Unfair skew in data, often from sampling bias (unrepresentative sample) or confirmation bias (favouring data that supports existing beliefs).

Data sources and limitations

Sources include surveys, experiments, existing reports and social media. Each has limitations, such as errors, uncertainty and bias, that affect how far results can be trusted.

Blockchain

Each block holds a hash and references adjacent blocks to form a chain. Tampering is rejected because the network compares it against existing chains, and a hacker would need to alter 51% of nodes.

Blockchain uses

Online voting, online identities, tracking items of value and recordkeeping. These rely on its tamper-resistant, verifiable records.

Software features affecting privacy and security

Autofill can expose personal data, public connections are easier to intercept than private ones, checkboxes can mean unknowingly agreeing, and terms of agreement are often unread.

Big data

Vast amounts of data produced daily and analysed with modern technology. Defined by volume (amount), variety (types) and velocity (speed of generation).

Data warehouse

A central repository that stores data from many sources for an organisation to analyse and use.

Data mining

Analysing large datasets to uncover trends and patterns. Benefits: better decisions, customer insights, fraud detection. Risks: privacy concerns, data bias, misinterpretation.

Impact of data scale

Larger volumes need more storage, and growing storage needs scalable solutions. Real-time and continuous streaming gives instant insights and opens opportunities for machine learning.

Data scale: behaviour and ethics

Data-driven systems shape people's actions, decisions and opinions. Ethical issues include privacy, data ownership, constant surveillance and digital footprints.

Evaluating data storage methods

Options are local, portable and cloud storage and data warehouses. Compare them on budget, volume of data, real-time access and security.

Ethical use of data

Data should be fair and unbiased and respect individuals' rights. Achieved by acknowledging the source, obtaining consent and anonymising data.

Social, ethical and legal issues

Bias leads to unfair representation and can breach anti-discrimination law. Copyright misuse breeds distrust and risks infringement. ICIP misuse harms communities and breaches laws protecting Indigenous cultural property.

Privacy Act 1988

Regulates how businesses handle personal information, requiring consent and following the Australian Privacy Principles (APPs) for handling, storing and sharing data.

Data protection authorities

OAIC enforces the Privacy Act, oversees the APPs, handles violations and gives guidance. ACSC focuses on cybersecurity and protecting individuals and organisations.

Indigenous data sovereignty

The right of Aboriginal and Torres Strait Islander Peoples to govern how data about their communities is collected, owned, used and shared.

Data literacy and timeframes

Data literacy is the ability to interpret, evaluate and apply information, which helps counter misleading curated content. Outdated data can lead to poor decisions or unnecessary risks.

Signals and data swamps

Signals (notifications, recommendations, targeted ads) sway behaviour and decisions. A data swamp is a mass of unorganised or irrelevant data, fixed with cleaning, quality standards and metadata so it becomes a structured data lake.

Summarising and analysing in spreadsheets

Summarise with statistical measures (mean, median, mode), aggregation and visual summaries. Analyse with charts (line for time, bar for categories, scatter for relationships), standard deviation, trendlines and what-if analysis.

Sorting, filtering, pivot tables and slicers

Sorting arranges data and filtering shows only data meeting criteria. Pivot tables dynamically organise data with real-time calculations, and slicers add interactive filters for dashboards.

Flat-file vs relational database

A flat-file database stores data in a single table and is simple and easy to access. A relational database uses multiple linked tables, supports complex queries and removes redundant data.

Machine learning vs statistical modelling

ML trains a model on data to learn input-output relationships, test it and then predict, without explicit programming. Statistical modelling uses maths (e.g. regression) to test hypotheses and find relationships between variables.