Privacy Engineering, Differential Privacy, and Secure Computation Review

0.0(0)
Studied by 0 people
call kaiCall Kai
learnLearn
examPractice Test
spaced repetitionSpaced Repetition
heart puzzleMatch
flashcardsFlashcards
GameKnowt Play
Card Sorting

1/38

flashcard set

Earn XP

Description and Tags

Comprehensive vocabulary flashcards covering privacy engineering objectives, differential privacy parameters, secure multi-party computation, federated learning attacks and defenses, k-anonymity, and pipeline security controls.

39 Terms

1
New cards

Workload Identity Least Privilege

The access control practice of provisioning a dedicated, scoped identity per data pipeline job (such as ingest, validate, and publish) rather than sharing a single privileged account capable of dropping tables.

2
New cards

Ephemeral Build Runner

A temporary compute environment that is provisioned on demand and automatically wiped after job completion, preventing sensitive residual artifacts (such as temporary files containing SSNs) from remaining accessible on shared build infrastructure.

3
New cards

Pipeline Quarantine

A data validation pattern in which records with invalid or impossible values (such as ages of 240240 or negative GPAs) are automatically isolated into a separate store without crashing the entire processing pipeline.

4
New cards

Data Lineage Records

Metadata linking each published pipeline output to its specific input dataset version, transform code version, and job run ID, allowing teams to audit, reproduce outputs, and trace the downstream impact of bad inputs.

5
New cards

Additive Secret Sharing Modulo 100100

A multi-party computation technique where a numerical secret ss is divided into shares sis_i that satisfy \big(\begin{matrix} \bigsum_{i} s_i \big) \bmod 100 = s\text{,} ensuring an isolated individual share reveals zero information about the true input value.

6
New cards

Malicious Adversary Model

An adversary model in secure multi-party computation where compromised participants actively deviate from the prescribed protocol, such as by transmitting falsified values to manipulate the reconstructed output.

7
New cards

Additively Homomorphic Encryption

A cryptosystem that allows a third-party compute service to perform additions directly over encrypted ciphertexts without decrypting or revealing the underlying plaintext inputs.

8
New cards

Model Poisoning

An adversarial machine learning attack where malicious participants inject corrupted weights or updates into a federated training pool, degrading model accuracy for targeted subgroups or planting backdoors.

9
New cards

Robust Aggregation

A federated learning defense mechanism that analyzes, clips, or excludes anomalous client updates to neutralize model poisoning attacks before model parameter averaging.

10
New cards

Membership Inference Attack

A privacy attack in which an adversary queries a trained machine learning model with a target individual's record to determine whether that record was included in the training set, exploiting abnormally high prediction confidence or low loss.

11
New cards

Differentially Private Training

A training framework (such as DP-SGD) that bounds and injects calibrated noise into weight gradients, preventing the trained model from memorizing individual records and mitigating membership inference attacks.

12
New cards

Gradient Inversion Attack

An attack against federated learning systems where an untrusted server reconstructs raw client training records and features directly from observed individual model gradient updates.

13
New cards

Secure Aggregation

A cryptographic federated learning protocol ensuring the central server can only observe the aggregate sum of all client updates, blinding the server to any single client's individual gradient update.

14
New cards

kk-Anonymity

A property of a sanitized dataset requiring that every unique combination of quasi-identifiers appears at least kk times, grouping records into equivalence classes of size at least kk.

15
New cards

Distinct ll-Diversity

An extension of kk-anonymity requiring every quasi-identifier equivalence class to contain at least ll well-represented, distinct sensitive attribute values.

16
New cards

Homogeneity Attack

A vulnerability in kk-anonymous data where all records in an equivalence class share the identical sensitive attribute value, allowing an attacker to deduce a target's sensitive attribute with certainty despite k>1k > 1.

17
New cards

Generalization

A de-identification technique that transforms quasi-identifier values into broader ranges or categories (e.g., ages to intervals like 25–2925\text{--}29 or ZIP codes to 90∗∗∗90^{***}), increasing equivalence class sizes while decreasing data precision.

18
New cards

Predictability

A privacy engineering objective that enables reliable assumptions by individuals and operators about personal data processing and whether systems operate in an expected, disclosed manner.

19
New cards

Disassociability

A privacy engineering objective that enables the processing of personal data without allowing the linkage of data records to specific individuals outside the authorized context.

20
New cards

Manageability

A privacy engineering objective that provides granular operational controls to administer personal data, including enabling users to inspect, opt out of, or delete their stored historical information.

21
New cards

Privacy Risk Mitigation

A risk treatment approach that alters data collection practices or technical features (such as replacing granular borrowing titles with broad categories and requiring opt-in) to lessen privacy exposure.

22
New cards

Organizational Impact

The downstream operational, financial, legal, or reputational consequences borne by an entity—such as regulatory penalties, civil lawsuits, or loss of donor trust—following a privacy violation.

23
New cards

Differential Privacy Parameter δ\delta

The failure probability parameter in approximate differential privacy; for a dataset of size NN, δ\delta must be chosen to be substantially smaller than 1N\frac{1}{N} (e.g., δ=0.01\delta = 0.01 on N=10,000N = 10{,}000 is unacceptably large).

24
New cards

Query Repetition Attack

An attack where an adversary queries a differentially private interface repeatedly with the same question, allowing the average of returned answers to cancel out injected noise and continuously increase privacy loss.

25
New cards

Basic Composition Theorem

A differential privacy theorem stating that the overall privacy budget expended across multiple independent analyses on the same population equals the direct linear sum of the individual privacy parameters: ϵtotal=∑ϵi\epsilon_{\text{total}} = \sum \epsilon_i.

26
New cards

Longitudinal Pseudonym Linkage

A re-identification risk where assigning a persistent random token to an individual across multiple successive releases allows an adversary to track and link their records over time into an exhaustive personal profile.

27
New cards

Privacy Risk Statement

A structured risk articulation that explicitly binds a specific system data-handling action to its context and the resulting concrete harm experienced by affected individuals.

28
New cards

Targeted Re-Identification Attack

An adversary model where an attacker uses specific background knowledge about a single known individual (such as age, ZIP code, and hospital visit date) to search and locate that exact individual's record in a released dataset.

29
New cards

Function Creep

The gradual expansion of a software system or dataset's use beyond the originally specified operational purpose, such as when sensitive student risk scores are retained indefinitely in a multi-department query store.

30
New cards

Collusion in Multi-Party Computation

A vulnerability in multi-party computation where a subset of cooperating parties pool their private inputs and the final output (such as an average) to mathematically determine the private input of the remaining non-colluding party.

31
New cards

Unique Quasi-Identifier

A non-direct identifying data field or attribute value—such as the job title 'university president' appearing once in a staff survey—that uniquely isolates an individual within a dataset.

32
New cards

Short-Lived Workload Credentials

Temporary, auto-expiring authentication tokens issued to computational workloads, ensuring that stolen credentials become invalid quickly and cannot be exploited long-term.

33
New cards

Relative Error in Differential Privacy

The ratio of injected noise to the magnitude of the underlying true value; adding noise of identical scale causes a much larger relative distortion to small query counts (e.g., 2020) than to large query counts (e.g., 20,00020{,}000).

34
New cards

Laplace Mechanism Scale Parameter (bb)

The dispersion parameter of the Laplace noise distribution, computed as b=Δfϵb = \frac{\Delta f}{\epsilon}, where Δf\Delta f is the query's global sensitivity and ϵ\epsilon is the allocated privacy budget.

35
New cards

Sample Uniqueness

A condition where a record's quasi-identifiers appear only once within an extracted sample of a population, which does not guarantee that the individual is unique within the broader underlying population.

36
New cards

Raw Zone Access Isolation

A data architecture boundary enforcing least privilege by restricting read access to unvalidated files containing PII exclusively to the dedicated ingestion and validation jobs moving records to curated zones.

37
New cards

Data-Dependent Clipping Bound

A differential privacy design flaw where a clipping threshold is chosen based on the maximum observed value within private data, causing the clipping parameter itself to leak sensitive information about outlier records.

38
New cards

Client-Level Differential Privacy

A differential privacy standard that bounds the aggregate influence of all data generated across a user's entire device or session history, rather than bounding the influence of each individual message or event.

39
New cards

Erroneous Record Linkage Harm

A privacy harm where an incorrect match generated from outdated or inaccurate auxiliary registries falsely attributes a sensitive condition or negative flag to an individual who does not have it.