ch 1

Chapter 1: Measurement - What Is It and Why Do It?

• Measurement is considered an essential component of good software engineering.

  • Software developers measure various characteristics to ensure consistent and complete requirements, high design quality, and code readiness for release.

  • Effective project managers utilize measurement to assess project deliverability and budget adherence.

  • Organizations leverage process evaluation metrics to select software suppliers.

  • Informed customers assess the final product to ensure it meets user requirements and quality standards.

  • Software maintainers must evaluate existing products to determine necessary upgrades or improvements.

• This book addresses numerous measurement concerns and elaborates on the fundamentals of measurement and experimentation.

  • Chapters 1 to 7 cover foundational measurement and its relevance to software engineering tools and techniques.

  • Chapters 8 to 11 focus on specific metrics, their applications, and explore software engineering measurement in extensive detail.

  • The material aims to provide comprehensive coverage that allows practical application of appropriate metrics across software processes, products, and resources.

• The chapter begins with a discussion of measurement in everyday life.

  • Measurement is portrayed as a common practice necessary for understanding, controlling, and improving our environment.

  • Measurement is characterized by the requirement for rigor and meticulousness.

  • The chapter further describes the role of measurement in software engineering, linking measurement needs to project goals and necessary inquiries in software development.

  • It examines the contrast between software engineering measurement and that in other engineering disciplines, proposing specific objectives for measurement in the software context.

• Lastly, the chapter offers a roadmap for upcoming measurement-related topics in the book.

1.1 Measurement in Everyday Life

• Measurement influences various systems in our daily activities.

  • Economic measurements regulate pricing and salary adjustments.

  • Radar system measurements aid in aircraft detection.

  • Medical system measurements assist in diagnosing illnesses.

  • Atmospheric measurements form the basis for weather forecasting.

  • Everyday interactions involve measurement, whether calculating total bills in shops or ensuring clothing fits properly.

• Measurement enhances our understanding of the world and interaction with our surroundings.

  • Examples of common measurements include:

  • Prices for comparing item values in retail.

  • Size measurements for clothing.

  • Distance and speed evaluations for travel.

1.1.1 What Is Measurement?

• The chapter defines measurement as:

  • Measurement: The process of assigning numbers or symbols to attributes of real-world entities according to clearly defined rules, thus describing them.

  • Important concepts:

  • Entity: Any object (e.g., a person or room) or event (e.g., a journey or software testing phase).

  • Attribute: A feature or property of an entity (e.g., area, cost, elapsed time).

• The distinction between measuring entities and their attributes is clarified:

  • For instance, one does not measure “a room,” but rather specific attributes like length or temperature.

  • In precise scientific discourse, it is inappropriate to use terms loosely, as clarity is paramount in engineering and science.

• Measurement often employs numbers or symbols to express attributes, enabling better understanding and comparisons.

  • Examples:

  • A price may represent dollars or pounds.

  • Height can be expressed in inches or centimeters.

  • Clothing sizes may be denoted as “small,” “medium,” or “large.”

• Measurement possesses a complexity; various authoritative interpretations exist regarding its definition.

• The chapter asks confronting questions about measurement, such as:

  • Is the color "blue" a true measure of room color or merely a description?

  • Can IQ tests accurately measure intelligence? Are wine quality ratings valid?

  • How reliable are measuring devices, and how can error margins be established?

  • Which units of measurement are appropriate for specific attributes?

  • What manipulations are permissible with measurement results?

1.1.2 “What Is Not Measurable Make Measurable”

• This phrase, attributed to Galileo, stresses the scientific obligation to find ways to measure previously unmeasurable attributes.

  • Measurement fosters visibility and understanding of the world.

  • Achievements in the physical sciences, medicine, and social sciences have enabled measurement of previously intangible attributes like intelligence, quality of air, and economic indices.

  • Measurement informs critical decisions and has become essential to various fields of research and practice.

• Hubbard's definition of measurement is:

  • Measurement: A quantitatively expressed reduction of uncertainty, based on one or more observations.

• Example statements from a business perspective illustrate potential measurements:

  • “There’s an 85% chance we will win our patent dispute.”

  • “We’re 93% certain our public image will improve after the merger.”

• The Rule of Five demonstrates how minimal data can reduce uncertainty:

  • For a population sample of five, there’s a 93.75% chance the median falls between the smallest and largest values.

  • Illustrative case: Assessing code method lengths using a small sample can yield acceptable estimations about the overall codebase.

1.2 Measurement in Software Engineering

• The importance of measurement in software engineering is discussed:

  • Software engineering encompasses techniques to build and maintain software products reliably and effectively.

  • Essential activities include managing, costing, planning, modeling, analyzing, specifying, designing, implementing, testing, and maintaining software.

• While computer science provides the theoretical foundation, software engineering applies those theories in a controlled and tested manner.

• The role of measurement is critical but often seen as an afterthought in software engineering, highlighting:

  1. A lack of well-defined measurable targets.

  2. Difficulty in quantifying or predicting product quality.

  3. Reliance on anecdotal evidence over rigorous studies for new technologies.

1.2.1 Neglect of Measurement in Software Engineering

• Measurement has been disregarded in software engineering compared to other disciplines despite its significance in ensuring success. Major issues include:

  • Failure to set clear measures leads to vague definitions of qualities like reliability and maintainability.

  • Lack of quantification hampers understanding cost distribution across development phases.

  • Absence of effective quality assessments limits the ability to articulate product reliability.

  • Decisions based on anecdotal evidence risk adoption of inefficient methodologies.

1.2.2 Objectives for Software Measurement

• Measurements in software projects serve critical roles, providing insights into project health and direction.

• Key objectives observed in managers include:

  • Understanding process costs and productivity.

  • Evaluating software quality and performance systems.

  • Assessing customer satisfaction implications based on requirements fulfillment and functional performance.

  • Identifying areas for improvement and assessing cost-benefit analysis of practices.

• Developer perspectives involve:

  • Formulating testable requirements, tracing fault origins, assessing goal achievements, and forecasting future trends.

1.2.3 Measurement for Understanding, Control, and Improvement

• Measurement serves to:

  1. Understand the current software situation through baseline metrics, setting goals for future enhancements.

  2. Control project dynamics by predicting outcomes and making iterative adjustments to processes and products.

  3. Promote continual improvement by incentivizing reviews based on measured quality.

• Development of an effective measurement program is crucial, but it's pertinent to manage expectations around accuracy.

1.3 Scope of Software Metrics

• Software metrics encompasses a range of measurement activities:

  • Cost and effort estimation models.

  • Data collection protocols.

  • Quality, reliability, and security metrics.

  • Complexity metrics and capability maturity assessments.

  • Management by metrics and method/tool evaluations.

1.3.1 Cost and Effort Estimation

• Achieving reliable cost and effort estimations during the software lifecycle is paramount.

  • Models like COCOMO II and function points are cited as common methods for predicting efforts based on various factors (such as product size and development capability).

1.3.2 Data Collection

• Data collection must be well-planned, ensuring measures are clear, consistent, and secure.

• Chapters delve into effective data collection strategies vital for robust scientific investigations.

1.3.3 Quality Models and Measures

• Diverse models exist to quantify software quality, often through a tree-like structure defining relationships between high-level quality aspects and their actionable criteria.

1.3.4 Reliability Models

• Reliability modeling serves to enhance the prediction and measurement of software system reliability, a sub-component of quality metrics.

1.3.5 Security Metrics

• The necessity of security measures grows as technology advances; assessing impact, likelihood, vulnerabilities, and threats holds paramount importance.

1.3.6 Structural and Complexity Metrics

• Early prediction of software reliability and maintenance demands measurement of structural attributes.

  • Usage of control flow graphs and UML diagrams serves to establish empirical theories supporting quality assurance.

1.3.7 Capability Maturity Assessment

• Capability Maturity Model Integration (CMMI) evaluates the maturity of software development practices and is impactful in measuring capability effectively.

1.3.8 Management by Metrics

• Measurement is integral for effective software project management, fostering informed decision-making through universally understandable reporting techniques.

1.3.9 Evaluation of Methods and Tools

• A structured evaluation process concerning new methods and tools is crucial for transparency and effectiveness in software development improvements.

1.4 Summary

• Measurement is fundamental for engineering disciplines; it is an essential part of software engineering practice.

  • Objectives for measurement must be clear and definitions for measures need precision.

  • A progressive approach toward measurement can enhance comprehension and improve the engineering discipline overall.

Exercises

1. Discuss measurement roles in sports performance evaluation.

2. Analyze how universities utilize measurements for student assessment and related issues.

3. Propose measurement strategies for evaluating software product quality.

4. Consider everyday measurements, evaluating their margins of error and impact on the entities measured.

5. Understand user perspectives on software measurement objectives.

6. Compare the relevance of errors per thousand lines of code for users and developers regarding software quality evaluation.

7. Suggest quality measurement methods for web browsers from an end-user's viewpoint.

Based on the notes provided and general engineering principles, here are the answers to the exercises:

1. Sports Performance: Measurement in sports provides a quantitative basis for evaluating athlete prowess and team efficiency. Metrics such as velocity ($v$) in sprinting, batting averages in baseball, or heart rate during training allow for objective benchmarking, progress tracking over time, and data-driven recruitment.

2. University Assessments: Universities utilize measurements like $GPA$, standardized test scores, and graduation rates to assess student performance and institutional effectiveness. Issues arise regarding whether these metrics truly capture 'intelligence' or 'learning' (mirroring the IQ test debate in Section 1.1.1) and whether they incentivize 'teaching to the test' rather than deep understanding.

3. Software Product Quality Strategies: Strategies include measuring defect density ($defects/KLOC$), assessing maintainability through structural complexity metrics (such as cyclomatic complexity in Section 1.3.6), and using reliability models to predict Mean Time To Failure ($MTTF$).

4. Everyday Measurements: A common example is a thermometer with an error margin of $\pm 0.1$ degrees. Regarding impact, measuring the temperature of a small liquid sample with a large thermometer can actually change the liquid's temperature by absorbing some of its heat, thus altering the entity being measured.

5. User Perspectives: Users primarily see measurement as a means to verify that software is fit for purpose. Their objectives focus on reliability, usability, and the fulfillment of functional requirements within budget, rather than internal structural complexity (Section 1.2.2).

6. Errors per KLOC Comparison: For developers, $errors/KLOC$ is a vital metric for identifying prone-to-failure modules and evaluating the development process. For users, however, this metric is less relevant than 'perceived reliability'; a user cares if the software crashes during their specific tasks, regardless of the statistical error density in the code.

7. Web Browser Quality for End-Users: End-users can measure quality through: 1. Page load latency (seconds). 2. Memory usage ($MB$) per tab. 3. Frequency of crashes or 'hangs'. 4. Compatibility percentage with modern web standards.