COMPSCI 169A Quiz 4

Modules 7 + 8

does behavior driven design emphasize validation or verification

validation (building the right thing)

what is the Connextra format for user stories

As a ( type of stakeholder )

So that I can ( achieve some goal )

I want to ( do some task )

what does the SMART acronym stand for in relation to user stories

Specific, Measurable, Achievable, Relevant, Time-Boxed

how do we most effectively conceptualize a UI and simultaneously work with customers

lo-fi UI mockups via pen and paper or whiteboard → storyboards to facilitate clicking on links and moving between pages

how do we measure productivity when working on user stories

assign points to each and measure velocity in each iteration

what is a spike in relation to Agile development

a short investigation into a technique or problem (eg. new tech or library) that the team wants explored before starting to code

what is the backlog in terms of user stories and Agile development

collection of stories that have been prioritized and assigned points but have not been started in this iteration

what is the “icebox” category in Pivotal Tracker

contains user stories that have not yet been prioritized or started

what is the “current” category in Pivotal Tracker

stories in progress

when is a task in Pivotal Tracker moved from the icebox into the backlog

when it goes from estimated to prioritized

when is a task in Pivotal Tracker moved from the backlog into current

when it goes from being assigned to being started

true or false: you should track bugs/debugging in Pivotal Tracker and assign points to it

false

true or false: you should not use Pivotal Tracker as a to-do list

true

what is the framework that allows you to transform user stories into real tests

Cucumber

how many scenarios should a singular feature have

at least one

a user story typically maps to how many features

one

how many steps does each scenario usually have

3-8

how do step definitions use Ruby code to test steps in Cucumber user story tests

matching regexes

what are the five step keywords to make scenarios specific

Given, When, Then, And, But

true or false: the five step keywords in Cucumber scenarios are interchangeable because they are all aliases for the same methods

true

what is a happy path in a Cucumber scenario

when everything goes according to plan

what is a sad path in a Cucumber scenario

when things do not go according to plan; it simulates when a user makes a mistake or does not have proper set up or accesses for a given feature

what is the tool that simulates a headless browser to help with testing

Capybara

how do you form capture groups in Regex so that you can capture parts of a string or step def so that you can use it in the “do” block of your test?

([^”]*) → looks for any sequence of characters that is not a double quote

eg. When /^(?:|I)fill in "([^"]*)" with "([^"]*)"/ do |fld, val|

what is the object in Capybara that returns a representation of a top-level HTML element that supports expectations (eg. expect(...).to have_css('#some_div') → what is ...?

page

what tool allows you to “time travel” to support time- or date-sensitive features

Timecop

what tag must you include before every scenario that tests a page whose functionality relies on JavaScript

@javascript

what is an imperative scenario

long, includes specifics about implementation and simulating what a user would do

what is a declarative scenario

describes state and high-level actions, often in terms of domain language (think “I declare bankruptcy!” from The Office → If Michael had his way with things, he would be able to say this and things would happen automatically behind the scenes without him actually having to do something)

descriptive declarative scenarios tend to ___ steps from imperative scenarios to make themselves more concise

reuse

what is an explicit requirement

in BDD, user stories developed by stakeholders. usually captured by acceptance tests

what are implicit requirements

logical consequences of explicit requirements. usually captured by integration tests

steps written in “domain language” (informal language specific to your app) tend to be more [ imperative / declarative ]

declarative

what does the requirements elicitation step involve in P&D perspective on documentation?

• understanding functional feature and non-functional performance requirements

• project manager must learn what the customer wants via interviewing, cooperatively creating scenarios, or creating use cases

what does the requirements documentation step involve in P&D perspective on documentation?

• equivalent to the writing of user stories in Agile

• document requirements laid out in Software Requirements Specification (SRS) → may have one high-level SRS for management/marketing and one low-level SRS for dev team

• have stakeholders read SRS, build a basic prototype, or generate test cases to test validity, consistency, completeness, and feasibility of requirements

what does the cost estimation step involve in P&D perspective on documentation?

• manager decomposes SRS into tasks

• estimates weeks per tasks (one should be bound between 1 and 8 weeks)

  • estimate via experiential estimate → based on PM’s experience

  • or via quantitative estimate → lines of code or constructive cost model

what does the scheduling and monitoring step involve in P&D perspective on documentation?

• equiv to Agile assigning points and velocity

• use a chart to show task parallelism and find critical (longest) path to create optimal schedule

• monitor progress by comparing predicted time for tasks and expenditures to the actual time and expenditures

what does the change management for requirements, cost, and schedule step involve in P&D perspective on documentation?

• use version control systems for both the code and SRS in case customers ask for changes to requirements as project evolves

what does the requirements checking step involve in P&D perspective on documentation?

• PM uses tools for requirements traceability to check if the code is passing the tests that deal with all the requirements in the SRS

• forward traceability → traditional path checking from requirements to implementation

• backwards traceability → mapping from a detailed requirement back to a high-level requirement

what does the risk analysis & management step involve in P&D perspective on documentation?

• improve accuracy of budget and schedule by identifying risks early

• technical risk → eg. database can’t scale

• organizational risk → eg. team is unfamiliar with a tool

• business risk → eg. we will deploy too late for the market

• might create risk table to identify and prioritize risks

Cucumber is associated with [ BDD / TDD ] and RSpec is associated with [ BDD / TDD ]

BDD; TDD

what is the BDD + TDD cycle

1. fail a Cucumber step

2. write a failing RSpec test

3. pass the RSpec test

4. pass the Cucumber step

what is the Red-Green-Refactor testing framework

• red → capture something your code should do in a test, and fail the test because you haven’t written code for it yet

• green → write out the simplest possible code to make the test pass

• refactor → DRY out with commonality with other tests

what does the acronym FIRST stand for in testing?

Fast, Independent, Repeatable, Self-Checking, Timely

in the Agile lifecycle, devs are expected to write their own automated tests, but sometimes a QA team may conduct manual human-run [ unit / functional / integration / acceptance ] tests

acceptance

adding a new feature requires you to write three new things

• new route

• new controller method

• new view

what is a unit test

checks the result of calling a function that needs no preconditions or setup, has no side effects, and doesn’t depend on or call other functions

what is a pure funciton

functions that have no side effects and whose return value is always the same for the same arguments

what does AAA stand for in RSpec testing

Arrange (like Given), Act (like When), Assert (like Then)

what does an expectation look like in RSpec?

expect(expression).to(matcher)

what is the difference between before :each and before :all

• both set up testing environment (eg. creates variables)

• before :each → setup before each test

• before :all → setup once before all tests are run

how to isolate unit tests that have depended-on components (relies on results from calling other methods)

• Arrange → isolate SUT from dependencies via doubles that force desired behavior

how to isolate unit tests that have side effects (causes change in application state visible outside of test code itself)

• Arrange → observe relevant state before executing test code

• Assert → observe state again and check for side effect

how to isolate unit tests that are not pure functions (output depends on other factors like time of day or randomness)

• Arrange → isolate non-determinism in stubbed method call using a double; use Timecop to stub the system clock

what is a seam in testing

a place where we can change the app’s behavior without changing the source code of the app → allows for isolation of behavior in testing

what does expect(Movie).to receive(:find_in_tmdb).with('hardware') do?

• expects the Movie class to receive a call to a method named find_in_tmdb with a particular argument

• key is that it doesn’t actually matter whether or not the method exists or works → the test will stub the method call

• have this line before the Action is taken, because it creates a spy on the method which reports whether or not the method got called

what is a double in testing

something that stands in for the real thing, like a controller method for searching TMDb that doesn’t exist yet

what is an mock / instance double in testing

creates a stunt double instance of a class that says that it is an instance of Movie, but it can’t do anything a Movie can do

• need to endow it with desired behaviors for testing via allow(double).to receive(:title).and_return('Snowden')

• use this when you need a stand-in with limited ability

what is the difference between expect(obj).to receive and allow(obj).to receive

expect errors out if the test finishes with obj never having received a call, while allow merely lets obj receive the call, but we do not care if it is called at all because that is not the test’s purpose

unit testing should sub [ close to / far from ] the top of the stack and integration testing should stub [close to / far from ] the top of the stack

close to; far from

what is a fixture

statistically preload some known data into database tables before testing

why are fixtures usually undesirable

dependency issue (not necessarily independent)

when might using fixtures be good

for testing configuration data or data that does not change often or at all (eg. timezone, language). something that is not manipulated by models

what is a factory

allows you to create objects on a need-by basis and selectively override attributes

when might using factories be good

when you want to easily set up model objects independently → they will be unaffected by the presence of objects they don’t care about

what is the gem that helps you build and use factories

FactoryBot

when FactoryBot builds a new object, such as FactoryBot.build(:movie, :title => 'Milk'), what happens

it creates movie in memory without adding to the DB, and overrides the regular :movie factory’s title with ‘Milk’ → allows you to create a Movie object without having to worry about the details

true or false: factories make has_many and belongs_to relationships easy in testing

true → you can create a Review without first having to create a Movie if you set up the factory nicely

what kind of tests (3) are Cucumber scenarios?

system, acceptance, and full-stack tests

what kind of test are RSpec tests?

integration tests

what is a module / function test

tests behavior across methods and classes

what is a smoke test

minimal attempt to operate software → see if anything is obviously wrong before running the rest of tests

what is a compatibility test

testing an app’s UI in different browsers

what is a regression test

ensures that previously-fixed bugs do not reappear

performance / stress / computer security tests are all examples of what kind of tests?

non-functional tests → ensures software meets operational criteria

what is an accessibility test

ensures software is usable by persons with disabilities

what is method coverage

call every method

what is call coverage

call every method from every call site

what is statement coverage

have I touched every statement → eg. touch every if statement (does not have to execute both true and false cases)

what is branch coverage

test every branch in both directions

what is branch coverage + decision coverage

evaluate every subexpression in the conditional such that they are true and false

what is all-paths coverage

tests every possible route (eg. if statement with variables x, y, and z → test 2³ = 8 possible paths)

what is modified condition / decision coverage (MCDC)

• every point of entry and exit in the program has been invoked at least once

• every decision in the program has taken all possible outcomes at least once

• each condition in a decision has been shown to independently affect that decision’s outcome

what is the level of coverage that is generally considered good enough

high statement coverage

basic blocks are executable blocks of code with no possibility of branching. how is this related to control flow coverage

if blocks are joined in a graph via conditionals in the code, then control flow coverage is just covering the graph

what is define-use coverage

given a variable x, a define-use pair consists of a site where it is assigned a value and a place that value is used. DU-coverage checks the fraction of all pairs that are covered

what are black-box tests

based on software’s external specification

what are white/glass-box tests

design reflects knowledge about software’s implementation

what are mutation tests

small, syntactically-legal changes are automatically made to program source code → small changes should cause at least one test to fail. otherwise, lack of test coverage

what is random / black-box fuzz testing

generates completely random data or randomly mutates valid input data

what is smart fuzzing

incorporates knowledge about the app’s structure to construct realistic but fake fuzz data

what is white-box fuzzing

uses symbolic execution to simulate program execution and observes the conditions under which each branch is taken or not. then generates fuzzed inputs to exercise not-taken branch paths 

in P&D, who performs unit testing

devs

in P&D, who performs high-level tests like module, integration, system, and acceptance tests?

QA team

what is top-down integration for QA testing in P&D

starts at the top of the dependency graph to get high-level functions like UI working early. con is that it requires many stubs to get the app to work

what is bottom-up integration for QA testing in P&D

start at the bottom of the dependency graph to avoid having to create stubs and be able to integration everything in a module. con is that you can’t see app working until all code is written and integrated

what is sandwich integration for QA testing in P&D

happy medium between top-down and bottom-up integration → reduce stubs by integrating some units bottom up, and try to get UI operational by integrating some units top down

how is the code verified in P&D (3 options)

• human does proof

• computer does proof via automatic theorem proving

• computer does proof via model checking