311 (2) System Programming

Page 2: Systems Programming in 2014 and Beyond

  • Bjarne Stroustrup (C++):

    • Deal with hardware

    • Resource constraints

    • Fine-grained control

  • Rob Pike (Go):

    • Server writing

    • Cloud computing

  • Niko Matsakis (Rust):

    • High security requirements

  • Andrei Alexandrescu (D):

    • Write your own memory allocator

    • Forge a number into a pointer

Page 3: 10,000-foot View of Systems

  • Layers of a system:

    • Hardware

    • Operating system

    • HW/SW interface

    • CPU, memory, storage, network, GPU, clock, audio, radio, peripherals

    • OS/app interface

    • C standard library

    • C application

    • C++ STL/boost/standard library

    • C++ application

    • JRE Java application

Page 4: A layered view

  • Layers in a system:

    • Layer below the system

    • Client layer below

    • Client layer

    • Provides service to layers above

Page 5: A layered view

  • Layers in a system:

    • Layer below the system

    • Client layer below

    • Client layer

    • Constrained by performance, footprint, behavior of the layers below

    • More useful, portable, reliable abstractions

Page 6: Example layered system

  • Example of a layered system:

    • Hardware

    • Software

    • File System

    • Block Interface

    • Application

    • Files and directories

    • File System Interface

Page 7: Example system: operating systems

  • Operating system:

    • Abstracts away messy details of hardware

    • Modules: file system, virtual memory system, network stack, protection system, scheduling subsystem

    • Major system with engineering tradeoffs

Page 8: Example system: web server

  • Web server:

    • Abstracts away messy details of OSs, HTTP protocols, database and storage systems

    • Modules: HTTP server, HTML template system, database storage, user authentication system

    • Many tradeoffs: programmer convenience vs. performance, simplicity vs. extensibility

Page 9: Systems and Layers

  • Layers are collections of system functions that support some abstraction

  • Abstraction may be provided by software or hardware

  • Examples from the OS layer: processes, files, virtual memory

Page 10: A real world abstraction

  • Abstraction example: P R N 4-D 3 2-L

Page 11: What makes a good abstraction?

  • An abstraction should match the cognitive model of users

  • Cognitive science is concerned with understanding the processes that the brain uses to accomplish complex tasks

Page 12: How humans think

  • Our brains receive sensor data to perceive and categorize the environment

  • Things that are easy to assimilate are close to things we already know

  • The simpler and more generic the object, the easier it is to classify

Page 13: A good abstraction

  • Computers have a desktop with files, folders, trash bins, panels, switches instead of streets with buildings, rooms, alleys, dump-trucks, levers

Page 14: Computer system abstractions

  • Basic abstractions used for modern computer systems

Page 15: Processes

  • Processes are hardware supported structures that form independent programs running concurrently within operating systems

  • Execution abstraction provides sole control of the entire computer

Page 16: Files

  • A file is an abstraction of a data object

  • Files can be much more than just data, in UNIX nearly everything is a file

Page 17: Virtual Memory

  • Virtual memory abstraction provides control over an imaginary address space

  • OS/hardware work together to map the address onto physical memory addresses or addresses on disk

Page 18: Byte-Oriented Memory Organization

  • Programs refer to virtual addresses

  • Implemented with a hierarchy of different memory types

  • Program has a private address space

  • Compiler and run-time system control allocation

Page 19: Machine Words

  • Machine has a "word size"

  • Old machines use 32-bit words, current systems use 64-bit words

  • Machines support multiple data formats

Page 20: Word-Oriented Memory Organization

  • Addresses specify byte locations

  • Addresses of successive words differ by 4 (32-bit) or 8 (64-bit)

Page 21: APIs

  • Applications Programmer Interface is a set of methods used to manipulate an abstraction

  • Mastering systems programming involves understanding APIs and layers

Page 22: Example: Java Input/Output

  • Set of abstractions for different kinds of input and output

  • Professional Java programmers know when and how to use these abstractions

Page 23: Systems programming

  • Programming skills, engineering discipline, and knowledge needed to build a system using abstractions

  • Deep understanding of the "layer below" is important

Page 24: Programming Languages

  • Low-level languages (C, C++)

    • Hides some architectural details

    • Kind of portable

    • Useful abstractions like types, arrays, procedures, objects

    • Requires handling low-level details like memory management, locks, threads

    • Gives control over resources

    • Complex and error-prone

    • Requires engineering discipline

  • High-level languages (Python, Ruby, JavaScript, ...)

    • Focus on productivity and usability over performance

    • Powerful abstractions shield from low-level details

    • Bounded arrays, garbage collection, rich libraries

    • Usually interpreted, translated, or compiled via an intermediate representation

    • Sl