Chapter 4: Network Layer Data Plane

Computer Networking: A Top-Down Approach - Chapter 4: Network Layer Data Plane

Introduction

• These slides are based on the "Computer Networking: A Top-Down Approach" 8th edition by Jim Kurose and Keith Ross, 2020.

• The material is available for free use (faculty, students, readers) with the condition of mentioning the source when used or adapted.

• All material is copyrighted by J.F Kurose and K.W. Ross.

Network Layer Goals

• Understand the principles behind network layer services, focusing on the data plane:

  • Network layer service models

  • Forwarding versus routing

  • How a router works

  • Addressing

  • Generalized forwarding

  • Internet architecture

• Instantiation and implementation in the Internet:

  • IP protocol

  • NAT, middleboxes

Data Plane Roadmap

• Network layer overview: data plane, control plane

• What’s inside a router: input ports, switching, output ports, buffer management, scheduling

• IP: the Internet Protocol: datagram format, addressing, network address translation, IPv6

• Generalized Forwarding, SDN: Match+action, OpenFlow: match+action in action

• Middleboxes

Network-Layer Services and Protocols

• Transport segment from sending to receiving host

  • Sender: encapsulates segments into datagrams, passes to link layer

  • Receiver: delivers segments to transport layer protocol

• Network layer protocols in every Internet device: hosts, routers

• Routers:

  • Examines header fields in all IP datagrams passing through it

  • Moves datagrams from input ports to output ports to transfer datagrams along end-end path

Key Network-Layer Functions

• Forwarding: Move packets from a router’s input link to the appropriate router output link

  • Analogy: taking a trip - forwarding is the process of getting through a single interchange

• Routing: Determine the route taken from source to destination by packets from source to destination

  • Analogy: taking a trip - routing is the process of planning the trip

  • Routing algorithms

Data Plane vs. Control Plane

Data Plane

• Local, per-router function

• Determines how a datagram arriving on a router input port is forwarded to a router output port

Control Plane

• Network-wide logic

• Determines how a datagram is routed among routers along an end-to-end path from source host to destination host

• Two control-plane approaches:

  • Traditional routing algorithms: implemented in routers

  • Software-defined networking (SDN): implemented in (remote) servers

Per-Router Control Plane

• Individual routing algorithm components in each and every router interact in the control plane

Software-Defined Networking (SDN) Control Plane

• Remote controller computes and installs forwarding tables in routers

Network Service Model

• Examples of services for individual datagrams:

  • Guaranteed delivery

  • Guaranteed delivery with less than 40 msec delay

• Examples of services for a flow of datagrams:

  • In-order datagram delivery

  • Guaranteed minimum bandwidth to flow

  • Restrictions on changes in inter-packet spacing

• Question: What service model for “channel” transporting datagrams from sender to receiver?

Network-Layer Service Models

• A comparison of Internet, ATM, and IntServ/DiffServ models is provided based on several parameters:

  • Bandwidth: Internet offers none, ATM: Constant Bit Rate (CBR) and Available Bit Rate (ABR), Intserv: Guaranteed Minimum (RFC 1633), Diffserv: possibly provide bandwidth.

  • Loss: Internet no guarantee, ATM yes, Intserv no, Diffserv yes, possibly

  • Order: Internet no guarantee, ATM yes, Intserv yes, Diffserv yes possibly

  • Timing: Internet no guarantee, ATM yes, Intserv no, Diffserv yes, no

• Internet “best effort” service model provides no guarantees on:

  • Successful datagram delivery to destination

  • Timing or order of delivery

  • Bandwidth available to end-end flow

• Quality of Service (QoS) Guarantees?

Reflections on Best-Effort Service

• Simplicity of mechanism has allowed Internet to be widely deployed and adopted

• Sufficient provisioning of bandwidth allows performance of real-time applications (e.g., interactive voice, video) to be “good enough” for “most of the time”

• Replicated, application-layer distributed services (datacenters, content distribution networks) connecting close to clients’ networks allow services to be provided from multiple locations

• Congestion control of “elastic” services helps

• The success of the best-effort service model is hard to argue with.

Router Architecture Overview

• High-level view of generic router architecture:

  • Input ports, high-speed switching fabric, output ports, routing processor

  • Data plane (hardware) operates in nanosecond time frame (forwarding)

  • Control plane (software) operates in millisecond time frame (routing, management)

Input Port Functions

• Line termination (physical layer: bit-level reception)

• Link layer protocol (receive): e.g., Ethernet (chapter 6)

• Lookup, forwarding, queueing

• Decentralized switching:

  • Using header field values, lookup output port using forwarding table in input port memory (“match plus action”)

  • Goal: complete input port processing at ‘line speed’

  • Input port queuing: if datagrams arrive faster than forwarding rate into switch fabric

Destination-Based vs Generalized Forwarding

• Decentralized switching:

  • Using header field values, lookup output port using forwarding table in input port memory (“match plus action”)

  • Destination-based forwarding: forward based only on destination IP address (traditional)

  • Generalized forwarding: forward based on any set of header field values

Destination-Based Forwarding

• Forwarding table example showing destination address ranges and corresponding link interfaces.

• It raises a question about what happens if ranges don't divide up so nicely?

Longest Prefix Matching

• When looking for forwarding table entry for a given destination address, use the longest address prefix that matches the destination address.

• Examples provided to illustrate which interface to select based on destination address and prefixes.

Longest Prefix Matching (Continued)

• Longest prefix matching is used when studying addressing.

• Longest prefix matching: often performed using ternary content addressable memories (TCAMs):

  • Content addressable: present an address to TCAM and retrieve the address in one clock cycle, regardless of table size.

  • Cisco Catalyst: ~1M routing table entries in TCAM.

Switching Fabrics

• Transfer packet from input link to the appropriate output link.

• Switching rate: rate at which packets can be transferred from inputs to outputs

  • Often measured as a multiple of input/output line rate

  • $N$ inputs: switching rate $N$ times line rate desirable

Switching Fabrics Types

• Three major types of switching fabrics: memory, bus, interconnection network.

Switching via Memory

• First-generation routers:

  • Traditional computers with switching under direct control of CPU

  • Packet copied to system’s memory

  • Speed limited by memory bandwidth (2 bus crossings per datagram)

Switching via a Bus

• Datagram from input port memory to output port memory via a shared bus.

• Bus contention: switching speed limited by bus bandwidth.

• 32 Gbps bus, Cisco 5600: sufficient speed for access routers.

Switching via Interconnection Network

• Crossbar, Clos networks, other interconnection nets initially developed to connect processors in multiprocessor systems.

• Multistage switch: nxn switch from 3x3 crossbar with multiple stages of smaller switches.

• Exploiting parallelism – fragment datagram into fixed-length cells on entry, switch cells through the fabric, reassemble datagram at exit.

Switching via Interconnection Network - scaling

• Scaling, using multiple switching “planes” in parallel: speedup, scaleup via parallelism.

• Cisco CRS router:

  • Basic unit: 8 switching planes

  • Each plane: 3-stage interconnection network

  • Up to 100’s Tbps switching capacity

Input Port Queueing

• If switch fabric is slower than input ports combined, queueing may occur at input queues.

  • Queueing delay and loss due to input buffer overflow!

• Head-of-the-Line (HOL) blocking: queued datagram at the front of the queue prevents others in the queue from moving forward.

Output Port Queueing

• Buffering is required when datagrams arrive from the fabric faster than the link transmission rate.

• Drop policy: which datagrams to drop if no free buffers?

• Scheduling discipline chooses among queued datagrams for transmission.

• Datagrams can be lost due to congestion, lack of buffers

• Priority scheduling – who gets best performance, network neutrality – this is a really important slide

Output Port Queueing (Continued)

• Buffering occurs when the arrival rate via switch exceeds output line speed.

• Queueing (delay) and loss due to output port buffer overflow!

How Much Buffering?

• RFC 3439 rule of thumb: average buffering equal to “typical” RTT (say 250 msec) times link capacity C

  • e.g., $C = 10 \, \text{Gbps}$ link: $2.5 \, \text{Gbit}$ buffer

• But too much buffering can increase delays (particularly in home routers):

  • Long RTTs: poor performance for real-time apps, sluggish TCP response

  • Recall delay-based congestion control: “keep bottleneck link just full enough (busy) but no fuller”

• More recent recommendation: with $N$ flows, buffering equal to $\frac{RTT \cdot C}{\sqrt{N}}$

Buffer Management

• drop: which packet to add, drop when buffers are full

  • tail drop: drop arriving packet

  • priority: drop/remove on priority basis

• marking: which packets to mark to signal congestion (ECN, RED)

Packet Scheduling: FCFS

• Packet scheduling: deciding which packet to send next on the link

  • First come, first served (FCFS)

  • Priority

  • Round robin

  • Weighted fair queueing

• FCFS: packets transmitted in order of arrival to output port

  • Also known as: First-in-first-out (FIFO)

Scheduling Policies: Priority

• Arriving traffic classified, queued by class (any header fields can be used for classification)

• Send packet from highest priority queue that has buffered packets

• FCFS within priority class

Scheduling Policies: Round Robin (RR)

• Arriving traffic classified, queued by class (any header fields can be used for classification)

• Server cyclically, repeatedly scans class queues, sending one complete packet from each class (if available) in turn

Scheduling Policies: Weighted Fair Queueing (WFQ)

• Generalized Round Robin

• Minimum bandwidth guarantee (per-traffic-class)

• Each class, $i$, has weight, $w_i$, and gets a weighted amount of service in each cycle:

  • $\frac{w<em>i}{\Sigma</em>j w_j}$

Sidebar: Network Neutrality

• Technical: how an ISP should share/allocate its resources

  • Packet scheduling, buffer management are the mechanisms

• Social, economic principles:

  • Protecting free speech

  • Encouraging innovation, competition

• Enforced legal rules and policies; different countries have different “takes” on network neutrality

Sidebar: Network Neutrality - US FCC Order

• 2015 US FCC Order on Protecting and Promoting an Open Internet: three “clear, bright line” rules:

  • No blocking: shall not block lawful content, applications, services, or non-harmful devices, subject to reasonable network management.

  • No throttling: shall not impair or degrade lawful Internet traffic on the basis of Internet content, application, or service, or use of a non-harmful device, subject to reasonable network management.

  • No paid prioritization: shall not engage in paid prioritization.

Sidebar: Network Neutrality - ISP

• US Telecommunication Act of 1934 and 1996:

  • Title II: imposes “common carrier duties” on telecommunications services: reasonable rates, non-discrimination and requires regulation

  • Title I: applies to information services: no common carrier duties (not regulated), but grants FCC authority “… as may be necessary in the execution of its functions”4

• Is an ISP a “telecommunications service” or an “information service” provider?

  • The answer really matters from a regulatory standpoint!

IP protocol

• Network Layer: Internet host, router network layer functions: IP protocol

  • Datagram format, addressing, packet handling conventions

• ICMP protocol

  • Error reporting, router “signaling”
    Transport layer: TCP, UDP, link layer physical layer network layer forwarding table

• Path-selection algorithms: implemented in routing protocols (OSPF, BGP) and SDN controller

IP Datagram format

• ver, header length, type of service, length, 16-bit identifier, flgs, fragment offset, time to live, upper layer, header checksum, source IP address, destination IP address, options (if any), payload data(variable length, typically a TCP or UDP segment)

  • version number

  • header length (bytes)

  • upper layer protocol (e.g., TCP or UDP)

  • total datagram length (bytes)

  • “type” of service:

    • Diffserv (0:5)

    • ECN (6:7)

  • fragmentation/ reassembly

  • TTL: remaining max hops (decremented at each router)

  • 32-bit source IP address

  • 32-bit destination IP address

  • header checksum

  • Maximum length: 64K bytes

  • Typically: 1500 bytes or less

  • e.g., timestamp, record route taken

• e.g., timestamp, record route taken

• 20 bytes of TCP

• 20 bytes of IP

• = 40 bytes + app layer overhead for TCP+IP overhead

IP addressing: introduction

• IP address: 32-bit identifier associated with each host or router interface

• interface: connection between host/router and physical link

  • router's typically have multiple interfaces

  • host typically has one or two interfaces (e.g., wired Ethernet, wireless 802.11)

• dotted-decimal IP address notation:

Interconnecting Interfaces

• Wired Ethernet interfaces connected by Ethernet switches

• Wireless WiFi interfaces connected by WiFi base station

• For now: don’t need to worry about how one interface is connected to another (with no intervening router)

• A: we’ll learn about that in chapters 6, 7

Subnets

• What’s a subnet?

  • Device interfaces that can physically reach each other without passing through an intervening router

• IP addresses have structure:

  • Subnet part: devices in the same subnet have common high-order bits

  • Host part: remaining low-order bits

Recipe for Defining Subnets

• Detach each interface from its host or router, creating “islands” of isolated networks.

• Each isolated network is called a subnet.

• Subnet mask: /24 (high-order 24 bits: subnet part of IP address).

Identifying Subnets

• Identify the subnets and their /24 subnet addresses within a given network configuration showing multiple hosts and routers.

IP addressing: CIDR

• CIDR: Classless InterDomain Routing (pronounced “cider”)

  • subnet portion of address of arbitrary length

  • address format: a.b.c.d/x, where x is # bits in subnet portion of address

IP Addresses - How to Get One?

• Two questions:

  • How does a host get an IP address within its network (host part of address)?

  • How does a network get an IP address for itself (network part of the address)?

• How does a host get an IP address?

  • Hard-coded by sysadmin in config file (e.g., /etc/rc.config in UNIX)

  • DHCP: Dynamic Host Configuration Protocol: dynamically get address from as server. “plug-and-play”

DHCP: Dynamic Host Configuration Protocol

Goal

• Host dynamically obtains IP address from network server when it “joins” network:

  • Can renew its lease on the address in use

  • Allows reuse of addresses (only hold address while connected/on)

  • Support for mobile users who join/leave network

DHCP overview

• Host broadcasts DHCP discover msg [optional]

• DHCP server responds with DHCP offer msg [optional]

• Host requests IP address: DHCP request msg

• DHCP server sends address: DHCP ack msg

DHCP Client-Server Scenario

• DHCP server is typically co-located in a router, serving all subnets to which the router is attached.

• Arriving DHCP client needs address in this network

DHCP Client-Server Message Sequence

• DHCP discover: Broadcast to find a DHCP server.

• DHCP offer: DHCP server offers an IP address.

• DHCP request: Host requests the offered IP address.

• DHCP ACK: DHCP server acknowledges and assigns the IP address.

• The first two steps can be skipped if a client remembers and wishes to reuse a previously allocated network address [RFC 2131].

DHCP - Beyond IP Addresses

• DHCP can return more than just the allocated IP address on a subnet:

  • Address of first-hop router for client

  • Name and IP address of DNS server

  • Network mask (indicating network versus host portion of address)

  • DHCP REQUEST message encapsulated in UDP, encapsulated in IP, encapsulated in Ethernet

  • Ethernet frame broadcast (dest: FFFFFFFFFFFF) on LAN, received at router running DHCP server

  • Ethernet demux’ed to IP demux’ed, UDP demux’ed to DHCP

DHCP: Example (Continued)

• DHCP server formulates DHCP ACK containing client’s IP address, IP address of first-hop router for client, name & IP address of DNS server

• Encapsulated DHCP server reply forwarded to client, demuxing up to DHCP at client

• Client now knows its IP address, name and IP address of DNS server, IP address of its first-hop router.

IP Addresses - Network part

• How does a network get the subnet part of the IP address?

  • A: Gets allocated a portion of its provider ISP’s address space.

• ISP can then allocate out its address space in 8 blocks

Hierarchical Addressing: Route Aggregation

• Hierarchical addressing allows efficient advertisement of routing information.

Hierarchical Addressing: More Specific Routes

• Organization 1 moves from Fly-By-Night-ISP to ISPs-R-Us

• ISPs-R-Us now advertises a more specific route to Organization 1

IP Addressing - Last Words

• How does an ISP get a block of addresses?

  • A: ICANN: Internet Corporation for Assigned Names and Numbers.

  • Allocates IP addresses through 5 regional registries (RRs) (who may then allocate to local registries).

  • Manages DNS root zone, including delegation of individual TLD (.com, .edu , …) management

• Are there enough 32-bit IP addresses?

  • ICANN allocated the last chunk of IPv4 addresses to RRs in 2011

  • NAT (next) helps IPv4 address space exhaustion IPv6 has 128-bit address space

• Vint Cerf (reflecting on decision to make IPv4 address 32 bits long) quote: "Who the hell knew how much address space we needed?"

NAT: Network Address Translation

• All devices in a local network share just one IPv4 address as far as the outside world is concerned.

NAT: Address spaces

• All devices in the local network have 32-bit addresses in a “private” IP address space (10/8, 172.16/12, 192.168/16 prefixes) that can only be used in the local network.

• Advantages:

  • Just one IP address needed from provider ISP for all devices

  • Can change addresses of host in local network without notifying the outside world

  • Can change ISP without changing addresses of devices in the local network

  • Security: devices inside local net not directly addressable, visible by outside world

NAT Implementation

• NAT router must (transparently):

  • Outgoing datagrams: replace (source IP address, port #) of every outgoing datagram to (NAT IP address, new port #)

    • Remote clients/servers will respond using (NAT IP address, new port #) as destination address

  • Remember (in NAT translation table) every (source IP address, port #) to (NAT IP address, new port #) translation pair

  • Incoming datagrams: replace (NAT IP address, new port #) in destination fields of every incoming datagram with the corresponding (source IP address, port #) stored in NAT table

NAT Translation scenario

• Host 10.0.0.1 sends datagram to 128.119.40.186, 80

• NAT router changes datagram source address from 10.0.0.1, 3345 to 138.76.29.7, 5001, updates table

• Reply arrives, destination address: 138.76.29.7, 5001

NAT Controversy

• NAT has been controversial:

  • Routers “should” only process up to layer 3

  • Address “shortage” should be solved by IPv6

  • Violates the end-to-end argument (port # manipulation by network-layer device)

  • NAT traversal: what if a client wants to connect to a server behind NAT?

• But NAT is here to stay:

  • Extensively used in home and institutional nets, 4G/5G cellular nets

IPv6: Motivation

• Initial motivation: the 32-bit IPv4 address space would be completely allocated

• Additional motivation:

  • Speed processing/forwarding: 40-byte fixed-length header

  • Enable different network-layer treatment of “flows”

IPv6 Datagram Format

• Priority: identify priority among datagrams in flow

• Flow label: identify datagrams in the same "flow” (concept of “flow” is not well defined).

What’s missing (compared with IPv4):

• No checksum (to speed processing at routers)

• No fragmentation/reassembly

• No options (available as upper-layer, next-header protocol at router)

Transition from IPv4 to IPv6

• Not all routers can be upgraded simultaneously

  • No “flag days”

  • How will network operate with mixed IPv4 and IPv6 routers?

• Tunneling: IPv6 datagram carried as payload in IPv4 datagram among IPv4 routers (“packet within a packet”)

  • Tunneling used extensively in other contexts (4G/5G)

Tunneling and Encapsulation

• Tunneling: IPv6 datagram as payload in an IPv4 datagram

Generalized Forwarding: Match plus action

• Review: Each router contains a forwarding table

  • “match plus action” abstraction: match bits in arriving packet, take action generalized forwarding:

  • Many header fields can determine action; many actions possible: drop/copy/modify/log packet

  • Destination-based forwarding: forward based on dest. IP address

Flow Table abstraction

• Flow: defined by header field values (in link-, network-, transport-layer fields)

• Generalized forwarding: simple packet-handling rules

  • Match: pattern values in packet header fields Actions: for matched packet: drop, forward, modify, matched packet, or send matched packet to controller

  • Priority: disambiguate overlapping patterns

  • Counters: #bytes and #packets

OpenFlow: Flow Table Entries

• Header fields to match: Ingress Port, Src MAC, Dst MAC, Eth Type, VLAN ID, VLAN Pri, IP ToS, IP Prot, IP Src, IP Dst, TCP/UDP Src Port, TCP/UDP Dst Port.

• Action: Forward packet to port(s), drop packet, modify fields in header(s), encapsulate and forward to the controller

OpenFlow: Examples

• Destination-based forwarding: IP datagrams destined to IP address 51.6.0.8 should be forwarded to router output port 6

• Firewall: Block (do not forward) all datagrams destined to TCP port 22 (ssh port #); block (do not forward) all datagrams sent by host 128.119.1.1

OpenFlow: More L2 Examples

• Layer 2 destination-based forwarding: layer 2 frames with destination MAC address 22:A7:23:11:E1:02 should be forwarded to output port 3

OpenFlow Abstraction

• Match+action: abstraction unifies different kinds of devices

  • Router – match longest destination IP prefix, action forward out a link

  • Switch – match destination MAC address, action forward or flood

  • Firewall – match IP addresses and TCP/UDP port numbers, action permit or deny

  • NAT – match IP address and port, action rewrite address and port

OpenFlow Example Topology

• Orchestrated tables can create network-wide behavior, e.g., datagrams from hosts h5 and h6 should be sent to h3 or h4, via s1 and from there to s2.

OpenFlow Example Instructions

• Illustrates how to set the forwarding actions.

Generalized Forwarding: Summary

• “Match plus action” abstraction: match bits in arriving packet header(s) in any layers, take action

  • Matching over many fields (link-, network-, transport-layer)

  • Local actions: drop, forward, modify, or send matched packet to controller

  • “Program” network-wide behaviors

• Simple form of “network programmability”

  • Programmable, per-packet “processing”

  • Historical roots: active networking

  • Today: more generalized programming: P4

Roadmap: Middleboxes

• Network layer: overview

• What’s inside a router

• IP: the Internet Protocol

• Generalized Forwarding

• Middleboxes

  • Middlebox functions

  • Evolution, architectural principles of the Internet

Middleboxes Defined

• “Any intermediary box performing functions apart from normal, standard functions of an IP router on the data path between a source host and destination host”

Middleboxes Everywhere!

• Examples of middleboxes include NAT, Firewalls/IDS, Load Balancers, Caches, and Application-specific boxes

Middleboxes: Software and Hardware

• Initially: proprietary (closed) hardware solutions

• Move towards “whitebox” hardware implementing open API

• Move away from proprietary hardware solutions

• Programmable local actions via match+action

• Move towards innovation/differentiation in software

• SDN: (logically) centralized control and configuration management often in private/public cloud

• Network functions virtualization (NFV): programmable services over white box networking, computation, storage

The IP Hourglass

• Internet’s “thin waist”: one network layer protocol: IP, must be implemented by every (billions) of Internet-connected devices

• Many protocols in physical, link, transport, and application layers

The IP Hourglass, at Middle Age

• Internet's middle-age “love handles” include middleboxes, operating inside the network with NAT, NFV, Firewalls, and caching.

Core Beliefs

• Simple connectivity

• IP protocol: that narrow waist

• Intelligence, complexity at network edge

The End-End Argument

• Some network functionality (e.g., reliable data transfer, congestion) can be implemented in the network, or at the network edge

The End-End Argument (Continued)

• Saltzer, Reed, Clark 1981 “The function in question can completely and correctly be implemented only with the knowledge and help of the application standing at the end points of the communication system. Therefore, providing that questioned function as a feature of the communication system itself is not possible. (Sometimes an incomplete version of the function provided by the communication system may be useful as a performance enhancement.) We call this line of reasoning against low-level function implementation the “end-to-end argument.”

Where’s the Intelligence?

• 20th-century phone net: intelligence/computing at network switches

• Internet (pre-2005): intelligence, computing at edge

• Internet (post-2005): programmable network devices, intelligence, computing, massive application-level infrastructure at edge

Conclusion of Chapter 4

• Question: How are forwarding tables (destination-based forwarding) or flow tables (generalized forwarding) computed?

• Answer: By the control plane

Additional Chapter 4 slides

IP Fragmentation/Reassembly

• Network links have MTU (max. transfer size) - largest possible link-level frame

  • Different link types, different MTUs

• Large IP datagram divided (“fragmented”) within the net:

  • One datagram becomes several datagrams

  • “Reassembled” only at destination

  • IP header bits are used to identify, order related fragments

IP Fragmentation/Reassembly

• An example illustrates how a 4000 byte datagram is fragmented into smaller datagrams with an MTU of 1500 bytes including the byte offsets and flags

DHCP: Wireshark Output

• Illustrative examples of DHCP request and reply messages are given.