Fundamentals of Web Development: Chapter 2 - How the Web Works

Fundamentals of Web Development: Chapter 2 - How the Web Works

Learning Objectives

  • Understand the fundamental protocols that make the web possible.

  • Learn how the domain name system functions.

  • Comprehend why HTTP is more than just a four-letter abbreviation.

  • Explore how browsers and servers work to exchange and interpret HTML.

Internet Protocols

  • Definition: A protocol is a set of rules used by partners to communicate.

  • Key Protocols: TCP/IP is an essential internet protocol.

  • Implementation: These protocols are integrated into every operating system, enabling fast web development with minimal complexity for developers in tracking elements like:
      - Packet routing
      - Transmission details
      - Domain resolution
      - Checksums

Starting Note

  • Importance of Knowledge: Understanding web operations, from low-level protocols to high-level JavaScript libraries, enhances web development skills.

  • Holistic Development: Modern web development encourages blending the roles of developers and designers in a DevOps approach, ensuring familiarity with various elements like:
      - CSS
      - HTML
      - IP addresses
      - Domain names
      - Web servers
      - Browsers

  • Revisiting Material: The material can be revisited as needed.

Layered Architecture

  • TCP/IP Model: Originally, TCP/IP internet protocols were abstracted as a four-layer stack, but modern models split this into five or seven layers.

  • Focus on Simplicity: The simpler four-layer model will be utilized in this chapter.

Link Layer
  • Role: The link layer is the lowest layer responsible for:
      - Physical data transmission
      - Establishing logical links

  • Functions include:
      - Packet creation
      - Transmission
      - Reception
      - Error detection
      - Handling of collisions and line sharing

  • MAC Addresses: Media Access Control addresses are sometimes referenced in this context.

Internet Layer
  • Role: The Internet layer, also called the IP Layer, routes packets between networks and partners.

  • Communication: It provides “best effort” communication, meaning:
      - Sent messages are not guaranteed to arrive intact or at all.

  • IP Addresses: The layer utilizes IP addresses, which are unique numeric codes for identifying destinations on the internet. Every internet-connected device possesses an IP address.

IP Addressing
  • Types of IP addresses:
      - IPv4: Composed of four 8-bit integers separated by dots (e.g. XX.XX.XX.XX).
      - IPv6: Comprises eight 16-bit integers, significantly expanding the possible address space.
      - Note: IPv6 allows for more than a billion times the number of addresses available with IPv4.

Port Address Translation (PAT)
  • Issue: The IPv4 address space was depleted in 2011, yet the number of internet devices continued to grow.

  • Solution: Port Address Translation (PAT) enables multiple unrelated networks to utilize the same IP address by differentiating through unique port numbers.

  • Example: Using a coffee shop’s wireless network may involve PAT.

  • Future Outlook: Transitioning to IPv6 is a necessity for continued growth in connectivity.

Transport Layer
  • Function: Manages the orderly and error-free transmission of packets.

  • **Divisions of Data:
      - Data packets are formatted via the Transmission Control Protocol (TCP).
      - Each packet features a header with a sequence number, allowing the receiver to reorder the packets as needed.
      - Receivers acknowledge receipt of packets (ACK), enabling resend of lost packets.
      - This structure guarantees message delivery and order.

User Datagram Protocol (UDP)
  • Context: Certain applications, like live broadcasts, do not require guaranteed packet delivery.

  • Examples: UDP is ideal for services where minor data loss is acceptable, such as:
      - Live game streaming
      - Voice Over IP (VoIP)
      - Domain Name System (DNS).

Application Layer

  • Description: The application layer focuses on process-to-process communication familiar to developers, utilizing several protocols such as:
      - HTTP: Hypertext Transfer Protocol for web communication.
      - SSH: Secure Shell for remote connections.
      - FTP: File Transfer Protocol for file movement between systems.
      - POP/IMAP/SMTP: Email-related protocols.
      - DNS: Domain Name System for resolving domain names into IP addresses.

Domain Name System (DNS)

  • Utility of DNS: As IP addresses can be cumbersome for users, DNS provides a more human-friendly way to identify devices.

  • Historical Context: Initially, ARPANET allowed domain names to be assigned to IP addresses with a manageable hosts file.

  • Evolution: With increasing users, hosting systems transitioned to a scalable and distributed DNS framework.

DNS Overview
  • Function: DNS maps domain names to IP addresses, meaning websites can switch hosts without altering their names.

  • Speed: DNS request-response cycles are often under a second, making them largely unnoticed.

Domain Name Structure
  • Components: A domain name consists of multiple parts forming a hierarchy, at minimum having:
      - A top-level domain (TLD)
      - A second-level domain (SLD)

Top-Level Domains (TLDs)
  • Definition: The rightmost portion of the domain (e.g., .com, .org).

  • Types of TLDs:
      - Generic TLDs (gTLD): Including common types like .com and .org.
      - Country Code TLDs (ccTLD): Administered by specific countries.
      - Special TLD: .arpa is reserved for reverse DNS lookups.

Generic Top-Level Domains (gTLD)
  • Types of gTLDs: Sub-categories consist of:
      - Unrestricted: Examples include .com, .net, .org.
      - Sponsored: Includes .gov, .mil, .edu.
      - New: Over 1000 new gTLDs like .art or .tax were introduced after June 2012.

Country Code Top-Level Domain (ccTLD)
  • Management: Each ccTLD is governed by the associated country, which influences the registration rules.

  • Examples: In the UK, specific regulations for '.co.uk'; Canada allows broader access to '.ca'.

  • Innovations: Internationalized domain names (IDN) permit non-ASCII characters, with over nine million registered since their introduction in 2009.

Domain Name Registration

  • Management and Registrar Role:
      - Registrars are authorized by the TLD registry or ccTLD registry to manage domain registrations.
      - The Internet Corporation for Assigned Names and Numbers (ICANN) oversees and accredits these registrars.

Domain Name Registration Process
  1. Registrant searches for domain using registrar or reseller portal.

  2. Registrar queries TLD registry to check domain availability.

  3. If available, the registrant pays and submits WHOIS information.

  4. Registrar forwards WHOIS information to TLD registry operator.

  5. Registry operator updates records with new WHOIS information.

  6. DNS information is disseminated to relevant name servers.

Address Resolution

  • DNS Query Process:
      - A request for a domain prompts the client to:
        - Check local DNS cache.
        - If absent, request an IP address from the primary DNS server, which escalates to a Root Name Server if necessary.
      - Ultimately, the DNS server retrieves the correct IP address, allowing the client to access the domain.

Uniform Resource Locators (URL)

  • Definition: URLs define a way for clients to request specific resources from a server.

  • Components: A URL contains required components:
      - Protocol type (e.g., HTTP)
      - Domain name or IP address.

Optional URL Components
  • Path: Designates a file or directory on the server.

  • Port: Defines a connection point, often defaulting to port 80 for HTTP.

  • Query String: Provides additional data to the server, formatted as key-value pairs (e.g., ?name=value).

  • Fragment Identifier: References a section within a page, indicated by a # in the URL.

Hypertext Transfer Protocol (HTTP)

  • Significance: HTTP is a foundational component of the web, establishing TCP connections on port 80 by default.

  • Server Response Structure: Upon receiving a request, the server sends:
      - Headers
      - Response Code
      - Optional message content

HTTP Headers
  • Types of headers:
      - Request Headers: Contain information from the client regarding the request (e.g., Host, User-Agent).
      - Response Headers: Provide details about the server's response (e.g., Server, Last-Modified, Content-Type).

HTTP Request Methods
  • Common Methods:
      - GET: To retrieve a resource (e.g., a web page).
      - POST: Primarily used for submitting data to the server.

  • Rare Methods: Additional less commonly used methods include CONNECT, TRACE, and OPTIONS, but are not elaborated upon in this section.

Response Codes
  • Explanation: Response codes signal the request's status, with categories:
      - 2##: Successful responses
      - 3##: Redirection messages
      - 4##: Client errors
      - 5##: Server errors.

  • **Examples of Common Response Codes: **
      - 200: OK - Request successful.
      - 301: Moved Permanently - Resource location has shifted forever.
      - 304: Not Modified - Resource has not changed since the last request.
      - 401: Unauthorized - Access requires credentials.
      - 404: Not Found - Resource could not be located.
      - 500: Internal Server Error - General server-side error reported.

Web Browsers

  • Functionality: The user experience on the web differs significantly from traditional software applications.

  • Operation: Users access web pages via URLs, which initiate requests fulfilled by browsers.

  • Browser Knowledge: Understanding browser functions is vital for effective web development.

Fetching a Web Page
  • Process Summary: Browsers fetch a web page through:
      1. Requesting the primary HTML page.
      2. Parsing HTML for additional resources (images, style sheets, scripts).
      3. Completing all resource retrieval before fully loading the web page.

Browser Rendering
  • Definition: The series of algorithms browsers use to manage the download, parse, layout, fetch assets, and create user-facing interfaces.

  • Performance Metrics: Key metrics for assessing rendering performance include:
      - Time to First Byte (TTFB)
      - First Paint (FP)
      - First Contentful Paint (FCP)
      - First Meaningful Paint (FMP)
      - Largest Contentful Paint (LCP)
      - Time to Interactive (TTI)
      - On Load
      - Cumulative Layout Shift (CLS)

Browser Caching
  • Functionality: Enables browsers to store web pages so subsequent requests can load faster without redownloading unchanged elements.

  • Impact: Browser caching significantly reduces network traffic, enhancing overall performance.

Web Servers

  • Core Definition: A web server responds to HTTP requests, functioning essentially as a computer.

  • Web Farms: Many web servers are often integrated into a web farm to handle increasing traffic and load.

  • Application Stack Selection: Each website operates using an application stack that includes:
      - Operating system
      - Web server software
      - Database
      - Scripting language for dynamic server requests.

LAMP Stack
  • Components: The LAMP stack includes:
      - Linux operating system
      - Apache web server
      - MySQL database
      - PHP scripting language.

  • Variations:
      - MAMP (for OSX) is similar to LAMP.
      - WAMP uses Windows in place of Linux.

Alternative Stacks
  • Other Configurations:
      - MERN: Includes MongoDB, Express, React, and Node.js.
      - WISA: Utilizes Windows, IIS, SQL Server, and ASP.NET.
      - JAM Stack: Focuses on JavaScript, APIs, and Markup.

Key Terms

  • Address resolution

  • Apache

  • Application stack

  • Application layer

  • Country code top-level domain (ccTLD)

  • Cumulative Layout Shift (CLS)

  • DNS resolver

  • DNS server

  • Domain names

  • Domain name registrars

  • Domain Name System (DNS)

  • First Contentful Paint (FCP)

  • First Meaningful Paint (FMP)

  • First Paint (FP)

  • Four-layer network model

  • Generic top-level domain (gTLD)

  • GET request

  • HEAD request

  • Hypertext Transfer Protocol (HTTP)

  • Internet Corporation for Assigned Names and Numbers (ICANN)

  • Internet Assigned Numbers Authority (IANA)

  • Internationalized top-level domain name (IDN)

  • Internet layer

  • Internet Protocol (IP) addresses

  • IP address

  • IPv4

  • IPv6

  • JAM stack

  • Largest Contentful Paint

  • LAMP software stack

  • Link layer

  • MAC addresses

  • MEAN software stack

  • On Load

  • Packet

  • Protocol

  • Punycode

  • Port

  • Port Address Translation (PAT)

  • POST request

  • Protocol

  • Request

  • Request headers

  • Response codes

  • Response headers

  • Reverse DNS lookups

  • Root name server

  • Second-level domain

  • Subdomain

  • Time to First Byte (TTFB)

  • Time to Interactive (TTI)

  • Transport layer

  • Transmission Control Protocol (TCP)

  • Top-level domain (TLD)

  • TLD name server

  • User Datagram Protocol (UDP)

  • Uniform Resource Locator (URL)

  • Web server

  • WISA software stack