Fundamentals of Web Development: Chapter 2 - How the Web Works
Fundamentals of Web Development: Chapter 2 - How the Web Works
Learning Objectives
Understand the fundamental protocols that make the web possible.
Learn how the domain name system functions.
Comprehend why HTTP is more than just a four-letter abbreviation.
Explore how browsers and servers work to exchange and interpret HTML.
Internet Protocols
Definition: A protocol is a set of rules used by partners to communicate.
Key Protocols: TCP/IP is an essential internet protocol.
Implementation: These protocols are integrated into every operating system, enabling fast web development with minimal complexity for developers in tracking elements like:
- Packet routing
- Transmission details
- Domain resolution
- Checksums
Starting Note
Importance of Knowledge: Understanding web operations, from low-level protocols to high-level JavaScript libraries, enhances web development skills.
Holistic Development: Modern web development encourages blending the roles of developers and designers in a DevOps approach, ensuring familiarity with various elements like:
- CSS
- HTML
- IP addresses
- Domain names
- Web servers
- BrowsersRevisiting Material: The material can be revisited as needed.
Layered Architecture
TCP/IP Model: Originally, TCP/IP internet protocols were abstracted as a four-layer stack, but modern models split this into five or seven layers.
Focus on Simplicity: The simpler four-layer model will be utilized in this chapter.
Link Layer
Role: The link layer is the lowest layer responsible for:
- Physical data transmission
- Establishing logical linksFunctions include:
- Packet creation
- Transmission
- Reception
- Error detection
- Handling of collisions and line sharingMAC Addresses: Media Access Control addresses are sometimes referenced in this context.
Internet Layer
Role: The Internet layer, also called the IP Layer, routes packets between networks and partners.
Communication: It provides “best effort” communication, meaning:
- Sent messages are not guaranteed to arrive intact or at all.IP Addresses: The layer utilizes IP addresses, which are unique numeric codes for identifying destinations on the internet. Every internet-connected device possesses an IP address.
IP Addressing
Types of IP addresses:
- IPv4: Composed of four 8-bit integers separated by dots (e.g. XX.XX.XX.XX).
- IPv6: Comprises eight 16-bit integers, significantly expanding the possible address space.
- Note: IPv6 allows for more than a billion times the number of addresses available with IPv4.
Port Address Translation (PAT)
Issue: The IPv4 address space was depleted in 2011, yet the number of internet devices continued to grow.
Solution: Port Address Translation (PAT) enables multiple unrelated networks to utilize the same IP address by differentiating through unique port numbers.
Example: Using a coffee shop’s wireless network may involve PAT.
Future Outlook: Transitioning to IPv6 is a necessity for continued growth in connectivity.
Transport Layer
Function: Manages the orderly and error-free transmission of packets.
**Divisions of Data:
- Data packets are formatted via the Transmission Control Protocol (TCP).
- Each packet features a header with a sequence number, allowing the receiver to reorder the packets as needed.
- Receivers acknowledge receipt of packets (ACK), enabling resend of lost packets.
- This structure guarantees message delivery and order.
User Datagram Protocol (UDP)
Context: Certain applications, like live broadcasts, do not require guaranteed packet delivery.
Examples: UDP is ideal for services where minor data loss is acceptable, such as:
- Live game streaming
- Voice Over IP (VoIP)
- Domain Name System (DNS).
Application Layer
Description: The application layer focuses on process-to-process communication familiar to developers, utilizing several protocols such as:
- HTTP: Hypertext Transfer Protocol for web communication.
- SSH: Secure Shell for remote connections.
- FTP: File Transfer Protocol for file movement between systems.
- POP/IMAP/SMTP: Email-related protocols.
- DNS: Domain Name System for resolving domain names into IP addresses.
Domain Name System (DNS)
Utility of DNS: As IP addresses can be cumbersome for users, DNS provides a more human-friendly way to identify devices.
Historical Context: Initially, ARPANET allowed domain names to be assigned to IP addresses with a manageable hosts file.
Evolution: With increasing users, hosting systems transitioned to a scalable and distributed DNS framework.
DNS Overview
Function: DNS maps domain names to IP addresses, meaning websites can switch hosts without altering their names.
Speed: DNS request-response cycles are often under a second, making them largely unnoticed.
Domain Name Structure
Components: A domain name consists of multiple parts forming a hierarchy, at minimum having:
- A top-level domain (TLD)
- A second-level domain (SLD)
Top-Level Domains (TLDs)
Definition: The rightmost portion of the domain (e.g., .com, .org).
Types of TLDs:
- Generic TLDs (gTLD): Including common types like .com and .org.
- Country Code TLDs (ccTLD): Administered by specific countries.
- Special TLD:.arpais reserved for reverse DNS lookups.
Generic Top-Level Domains (gTLD)
Types of gTLDs: Sub-categories consist of:
- Unrestricted: Examples include .com, .net, .org.
- Sponsored: Includes .gov, .mil, .edu.
- New: Over 1000 new gTLDs like .art or .tax were introduced after June 2012.
Country Code Top-Level Domain (ccTLD)
Management: Each ccTLD is governed by the associated country, which influences the registration rules.
Examples: In the UK, specific regulations for '.co.uk'; Canada allows broader access to '.ca'.
Innovations: Internationalized domain names (IDN) permit non-ASCII characters, with over nine million registered since their introduction in 2009.
Domain Name Registration
Management and Registrar Role:
- Registrars are authorized by the TLD registry or ccTLD registry to manage domain registrations.
- The Internet Corporation for Assigned Names and Numbers (ICANN) oversees and accredits these registrars.
Domain Name Registration Process
Registrant searches for domain using registrar or reseller portal.
Registrar queries TLD registry to check domain availability.
If available, the registrant pays and submits WHOIS information.
Registrar forwards WHOIS information to TLD registry operator.
Registry operator updates records with new WHOIS information.
DNS information is disseminated to relevant name servers.
Address Resolution
DNS Query Process:
- A request for a domain prompts the client to:
- Check local DNS cache.
- If absent, request an IP address from the primary DNS server, which escalates to a Root Name Server if necessary.
- Ultimately, the DNS server retrieves the correct IP address, allowing the client to access the domain.
Uniform Resource Locators (URL)
Definition: URLs define a way for clients to request specific resources from a server.
Components: A URL contains required components:
- Protocol type (e.g., HTTP)
- Domain name or IP address.
Optional URL Components
Path: Designates a file or directory on the server.
Port: Defines a connection point, often defaulting to port 80 for HTTP.
Query String: Provides additional data to the server, formatted as key-value pairs (e.g.,
?name=value).Fragment Identifier: References a section within a page, indicated by a
#in the URL.
Hypertext Transfer Protocol (HTTP)
Significance: HTTP is a foundational component of the web, establishing TCP connections on port 80 by default.
Server Response Structure: Upon receiving a request, the server sends:
- Headers
- Response Code
- Optional message content
HTTP Headers
Types of headers:
- Request Headers: Contain information from the client regarding the request (e.g., Host, User-Agent).
- Response Headers: Provide details about the server's response (e.g., Server, Last-Modified, Content-Type).
HTTP Request Methods
Common Methods:
- GET: To retrieve a resource (e.g., a web page).
- POST: Primarily used for submitting data to the server.Rare Methods: Additional less commonly used methods include CONNECT, TRACE, and OPTIONS, but are not elaborated upon in this section.
Response Codes
Explanation: Response codes signal the request's status, with categories:
- 2##: Successful responses
- 3##: Redirection messages
- 4##: Client errors
- 5##: Server errors.**Examples of Common Response Codes: **
- 200: OK - Request successful.
- 301: Moved Permanently - Resource location has shifted forever.
- 304: Not Modified - Resource has not changed since the last request.
- 401: Unauthorized - Access requires credentials.
- 404: Not Found - Resource could not be located.
- 500: Internal Server Error - General server-side error reported.
Web Browsers
Functionality: The user experience on the web differs significantly from traditional software applications.
Operation: Users access web pages via URLs, which initiate requests fulfilled by browsers.
Browser Knowledge: Understanding browser functions is vital for effective web development.
Fetching a Web Page
Process Summary: Browsers fetch a web page through:
1. Requesting the primary HTML page.
2. Parsing HTML for additional resources (images, style sheets, scripts).
3. Completing all resource retrieval before fully loading the web page.
Browser Rendering
Definition: The series of algorithms browsers use to manage the download, parse, layout, fetch assets, and create user-facing interfaces.
Performance Metrics: Key metrics for assessing rendering performance include:
- Time to First Byte (TTFB)
- First Paint (FP)
- First Contentful Paint (FCP)
- First Meaningful Paint (FMP)
- Largest Contentful Paint (LCP)
- Time to Interactive (TTI)
- On Load
- Cumulative Layout Shift (CLS)
Browser Caching
Functionality: Enables browsers to store web pages so subsequent requests can load faster without redownloading unchanged elements.
Impact: Browser caching significantly reduces network traffic, enhancing overall performance.
Web Servers
Core Definition: A web server responds to HTTP requests, functioning essentially as a computer.
Web Farms: Many web servers are often integrated into a web farm to handle increasing traffic and load.
Application Stack Selection: Each website operates using an application stack that includes:
- Operating system
- Web server software
- Database
- Scripting language for dynamic server requests.
LAMP Stack
Components: The LAMP stack includes:
- Linux operating system
- Apache web server
- MySQL database
- PHP scripting language.Variations:
- MAMP (for OSX) is similar to LAMP.
- WAMP uses Windows in place of Linux.
Alternative Stacks
Other Configurations:
- MERN: Includes MongoDB, Express, React, and Node.js.
- WISA: Utilizes Windows, IIS, SQL Server, and ASP.NET.
- JAM Stack: Focuses on JavaScript, APIs, and Markup.
Key Terms
Address resolution
Apache
Application stack
Application layer
Country code top-level domain (ccTLD)
Cumulative Layout Shift (CLS)
DNS resolver
DNS server
Domain names
Domain name registrars
Domain Name System (DNS)
First Contentful Paint (FCP)
First Meaningful Paint (FMP)
First Paint (FP)
Four-layer network model
Generic top-level domain (gTLD)
GET request
HEAD request
Hypertext Transfer Protocol (HTTP)
Internet Corporation for Assigned Names and Numbers (ICANN)
Internet Assigned Numbers Authority (IANA)
Internationalized top-level domain name (IDN)
Internet layer
Internet Protocol (IP) addresses
IP address
IPv4
IPv6
JAM stack
Largest Contentful Paint
LAMP software stack
Link layer
MAC addresses
MEAN software stack
On Load
Packet
Protocol
Punycode
Port
Port Address Translation (PAT)
POST request
Protocol
Request
Request headers
Response codes
Response headers
Reverse DNS lookups
Root name server
Second-level domain
Subdomain
Time to First Byte (TTFB)
Time to Interactive (TTI)
Transport layer
Transmission Control Protocol (TCP)
Top-level domain (TLD)
TLD name server
User Datagram Protocol (UDP)
Uniform Resource Locator (URL)
Web server
WISA software stack