HTTP Protocol Review


HTTP Protocol

  • HTTP is the protocol for the World Wide Web.

  • It operates at the application layer.

  • Messages exchanged between web client (browser) and web server.

  • Servers are unaware of IP addresses; rely on the underlying protocol stack.

  • Plain English is used in the headers

Request and Response

  • Two types of messages: request and response.

Request Format

  • Header and body separated by a blank line.

  • Human-readable headers.

  • The initial line is the request line, followed by optional headers.

  • Optional message body.

Request Line

  • Contains the method, URL, and HTTP version.

  • Example: GET index.html HTTP/1.1

Demonstrating HTTP with Telnet

  • Telnet overview:

    • An old program available on Linux and Windows.

    • Insecure protocol (no encryption).

    • Not installed by default on Windows.

  • Telnet usage:

    • Can connect to a web server port and manually type HTTP commands.

    • Useful for testing.

  • Connecting to the server:

    • Using the command telnet prac.d0dji.net 80.

    • Default Telnet port is 23 (insecure, rarely used).

  • Typing a request:

    • Typing GET index.html HTTP/1.1.

    • Requires a blank line after the header to send the request.

  • Server response:

    • May provide a permanent redirect (e.g., to HTTPS).

  • Functionality:

    • Telnet simulates what a web browser does by building and sending request headers.

    • Can also be used with email servers via port 25 (SMTP), composing emails manually.

  • Telnet Alternative

    • Use SSH or other secured protocols
      *Reason to keep Telnet for now:

    • GNS3 program that we’re going to be using later on uses Telnet to form its consoles.

Request Message Details

  • Requires at least the request line.

  • HTTP 1.1 requires at least one header.

Headers

  • Provide server information about the client.

  • Examples: user-agent (browser type), connection (persistent or close).

  • Mandatory in HTTP 1.1: host header.

    • Specifies the DNS name of the requested host.

    • Needed for multiple websites per host (unlike HTTP 1.0).

Methods

  • get: requests a resource.

  • head: like get, but only returns the header (for debugging).

  • post: sends data to the server for processing (e.g., form submission).

  • put: uploads a resource to the server.

  • delete: deletes a resource (often disabled on web servers).

  • trace/options: debugging purposes

  • connect: establishes a two-way communication channel (tunneling).

HTTP Headers

  • Optional, but generally present.

  • Give the server more information about the request.

User Agent

  • Tells the server the browser being used.

  • Historically used to serve specific content to different browsers.

  • Less critical now due to improved browser compatibility.

Connection

  • Specifies whether to maintain a persistent connection or close after the request.

  • HTTP 1.1 uses persistent connections by default.

    • Reduces network overhead.
      *Host

    • mandatory header in HTTP 1.1

    • Contains the DNS name of the requested host.

    • Allows multiple websites on a single host.

    • The URL is just the document, not the host.

Response Packets

  • Same format as requests: header and body separated by a blank line.

  • Mandatory status line and optional headers.

  • Message body is optional.

  • If it sends an error back, it just sends a header

Status Line

  • Includes the HTTP version, status code, and phrase.

  • Example: HTTP/1.1 200 OK

Status Codes

  • Codes inform client of request status.

  • Five categories:

    • 1xx: informational.

    • 2xx: success (e.g., 200 OK).

    • 3xx: redirection (resource moved).

    • 4xx: client error (e.g., 404 Not Found).

    • 5xx: server error (e.g., 502 Bad Gateway).

Response Headers

  • Content-Type: specifies the type of content (e.g., image, movie, text).

  • Server: provides information about the server software and version (often disabled for security).

  • Etag: a randomly generated string that changes if a resource is updated (used for caching).

Telnet for Diagnosing HTTP Issues

  • Telnet can be used to diagnose connectivity issues.

HTTP Extensions

  • Cookies: a way of maintaining state (e.g., for e-commerce).

  • Cookies can become nasty for tracking you on the Internet, doing advertising, things like that.

  • Can be used for legitimate purposes

Caching

  • A way to improve performance.

  • Uses cached copies using e-tags which provide the option to get a fresh copy from your internal cache

  • You can actually there are headers in the, HTTP protocol that can indicate whether a document should or can be cached or not.

Wireshark Practical

  • The website mentioned is set up with HTTP to inspect packets.

  • HTTPS is the same content as HTTP, just encrypted.

Initial Setup

  • Clear browsing history in Chromium browser (history -> clear browsing data).

  • Start capturing packets in Wireshark on ethernet zero interface.

  • Enter the URL: http://prac.d0dji.net/index.html

  • Stop capturing packets after the page loads.

  • Apply filter in Wireshark: http && ip.addr == 84.247.142.156

  • Check if the two captured packets are there.

Analyzing HTTP Packets

  • Note that the first GET requests the resource index.html results in a 200 response code which means successful

  • Note that the second GET requests for favicon.icove, an optional image that goes up to the side of the URL. The response code is 404, the file does not exist

  • Wireshark Overview

    • Outbound and Inbound IP adresses

    • There is relation from where to get the resource

Packet Details

  • View packet details in the middle frame.

  • Different layers: frame, Ethernet II, Internet Protocol, Transmission Control Protocol, Hypertext Transfer Protocol.

  • Focus on Hypertext Transfer Protocol (HTTP).

  • Expand hypertext transfer protocol to analyze details.

  • Human readable

Analyzing Protocol Information

  • Examine a few headers in order:
    *Request Line, host
    *Connection, and User-Agent
    *Accept, Accept-Encoding and Accept-Language

Analayzing Connection

  • See if the GET request requires to keep alive

  • Make sure it corresponds to your web request

  • The upgrade in secure requests might not be relevant but give it a scan
    *User Agent is the code that is running internally of your machine

  • Accept, tell the server what you like
    *Accept-encoding:
    *Gzip
    *Accept language
    *What language to communicate in
    *The Backslash R backslash N means that the header has ended
    *Extensible markup language. This is the data of the data

Actionable thing to open stream of conversation

  • Right click a packet and click follow then choose HTTP stream.

Testing for Resource Availability

*Close out the filter we were working with to now test for what happens when the server is not there
*Enter http://prac.d0dji.net/n0there.html
*N0t is a zero
*Filter the IP you are working with
*The output will show 404 This means NOT found

Testing for Images

*Close out the filter and open up http://prac.d0dji.net/embed.html
*Find the output. It should show that you get three requests. If not, then don’t worry its just how the stream of Wireshark works at sometimes
*HTML
*Image one
*Request again
*Image Two
*Request again
*This means that the website is stateless. Meaning images and web pages or requests

If you get Cached Issues

*Try to refresh or SHIFT refresh

HTTP 1.1

*This means the server maintain open connection with the clients.
HTTP 1.0 has to redial every time. It does not have main connection.

HTTP Size Limit

*Since everything to be put through the network, It needs to be packetted to the correct size
*Open up apache.html from your browser. Check if it shows assembled TCP contents.
*This can range from 5+ packets. Its due to wireless or wired LAN. Check MTU(maximum Transmission Unit is enabled)
*It can go to the 5000. In general 1500 is an average value. Not always accurate due to path and fragments

HTTP Authentication Request

*Let's close our capture and start a new one

  • Go to http://prac.d0dji.net/authindex.html, if its asking for something, then its a good thing
    *Login credentials
    *username
    *Pracuser
    *Password
    *Practicalpassword
    *Note that it shows that it get an error code being 401, it is intended
    *The difference is that it required Authorization under header

Base SixityFour Encoding

  • Highlight the password from the HTTP stream, then find base sixty four, copy the text to reveal the password. Its readable.

  • When using HTTP, be caution that there is no layer, hence its critical to have the HTTPs or a form.

Summary

  • Headers are required for requests

  • Servers can take the header to interpret information

  • Make your research on the network stack

  • You can actually set Wireshark up to decrypt on the fly the website if you have access to your own web certificates.

  • If you see in the filter a protocol in QUIC, it is related to HTTP three. Developed by Google, there has been a standard but it is newer
    Basically, its about page loading faster over 30 images that takes time to download on website