Deployment Notes

Deployment

  • This lecture covers deploying web applications to be accessible via a URL worldwide.
  • Avoid insecure practices as illustrated by xkcd.com/908.

Hosting Options

1. Running Your Own Server
  • Historically, running your own server was expensive.
  • Now, it's cheaper with wireless routers, home broadband, and devices like Raspberry Pi.
  • A Raspberry Pi: single-board computer (under $60), 1GB memory, 1.4 GHz processor which is adequate for small web servers.
  • Raspberry Pi uses Raspbian (a Linux variation) and is configured similarly to a Linux server.
  • Configuration typically requires a keyboard and monitor but can be accessed via SSH like a normal Linux server once on the network.
2. A Linux Virtual Machine
  • A Linux server is a common way to deploy web applications.
  • The server runs applications, listens for requests on ports, and serves those requests.
  • Many use hosting solutions to avoid managing physical infrastructure.
  • Providers like Amazon, Digital Ocean, A2, Azure, and AliBaba offer hosted virtual machines for around $10/month.
  • Users register, request an instance, and use SSH to configure and deploy applications from the command line.
3. Platform-as-a-Service (PaaS)
  • PaaS solutions simplify deployment, especially for early-stage projects, by managing the server machine.
  • PaaS handles server setup, storage management, and infrastructure maintenance.
  • Typically, PaaS solutions are commercial and scale with usage, though free or hobby tiers may be available.
  • PaaS offerings may be limited to specific OS/languages.

Containers

  • Containers address versioning, package management, and cross-platform reliability issues in software development.
  • A container packages a web app and all its dependencies into a transportable unit.
  • Containers standardize software, ensuring consistent operation across different environments.
  • Example container stack: everything above the Host OS is bundled into the container and deployed on the server.
  • [1] https://www.msystechnologies.com/blog/a-complete-guide-to-cloud-containers/

Containers in the Cloud

  • Docker and Heroku are two micro-hosting solutions.
  • They offer containers to run a single process as a service (Software as a Service).
  • Containers install code from a Git repository, install dependencies, and initialize the app.
  • Since containers lack persistent memory, they rely on 3rd party cloud databases.
Docker
  • Docker software includes functionality to create Docker container "images" and the Docker Engine as a run environment.
  • Container images run on the Docker Engine, ensuring consistent app behavior regardless of the environment.
  • Docker offers different container tiers based on needs (security, space, etc.).
  • Docker can automatically create an environment from a Dockerfile script.
  • Each project can have multiple Dockerfiles for different environments (e.g., development, production).
  • docker build runs the script to create the container.
  • Further information: Dockerfile scripts and best practices.
Heroku: A Heroic Haiku
  • Heroku is a cloud platform to upload (ideally containerized) apps and use built-in functionality to build, maintain, and scale projects.
  • Heroku is popular due to its usability:
    • Commit and push projects to the host URL like a Git repo.
    • Good tutorials are available.
  • Heroku is a well-recognized platform, though it is considered a bit expensive.
  • Free alternatives with fewer features are available (e.g., PythonAnywhere).

Server Configuration

Obtaining a Domain Name
  • A domain name has two parts:
    • Top-level domain: e.g., ".com", ".au", ".net"
    • Second-level domain: e.g., "google", "uwa", "facebook"
  • Domain name choice:
    • Purchase cost varies based on the top-level domain.
    • Brand success depends on the second-level domain.
    • Short, memorable domain names are preferable.
  • Domain name availability can be checked online via domain availability checker tools.
  • Domain names are bought (rented) through a Domain Registrar.
  • Choose a reputable registrar affiliated with ICANN.
Obtaining an SSL Certificate
  • Always deploy websites using HTTPS, requiring an SSL certificate.
  • Steps:
    1. Generate a public/private key pair.
    2. Create a Certificate Signing Request (CSR) containing the public key, domain, etc.
    3. Submit the CSR to a Certificate Authority (some are free, e.g., LetsEncrypt), which validates the information and issues an SSL certificate.
    4. Install the SSL certificate on the server and redirect all traffic from HTTP to HTTPS.
Securing Your Server
  • Be reasonably paranoid when using a publicly accessible machine.
  • Security steps:
    1. Remove passwords for login and use key files instead (generate a public-private key pairing).
    2. Disable root logins.
    3. Use a firewall to accept traffic only on ports 22 (SSH), 80 (HTTP), and 443 (HTTPS).
    4. Route all web requests through HTTPS (HTTP traffic is transmitted in plaintext).
Logging
  • Continuous monitoring of a deployed server is not feasible.
  • Logging requests and actions is invaluable for:
    • Understanding user interaction.
    • Detecting application attacks.
  • Flask has built-in logging packages.
  • Considerations:
    • Log format.
    • Avoiding sensitive/private information.
    • Pruning log files.
    • Automated monitoring tools. https://xkcd.com/1495/

Scaling Your Architecture

Re-cap: Client Server Architecture
  • The client-server architecture involves one server serving many clients.
Specialised Database Hardware
  • The database is often stored on a separate computer.
  • Advantages:
    • Increased performance using optimized hardware.
    • Multiple servers can access the same database.
Load Balancer and Multiple Servers
  • As the application scales, multiple servers are needed to handle traffic.
  • A load balancer distributes requests to individual identical servers.
  • This setup offers higher throughput than a single server.
  • From the user's perspective, the load balancer acts as the server.
Reverse-Proxy Server
  • A reverse-proxy server takes on additional tasks alongside load balancing.
  • It can serve static content (e.g., CSS, JS, images).
  • It can handle encryption and decryption (sometimes with specialized hardware).
  • It can filter suspicious traffic and offer protection against DDoS attacks.

A (High-Level) Walkthrough for Deploying a Flask Application

Our Deployment Architecture
  • Deployment will be done using Heroku with the following architecture:
  • The reverse-proxy server and the main server (origin server) are separate processes running on the same machine.
Upgrading to Production-Grade Tools
  • Flask development webserver and SQLite were used for fast development, but they aren't suitable for public deployment due to security and scalability concerns.
  • SQL-Alchemy allows MySQL to be used instead of SQLite.
  • Gunicorn and nginx will be used as alternatives to the Flask development server to run the Flask app.
  • Only minor configuration tweaks are needed.
Why Replace SQLite?
  • MySQL is a server database, unlike the embedded SQLite.
  • MySQL can run on a separate machine and act as a server for multiple web servers.
  • MySQL performs better in production and scales well with multiple web servers.
  • It has built-in authentication methods for data security.
Replacing SQLite with MySQL
  • The database will initially be kept on the same machine.
  • Install MySQL on the web server using apt (a Linux package manager).
  • Create a special user in MySQL to handle database transactions.
  • Set the username to the app name and use an appropriate password.
Running MySQL on the Webserver
  • Install a driver for MySQL: pip install pymysql
  • Set the DATABASE_URL to the new database.
  • Flask automatically reads the DATABASE_URL variable.
  • Run flask db migrate and flask db upgrade to create the database.
  • The app will function as before but now uses a full MySQL database server.
Gunicorn as the Origin Server
  • Gunicorn will be used as the origin server (install with pip).
  • Gunicorn implements the Web Service Gateway Interface, a Python standard for accessing web requests.
  • Gunicorn can configure the webserver itself via a .conf file, similar to Flask.
Nginx as the Reverse-Proxy Server
  • Nginx will be used as the outward-facing reverse-proxy server (install with apt) for external traffic and serving static content.
  • Nginx does the following:
    1. Routes all traffic through HTTPS (port 443) for encryption.
    2. Caches static data to improve efficiency.
    3. Handles public-key encryption/decryption using a secure certificate (offloading this from the origin server).
Configuring Nginx
  • Nginx is configured using a configuration file.
  • SSL certificates can be generated locally or obtained from LetsEncrypt.org.
  • Obtaining an externally signed certificate requires knowing the final domain name for the server.
Deploying the Website
  • The reverse-proxy and origin servers can be deployed on the same machine, even though they're often on different machines.
  • When deploying, the server listens for requests on port 80 and forwards them through port 443 for end-to-end encryption: sudo nginx start
  • The servers run in a daemon thread, so they persist after the session ends.
Setting up Heroku
  • To launch the app on Heroku, a Heroku account and a Git repo of the project are needed.
  • Install Heroku CLI to set up and configure the Heroku instance from the local machine.
  • Build the app as usual using Git.
  • Heroku does not have persistent memory but offers free Postgres databases as a service; psycopg2 and Gunicorn are required.
  • Freeze the requirements and create the app, initializing a Git remote to push code to.
Databases as a Service
  • Heroku containers lack persistent memory, requiring an external database.
  • Use Heroku add-ons to add a PostgreSQL database (similar to MySQL).
  • Heroku initializes the database and sets DATABASE_URL in the project settings.
  • Many other Database-as-a-service providers exist: MLab for mongo databases, Azure, and AWS for every database type.
Deploying on Heroku
  • Deploying to Heroku is relatively easy after setting up the Flask environment locally.
  • Set system variables (e.g., FLASK_APP); Heroku sets others (DATABASE_URL and PORT).
  • Provide basic initialization commands in a Procfile (in the root of the Git repo).
  • Push the Git remote to the remote created with the project to launch the application.
  • Heroku detects the language, installs Python, pip, and requirements from requirements.txt, and runs commands in the Procfile.
Full Deployment
  • Example Heroku commands and output are shown in the transcript.

The Future of the Web

The Future of the Web
  • The course focused on fundamental web technologies: HTML, CSS, JavaScript, web application frameworks, RESTful architectures, AJAX, Sockets.
  • It also covered Agile Software Development and tools like GitHub.
  • While core technologies like REST, HTTP, and AJAX are persistent, the web evolves rapidly.
  • LLMs are reshaping the content and structure of the web.
  • The future web is uncertain.
Cloud Computing
  • Cloud services are prevalent.
  • High bandwidth, permanent online devices, and REST principles blur the line between personal devices and cloud services.
  • Cloud is merging into desktop apps, and desktop apps are disappearing.
  • Streaming services are changing the notion of libraries and media ownership.
  • Software as a service is a significant industry with many emerging business models.
  • Further study: CITS5503, Cloud Computing
Rise of the LLMs
  • LLM summaries are decreasing click-throughs to webpages, threatening advertising revenue.
  • Users are increasingly searching the web by voice, requiring voice replies.
  • LLMs may make writing JavaScript, HTML, CSS rarer, increasing the availability of "No Code" platforms.
  • However, purpose-built applications will still require skilled developers with architectural knowledge.
  • Further study: CITS4012 - Natural Language Processing
The Semantic Web
  • Sir Tim Berners-Lee champions the semantic web which highlights the difference between content and meaning.
  • Semantics = Meaning.
  • HTML and CSS differentiate content and presentation.
  • A standard for meaning enables smart search engines to understand web services (selling tickets, promoting events, documenting history, etc.).
  • This has significant implications for AI and automatic assistants.
  • Further study: CITS3011 – Intelligent Agents, and CITS3005 – Knowledge Representation.
The Internet of Things
  • The Internet of Things (IoT) extends the internet to smart devices which focuses on message passing interfaces and infrastructure.
  • Smart homes, adaptive lighting, smart vehicles, and connected devices use basic web technologies for remote control.
  • These devices gather analytics and adapt to usage to improve user experience.
  • This has privacy implications, as the web extends seamlessly into daily life.
  • Further study: CITS5506, Internet of Things
Cybersecurity
  • Security is an ongoing challenge of the web.
  • The web evolved without built-in security, making securing services a constant battle.
  • Small flaws can have major implications in defense and society, with cyberwarfare consuming significant defense spending.
  • Examples: Stuxnet attacks, Ashley Madison hack, Anonymous's attack on HB Gary.
  • Further study: CITS3006 - Penetration Testing

Final Exam

  • The exam is 2 hours long, in-person, on Wednesday, June 4th at 9 am.
  • See the "Exams" LMS page for examinable topics.
  • The group project assessed coding ability.
  • The exam will assess understanding of theoretical topics and web functioning.
  • No calculators or cheatsheets are allowed.
  • Good luck!