PAF (IT3030) introduces full-stack web development fundamentals with an emphasis on good software engineering practices and industry standards.
Learning Outcomes:
Know the basics of Software Frameworks.
Apply Industry standard software development best practices.
Understand the REST architectural style.
Develop Java-based REST web services (Back-end).
Develop JavaScript-based Interactive web applications (Front-end).
Programming vs Software Engineering
Programming: Writing Code to create an application.
Software Engineering: The end-to-end process of applying good engineering principles and software development practices to produce quality software. Coding is just a part of this large process.
Practices encompassed by software engineering:
Analyzing user requirements and designing the software to meet those requirements.
Communicating with all the stakeholders to resolve issues and update them on the progress.
Choosing the software architectures, patterns, algorithms, coding techniques, technologies used to realize the software design based on sound design considerations.
Quality Assurance (QA) of the developed software to ensure that the software meets the client requirements and acceptable reliability, performance and security considerations.
Deploying/ releasing the software for actual use.
Maintaining the developed software as needed after release/ deployment.
Generic Functionalities in Software Development
Most (web) applications have a lot of features in common e.g. User Authentication, routing, database connectivity, scheduled jobs etc.
There is little point in rewriting all the functionalities (including the common ones) from scratch every time new software is developed.
What if the generic functionalities can be collected and made reusable?
Frameworks: What Are They?
An integrated set of software artifacts (such as classes, objects, and components) that collaborate to provide a reusable architecture for a family of related applications.
A framework enables a coding environment that contains low-level libraries to address conventional coding issues. The objective of a framework is to deliver faster development of an application. This includes everything we need to build large-scale applications, such as templates based on best practices.
A framework is a foundation for developing software applications. Software engineers and developers use a framework as a template to create websites and applications. Developers do this by adding code to a framework, then personalizing it for their specific purpose.
A software framework is an abstraction in which software, providing generic functionality, can be selectively changed (extended) by additional user-written code, thus providing application-specific software.
How do Frameworks help with the Software Engineering Approach?
Architectural Patterns (e.g.: MVC, MVVM)
Design Patterns
Inversion of Control
Standards/ Best Practices (Naming conventions, Coding standards, Directory structures etc.)
Tooling Support (Package managers, Dependency managers, Boilerplate code generation)
Testing Support
Framework vs. Libraries
Certain features make a framework different from other library forms, including the following:
Default Behavior: Before customization, a framework behaves in a manner specific to the user’s action.
Inversion of Control: Unlike other libraries, the global flow of control within a framework is employed by the framework rather than the caller.
Extensibility: A user can extend the framework by selectively replacing default code with user code.
Non-modifiable Framework Code: A user can extend the framework, but not modify the code.
Why Use Frameworks?
The designers of software frameworks aim to facilitate software developments by allowing designers and programmers to devote their time to meeting software requirements rather than dealing with the more standard low-level details of providing a working system, thereby reducing overall development time.
The aim of frameworks is to provide a common structure so that developers don’t have to redo it from scratch and can reuse the code provided. In this way, frameworks allow us to cut out much of the work and save a lot of time.
To summarize: there’s no need to reinvent the wheel.
Advantages of Using Frameworks
Improved coding, easy code reusability and ease of debugging code compared with from-scratch developments.
Community support.
Accelerated development (if the programmer is familiar with the framework).
Often provides caching and optimized processes for resource intensive tasks.
Enables faster methods for development with less code (boilerplate code etc.).
Better compliance to accepted security standards.
Limitations of Using Frameworks
Learning the Framework and not the programming language prevent programmers from gaining an in-depth understanding of the programming language.
Users of a framework are forced to respect the limitations and conventions imposed by its design.
Options to tweak functionalities are limited.
A framework will come with everything required to satisfy a wide range of use cases, not all these features will be used for a given project.
The right framework for the application should be chosen, or else performance and user experience may be impacted negatively.
We must be up-to-date with new/deprecated features in every version.
Some Examples
Spring/ Spring Boot (Java)
Express.js (with Node.js)
Laravel (PHP)
.NET/ .NET Core (C# etc.)
Django (Python)
Angular (Typescript)
Vue.js (JavaScript)
Software Quality Assurance (SQA) – An Overview
Learning Outcomes
Define what Software Quality Assurance means.
Differentiate between SQA and testing.
Discuss the importance of SQA in today’s software industry.
Apply SQA practices in your development work.
What is Software Quality?
According to the IEEE Computer Society,software quality refers to how well a software product conforms to the given requirements and meets the needs of its users. It involves both the software product itself as well as the processes used to develop it.
The American Society for Quality defines Software Quality Assurance (SQA) as a systematic approach to evaluating the quality of and adherence to software product standards, processes and procedures. SQA includes ensuring standards and procedures are established and followed throughout the software acquisition life cycle.
Why is Software Quality Assurance (SQA) Important?
SQA is a crucial part of the software development process, ensuring a final product that meets expectations, that involves people, processes, and technology.
SQA leads to a positive user experience by minimising failures and ensuring conformity with the user requirements. This translates to satisfied users who are more likely to recommend the product.
SQA helps businesses save money, time and reputation by preventing or catching defects early. Fixing bugs later in the SDLC is significantly more expensive in all three aspects.
SQA helps businesses comply with industry standards and regulations, reducing legal risks.
SQA emphasizes documentation, ensuring that the requirements, processes, design decisions, technical information, test results and reports are well documented, which aids in understanding the base code and makes maintenance easier.
SQA is NOT just testing!
SQA is a process that focuses on preventing defects through proactive measures throughout the SDLC and ensures overall quality.
Software Testing is a specific activity within SQA. It's a reactive approach that helps find defects, but SQA goes beyond testing to prevent them from occurring in the first place.
Real-world examples of software failures
Therac-25 (6 deaths due to radiation overdose)
Boeing 737 Max Crashes (346 deaths, US$20-80 Billion in costs)
Y2K Bug (US$300 billion)/ Year 2038 problem (yet to occur)
Loss of Mars Climate Orbiter (US$93 million)
Ariane 5 (US$8 million)
The SQA Process
Identify Requirements
Establish quality standards based on the requirements.
Clearly define the quality standards that the software product must meet. This includes defining requirements, acceptance criteria, and performance metrics.
Plan SQA activities
Process for problem reporting and corrective action
Software artefact management
Process reviews/ audits
Testing methodology
Execute the SQA plan
Conduct reviews of software artefacts
Perform testing (repeatedly) – use automation to increase efficiency.
Conduct process reviews
Monitor the quality of the product
This should be done throughout the SDLC.
Defects must be tracked
Analyze metrics such as code coverage and defect density
Conduct root cause analysis
Continuously improve the SQA process
Use the findings from the process reviews and monitoring data to identify areas for improvement and implement changes to the SQA process.
Follow the Plan-Do-Check-Act (PDCA) Cycle.
Building Quality into the Software Development Process
Define Clear Requirements
Start by clearly defining and documenting the requirements for the software. This ensures that everyone involved understands what needs to be built and what constitutes success. Effective requirement management is critical for delivering a successful software product.
Adopt Agile Methodologies
Agile methodologies like Scrum or Kanban promote iterative development, frequent collaboration, and continuous improvement. This allows for early detection and resolution of issues, leading to higher-quality software.
Design Reviews
Design reviews involve a methodical examination of the software design. The goal is to identify and address any defects, inconsistencies, or areas for improvement in the design before coding begins. This proactive approach helps to prevent errors from being coded into the software and reduces the time and effort required for fixing problems later in the development process.
Implement Code Reviews
Regular code reviews help catch issues early in the development process. They also facilitate knowledge sharing among team members and ensure adherence to coding standards and best practices.
Automate Testing
Automated testing, including unit tests, integration tests, and end-to-end tests, helps identify defects quickly and consistently. Continuous integration and continuous deployment (CI/CD) pipelines can automate the testing process, enabling faster feedback loops.
Use Version Control
Version control systems like Git enable teams to track changes to the codebase, collaborate effectively, and revert to previous versions if needed. This promotes transparency and reduces the risk of introducing errors.
Prioritize Security
Incorporate security practices throughout the development process, including threat modelling, code scanning, penetration testing, and security reviews. Security should be considered at every stage, from design to deployment.
Focus on User Experience (UX)
Design the software with the end user in mind. Conduct user research, gather feedback, and iterate based on user needs and preferences. A well-designed user interface enhances usability and overall satisfaction.
Document everything
In addition to the standard documentation (SRS, test plans etc.), document the thought process behind technical decisions. This should be a continuous process. This enables proactive identification of any issues during reviews as well as making it easy for anybody new to get up to speed quickly.
Continuous Monitoring and Feedback
Implement monitoring and logging mechanisms to track system performance, errors, and user behaviour in real time. This enables proactive identification of issues and opportunities for optimization.
Invest in Training and Education
Provide ongoing training and professional development opportunities for team members to stay updated on emerging technologies, best practices, and industry trends. Knowledgeable and skilled teams are better equipped to deliver high-quality software.
Promote a Culture of Quality
Foster a culture where quality is everyone's responsibility. Encourage open communication, collaboration, and accountability among team members. Recognize and reward efforts that contribute to maintaining and improving software quality.
SQA, Agile and DevOps
Traditional approach (Waterfall SDLC)
It is a sequential process where each phase of the software development lifecycle is completed before moving on to the next phase.
SQA is performed at the end of each phase to ensure that the requirements have been met before moving to the next phase.
This approach involves requirement analysis, design, coding, testing, and maintenance to ensure that the software product is developed with minimal errors and defects and meets the desired quality standards.
Agile approach
The Agile approach to SQA is an iterative, incremental, and flexible approach that focuses on delivering software products in small increments.
This approach emphasizes collaboration between the development team and the stakeholders for a seamless and quick development process.
Agile SQA is quite popular and focuses on self-organizing teams, continuous integration and testing, continuous delivery, and continuous feedback to ensure a high- quality software product.
DevOps approach
This approach promotes collaboration between development and IT operations to ensure that the software product meets the requirements of the customers.
DevOps is focused heavily on automation, such as continuous integration, continuous testing, and continuous deployment to deliver a high-quality product.
This approach is great for projects that require frequent updates.
Version Controlling & Workflows with Git - I
Version Controlling: What?
“Version control is a system that records changes to a file or set of files over time so that you can recall specific versions later."
What you know about Version Controlling
Methods of Version Controlling:
Keeping multiple copies of files with different names
Google Drive/ OneDrive
Undo/ Redo Buffer
Version Controlling: Scenario Discussion
Amith is working on numbers.py and saves multiple copies: numbers1.py, numbers2.py, numbers3.py, and numbers.py.
numbers.py is the latest version. Before updating the file, the existing numbers.py is renamed to numbersX.py (X = {1, 2, 3, 4…}).
What can Amith do?
Submit the previous version (e.g.: numbers10.py).
Compare numbers10.py and numbers.py, copy the good parts, and paste as fit in the appropriate places in numbers10.py.
Replace the numbers.py with this modified file.
What can Amith do to address this very valid concern?
Copy all the versions of numbers.py to a Cloud storage location (OneDrive/ Google Drive).
The pros of above approach?
All the versions are safely stored.
The files can be accessed from anywhere now.
The cons?
Considering accessing from anywhere: If Amith develops two versions of numbers.py (both named the same), one at the computer lab in his university (then uploaded to the cloud) and another numbers.py separately on his laptop, Amith runs the risk of replacing the version on laptop with the other if he uploads the first file without the name being changed when his laptop syncs with the cloud storage.
Version Controlling: Scenario Discussion
Amith now has a working Version Controlling System that can:
Revert to a past version.
Compare two file versions.
Push changes done from anywhere to a secure remote location on the cloud.
Pull changes from the cloud to anywhere.
Merge different versions of files.
Binesh joins Amith as future assignments are to be done as pair work; it is more important than ever to follow a strict discipline when using Amith’s system of version controlling.
Solution? A log file! The log file should contain information on who did what when so everybody would be aware of what is happening.
Does Amith’s method solve the problem of version controlling he had earlier? Yes!
Does that mean it is an easy system to follow? Not really. There are several specific rules the users of the system must follow in order to make it work.
Version Controlling Systems (VCS) History
1973: Source Code Control System (SCCS) was released by Bell Labs.
First-generation VCS, now known as Local Version Control Systems.
Track changes for individual files and checked-out files could only be edited locally by one user at a time.
Facilitating collaboration among large teams was problematic.
Centralized Version Control Systems (Client-Server approach).
Allowed users to refer files stored on a centralized repository through a network and allowed multiple users to use the files concurrently.
However, they would all have to commit their code to the centralized repository, which would require network access.
A central authority is required to maintain the Centralized repository.
Distributed Version Control Systems (peer-to-peer (P2P) approach to version controlling).
Contributors may develop features which may or may not be relevant to the main open-source project.
Distributed VCS are widely used today.
De-facto standard in Distributed VCS now? Git!
Git: the De-Facto Standard Today in VCS
Created by Linus Torvalds (creator of Linux) in 2005.
Originally designed to do version control on Linux kernel.
Git supports non-linear development (thousands of parallel branches) and is fully distributed.
Git Basic Concepts
Git does not necessarily need a Centralized Repository to work.
Every user maintains a local repository of their own.
They will obtain this local repository from another user, or clone it from a central location, or initialize a brand-new repository from scratch.
This repository will contain what others have done originally, what the user has done themselves and even work of others as selected by the user.
They can commit and update their local repository without any interference from any others.
Git Concepts
Git has two types of repositories:
Local repositories – which is owned by each user.
Remote repository – a centralized location accessible to all users. A single source of truth.
The users will push their new updates to this repository so those can be shared with others as well. The users will pull new updates from others from this repository. Which project is the “right” project when multiple versions of the project exist?
Solution? Remote repositories as a single source of truth!
Git != GitHub (Git is not Github)!
Git works on Commits.
A Commit is very simply a snapshot of all changes in the local repository (more correctly, the working directory).
Each Commit has a unique Commit ID.
A Commit ID is a unique SHA-1 hash that is created whenever a new commit is recorded.
The most recent commit is called the Head (more correctly, it refers to the currently checked-out branch's latest commit).
For now, we will only deal with main/ master Branch.
We will extensively deal with Commit IDs for many operations on git (more on this later).
Add a new file to the local Repo.
Do the required changes to the file.
Commit the file to the local repo.
Pull the changes from the remote repo.
Push the local changes to the remote repo.
Scenario Revisited
Refer back to Amith’s version controlling method in the beginning. All these actions can be easily accomplished using Git.
Revert – git revert (this reverts a commit that was made to the repository).
Compare & Merge – Automatically handled by git. No user intervention required most of the time (unless a conflict occurs).
Push and Pull – Update the Upstream and get updates from upstream.
Git has many ways of automatically keeping track of all the events (e.g.: git status, log, blame etc.)
Version Controlling & Workflows with Git - II
Learning Outcomes
Apply the basic concepts of Git for version controlling
Apply an efficient branching strategy for a team
Describe what a git workflow is and why a workflow is important.
Apply a simple git workflow to your own projects.
From the Previous Week
Version Controlling: Scenario Discussion
A bit of history on Version Controlling Systems (VCS)
Git Basic Concepts
Repositories – Local and Remote
Each user has their own local repository
This local repository can be obtained by:
Copying from another user (P2P)
Cloning it from a remote repository (on GitHub, BitBucket etc.)
A branch is a detour you make from the main path of development.
The default branch in Git is the master/ main branch
Why is branching needed?
Branching lets developers work independently inside a single code repository.
Allows a developer to switch among different tasks easily.
Using git branches during development time is clearly more efficient rather than not using any.
Using branches is a must for collaborations with multiple developers.
Whatever you do on a branch is isolated from the rest of the code – great when you want to experiment!
It is not a bad idea to use branching for individual projects too.
Branching on Git is a very lightweight operation; therefore, use it whenever possible.
How do we create a branch on Git?
git branch <branch_name> - just creates the branch based on your current branch.
git checkout <branch_name> - checks out the branch.
git checkout –b <branch_name> - creates the branch, and switches to the new branch (checks it out) from your current branch immediately.
Workflows (Branching Strategies)
Git is very flexible when it comes to how-to branch. There is no one standard method, but some well accepted how-to branch ‘recipes’ are there; they are called workflows or branching strategies.
A Git workflow is a recipe or recommendation for how to use Git to accomplish work in a consistent and productive manner.
Some popular workflows are:
GitFlow
GitHub Flow
GitLab Flow
Trunk Based Development (goes with CI/ CD)
GitFlow Workflow
GitFlow is very comprehensive.
Can become unnecessarily complex for small projects.
It is more suited to Enterprise grade software development efforts.
Requires enforcement of the practices for the proper functioning of the workflow.
For most use cases, it is overkill.
GitHub Workflow
A simpler alternative is preferable for most use cases.
One such workflow is GitHub Flow which can be used by anybody.
Create a new feature branch from the main (master) branch.
Make the necessary changes to the feature branch – continue until you think your feature is ready.
Once you think the feature is done, ask for feedback from the other developers – create a Pull Request (PR) for this.
Address their review comments received on the Pull Request (PR).
Merge your Pull Request (PR) to the main branch.
Delete your feature branch.
Pull Requests (PR)
A Pull Request (PR) is used to propose a new change to a repository (usually to the main branch).
Usually, the proposed changes are in a feature branch, which ensures that the main branch will only contain code which is tested and working.
The other collaborators will have to verify that the proposed change (PR) is OK first.
If the PR is OK, they will accept it, or else, they will suggest changes until it is acceptable.
Once the PR is accepted, the changes in the feature branch are merged automatically to the main branch, bringing the proposed change to the main branch.
Git Branching: Best Practices
Create a branch for each feature. Do not work on multiple features in a single branch.
Ideally, only one person should work on one feature branch.
Feature branches should be short-lived. They should not exist once the feature is complete.
Delete the local and remote feature branch after merging to the master branch.
Merge early and often to avoid merge conflicts.
Keep the master branch clean. Only production ready code should be in the master branch. It is better to avoid working directly on the master branch.
Use a good naming convention to name your feature branches:
Make small, incremental changes. Don’t make huge changes in one go. If you have a big feature to add, break it down to smaller features and add one at a time.
Keep commits atomic. It is better if each commit is focused on one part of the feature. This will minimize the damage if you must revert a commit.
Commit often. Commit your changes often, even if the work is not done.
Pull the changes in origin/main branch before you create a new branch from your local/main. This ensures that the new feature branch is created with all the latest changes in master. This reduces the possibility of merge conflicts later.
Push local feature branch commits to Remote often. Even if it is an unfinished feature, push it. That’s good for safekeeping.
Use branches. See the section on branching for a comprehensive set of reasons.
Write good commit messages. Make the lives of other Software Engineers, DevOps engineers, QA engineers easier.
Select one branching strategy (Git Workflow) and stick with it. Select one that suits you and use it well.
Web Application Architecture - An Overview
Learning Outcomes
Describe the main components of three-tier architecture for web development.
Identify some suitable technologies to develop the presentation layer and the application layer of a web application.
Apply suitable architectural components to design the presentation layer and the application layer of a Three-tier web application.
Website vs. Web Application
A Website provides static content which can be consumed by the end users.
A Web Application is designed for interaction with the end users.
Focus is on Web Applications.
Three-Tier Architecture
Modern web applications are engineered based on this architecture.
Has three tiers:
Presentation tier/ layer
Application tier/ layer
Database tier/ layer
The most popular form of n-tier (multitier) architecture.
Three-tier architecture is a good choice for the development of web applications, because:
Each tier/ layer runs on its own infrastructure.
Each tier/ layer can be developed independently/ in parallel by different teams.
Allows updating and scaling up/ down each tier independently without impacting the other tiers.
Better security when compared with two-tier architecture (due to the isolation of data and logic from the presentation tier).
Presentation Layer
Also called the client-side or the frontend.
Technologies used to implement include:
HTML/ CSS/ JavaScript
React.js
Angular
Vue.js
Some legacy technologies.
jQuery and AJAX
Architecting the Presentation Layer
Some well-known examples:
Model-view-controller (MVC)
Model-view-viewmodel (MVVM)
Some more examples:
Model-view-presenter (MVP)
Component Architecture
Micro frontends
Application Layer
Also called the server-side or the backend.
Technologies used to implement include but not limited to:
Programming languages and frameworks
Java (frameworks: Spring/ Spring Boot)
Node.js (frameworks: express.js/ Koa)
Python (frameworks: Django/ Flask)
PHP (frameworks: Laravel, Symphony)
.Net Framework
Virtual Machines (VMs)
Serverless Computing
Containers
How does the Backend and the frontend communicate?
Application Layer: APIs
Application Programming Interface (API):
A software interface which facilitates communication between two or more applications.
Abstracts away the complexities of application integrations.
A contract of services offered.
Benefits of APIs:
Developers don’t have to create everything from scratch - can use existing functions exposed as an API resulting in more productivity.
APIs significantly reduce development costs by reducing development efforts.
Improves collaboration and connectivity across the ecosystem.
Different types of APIs?
Representational State Transfer API (REST API)
Simple Object Access Protocol API (SOAP API)
Remote Procedure Calls (RPC)
WebSockets
GraphQL APIs
Architecting the Application Layer
Some well-known examples:
Monolithic Architecture
Microservices Architecture
Service-Oriented Architecture (SOA)
Architecting the Application Layer: Monolithic
Drawbacks in Monolithic Architecture:
Slower development speed due to local complexity.
Can’t scale individual components.
Reliability – if there’s an error in any module, it could affect the entire application’s availability.
Barrier to technology adoption – any changes in the framework or language affects the entire application, making changes often expensive and time-consuming.
Lack of flexibility – a monolith is constrained by the technologies already used in the monolith.
A small change to a monolithic application requires the redeployment of the entire monolith.
Architecting the Application Layer: Microservices
“Microservices architecture is an approach in which a single application is composed of many loosely coupled and independently deployable smaller services.”
Some benefits of Microservices:
Microservices are small, independent, and loosely coupled (therefore, a single team can write/ maintain a single service).
Services can be deployed independently (can update an existing service without rebuilding/ redeploying the entire application).
Can scale each service independently.
High reliability.
Supports polyglot programming.
High global complexity is the biggest problem when it comes to Microservices.
Local complexity depends on the implementation of a service; global complexity is defined by the interactions and dependencies between the services.
Managing complexity is a constant challenge in Microservices.
What are some other downsides of Microservices:
High infrastructure costs.
Debugging challenges.
Big Ball of Mud: An Architectural Anti-Pattern
Other Helpful Technologies
API Gateways (authentication, throttling, orchestration, caching, statistics etc.)
Content Delivery Networks (CDN)
Caching Tools (e.g., Redis)
Load Balancers
Message Queues
Cloud Computing
Cloud Storage
Virtual Machines (VMs)
REST APIs
Learning Outcomes
Describe the six REST architectural constraints.
Apply the knowledge gained in this lecture in designing proper REST APIs.
Evaluate if an API is a REST API or not.
Evaluate a given API to identify how much it conforms to the REST constraints using the Richardson Maturity Model.
Consuming Web Resources - Human vs. Programs
How do you consume web resources?
Web browsers
Web applications
Mobile applications etc.
How do programs consume web resources?
Web APIs!
Application Programming Interface (API)
What is an API?
A software interface which facilitates communication between two or more applications.
Abstracts away the complexities of application integrations.