Back End Development and APIs Review

## Working with Node.js and Event-Driven Architecture - **Node.js**: An open-source, cross-platform JavaScript runtime environment that allows you to run JavaScript code outside of a browser. It is widely used to build web servers, APIs, and back-end systems. - **Client-side limitations**: Client-side JavaScript has restricted file system access, limited support for complex application logic, and potential security risks (e.g., you must not expose database credentials in client-side code). - **Browser vs. Node.js global object**: In the browser, the global object is `window`, which exposes DOM manipulation, cookies, and browser events. In Node the global object is `global`, which exposes built-in modules for files, networking, and OS interaction. - **DOM access**: The browser provides access to the DOM API, but Node does not. Conversely, Node can access almost all system resources, including the file system, while the browser cannot. - **Version control**: You can choose which version of Node runs on your server. You have no control over which browser version your users run. - **Non-blocking, event-driven architecture**: Node uses a single thread and an event loop that efficiently handles many simultaneous requests and I/O operations without blocking, making it ideal for real-time applications. - **Callbacks**: Functions that define what happens once an asynchronous operation completes. Heavy reliance on them is part of why asynchronous Node code can be harder to read and debug. - **Advantages of Node on the back-end**: Single language (JavaScript) across front-end and back-end, non-blocking architecture suited to high concurrency, large npm ecosystem, free and open-source. - **Disadvantages of Node on the back-end**: Single-threaded, so CPU-intensive tasks (image processing, cryptography, complex math) can block the event loop, heavy reliance on asynchronous/callback-based code can reduce readability, npm packages vary in quality and maintenance. - **NVM (Node Version Manager)**: A tool for managing and switching between multiple versions of Node.js on the same machine. On macOS/Linux install with: ```bash curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.3/install.sh | bash ``` - **Installing the LTS version of Node**: `nvm install --lts` - **Switching Node versions**: `nvm use 20` - **Listing all installed Node versions**: `nvm ls` - **Setting a default Node version (macOS/Linux)**: `nvm alias default 22.20.0` - **nvm-windows**: A separate project for managing Node versions on Windows (not WSL). Download the installer from the official repository and run `nvm install lts` then `nvm use <version>`. - **Running a JavaScript file with Node**: `node app.js` ## Working with Node Core Modules ### The `fs` Module - **Purpose**: Provides methods for creating, reading, writing, appending, and deleting files and directories. - **Three usage styles**: Callback-based (asynchronous), promise-based (`fs/promises`), and synchronous (`Sync` suffix). Synchronous methods block the event loop and are suited only for scripts. - **`writeFile()`**: Writes content to a file, creating it if it does not exist: ```js const fs = require("fs/promises"); async function writeToFile() { try { await fs.writeFile("article.md", "## Node `fs` Module", "utf8"); console.log("File written to!"); } catch (err) { console.error("Error writing to file:", err); } } ``` - **`appendFile()`**: Appends content to an existing file without overwriting it. - **`readFile()`**: Reads file content. Pass `"utf8"` as the encoding to get a string, or omit it to get a `Buffer`. - **`unlink()`**: Deletes a file. Throws `ENOENT` if the file does not exist. ### The `Buffer` Module - **Purpose**: Handles binary data (files, images, network streams) directly in memory. - **`Buffer.from()`**: Creates a buffer from a string: ```js const { Buffer } = require("buffer"); const myStrBuffer = Buffer.from("freeCodeCamp"); console.log(myStrBuffer); // <Buffer 66 72 65 65 43 6f 64 65 43 61 6d 70> console.log(myStrBuffer.toString()); // freeCodeCamp ``` - **`Buffer.alloc(n)`**: Creates a zero-filled buffer of `n` bytes: ```js const someBuffer = Buffer.alloc(10); someBuffer.write("Hello fCC"); console.log(someBuffer.toString()); // Hello fCC ``` - **Truncation**: Writing more data than the buffer can hold silently truncates the excess. - **`Buffer.byteLength(str)`**: Returns the number of bytes needed to store a string. ### The `crypto` Module - **Purpose**: Provides cryptographic tools such as hashing, HMAC, encryption/decryption, digital signatures, and secure random number generation. - **`createHash(algorithm)`**: Creates a one-way hash (irreversible). Use `.update(data).digest("hex")` to provide the data to hash and return the hash as a hexadecimal string: ```js const crypto = require("crypto"); const hashedPassword = crypto .createHash("sha256") .update("myStrongPassword") .digest("hex"); ``` - **`createHmac(algorithm, secret)`**: Like `createHash` but requires a secret key, suitable for verifying data integrity and authentication. - **`createCipheriv()` / `createDecipheriv()`**: Encrypt and decrypt data. Both require an algorithm, a key, and an initialization vector (IV): ```js const cipher = crypto.createCipheriv("aes-256-cbc", key, iv); let encrypted = cipher.update("Hello campers!", "utf8", "hex"); encrypted += cipher.final("hex"); ``` - **`randomBytes(size)`**: Generates cryptographically secure random bytes, safer than `Math.random()` for tokens and keys. - **`randomInt(min, max)`**: Generates a secure random integer in the given range. Useful for OTPs. - **`sign()` / `verify()`**: Create and verify digital signatures using a private/public key pair. ### The `os` Module - **Purpose**: Exposes information about the operating system Node is running on. - **`os.platform()`**: Returns the OS identifier, e.g. `'darwin'`, `'win32'`, `'linux'`. - **`os.arch()`**: Returns the CPU architecture, e.g. `'x64'`, `'arm64'`. - **`os.type()`**: Returns the official OS name, e.g. `'Darwin'`. - **`os.release()`**: Returns the OS kernel version string. - **`os.cpus()`**: Returns an array of objects describing each logical CPU core (model, speed, times). - **`os.uptime()`**: Returns the system uptime in seconds. - **`os.totalmem()` / `os.freemem()`**: Return total and available system memory in bytes. - **`os.userInfo()`**: Returns an object with the current user's `uid`, `gid`, `username`, `homedir`, and `shell`. - **`os.networkInterfaces()`**: Returns network interface objects with address, family, MAC, and CIDR details. ### The `path` Module - **Purpose**: Provides utilities for working with file and directory paths in a platform-independent way. - **`__filename`**: A global variable, the absolute path of the current file. - **`__dirname`**: A global variable, the absolute path of the directory containing the current file. - **`path.basename(p)`**: Returns the last component of a path (the filename). - **`path.dirname(p)`**: Returns the directory portion of a path. - **`path.extname(p)`**: Returns the file extension, e.g. `'.js'`. - **`path.join(...segments)`**: Joins path segments into a normalized path and fixes erroneous slashes. - **`path.resolve(...segments)`**: Resolves segments into an absolute path starting from the current working directory. - **`path.parse(p)`**: Returns an object with `root`, `dir`, `base`, `ext`, and `name` properties. - **`path.format(obj)`**: Builds a path string from an object with `dir`, `name`, and `ext` properties. ### The `process` Module - **Purpose**: Exposes information and control over the current Node.js process. It is a global object, so no import required. - **`process.env`**: Object containing all environment variables. Access specific values like `process.env.NODE_ENV`. - **`process.argv`**: Array of command-line arguments. Index 0 is the Node executable, index 1 is the script path, index 2+ are user arguments. - **`process.cwd()`**: Returns the current working directory. - **Process events**: ```js process.on("exit", (code) => { console.log(`Process exiting with code: ${code}`); }); process.on("uncaughtException", (err) => { console.error("Uncaught error:", err.message); }); ``` - **`process.emitWarning(message, type)`**: Triggers a custom warning catchable by a `warning` listener. ### The `stream` Module - **Purpose**: Processes large amounts of data in chunks rather than loading everything into memory at once. - **Four stream types**: Readable (read data in chunks), Writable (write data in chunks), Duplex (both), Transform (a Duplex that modifies data as it flows through). - **Piping streams**: ```js const readInputFileStream = fs.createReadStream(inputFilePath); const writeOutputFileStream = fs.createWriteStream(outputFilePath); readInputFileStream.pipe(writeOutputFileStream); writeOutputFileStream.on("finish", () => { console.log("All data has been written to the file"); }); ``` - **`finish` event**: Fires on a writable stream when all data has been flushed. - **`error` event**: Fires when a stream encounters a problem. ## Introduction to npm - **npm**: Comprises three parts: the website `npmjs.com`, the public registry (a database of packages), and the CLI tool installed with Node.js. - **Purpose**: Lets you install, manage, and publish reusable JavaScript packages. npm is the most widely used package manager. - **Alternatives**: Yarn, PNPM, Bun. ### The `package.json` File - **Purpose**: The npm configuration manifest. It tracks the project name, version, description, entry point (`main`), author, license, scripts, and dependencies. - **Creating `package.json` interactively**: `npm init` - **Creating with defaults**: `npm init -y` - **`dependencies`**: Packages required for the app to run in production (e.g., `express`, `react`). - **`devDependencies`**: Packages only needed during development and testing (e.g., `jest`, `nodemon`, `eslint`). - **`license`**: Legal terms for usage. Common values: `MIT`, `ISC`, `GPL`. ### Semantic Versioning (SemVer) - **Format**: `MAJOR.MINOR.PATCH`, e.g., `4.17.21`. - **MAJOR**: Incremented for breaking changes. - **MINOR**: Incremented for new backwards-compatible features. - **PATCH**: Incremented for backwards-compatible bug fixes. - **Caret `^`**: Allows updates to minor and patch versions but not major. Example: `^4.17.21` accepts `4.18.0` but not `5.0.0`. - **Tilde `~`**: Allows updates to patch version only. Example: `~1.2.3` accepts `1.2.4` but not `1.3.0`. - **Asterisk `*`**: Matches any version, suitable only for testing, never production. - **Exact version**: No prefix, e.g. `1.2.3` pins to exactly that version. ### Installing and Removing Dependencies - **Install a package**: `npm install express` (shorthand: `npm i express`) - **Install as a devDependency**: `npm install nodemon -D` - **Uninstall a package**: `npm uninstall chalk` - **`node_modules/`**: Folder created after installation containing actual package code. Not committed to version control. - **`package-lock.json`**: Auto-generated file that locks exact versions of every installed package (including child dependencies). Always commit this file and do not edit it manually. - **Install a specific version**: `npm install express@4.21.2` - **View available versions**: `npm view express versions` - **Check for outdated packages**: `npm outdated` - **Update a single package** (within allowed range): `npm update express` - **Force install the latest version**: `npm install express@latest` ### Publishing to the npm Registry - **Login**: `npm login` - **Scoped packages**: Use `@username/package-name` to avoid name collisions. - **`.npmignore`**: Lists files to exclude from the published package (similar to `.gitignore`). - **Publish an unscoped package**: `npm publish` - **Publish a scoped package publicly**: `npm publish --access public` ## Working with npm Scripts - **npm scripts**: Custom commands defined in the `"scripts"` object of `package.json`. They automate repetitive tasks such as starting a server, running tests, linting, and building. - **Defining a script**: ```json "scripts": { "start": "node app.js" } ``` - **Running a script**: `npm run start` - **Shorthand for built-in script names** (`start`, `test`, `stop`, `restart`): `npm start` - **Sequential commands with `&&`** (next command runs only if the previous succeeds): ```json "start": "npm run build && node server.js" ``` - **Sequential commands with `;`** (next command always runs): ```json "start": "npm run build; node server.js" ``` - **Concurrent commands with `&`**: `"dev": "npm run start:server & npm run start:client"` - **Passing arguments to a script** (use `--` separator): `npm start -- 8000`; arguments accessible via `process.argv[2]`. ### CommonJS Modules - **CommonJS**: The original Node.js module system. Loads modules synchronously. - **Importing** with `require()`: ```js const { multiply } = require('./math'); ``` - **Exporting** with `module.exports`: ```js function multiply(a, b) { return a * b; } module.exports = { multiply }; ``` - CommonJS is the default when `package.json` has `"type": "commonjs"` (or no `"type"` field), or the file has a `.cjs` extension. ### ES Modules (ESM) - **ES Modules**: The modern, standardized module format. Loaded asynchronously. - **Named export**: `export function multiply(a, b) { return a * b; }` - **Default export**: `export default function multiply(a, b) { return a * b; }` - **Import**: `import { multiply } from './math.mjs';` - ESM is used when the file has an `.mjs` extension or `package.json` contains `"type": "module"`. ## Understanding How HTTP, DNS, and TCP/IP Work ### Servers and the Client-Server Model - **Client-Server Model**: A distributed system architecture where components are either clients (processes that send requests, e.g., a web browser) or servers. - **Server**: A computer or software program that receives requests from clients and returns data or services over a network. - **Types of servers**: Web servers (HTML/CSS/assets), application servers (business logic), database servers, mail servers, file servers, proxy servers. ### DNS - **DNS (Domain Name System)**, often called the "phone book of the internet": Translates human-readable domain names (e.g., `freecodecamp.org`) into IP addresses. - **Domain name hierarchy** (right to left): Top-Level Domain (TLD, e.g. `.org`) → Second-Level Domain (e.g. `freecodecamp`) → Subdomain (e.g. `www`). - **IP address**: A unique numerical identifier for a device on a network. IPv4 example: `127.0.0.1`. IPv6 example: `0:0:0:0:0:0:0:1`. - **DNS resolution process**: Browser → Recursive Resolver → Root Servers → TLD Server → Authoritative Name Server → returns IP address → Recursive Resolver caches and delivers to client. ### TCP/IP and HTTP - **IP (Internet Protocol)**: Routes data packets from source to destination using IP addresses. Connectionless, each packet is sent independently. - **TCP (Transmission Control Protocol)**: Breaks data into numbered segments, reassembles them at the destination, checks for errors, and requests retransmission if needed. Ensures reliable, ordered delivery. - **HTTP (HyperText Transfer Protocol)**: Application-layer protocol defining how clients and servers exchange web content. - **HTTPS**: A secure version of HTTP where data is encrypted using SSL/TLS. ### HTTP Request Structure ```http GET / HTTP/1.1 Host: example.com User-Agent: Mozilla/5.0 Accept: text/html Connection: keep-alive ``` - **Request line**: HTTP method + path/URI + HTTP version. - **Headers**: Key-value metadata pairs. - **Body** (optional): Sent with `POST`, `PUT`, `PATCH`, `DELETE` to carry data. ### HTTP Response Structure ```http HTTP/1.1 200 OK Content-Type: text/html Content-Length: 105 <!DOCTYPE html>... ``` - **Status line**: HTTP version + status code + reason phrase. - **Headers**: Metadata about the response (content type, length, caching directives). - **Body** (optional): The requested data. ### HTTP Status Code Categories - **1xx Informational**: Request received, processing continues. Example: `101 Switching Protocols`. - **2xx Success**: `200 OK`, `201 Created`, `204 No Content`. - **3xx Redirection**: `301 Moved Permanently`, `302 Found`, `304 Not Modified`. - **4xx Client Errors**: `400 Bad Request`, `401 Unauthorized`, `403 Forbidden`, `404 Not Found`. - **5xx Server Errors**: `500 Internal Server Error`, `502 Bad Gateway`, `503 Service Unavailable`. ## Understanding the HTTP Request-Response Model - **HTTP methods**: `GET` (retrieve), `POST` (create), `PUT` (full update), `PATCH` (partial update), `DELETE` (remove). - **Request-response cycle**: Client sends request → Server validates auth, checks method and URL, queries database → Server builds response (status code + optional body) → Client handles response. ### Assets Returned in HTTP Responses Every response includes a `Content-Type` header telling the browser how to handle the asset: - **HTML**: `text/html` - **CSS**: `text/css` - **JavaScript**: `application/javascript` - **JSON**: `application/json` - **PNG/JPEG/WebP**: `image/png`, `image/jpeg`, `image/webp` - **MP4/MP3**: `video/mp4`, `audio/mpeg` - **PDF**: `application/pdf` - **WOFF2**: `font/woff2` ### HTML Form Submissions - **`action` attribute**: Specifies the URL where form data is sent. - **`method` attribute**: Specifies the HTTP method (`GET` or `POST`). HTML does not natively support `PUT`, `PATCH`, or `DELETE`, use the Fetch API or the `method-override` package in Express instead. - **Client-side validation**: Use `required` and `pattern` attributes for immediate feedback. Always validate on the server too. - **`GET` form**: Data appended to the URL as query parameters (e.g., search forms). - **`POST` form**: Data sent in the request body (e.g., login and registration forms). ## Understanding the Web Standards Model ### Standards Bodies - **WHATWG**: Maintains HTML and the DOM as a continuously updated "living standard." Decision-making is driven by major browser engine teams (Chromium, Gecko, WebKit). - **W3C (World Wide Web Consortium)**: Primary standards body. Oversees CSS, SVG, accessibility guidelines (WCAG), and numerous web APIs. - **ECMA International / TC39**: Ecma publishes the official ECMAScript specification. TC39 develops the language through a five-stage proposal process (Stage 0 rough idea → Stage 4 fully specified). - **Khronos Group**: Focuses on graphics, compute, and media. Produces WebGL and WebGL 2 for GPU-accelerated 3D rendering in browsers. ### Creating Web Standards 1. **Identifying a need**: A developer or organization encounters friction and writes it down. 2. **Proposal and early discussion**: Informal GitHub issue or explainer shared to gauge interest. In TC39 this is Stage 0/Stage 1. 3. **Writing the specification**: A precise formal document covering every edge case, error condition, and security implication. 4. **Review, objection, and iteration**: Draft reviewed publicly by browser vendors, features that a major browser refuses to implement rarely become standards. 5. **Implementation and testing**: Vendors implement behind a feature flag, a shared test suite ensures all browsers behave identically. 6. **Finalization**: W3C calls it a Recommendation, TC39 calls it Stage 4. ### Lifecycle of a Web Standards Feature 1. **Proposal**: Problem identified and written up. 2. **Incubation**: Open discussion, many proposals die here. 3. **Specification**: Formal spec written with exact rules. 4. **Interoperability testing**: W3C Web Platform Tests suite verifies cross-browser consistency. 5. **Finalization**: Feature ships without flags as part of the baseline web platform. 6. **Deprecation**: Superseded or unsafe features marked for removal, but browsers keep them for backwards-compatibility. ### Key Principles of Web Standards - **Openness**: Standards are developed in the open and implemented royalty-free. - **Accessibility**: Baked into HTML, CSS, and browser behavior, WCAG provides formal guidelines. - **Interoperability**: A page built to standard should behave identically across all browsers. - **Backwards compatibility**: Old pages must keep working in modern browsers. - **Privacy and security**: New APIs that could expose user data face significant scrutiny. - **Layering**: HTML for structure, CSS for presentation, JavaScript for behavior. Each layer can be adopted independently. - **Progressive enhancement**: Features must work for everyone first, then improve for users with more capable browsers. ## Understanding REST APIs and Web Services ### Web Services - **API (Application Programming Interface)**: A general mechanism for software to communicate with other software. - **Web service**: A specific type of API that uses web protocols (HTTP/HTTPS) to exchange data over a network. Every web service is an API, but not every API is a web service. - **Characteristics**: Runs on a server, enables machine-to-machine communication, language-agnostic, platform-agnostic, uses standard web protocols. - **REST**: Representational State Transfer, the most widely used web service style. Simple, flexible, works naturally with HTTP. - **SOAP**: Simple Object Access Protocol, uses XML only, common in enterprise systems needing strict standards. - **XML-RPC**: Uses XML to encode lightweight requests and responses. ### REST Architecture - **Statelessness**: Each request is handled independently. The server stores no session state between requests. Every request must include all necessary information (e.g., an auth token on every call). - **Resources**: Everything is a resource identified by a URL, e.g., `/products` for a collection, `/products/1` for a specific item. - **HTTP methods in REST**: `GET` (retrieve), `POST` (create), `PUT`/`PATCH` (update), `DELETE` (remove). - **Response formats**: JSON is most common but XML and plain text are also supported. Clients specify format via the `Accept` header: ```bash Accept: application/json ``` ### Microservices - **Monolith**: One large codebase where the UI, database, and API all reside together. Difficult to maintain and scale as complexity grows. - **Microservices**: An architectural approach where a large application is broken into small, independent services, each responsible for a single business capability. Services communicate via APIs or message queues. - **Advantages**: Faster independent development and deployment, better fault isolation, different services can use different languages or frameworks. - **Disadvantages**: More complex debugging (issues span services), higher infrastructure overhead, increased network communication. ## Working with Express - **Express.js**: A minimal and flexible web framework built on top of Node.js. It simplifies building web servers and APIs with far less boilerplate than the native `http` module. - **A basic Express server**: ```js const express = require("express"); const app = express(); const port = 3000; app.get("/", (req, res) => { res.send("Hello World!"); }); app.listen(port, () => { console.log(`Example app listening on port ${port}`); }); ``` - **`express()`**: Factory function that creates an Express application instance (`app`). - **`app.listen(port, callback)`**: Starts the server and begins listening for HTTP requests on the specified port. - **`req`**: The request object, contains URL, headers, params, body, and query data. - **`res`**: The response object, used to send data back to the client. - **Middleware**: Code that runs between receiving a request and sending a response. ## Understanding Routing in Express.js - **Routing**: Defines how an application responds when a client visits a specific URL with a specific HTTP method. - **`app.get/post/put/delete(path, handler)`**: Registers handlers for the respective HTTP methods. - **Static route**: A fixed URL pattern, e.g., `app.get("/home", handler)`. - **Dynamic route with parameters**: Uses `:paramName` placeholder. Captured via `req.params`: ```js app.get("/post/:postId", (req, res) => { const postId = req.params.postId; res.send(`Viewing post with ID: ${postId}`); }); ``` - **`req.params`**: Route parameters (`:id` segments). - **`req.query`**: Query string parameters (`?key=value`). - **`req.body`**: Request body (requires body-parsing middleware). - **`res.send(data)`**: Sends a response (text, HTML, JSON, or binary). - **`res.json(obj)`**: Sends a JSON response, automatically sets `Content-Type: application/json`. - **`res.status(code).send(msg)`**: Sets the status code and sends a response. - **`res.redirect(url)`**: Redirects the client to another URL. - **`res.render(view, data)`**: Renders a template engine view and sends HTML. ### Chainable Route Handlers with `app.route()` Groups multiple HTTP methods for the same path to reduce repetition: ```js app .route("/user") .get((req, res) => { res.send("Fetching user data"); }) .post((req, res) => { res.send("Creating a user"); }) .put((req, res) => { res.send("Updating the user"); }) .delete((req, res) => { res.send("Deleting the user"); }); ``` ### `express.Router()` Creates a modular mini-application for grouping related routes: ```js const express = require("express"); const router = express.Router(); router.get("/", (req, res) => { res.send("List of users"); }); router.get("/:id", (req, res) => { res.send(`User details for user with id: ${req.params.id}`); }); module.exports = router; ``` ```js const userRoutes = require("./userRoutes"); app.use("/users", userRoutes); ``` ### Serving Static Files - **`express.static(directory)`**: Built-in middleware that serves files from a specified directory: ```js app.use(express.static(path.join(__dirname, "public"))); ``` - Serve from multiple directories with a mount path: `app.use("/images", express.static(path.join(__dirname, "images")));` ## Understanding Error Handling and Health Checks ### Error Handling in Express - **Error-handling middleware**: Identified by four parameters `(err, req, res, next)`. Always placed at the end of the middleware stack: ```js app.use((err, req, res, next) => { console.error(err.message); res.status(500).send("Internal Server Error"); }); ``` - **Express 4 async errors**: Must be caught manually with `try/catch` and forwarded with `next(err)`: ```js app.get("/user", async (req, res, next) => { try { const user = await getUserFromDatabase(); res.send(user); } catch (err) { next(err); } }); ``` - **Express 5 async errors**: Automatically forwarded to error-handling middleware, no manual `next(err)` needed. - **404 handler**: Place after all routes but before the error handler: ```js app.use((req, res, next) => { res.status(404).send("Sorry, that route does not exist."); }); ``` ### Debugging and Logging - **Built-in debug**: Enable by setting the `DEBUG` environment variable: `DEBUG=express:* node index.js` - **`morgan`**: A popular third-party HTTP request logger middleware. Logs method, URL, status code, and response time: ```js const morgan = require('morgan'); app.use(morgan('dev')); ``` ### Health Checks and Graceful Shutdowns - **Health check route**: Returns `200 OK` to signal the app is running. Used by load balancers and container orchestrators: ```js app.get('/health', (req, res) => { res.status(200).send('OK'); }); ``` - **`SIGTERM`**: Signal sent when a process should terminate. Handle it to allow in-progress requests to finish: ```js process.on('SIGTERM', () => { server.close(() => { console.log('Server closed. Cleaning up...'); }); }); ``` - **`SIGINT`**: Signal sent when pressing <kbd>Ctrl</kbd> + <kbd>C</kbd> during local development. Handle the same way as `SIGTERM`. - **Marking as unhealthy before shutdown**: Return a `503` from the health check endpoint during shutdown to stop the load balancer from routing new traffic: ```js let isShuttingDown = false; app.get('/health', (req, res) => { if (isShuttingDown) { res.status(503).send('Shutting down'); } else { res.status(200).send('OK'); } }); process.on('SIGTERM', () => { isShuttingDown = true; server.close(() => { console.log('Graceful shutdown complete'); }); }); ``` ## Express Middleware - **Middleware**: A function that executes during the lifecycle of a request. It has access to `req`, `res`, and `next`. Call `next()` to pass control to the next middleware, omit it to end the cycle. - **Application-level middleware**: Bound to the entire app via `app.use()`. Runs for every incoming request. - **Router-level middleware**: Bound to a specific `express.Router()` instance via `router.use()`. Applies only to routes handled by that router. - **Error-handling middleware**: Identified by four parameters `(err, req, res, next)`. Placed at the bottom of the stack. - **`express.json()`**: Parses incoming requests with JSON payloads. Makes data available on `req.body`. - **`express.urlencoded({ extended: true })`**: Parses URL-encoded form data. - **`express.static('public')`**: Serves static files from a directory. - **`cors`**: Enables Cross-Origin Resource Sharing. Install with `npm install cors`, use with `app.use(cors())`. - **`morgan`**: Logs HTTP requests. Use `app.use(morgan('tiny'))`. - Middleware is applied in the order it is registered with `app.use()`. ## Understanding WebSockets - **WebSockets**: A protocol that creates a persistent, full-duplex (bidirectional) connection between a client and a server, enabling real-time communication. - **Problem with HTTP for real-time**: HTTP requires the client to repeatedly poll the server for new data, so it is slow and inefficient for chat apps, live scoreboards, and multiplayer games. - **WebSocket handshake**: Every WebSocket connection begins with an HTTP request. The client requests an upgrade and the server responds with `101 Switching Protocols`. After that the connection stays open until either side sends a close signal. - **Use cases**: Chat applications, online multiplayer games, live sports dashboards, stock market feeds, collaborative tools. ### The `ws` Library - Node.js has no built-in WebSocket server; the `ws` library is the common way to implement one. ```js const WebSocket = require("ws"); const server = new WebSocket.Server({ port: 3000 }); server.on("connection", (socket) => { console.log("Client connected"); socket.on("message", (message) => { console.log(message.toString()); }); socket.on("close", () => { console.log("Disconnected"); }); socket.on("error", (error) => { console.error(error); }); }); ``` - **`socket.send(msg)`**: Send a message to a specific client. - **Broadcasting to all connected clients**: ```js server.on("connection", (socket) => { socket.on("message", (message) => { server.clients.forEach((client) => { if (client.readyState === WebSocket.OPEN) { client.send(message.toString()); } }); }); }); ``` - **`readyState`**: Property on every connection. `WebSocket.OPEN` means the connection is active. ### Pub/Sub Architecture - **Pub/Sub (Publish/Subscribe)**: A messaging pattern where **publishers** send messages to **topics**, and **subscribers** receive messages from topics they have subscribed to. Publishers and subscribers do not communicate directly. - **Advantages over direct communication**: Decouples components, a publisher does not need to know who its subscribers are. - **Pub/Sub vs. WebSockets**: WebSockets are the communication channel (the road). Pub/Sub is the messaging pattern that determines who receives which messages (the traffic system). ## Introduction to Authentication and Authorization ### Authentication vs. Authorization - **Authentication**: Verifies identity, answers the "Who are you?" question, must happen before authorization. - **Authorization**: Determines permissions, answers the "What are you allowed to do?" question. - **Three authentication factors**: - **Something you know**: Password, PIN, security question answer. - **Something you have**: Smartphone receiving an SMS code, authenticator app token. - **Something you are**: Fingerprint, facial recognition (biometrics). - **Common roles in authorization**: Standard User (limited self-service), Editor (content management), Admin (full system access). - In backend code, authorization is usually enforced via middleware that checks user roles or permissions before granting access. ### Cross-Site Request Forgery (CSRF) - **CSRF**: An attack where a malicious website tricks an authenticated user's browser into making an unwanted request to a backend server, exploiting the browser's automatic cookie attachment. - **Attack flow**: User logs into `trusted-bank.com` → visits `evil-hacker-site.com` → malicious site submits a hidden form to the bank → bank server sees valid session cookie and executes the request. - **`SameSite=Strict`**: Browser never sends the cookie if the request originates from a third-party website. - **`SameSite=Lax`**: Browser allows cookies only when a user safely navigates to the target site (clicking a link), but blocks them on cross-site form submissions. - **CSRF tokens**: Server generates a random unpredictable token embedded in forms. Client must include the token on every state-changing request. A malicious site cannot read the token from the frontend, so its forged requests will fail validation. ### JSON Web Tokens (JWT) - **JWT**: A digitally signed token issued by the server after successful login. The client stores it and sends it with every subsequent request, enabling stateless authentication (no server-side session storage needed). - **JWT structure**: Three Base64URL-encoded parts separated by dots: - **Header**: Token type (`JWT`) and signing algorithm (e.g., `HS256`). - **Payload**: Claims about the user (user ID, username, role). Not encrypted, so don't ever store passwords or sensitive data here. - **Signature**: Hash of header + payload + server secret key. Prevents tampering. ```bash eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3OCIsIm5hbWUiOiJKYW5lIERvZSIsImFkbWluIjp0cnVlLCJpYXQiOjE1MTYyMzkwMjJ9.Lfu39gE9OI8rhPXbmJMorXTtVappkNMw9xq-M4HXttA ``` - If a user tampers with the payload, the signature will not match and the server rejects the request with `401 Unauthorized`. ### Passport.js - **Passport.js**: Authentication middleware for Node.js. Standardizes the authentication flow and handles low-level details so developers focus on business logic. - **Strategies**: Modular plugins that implement specific authentication methods. Over 500 strategies available: `passport-local` (username/password), `passport-jwt` (JWTs), `passport-google-oauth20` (Google login), `passport-facebook`, `passport-github2`. - **Wiring into Express**: ```js const passport = require('passport'); const LocalStrategy = require('passport-local').Strategy; app.use(passport.initialize()); passport.use(new LocalStrategy((username, password, done) => { // database lookup and verification logic })); ``` - **Using as route middleware**: ```js app.post('/login', passport.authenticate('local'), (req, res) => { res.send(`Welcome back, ${req.user.username}!`); }); ``` - After successful authentication, Passport stores a piece of user data in the session cookie and attaches the authenticated user object to `req.user`. ### Helmet.js - **Helmet.js**: A security middleware collection that protects Express apps by automatically setting, modifying, or removing HTTP response headers. ```js const helmet = require('helmet'); app.use(helmet()); ``` - **Removes `X-Powered-By`**: Prevents attackers from knowing the server technology. - **Sets `Content-Security-Policy`**: Restricts where scripts, images, and styles can be loaded from, mitigating XSS attacks. - **Sets `X-Frame-Options: SAMEORIGIN`**: Prevents clickjacking by blocking your site from being embedded in external iframes. - **Sets `Strict-Transport-Security`**: Forces browsers to use HTTPS, protecting data from interception. ## Understanding Security and Privacy in Web Applications ### Security vs. Privacy - **Security**: The technical defense mechanisms that protect systems and data from unauthorized access. Security provides the lock on the door. - **Privacy**: The governance of how personal data is collected, used, and shared. Privacy provides the curtain on the window. - You can have security without privacy (a locked-down database can still sell your data), but you cannot have privacy without security (unprotected data cannot stay private). ### Same-Origin Policy and CORS - **Origin**: Defined by three components: the protocol (e.g., `https`), the domain (e.g., `example.com`), and the port (e.g., `443`). All three must match for two URLs to share the same origin. - **Same-origin policy**: A browser security rule that blocks scripts on one origin from reading data from a different origin, preventing malicious sites from stealing data from other open tabs. - **CORS (Cross-Origin Resource Sharing)**: A mechanism that lets servers relax the same-origin policy for trusted origins by setting HTTP headers. - **In Express (development)**: `app.use(cors())` allows all origins. - **In Express (production)**: Pass a `corsOptions` object with an `origin` allowlist: ```js const corsOptions = { origin: ["https://mysite.com", "https://api.mysite.com"] }; app.use(cors(corsOptions)); ``` ### HTTPS - **HTTPS**: HTTP with encryption provided by TLS (Transport Layer Security). TLS replaced the older SSL protocol. - **Three guarantees of HTTPS**: - **Encryption**: Data sent between client and server is unreadable to third parties. - **Authentication**: A digital certificate from a trusted Certificate Authority confirms the server is who it claims to be. - **Data integrity**: Messages cannot be altered in transit without detection. - HTTPS is required for modern browser APIs such as geolocation, push notifications, and camera access. ### Why Security Issues Occur - **Rapid delivery pressure**: Teams that move too fast to meet deadlines skip security reviews and ship vulnerabilities. - **Complex system integration**: Supply chain risk arises when third-party libraries or cloud services introduce vulnerabilities the development team did not write. - **Poor configuration management**: Leaving default or factory settings in place (default admin passwords, open ports) exposes systems unnecessarily. - **Human error**: Phishing attacks exploit staff who click malicious links or attachments and hand over credentials. - **Organizational mergers**: Combining different codebases and infrastructure surfaces mismatched security policies and legacy vulnerabilities. ### Cookies: Storage, Security Flags, and Privacy - **How cookies are stored**: The server sends a `Set-Cookie` header. The browser saves the file to disk and returns it automatically on every subsequent request to that origin. - **`HttpOnly` flag**: Blocks JavaScript from reading the cookie value, preventing XSS-based session theft. - **`Secure` flag**: Restricts the cookie to HTTPS connections only, protecting it from interception on public networks. - **`SameSite=Strict`**: Prevents the browser from sending the cookie on any cross-site request, fully blocking CSRF attacks. - **`SameSite=Lax`**: Allows cookies on top-level navigations (e.g., clicking a link) but blocks them on cross-site form submissions and API calls. - **First-party cookies**: Scoped to the site you are visiting. - **Third-party cookies**: Set by advertising networks across thousands of sites to track behavior and build user profiles. Modern browsers now block them by default. - **GDPR compliance**: Privacy laws require cookie consent banners giving users the legal right to reject non-essential tracking cookies. ### Common Security Threats and Mitigations - **Phishing**: Fraudulent emails that impersonate trusted companies to steal credentials. Mitigations include security awareness training and deploying email authentication protocols to block malicious messages before they reach inboxes. - **Malware** (viruses, spyware, ransomware): Ransomware encrypts files and demands payment to unlock them. Mitigations include automated updates and antimalware software. - **Credential stuffing**: Attackers test millions of stolen username and password combinations across different websites. The primary mitigation is multi-factor authentication (MFA), which requires a second verification step such as an OTP or authenticator app code. - **DDoS (Distributed Denial of Service)**: Attackers flood a server with fake traffic to crash it and block legitimate users. Mitigations include cloud-based traffic filtering services such as Cloudflare, Akamai, and AWS Shield. - **Insider threats**: Malicious or careless employees who steal or leak data. Mitigations include enforcing the **principle of least privilege** (workers only access what their job requires) and monitoring system logs for unusual activity. ### Content Security Policy and Permissions-Policy - **CSP (Content Security Policy)**: An HTTP response header that whitelists approved source domains for scripts, styles, and images. When a browser receives a CSP header, it blocks any resource loaded from a domain not on the list. The `script-src` directive specifies which domains may execute JavaScript, stopping XSS injection attacks. - **Permissions-Policy**: An HTTP header that enables or disables browser features and device hardware for a page and any embedded iframes. It replaced the older `Feature-Policy` header. - `camera=(self)` restricts webcam access to the page's own origin only. - `geolocation=()` disables geolocation entirely for all users. - Combining both headers provides layered protection: CSP prevents code injection and Permissions-Policy prevents unauthorized hardware access. ### Fundamental User Privacy Concepts - **PII (Personally Identifiable Information)**: Any data that can identify a person directly or indirectly. - **Direct identifiers**: Full name, home address, national ID number. - **Indirect identifiers**: IP address, device serial number, precise location history. - **Confidentiality**: Keeping PII safe from unauthorized access through encryption and access controls. - **Data minimization**: Only collect the absolute minimum PII required to complete a specific task. - **Purpose limitation**: Data collected for one reason cannot be repurposed (for example, sold to advertising networks) without fresh user consent. - **Tracking**: Continuous monitoring of user behavior online using scripts, device fingerprinting, and browser history logs. - **Right to be Forgotten (data erasure)**: The legal right to request permanent deletion of all personal records from a company's databases after closing an account. ### The `.env` File and `dotenv` - **`.env` file**: A plain-text file at the project root that stores configuration secrets such as API keys, database passwords, and port numbers as key-value pairs. Always add it to `.gitignore` to keep secrets out of version control. - **`dotenv` package**: Reads the `.env` file at startup and injects the values into `process.env`: ```js require('dotenv').config(); const port = process.env.PORT || 5000; const apiKey = process.env.API_KEY; ``` - **Two key benefits**: Credentials stay out of source code (security), and the same codebase can behave differently across development, staging, and production by swapping the `.env` file on each server (flexibility). ### Regional Privacy Laws and Compliance - **GDPR (General Data Protection Regulation)**: Protects individuals in the EU. Requires explicit opt-in consent before tracking, with no pre-ticked boxes allowed. Grants users the right to view and permanently delete their data. Carries heavy financial penalties for violations. - **CCPA (California Consumer Privacy Act) / CPRA (California Privacy Rights Act)**: Protects California residents. Focuses on opt-out rights rather than opt-in. Requires a visible "Do Not Sell My Personal Information" link on the homepage. - **COPPA (Children's Online Privacy Protection Act)**: US federal law protecting children under 13. Requires a clear privacy policy and verifiable parental consent before collecting personal data from a child. - **DPA (Data Protection Act)**: The UK's principal data privacy framework. Works alongside the UK GDPR and grants users the right to access and delete personal data. - Other countries have their own laws: Nigeria has the NDPA (Nigeria Data Protection Act) and India has the DPDP Act (Digital Personal Data Protection Act). - **Compliance steps for global teams**: - Build location-aware consent banners that display the appropriate cookie notice based on the user's geographic location. - Implement an automated DSAR (Data Subject Access Request) system to find, export, or permanently erase a user's data upon their legal request. - Practice data minimization and use age-verification gates to prevent unauthorized tracking of minors. - Designate a Data Protection Officer (DPO) to oversee the privacy architecture and act as the main contact for regulatory authorities.