diff --git a/backend/migrations/README.md b/backend/migrations/README.md new file mode 100644 index 0000000..f2af76d --- /dev/null +++ b/backend/migrations/README.md @@ -0,0 +1,303 @@ +# Database Migrations + +This project uses Sequelize CLI for database migrations. Migrations provide version control for your database schema. + +## Quick Reference + +```bash +# Run pending migrations +npm run db:migrate + +# Undo last migration +npm run db:migrate:undo + +# Undo all migrations +npm run db:migrate:undo:all + +# Check migration status +npm run db:migrate:status + +# Test all migrations (up, down, up) +npm run test:migrations +``` + +## Available Commands + +### `npm run db:migrate` + +**Purpose:** Runs all pending migrations that haven't been applied yet. + +- Checks the `SequelizeMeta` table to see which migrations have already run +- Executes the `up` function in each pending migration file in order +- Records each successful migration in the `SequelizeMeta` table +- **When to use:** Deploy new schema changes to your database + +### `npm run db:migrate:undo` + +**Purpose:** Rolls back the most recent migration. + +- Executes the `down` function of the last applied migration +- Removes that migration's entry from `SequelizeMeta` +- **When to use:** Quickly revert the last change if something went wrong + +### `npm run db:migrate:undo:all` + +**Purpose:** Rolls back ALL migrations, returning to an empty database. + +- Executes the `down` function of every migration in reverse order +- Clears the `SequelizeMeta` table +- **When to use:** Reset development database to start fresh, or in testing + +### `npm run db:migrate:status` + +**Purpose:** Shows the status of all migrations. + +- Lists which migrations have been executed (with timestamps) +- Shows which migrations are pending +- **When to use:** Check what's been applied before deploying or debugging + +### `npm run db:create` + +**Purpose:** Creates the database specified in your config. + +- Reads `DB_NAME` from environment variables +- Creates a new PostgreSQL database with that name +- **When to use:** Initial setup on a new environment + +### `npm run test:migrations` + +**Purpose:** Automated testing of migrations (to be implemented in Phase 4). + +- Will create a fresh test database +- Run all migrations up, then down, then up again +- Verify migrations are reversible and idempotent +- **When to use:** In CI/CD pipeline before merging migration changes + +## Environment Configuration + +All commands use the environment specified by `NODE_ENV` (dev, test, qa, prod) and load the corresponding `.env` file automatically. + +Examples: + +```bash +# Run migrations in development +NODE_ENV=dev npm run db:migrate + +# Check status in QA environment +NODE_ENV=qa npm run db:migrate:status + +# Run migrations in production +NODE_ENV=prod npm run db:migrate +``` + +## Creating a New Migration + +```bash +# Generate a new migration file +npx sequelize-cli migration:generate --name description-of-change +``` + +This creates a timestamped file in `backend/migrations/`: + +``` +20241125123456-description-of-change.js +``` + +### Migration File Structure + +```javascript +module.exports = { + up: async (queryInterface, Sequelize) => { + // Schema changes to apply + }, + down: async (queryInterface, Sequelize) => { + // How to revert the changes + }, +}; +``` + +## Naming Conventions + +Use descriptive names that indicate the action: + +- `create-users` - Creating a new table +- `add-email-to-users` - Adding a column +- `remove-legacy-field-from-items` - Removing a column +- `add-index-on-users-email` - Adding an index +- `change-status-enum-in-rentals` - Modifying a column + +## Zero-Downtime Patterns + +### Adding a Column (Safe) + +```javascript +up: async (queryInterface, Sequelize) => { + await queryInterface.addColumn("users", "newField", { + type: Sequelize.STRING, + allowNull: true, // Must be nullable or have default + }); +}; +``` + +### Adding a NOT NULL Column + +```javascript +// Step 1: Add as nullable +up: async (queryInterface, Sequelize) => { + await queryInterface.addColumn("users", "newField", { + type: Sequelize.STRING, + allowNull: true, + }); +}; + +// Step 2: Backfill data (separate migration) +up: async (queryInterface, Sequelize) => { + await queryInterface.sequelize.query( + `UPDATE users SET "newField" = 'default_value' WHERE "newField" IS NULL` + ); +}; + +// Step 3: Add NOT NULL constraint (separate migration, after code deployed) +up: async (queryInterface, Sequelize) => { + await queryInterface.changeColumn("users", "newField", { + type: Sequelize.STRING, + allowNull: false, + }); +}; +``` + +### Removing a Column (3-Step Process) + +1. **Deploy 1**: Update code to stop reading/writing the column +2. **Deploy 2**: Run migration to remove column +3. **Deploy 3**: Remove column references from model (cleanup) + +```javascript +// Migration in Deploy 2 +up: async (queryInterface) => { + await queryInterface.removeColumn('users', 'oldField'); +}, +down: async (queryInterface, Sequelize) => { + await queryInterface.addColumn('users', 'oldField', { + type: Sequelize.STRING, + allowNull: true + }); +} +``` + +### Renaming a Column (3-Step Process) + +```javascript +// Deploy 1: Add new column, copy data +up: async (queryInterface, Sequelize) => { + await queryInterface.addColumn("users", "newName", { + type: Sequelize.STRING, + }); + await queryInterface.sequelize.query( + 'UPDATE users SET "newName" = "oldName"' + ); +}; + +// Deploy 2: Update code to use newName, keep oldName as fallback +// (no migration) + +// Deploy 3: Remove old column +up: async (queryInterface) => { + await queryInterface.removeColumn("users", "oldName"); +}; +``` + +### Creating Indexes (Use CONCURRENTLY) + +```javascript +up: async (queryInterface) => { + await queryInterface.addIndex("users", ["email"], { + unique: true, + concurrently: true, // Prevents table locking + name: "users_email_unique", + }); +}; +``` + +## Testing Checklist + +Before committing a migration: + +- [ ] Migration has both `up` and `down` functions +- [ ] Tested locally: `npm run db:migrate` +- [ ] Tested rollback: `npm run db:migrate:undo` +- [ ] Tested re-apply: `npm run db:migrate` +- [ ] Full test: `npm run test:migrations` +- [ ] Application starts and works correctly +- [ ] No data loss in `down` migration (where possible) + +## Deployment Checklist + +Before deploying to production: + +- [ ] Backup production database +- [ ] Test migration on copy of production data +- [ ] Review migration for safety (no destructive operations) +- [ ] Schedule during low-traffic window +- [ ] Have rollback plan ready +- [ ] Monitor logs after deployment + +## Rollback Procedures + +### Undo Last Migration + +```bash +npm run db:migrate:undo +``` + +### Undo Multiple Migrations + +```bash +# Undo last 3 migrations +npx sequelize-cli db:migrate:undo --step 3 +``` + +### Undo to Specific Migration + +```bash +npx sequelize-cli db:migrate:undo:all --to 20241124000005-create-rentals.js +``` + +## Common Issues + +### "Migration file not found" + +Ensure the migration filename in `SequelizeMeta` matches the file on disk. + +### "Column already exists" + +The migration may have partially run. Check the schema and either: + +- Manually fix the schema +- Mark migration as complete: `INSERT INTO "SequelizeMeta" VALUES ('filename.js')` + +### "Cannot drop column - dependent objects" + +Drop dependent indexes/constraints first: + +```javascript +await queryInterface.removeIndex("users", "index_name"); +await queryInterface.removeColumn("users", "column_name"); +``` + +### Foreign Key Constraint Failures + +Ensure data integrity before adding constraints: + +```javascript +// Clean orphaned records first +await queryInterface.sequelize.query( + 'DELETE FROM rentals WHERE "itemId" NOT IN (SELECT id FROM items)' +); +// Then add constraint +await queryInterface.addConstraint("rentals", { + fields: ["itemId"], + type: "foreign key", + references: { table: "items", field: "id" }, +}); +```