Skip to content

hasirciogluhq/migrator

Repository files navigation

Migrator

Go Report Card GoDoc License: MIT

A production-ready PostgreSQL migration tool for Go, built around shadow database testing and per-migration transaction safety.

Why Migrator

Most migration tools apply changes directly to production and hope for the best. Migrator tests every pending migration against a temporary shadow database first, catching syntax errors and schema conflicts before they touch production. Each migration then runs in its own transaction, so a failure rolls back cleanly instead of leaving the schema half-applied.

Features

  • Transaction-safe: every migration runs in a transaction with automatic rollback on failure
  • Shadow database testing: new migrations are validated on a disposable copy of the schema before hitting production
  • Consistency validation: checks that applied migrations still exist on disk
  • PostgreSQL-native: built specifically for Postgres connection semantics
  • Idempotent: safe to re-run
  • Context-aware: respects context.Context timeouts and cancellation
  • Modular internals: tracker, validator, and shadow DB are separate internal packages

Installation

go get github.com/hasirciogluhq/migrator@latest

Releases are fully automated: every push to main runs the test suite, and a semantic version is cut only if all tests pass. See RELEASE.md for the commit message convention.

Quick Start

1. Create a migrations directory and add SQL files. Files run in alphabetical order.

migrations/001_create_users.sql

CREATE TABLE users (
    id SERIAL PRIMARY KEY,
    name VARCHAR(255) NOT NULL,
    email VARCHAR(255) UNIQUE NOT NULL,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
);

CREATE INDEX idx_users_email ON users(email);

2. Run migrations from your application.

package main

import (
    "context"
    "database/sql"
    "log"

    _ "github.com/lib/pq"
    "github.com/hasirciogluhq/migrator"
)

func main() {
    db, err := sql.Open("postgres",
        "postgres://user:password@localhost:5432/mydb?sslmode=disable")
    if err != nil {
        log.Fatal(err)
    }
    defer db.Close()

    m := migrator.New(db)
    if err := m.Migrate(context.Background()); err != nil {
        log.Fatal(err)
    }
}

Configuration

Variable Purpose Default
DATABASE_URL Connection string used for shadow database operations none
MIGRATIONS_PATH Directory containing .sql migration files ./migrations

Custom options can also be passed directly, which takes priority over environment variables:

m := migrator.NewWithOptions(db, migrator.Options{
    MigrationsPath: "./db/migrations",
    DatabaseURL:    "postgres://user:pass@localhost:5432/mydb",
    SkipShadowDB:   false,
})

If no database URL is provided (via options or environment variable), shadow database testing is skipped with a warning.

How It Works

  1. Ensures the _go_migrations tracking table exists
  2. Validates that previously applied migrations still exist in the filesystem
  3. Loads pending .sql files
  4. Creates a temporary shadow database, applies existing and pending migrations to it, then drops it
  5. Applies pending migrations to production, each in its own transaction
  6. Cleans up the shadow database on exit

API Reference

Function Description
New(db *sql.DB) *Migrator Creates a migrator with default options
NewWithOptions(db *sql.DB, opts Options) *Migrator Creates a migrator with custom configuration
Migrate(ctx context.Context) error Runs the full migration process
GetAppliedMigrations(ctx) ([]string, error) Lists applied migration names
GetPendingMigrations(ctx) ([]*validator.MigrationFile, error) Lists migrations not yet applied

Full example: examples/basic/main.go

Best Practices

Naming: use a numeric prefix so ordering is unambiguous, e.g. 001_create_users.sql, 002_create_posts.sql.

Do

  • Keep each migration small and focused
  • Test locally and take a production backup before deploying
  • Version control every migration file
  • Run migrations as part of deployment automation

Don't

  • Modify a migration that has already been applied
  • Delete an applied migration file
  • Skip shadow database testing in production
  • Use unreviewed DROP statements

Testing

11 test scenarios cover the core migration flow, idempotency, transaction rollback, concurrent migrations, and context cancellation.

make test-docker      # spins up Postgres in Docker automatically
make test              # uses an existing Postgres instance
make test-coverage

CI runs the same suite against Postgres 15 on every push and PR, including go vet and race detection.

Troubleshooting

Shadow DB warning / DATABASE_URL not set: pass DatabaseURL in Options or set the environment variable. Shadow testing is skipped, not blocking, if neither is present.

"Migration validation failed: X migrations missing from filesystem": an applied migration file was deleted. Restore it, this check exists to prevent state drift.

Shadow database fails to drop: remove it manually with DROP DATABASE IF EXISTS your_database_gi_mig_shadow_db;

Migration times out: the default timeout is 5 minutes. Split large migrations or move heavy operations outside the migration system.

Architecture

migrator/
├── migrator.go          # Public API
├── internal/
│   ├── tracker/         # Migration tracking and database operations
│   ├── validator/       # File validation and parsing
│   └── shadowdb/         # Shadow database lifecycle
├── migrator_test.go
└── examples/

Contributing

Pull requests are welcome. Fork the repo, branch off main, and open a PR against it.

License

MIT, see LICENSE.

About

Production-ready PostgreSQL migration tool for Go with shadow database testing, transaction safety, and auto rollback.

Topics

Resources

License

Stars

0 stars

Watchers

0 watching

Forks

Contributors