Migration Playbook

How to migrate a project into AIPM from an existing flat-file or MemoryAndSoul (MAS) source, and how to roll back a migration that went wrong.

This playbook satisfies transition requirements REQ-TRN-001 (import path), REQ-TRN-002 (rollback), and is itself REQ-TRN-003 (this page).

Prerequisites

• Python 3.10+ with the source tree on PYTHONPATH (export PYTHONPATH=src), or the installed aipm package (pip install .).
• PostgreSQL reachable via a DSN. Set AIPM_DATABASE_URL to your target AIPM database, e.g. postgresql://USER:PASS@HOST:PORT/aipm_dev.
• A target project already exists in AIPM. Create one first with python -m aipm.db create-project --name "My Project" and note the returned proj_… id.
• For backups/rollback: the PostgreSQL client binaries pg_dump, pg_restore, and psql on your PATH.
• An export of your source data as a flat file — a CSV or a JSON file. If you are coming from a MAS-only project, export its requirements to a JSON file first (see Migrating from MAS).

Input formats

The importer reads two flat-file shapes. Field names are matched case-insensitively, and several common aliases are accepted, so most existing exports import without reshaping.

A status, if given, must be one of AIPM's backlog statuses: candidate, selected, promoted, deferred, rejected. Any other value is rejected before anything is written.

CSV example

external_id,title,description,priority,status
REQ-001,Login with SSO,Users sign in via corporate SSO,5,candidate
REQ-002,Audit export,Export the audit log to JSONL,3,selected

JSON example

A top-level array, an {"requirements": [...]} / {"items": [...]} envelope, or a single object are all accepted:

{
  "requirements": [
    {"id": "REQ-001", "name": "Login with SSO", "priority": 5},
    {"id": "REQ-002", "name": "Audit export", "status": "selected"}
  ]
}

Migrating INTO AIPM (step by step)

1. Back up the target database first. Even an import into an empty project should start from a known-good, restorable snapshot:

   export PYTHONPATH=src
   export AIPM_DATABASE_URL="postgresql://USER:PASS@HOST:PORT/aipm_dev"
   python scripts/aipm_backup.py --dir backups
   # writes backups/aipm_<db>_<UTC-stamp>.dump

2. Dry-check your file. Confirm the format and that every row has an external id and a title. Malformed rows fail loudly with a clear message rather than importing half a file.
3. Run the import. Point it at your target project and file:

   python -m aipm.flatfile_import \
       --project proj_xxxxxxxxxxxxxxxx \
       --file reqs.csv \
       --source-system flatfile

   The importer prints a JSON summary: how many rows were created, unchanged, updated, or skipped_existing.
4. Re-run safely if needed. The import is idempotent: re-running the same file creates no duplicates — rows are keyed on their stable external id. Unchanged rows are reported as unchanged.
5. Updating changed rows. By default a row whose content changed since a previous import is left untouched and reported as skipped_existing (the importer never silently clobbers). To apply changes, re-run with --update:

   python -m aipm.flatfile_import --project proj_… --file reqs.csv --update

6. Verify the result. Confirm the new backlog items appear and that the audit chain is intact:

   python -m aipm.db verify-audit   # expect ok=True

Migrating from MAS (MemoryAndSoul)

There are two distinct MAS paths — pick the one that matches your situation:

• Live MAS server (the server is up and you want an ongoing mirror): use python -m aipm.mas_sync import --aipm-project proj_… . That pulls requirements from the running MAS over its MCP tools and mirrors them into AIPM, with conflict detection. See the MAS-sync documentation.
• MAS-only / offline export (you have a JSON dump on disk, not a live server): export the MAS requirements to a JSON file, then import that file with the flat-file importer using a distinct provenance tag so the rows are clearly marked as a one-time migration:

  python -m aipm.flatfile_import \
      --project proj_… \
      --file mas_export.json \
      --source-system mas-export

Using a different --source-system value (e.g. mas-export vs. flatfile) keeps the two migration lineages independent: re-importing one never disturbs rows imported from the other.

Rolling back a migration

A rollback is restoring the backup you took before the migration. The restore is recoverable and integrity-verified: AIPM re-checks the tamper-evident audit chain after the restore and fails loudly if it does not verify.

1. Create the rollback target (if restoring elsewhere). A restore overwrites its target. To roll back in place you restore over the same database; to validate a backup safely, restore into a fresh throwaway database first.
2. Restore the pre-migration backup.

   python scripts/aipm_restore.py \
       --target "postgresql://USER:PASS@HOST:PORT/aipm_dev" \
       backups/aipm_aipm_dev_20260615T120000Z.dump

3. The integrity check runs automatically. After loading the dump, the tool runs the audit-chain verification and only reports success if the chain verifies. A corrupt or tampered backup is rejected with a non-zero exit code — it is never silently accepted.
4. Confirm the rolled-back state. The unwanted migration rows are gone and the database matches the backup snapshot.

Reference