Operations

This section covers everything required to keep the SONAN DIGITAL CRM running reliably in production. It is written for the engineering and operations team — the people responsible for deploying, monitoring, maintaining, and responding to incidents on the platform.

Audience

Pages in This Section

Operations Manual

The authoritative system overview. Covers what the CRM is, who uses it, the full infrastructure table, versioning and branching strategy, deployment frequency, support responsibilities, and escalation contacts. Start here if you are new to the platform.

Environment Configuration

Complete listing of all environment variables used by the application — names, descriptions, where to find each value, which are public vs. server-only, and how to set up a local .env.local for development.

Secrets Management

How application secrets are stored, who has access, rotation schedules for every secret, the emergency rotation procedure for compromised credentials, and recommendations for upgrading to a secrets manager.

Monitoring & Logging

How to use Sentry for error tracking, how to access Vercel function logs, recommended UptimeRobot checks, alert escalation paths, cron job monitoring, and the weekly monitoring review checklist.

Cron Jobs

Documentation for all scheduled background jobs: the invoice overdue check, recurring invoice generation, and the appointments cron. Covers authentication, Vercel config, local testing, and what to do when a job fails.

Incident Response

The incident classification table (P1–P4), the end-to-end response process from detection through post-mortem, escalation paths, and ready-to-use communication templates for P1 client outages.

Maintenance Schedule

Checklists for monthly, quarterly, and annual maintenance tasks — dependency updates, security audits, RLS policy reviews, secret rotation, backup testing, and more.

Client Onboarding

The 8-step admin procedure for onboarding a new client: creating their CRM record, setting up portal access, assigning an account manager, and briefing the client. Includes a quick-reference checklist and common issue fixes.

Employee Onboarding

The step-by-step admin procedure for onboarding a new team member — sending the invite, configuring their role and MFA, assigning projects, and briefing on time logging. Also covers the offboarding checklist.

Operations Manual

This document is the authoritative reference for operating the SONAN DIGITAL CRM in production. It covers system purpose, infrastructure, versioning, deployment, support responsibilities, and escalation contacts.

1. System Overview

The SONAN DIGITAL CRM is a multi-tenant web application built to manage the full client lifecycle for a digital agency. It serves two classes of users:

• Admin / Staff users — the agency's internal team. They use the CRM to manage leads, clients, projects, tasks, time logs, proposals, invoices, contracts, and support tickets.
• Client users — agency clients who log in to a self-service portal to view proposals, sign contracts, pay invoices, track project progress, and submit support requests.

The application is a single Next.js 15 codebase deployed on Vercel. All data lives in a Supabase PostgreSQL database, with row-level security (RLS) enforcing tenant isolation. Authentication is handled by Supabase Auth. File uploads (contracts, attachments) go to Supabase Storage. Transactional emails are sent via Resend. Payments and subscriptions are handled by Stripe. Errors are tracked in Sentry.

The system is designed to be always-on — clients interact with the portal and expect real-time access to invoices and project status. Any unplanned downtime directly impacts client trust.

2. Infrastructure

3. Versioning

The CRM follows semantic versioning (MAJOR.MINOR.PATCH):

Current version: v1.0.0

Version is tracked in package.json. On each release, tag the commit in Git:

git tag -a v1.2.0 -m "Release v1.2.0: recurring invoices, wiki module"
git push origin v1.2.0

4. Branching Strategy

Rules:

• Never commit directly to main for non-emergency changes.
• All changes go through a pull request: feature/* → dev → main.
• dev is used for QA and client-facing preview URLs before production release.
• Emergency hotfixes can be pushed directly to main with a post-merge PR for review — document in the incident report.

5. Deployment

Deployment is triggered automatically by a push to the relevant branch:

Deployment frequency: As needed. There is no fixed release schedule — features and fixes are deployed when ready and tested on dev.

Deployment checklist before merging to main:

• [ ] All new API routes include export const runtime = 'edge'
• [ ] Database migrations have been applied to the Supabase production project
• [ ] Any new environment variables have been added to Vercel (Production + Preview)
• [ ] The change has been tested on the dev preview environment
• [ ] No new TypeScript errors (npx tsc --noEmit)
• [ ] Sentry is not reporting new errors from the preview environment

Rollback: Vercel retains all previous deployments. To roll back, go to the Vercel dashboard → Deployments → select the last known-good deployment → "Promote to Production". This is the fastest recovery path for bad deployments.

6. Support Responsibilities

7. Escalation Contacts

Environment Configuration

This page documents all environment variables used by the SONAN DIGITAL CRM, how they are organized across environments, where to find their values, and how to set up a local development environment.

1. Vercel Environments

The application runs in three Vercel environments. Each has its own set of environment variable values:

To manage environment variables, go to the Vercel Dashboard → Project → Settings → Environment Variables.

2. Environment Variable Reference

Supabase

Email — Resend

Payments — Stripe

Background Jobs

Error Tracking — Sentry

3. Public vs. Server-Only Variables

4. Where to Find Each Value

5. Startup Validation

The application performs implicit validation at build time via TypeScript and at runtime via the Supabase and Stripe SDK initializers. If a required variable is missing:

• NEXT_PUBLIC_SUPABASE_URL or NEXT_PUBLIC_SUPABASE_ANON_KEY missing → Supabase client will throw during initialization. The application will fail to render.
• SUPABASE_SERVICE_ROLE_KEY missing → createServiceClient() will throw on first admin API call.
• STRIPE_SECRET_KEY missing → Stripe SDK initialization will throw; all payment routes will return 500.
• CRON_SECRET missing → Cron endpoints will return 401 for all requests.
• SENTRY_AUTH_TOKEN missing → The build will succeed but source maps will not be uploaded. Stack traces in Sentry will show minified code.

To verify all variables are loaded in a Vercel deployment:

1. Go to Vercel Dashboard → Project → Deployments → select the deployment.
2. Click View Build Logs — any missing NEXT_PUBLIC_ variable will appear as undefined in the build output.
3. Click View Function Logs and trigger an authenticated API request — check for undefined or initialization errors in the log output.

6. Local Development Setup

Create a .env.local file in the project root. This file is git-ignored and should never be committed.

# .env.local — Local development only. Never commit this file.

# Supabase
NEXT_PUBLIC_SUPABASE_URL=https://your-project-id.supabase.co
NEXT_PUBLIC_SUPABASE_ANON_KEY=eyJhbGci...
SUPABASE_SERVICE_ROLE_KEY=eyJhbGci...

# Resend (use a test API key or the real key for local testing)
RESEND_API_KEY=re_...

# Stripe — use TEST keys for local development
STRIPE_SECRET_KEY=sk_test_...
STRIPE_WEBHOOK_SECRET=whsec_...
NEXT_PUBLIC_STRIPE_PUBLISHABLE_KEY=pk_test_...

# Cron protection
CRON_SECRET=your-local-random-secret-at-least-32-chars

# Sentry (optional for local dev — can be omitted)
NEXT_PUBLIC_SENTRY_DSN=https://...@sentry.io/...
SENTRY_AUTH_TOKEN=sntrys_...

Starting the dev server:

npm install
npm run dev
# App available at http://localhost:3000

stripe listen --forward-to localhost:3000/api/stripe/webhook

Secrets Management

This page documents how application secrets are stored, who has access to them, the rotation schedule for each secret, the procedure for emergency rotation, and recommendations for upgrading the secrets infrastructure.

1. Where Secrets Are Stored

All secrets are stored as Vercel Environment Variables, which are:

• Encrypted at rest using Vercel's infrastructure-level encryption.
• Scoped per environment (Production, Preview, Development).
• Never written to disk or committed to version control.
• Accessible only to authenticated Vercel team members with the appropriate role.

Rules that must never be broken:

• No secret is ever written into source code.
• No .env file containing real secrets is ever committed to git. .env.local is in .gitignore — verify this before any git add ..
• Secrets are never logged, printed, or included in error messages.
• Secrets are never passed as URL query parameters.

2. Secrets Inventory

3. Rotation Schedule

SUPABASE_SERVICE_ROLE_KEY — Every 6 months

This key bypasses all RLS policies and has full database access. It is the highest-risk secret in the system.

1. Go to Supabase Dashboard → Project Settings → API.
2. Scroll to Project API keys — click the service role key's Reveal button.
3. Click Reset (Supabase will generate a new key; the old key becomes invalid immediately).
4. Copy the new key and update it in Vercel (see rotation steps below).
5. Trigger a Vercel redeploy to load the new value.

STRIPE_SECRET_KEY — Every 12 months or on suspected compromise

1. Go to Stripe Dashboard → Developers → API keys.
2. Click Roll key next to the secret key. Stripe lets you set a 24-hour delay before the old key is invalidated — use this window to update and test before the old key expires.
3. Update STRIPE_SECRET_KEY in Vercel.
4. Redeploy and verify Stripe API calls succeed.

RESEND_API_KEY — Every 12 months or on suspected compromise

1. Go to Resend Dashboard → API Keys.
2. Create a new API key.
3. Update RESEND_API_KEY in Vercel.
4. Redeploy and send a test email to confirm delivery.
5. Delete the old API key from the Resend dashboard.

CRON_SECRET — Every 6 months

1. Generate a new secret locally: bash openssl rand -hex 32
2. Update CRON_SECRET in Vercel (Production and Preview).
3. Update the secret in vercel.json under the cron authorization header if it is stored there (it should not be — the cron routes read it from the environment).
4. Redeploy and verify a cron manual trigger works.

STRIPE_WEBHOOK_SECRET — On endpoint URL change

The webhook signing secret is tied to the registered endpoint URL in Stripe. If the production URL changes:

1. Go to Stripe Dashboard → Developers → Webhooks.
2. Add the new endpoint URL.
3. Copy the new signing secret.
4. Update STRIPE_WEBHOOK_SECRET in Vercel.
5. Delete the old webhook endpoint from Stripe.

SENTRY_AUTH_TOKEN — Every 12 months

1. Go to Sentry → Settings → Auth Tokens.
2. Create a new token with org:ci scope.
3. Update SENTRY_AUTH_TOKEN in Vercel.
4. Delete the old token.

4. Emergency Rotation Procedure

Use this procedure when a secret is known or suspected to be compromised — leaked in logs, committed to git, exposed in an error message, or accessed by an unauthorized party.

Treat every suspected compromise as a confirmed compromise. Do not wait for confirmation.

Step-by-Step

1. Revoke the old key immediately in the provider dashboard — do not wait.
2. Supabase service role key: Supabase Dashboard → API → Reset
3. Stripe key: Stripe Dashboard → Developers → API keys → Roll key (set 0 delay)
4. Resend key: Resend Dashboard → API Keys → Delete
5. Cron secret: Skip to step 2 (self-generated, just replace it)

6. Sentry token: Sentry → Settings → Auth Tokens → Revoke

7. Generate the new key/secret in the provider dashboard.

8. Update in Vercel — go to Vercel Dashboard → Project → Settings → Environment Variables:

9. Update the value for Production, Preview, and Development scopes as applicable.

10. Trigger a redeploy — in Vercel Dashboard → Deployments → click Redeploy on the latest production deployment. The new environment variable is picked up immediately.

11. Verify the application works — check the feature that uses the rotated secret (e.g., send a test email after rotating RESEND_API_KEY, make a test payment after rotating STRIPE_SECRET_KEY).

12. Check for unauthorized activity — review provider dashboards for any suspicious API calls during the window when the key may have been compromised:

13. Supabase: Database → Logs
14. Stripe: Dashboard → Developers → Events

15. Resend: Dashboard → Emails → Sent

16. Document the rotation — update the Last Rotated date in the Secrets Inventory table above, and record the incident in the incident log with: date discovered, suspected exposure window, scope of potential access, and actions taken.

17. Notify the team — send an internal notification to all engineers noting the rotated secret and confirming the old key is no longer valid.

5. Vault and Secrets Manager Options

Current Setup: Vercel Environment Variables

Vercel's built-in environment variables are encrypted at rest and access-controlled by Vercel team roles. This is appropriate for small teams and single-project setups.

Limitations: - No audit log of who accessed a secret's value. - No automatic rotation. - Rotation requires a manual redeploy.

Recommended Upgrade for Growing Teams

If the team expands beyond ~5 engineers or begins handling particularly sensitive client data, consider migrating to a dedicated secrets manager:

For the current scale, Vercel environment variables remain appropriate. Re-evaluate when the team adds a dedicated DevOps role.

6. Access Control

Access to Vercel environment variables is controlled by Vercel team roles:

To review current team access: Vercel Dashboard → Team Settings → Members.