nitrokite/deployment/README.md

# NitroKite Deployment

Simple deployment workflow using Ansible + Gitea Actions + Blue-Green deployment.

## Overview

```
┌──────────────┐
│ 1. Ansible   │  Run once to set up infrastructure
│   Setup      │  (Gitea, PostgreSQL, Runners, Nginx, Scripts)
└──────┬───────┘
       │
┌──────▼───────┐
│ 2. Push Code │  Push to Gitea triggers automatic build
│   to Gitea   │  - Backend (Go API)
│              │  - Frontend (React App)
└──────┬───────┘
       │
┌──────▼───────┐
│ 3. Promote   │  Manual promotion to production
│   to Prod    │  (sudo promote.sh <commit-sha>)
└──────────────┘
```

## Architecture

### Deployed Applications

1. **Marketing Website** - Static Astro site
   - Domain: https://nitrokite.com
   - Directory: `/var/www/html/`
   - Deployment: Manual via Ansible `synchronize` task

2. **Frontend App** - React TypeScript dashboard
   - Domain: https://app.nitrokite.com
   - Directory: `/var/www/app/`
   - Deployment: Automated via Gitea Actions → staging → promotion
   - Features: TanStack Query, Router, Forms, shadcn/ui

3. **Backend API** - Go Gin server
   - Domain: https://api.nitrokite.com
   - Directory: `/opt/api-artifacts/current/`
   - Deployment: Blue-Green with systemd service
   - Database: PostgreSQL with separate user/database

4. **Gitea** - Git hosting + CI/CD
   - Domain: https://git.nitrokite.com
   - Runners: 5 parallel runners in Docker containers

## One-Time Setup

### 1. Prepare Ansible Inventory

```bash
cd deployment/ansible
cp hosts.ini.example hosts.ini
```

Edit `hosts.ini`:
```ini
[git]
your-vps-ip ansible_user=ubuntu ansible_ssh_private_key_file=~/.ssh/your-key.pem ansible_python_interpreter=/usr/bin/python3
```

### 2. Set Database Passwords

```bash
export POSTGRES_ADMIN_PASSWORD="your-strong-password"
export API_SERVER_DB_PASSWORD="your-api-db-password"
```

### 3. Run Ansible Playbook

```bash
cd deployment/ansible

# Test connection
ansible -i hosts.ini git -m ping

# Deploy everything
ansible-playbook -i hosts.ini site.yml
```

This sets up:
- Gitea server (accessible at `http://your-vps-ip`)
- PostgreSQL databases (gitea + apiserver)
- 5 Gitea Actions runners
- Nginx reverse proxy
- API server deployment infrastructure at `/opt/api-artifacts/`
- Systemd service for the API server

### 4. Create Gitea Repository

1. Visit `http://your-vps-ip`
2. Create admin account (first-time setup)
3. Create repository: `membersyncpro`

### 5. Add Git Remote

```bash
git remote add production http://your-vps-ip/your-username/nitrokite.git
```

## Daily Deployment Workflow

### Deploy New Version

```bash
# 1. Push code (triggers both backend & frontend builds)
git push production main

# 2. Wait for build (check Gitea → Actions)
#    - Builds Go backend binary
#    - Builds React frontend bundle
#    - Stages backend to inactive slot (blue/green)
#    - Stages frontend to /opt/frontend-artifacts/staging/<commit-sha>

# 3. SSH to VPS and promote (pass the commit SHA)
ssh ubuntu@your-vps-ip
sudo /opt/api-artifacts/scripts/promote.sh <commit-sha>
# Type 'yes' to confirm

# This will:
# - Switch backend to new version (systemd reload)
# - Deploy frontend to /var/www/app/ (nginx serves it)
# - Run health checks
# - Backup previous versions

# 4. Verify
curl https://api.nitrokite.com/hello
curl https://api.nitrokite.com/version
curl https://app.nitrokite.com  # Frontend should load
```

**Note:** Get the commit SHA from the Gitea Actions output or via `git rev-parse HEAD`

### Rollback

If something goes wrong after promotion:

```bash
ssh ubuntu@your-vps-ip
sudo /opt/api-artifacts/scripts/rollback.sh
```

Instantly reverts the backend to the previous deployment. For frontend, restore from backup:
```bash
# List backups
ls -l /opt/frontend-artifacts/backups/

# Restore specific backup
sudo rm -rf /var/www/app/*
sudo cp -r /opt/frontend-artifacts/backups/<timestamp>/* /var/www/app/
sudo chown -R www-data:www-data /var/www/app
```

## VPS Directory Structure

```
/opt/api-artifacts/
├── builds/
│   ├── abc123.../              # Previous backend build
│   └── def456.../              # Current backend build
├── blue -> builds/abc123/      # Previous deployment
├── green -> builds/def456/     # Current deployment
├── current -> green            # Active (systemd uses this)
├── scripts/
│   ├── deploy.sh               # Auto-called by CI
│   ├── promote.sh              # Manual promotion (now handles frontend)
│   ├── rollback.sh             # Emergency rollback
│   ├── health-check.sh         # Health validation
│   └── migrate.sh              # Database migrations
└── config/
    └── .env.production         # Environment variables

/opt/frontend-artifacts/
├── staging/
│   ├── abc123.../              # Built frontend for commit abc123
│   └── def456.../              # Built frontend for commit def456
└── backups/
    └── 20251225_120000/        # Backup of previous frontend

/var/www/
├── html/                       # Marketing website (nitrokite.com)
└── app/                        # Frontend app (app.nitrokite.com)
```

## Common Tasks

### View Service Logs

```bash
# Backend API logs
journalctl -u api-server -f

# Nginx logs (frontend & marketing)
tail -f /var/log/nginx/access.log
tail -f /var/log/nginx/error.log

# Last 100 lines
journalctl -u api-server -n 100
```

### Check Active Deployment

```bash
# Backend: See which slot is active
readlink /opt/api-artifacts/current

# Backend: Full build path
readlink -f /opt/api-artifacts/current

# Frontend: Check current files
ls -lh /var/www/app/

# Frontend: Check staging builds available
ls -l /opt/frontend-artifacts/staging/
```

### Check Runners

```bash
# All runners
systemctl status gitea-runner-{1..5}

# Individual runner logs
journalctl -u gitea-runner-1 -f
```

### List All Builds

```bash
ls -lh /opt/api-artifacts/builds/
```

### Clean Old Builds

```bash
# Keep only last 10 builds
cd /opt/api-artifacts/builds
ls -t | tail -n +11 | xargs -I {} sudo rm -rf {}
```

## Configuration

### Environment Variables (on VPS)

Edit `/opt/api-artifacts/config/.env.production`:

```bash
DATABASE_URL=postgresql://apiserver:PASSWORD@localhost:5432/apiserver?sslmode=disable
AUTO_MIGRATE=false
PORT=8080
GIN_MODE=release
```

### Ansible Variables

Edit `deployment/ansible/site.yml`:

```yaml
vars:
  create_separate_apiserver_db: true  # Separate vs shared DB
  runner_count: 5                     # Number of runners
  gitea_version: "1.23.3"            # Gitea version
```

## Blue-Green Deployment Flow

1. **Current state:** `current -> blue` (active)
2. **New push:** Build creates `builds/new456/`
3. **Staging:** `green -> builds/new456/` (inactive)
4. **Migrations:** Run on green slot
5. **Promotion:** `current -> green` (switch!)
6. **Restart:** Systemd loads from current
7. **Rollback available:** Blue slot still has previous build

## Troubleshooting

### Build Not Triggering

```bash
# Check runners
systemctl status gitea-runner-1

# Runner logs
journalctl -u gitea-runner-1 -f
```

### Deployment Fails

```bash
# Check deploy script logs
journalctl -u api-server -n 50

# Verify permissions
ls -la /opt/api-artifacts/

# Check database connection
cat /opt/api-artifacts/config/.env.production
```

### Service Won't Start

```bash
# Service logs
journalctl -u api-server -n 50

# Verify binary exists
ls -la /opt/api-artifacts/current/api-server

# Check symlinks
readlink /opt/api-artifacts/current
readlink /opt/api-artifacts/blue
readlink /opt/api-artifacts/green
```

## Files Reference

- `ansible/site.yml` - Complete infrastructure setup playbook
- `ansible/hosts.ini` - Your VPS inventory
- `scripts/` - Deployment scripts (copied to VPS by Ansible)
- `systemd/api-server.service` - Service configuration
- `.gitea/workflows/build-deploy.yml` - CI/CD workflow

## Ansible Playbook Details

The `site.yml` playbook handles:

1. **Package installation:** Git, Nginx, PostgreSQL, Docker
2. **Gitea setup:** Binary download, database, systemd service
3. **Database provisioning:** Admin user, API server user, databases
4. **Runner deployment:** 5 self-hosted runners with auto-registration
5. **API infrastructure:** User, directories, scripts, systemd service
6. **Nginx configuration:** Reverse proxy for Gitea
7. **Firewall:** UFW with SSH, HTTP, PostgreSQL ports

Run it multiple times safely (idempotent).

## Security Notes

- Change default passwords (use environment variables)
- Use SSH keys for VPS access
- Keep `.env.production` file permissions at 600
- Review UFW firewall rules
- Consider adding SSL/TLS with Let's Encrypt
- PostgreSQL is exposed on 0.0.0.0 - restrict if not needed externally

## Next Steps

After basic deployment:
1. Add SSL/TLS certificates (Let's Encrypt)
2. Set up monitoring (Prometheus/Grafana)
3. Configure alerting for deployment failures
4. Automate PostgreSQL backups
5. Add cron job to clean old builds