# ZyloVPN — System Architecture

> Standalone commercial VPN SaaS platform.
> This document is the contract that Phases 1–7 implement against. Read it before changing schema, services, or the node protocol.

---

## 0. Scope and independence

ZyloVPN is an independent product. It shares no database, schema, authentication, API, UI, code, or infrastructure with any other project. Every table, route, guard, and service described here is created new for ZyloVPN.

---

## 1. High-level architecture

```
                         ┌──────────────────────────────┐
   Browser / Mobile app  │  Public site · Customer portal │
                         │  Admin panel · REST API v1     │
                         └───────────────┬────────────────┘
                                         │  HTTPS
                         ┌───────────────▼────────────────┐
                         │      ZyloVPN Control Plane      │
                         │  Laravel 13 · PHP 8.3 · MySQL 8 │
                         │  Redis · Queue workers · Cron   │
                         └───────────────┬────────────────┘
                                         │  Node API (token auth, HTTPS)
                                         │  ── nodes PULL desired state ──
             ┌───────────────────────────┼───────────────────────────┐
             │                           │                           │
      ┌──────▼──────┐            ┌───────▼─────┐             ┌───────▼─────┐
      │ VPN node US │            │ VPN node UK │             │ VPN node DE │
      │ zylo-agent  │            │ zylo-agent  │             │ zylo-agent  │
      │ WireGuard   │            │ WireGuard   │             │ WireGuard   │
      └──────┬──────┘            └───────┬─────┘             └───────┬─────┘
             └───────────────────────────┴───────────────────────────┘
                                   Customer VPN traffic
                                   (never touches control plane)
```

**Hard rule:** the control plane carries zero customer VPN traffic. It owns identity, subscriptions, devices, peers, IP allocation, server selection, config generation, health state, usage aggregation, billing, and administration. Data-plane forwarding is exclusively the nodes' job.

### 1.1 The node protocol is pull-based (declarative desired state)

This is the most consequential architectural decision here, so it is stated up front.

The control plane does **not** open connections to VPN nodes. Instead each node runs `zylo-agent`, which authenticates with its own bearer token and asks the control plane: *"what should my peer set be?"* The control plane replies with the full desired peer list plus a `state_hash`. If the hash matches what the agent already applied, it does nothing; otherwise it reconciles via `wg set` and acknowledges.

Why pull rather than push:

| | Push (control plane → node) | **Pull (chosen)** |
|---|---|---|
| Inbound ports on VPN node | Needs an HTTPS listener + public cert | **Only the WireGuard UDP port** |
| Nodes behind NAT / cloud firewalls | Breaks | Works |
| Node reboots, disk rollback, manual `wg` edits | Silent drift, permanent | **Self-heals on next sync** |
| Failure mode | Lost command = peer never created | Retry converges |
| Blast radius if node is compromised | Node holds an inbound admin surface | Node holds one scoped token, revocable |

The cost is latency: peer changes land within one poll interval (default 5s, configurable). That is acceptable because config *download* does not depend on node sync — the config file only needs the customer keypair, the assigned IP, and the server's public key and endpoint, all of which the control plane already knows. The peer is live on the node before a human can open their VPN client.

Idempotent full-state reconciliation also means adding a new node is not a special case: it syncs from empty to correct on first poll. This is what makes "add servers without changing core architecture" (spec §38) actually true.

A push mode is available behind config for low-latency single-tenant deployments, but pull is the default and the documented path.

---

## 2. Technology stack

| Layer | Choice | Notes |
|---|---|---|
| Language | PHP 8.3 | Verified: 8.3.30 present locally |
| Framework | Laravel 13 | Verified: 13.24.0 installed |
| Interactive UI | Livewire 3 + Alpine.js | Admin tables, dashboards, ticket threads |
| Styling | Tailwind CSS 4 + Vite | ZyloVPN design tokens, no Bootstrap |
| Database | MySQL 8.4 | FKs, `SKIP LOCKED` for IP allocation |
| Cache / queue / sessions | Redis (prod), database (local) | See §2.1 |
| API auth | Laravel Sanctum | Bearer tokens for future mobile/desktop apps |
| Node auth | Custom token guard | Hashed per-node tokens, separate guard |
| Crypto | ext-sodium | X25519 keypairs natively; verified working |
| VPN | WireGuard | Ubuntu 24.04 nodes |
| Provisioning | Ansible | Node + app-server playbooks |
| Web server | Nginx + PHP-FPM | Apache locally via Laragon |

### 2.1 Environment finding — Redis is not available locally

`ext-redis` is **not installed** in this Laragon PHP build and no Redis server is listening on 6379. Rather than let the app fail on first boot, the configuration is environment-driven:

- **Local:** `CACHE_STORE=database`, `QUEUE_CONNECTION=database`
- **Production:** `CACHE_STORE=redis`, `QUEUE_CONNECTION=redis`

No application code references Redis directly — everything goes through Laravel's cache/queue abstractions, so the swap is a `.env` change with no code impact. Rate limiting, locks, and queues all work on both drivers. Installing Redis locally is documented but optional.

**Sessions are the exception and stay on the database driver in every environment.** The spec requires customer-facing session management — "these are your active sessions, sign this one out". That requires session storage you can *enumerate by user* and *delete individual rows from*. Redis session storage is a flat keyspace of opaque session ids with no index by user, so the feature is not implementable on it without maintaining a second user→session index and keeping it consistent through expiry, which is strictly worse than just querying a table. Session volume is a few rows per customer, so the database cost is negligible.

### 2.2 Sessions record which guard authenticated them

Laravel's database session handler stores `user_id` taken from the *default* guard. With two guards over two tables that is actively wrong in both directions: customer #1 and admin #1 both write `user_id = 1`, so a customer's "my sessions" list would include — and be able to revoke — the administrator's session; and admin sessions store `user_id = null`, because the default `web` guard has nobody logged in, so staff sessions are invisible to session management entirely.

`App\Support\Session\GuardAwareSessionHandler` resolves the actually-authenticated session guard and writes its name to a `guard` column. **Every session query must filter on `guard`** — `user_id` alone is not an identity in this schema.

### 2.3 WireGuard keys are generated in PHP, not by shelling out

Verified locally: `sodium_crypto_scalarmult_base()` on a clamped 32-byte secret produces exactly the `wg genkey | wg pubkey` result (44-char base64). So the control plane generates customer keypairs natively — no `wg` binary, no `proc_open`, no Windows/Linux divergence, and key material never crosses a process boundary. Clamping is `k[0] &= 248; k[31] = (k[31] & 127) | 64`.

---

## 3. Application surfaces

Five distinct surfaces, each with its own routes, middleware, layout, and guard.

| Surface | Route file | Prefix | Guard |
|---|---|---|---|
| Public marketing site | `routes/web.php` | `/` | none |
| Customer portal | `routes/customer.php` | `/app` | `web` (users) |
| Admin panel | `routes/admin.php` | `/admin` | `admin` (admin_users) |
| Public REST API | `routes/api.php` | `/api/v1` | `sanctum` |
| Node agent API | `routes/node.php` | `/api/v1/node` | `node` |

### 3.1 Admins are a separate table, not a flag on `users`

The spec's suggested table list implies one `users` table with roles attached. ZyloVPN instead uses a separate `admin_users` table and a separate `admin` guard.

Rationale: a commercial VPN panel holds server credentials, every customer's peer data, and billing records. If administrators live in the customer table, then every customer-facing surface — public registration, password reset, email-change, OAuth, the mobile API — becomes a potential privilege-escalation path into that data. One missed `is_admin` mass-assignment guard is a full compromise. Separate tables make escalation structurally impossible rather than a matter of remembering a `$fillable` entry: there is no column to flip, and customer login simply cannot produce an admin session.

The cost is a second auth stack (~1 extra migration and guard config). That is a good trade. This is a deliberate deviation from the suggested table list and is called out here so it is not mistaken for an oversight.

---

## 4. Database schema

### 4.1 Table list (35)

**Identity & access**
| Table | Purpose |
|---|---|
| `users` | Customers. Soft-deletes, 2FA columns, status enum |
| `admin_users` | Staff. Separate guard, soft-deletes, 2FA |
| `roles` | Super Admin / Admin / Support |
| `permissions` | Granular capability strings |
| `permission_role` | Pivot |
| `admin_user_role` | Pivot |
| `password_reset_tokens` | Laravel standard |
| `sessions` | DB-backed sessions (enables remote session revocation) |
| `personal_access_tokens` | Sanctum, for mobile/desktop apps |
| `audit_logs` | Sensitive admin actions, searchable |

**Billing**
| Table | Purpose |
|---|---|
| `plans` | Configurable; price, interval, device limit, data cap, speed cap |
| `plan_vpn_location` | Which locations a plan may use |
| `subscriptions` | trial / active / past_due / cancelled / expired |
| `payments` | pending / paid / failed / refunded / cancelled |
| `invoices` | Issued documents |
| `invoice_items` | Line items |
| `transactions` | Gateway-level ledger entries |

**VPN**
| Table | Purpose |
|---|---|
| `vpn_locations` | Country, code, city, flag, display name, active, sort order |
| `vpn_servers` | Node record + agent token hash + capacity + status |
| `vpn_server_metrics` | Time-series heartbeat samples |
| `vpn_ip_pools` | One or more subnets per server |
| `vpn_ip_addresses` | Materialised allocatable addresses |
| `vpn_devices` | Customer devices |
| `vpn_peers` | WireGuard peer records |
| `vpn_connections` | Session records derived from handshakes |
| `vpn_usage` | Daily aggregated rx/tx per peer and per user |

**Support & operations**
| Table | Purpose |
|---|---|
| `support_categories` | Ticket categories |
| `support_tickets` | open / pending / answered / closed, priority |
| `support_messages` | Customer and admin replies |
| `support_attachments` | Private-disk files |
| `notifications` | Laravel notifications |
| `email_templates` | Admin-editable templates |
| `settings` | Typed key/value application settings |

**Framework**: `jobs`, `job_batches`, `failed_jobs`, `cache`, `cache_locks`

### 4.2 ERD — core relationships

```mermaid
erDiagram
    users ||--o{ subscriptions : has
    users ||--o{ vpn_devices : owns
    users ||--o{ support_tickets : opens
    users ||--o{ payments : makes

    plans ||--o{ subscriptions : defines
    plans }o--o{ vpn_locations : "grants access to"

    subscriptions ||--o{ invoices : bills
    invoices ||--o{ invoice_items : contains
    invoices ||--o{ payments : settled_by
    payments ||--o{ transactions : records

    vpn_locations ||--o{ vpn_servers : hosts
    vpn_servers ||--o{ vpn_ip_pools : owns
    vpn_ip_pools ||--o{ vpn_ip_addresses : contains
    vpn_servers ||--o{ vpn_server_metrics : reports
    vpn_servers ||--o{ vpn_peers : serves

    vpn_devices ||--o| vpn_peers : "has active"
    vpn_peers ||--o| vpn_ip_addresses : "holds lease"
    vpn_peers ||--o{ vpn_connections : produces
    vpn_peers ||--o{ vpn_usage : accumulates

    admin_users }o--o{ roles : assigned
    roles }o--o{ permissions : grants
    admin_users ||--o{ audit_logs : performs

    support_tickets ||--o{ support_messages : contains
    support_messages ||--o{ support_attachments : carries
```

### 4.3 Key schema decisions

**IP allocation is a materialised pool with row locking.** On server provisioning, each `vpn_ip_pool` subnet is expanded into `vpn_ip_addresses` rows (network, broadcast, and gateway addresses excluded). Allocation is:

```sql
SELECT id FROM vpn_ip_addresses
 WHERE vpn_ip_pool_id = ? AND status = 'available'
 ORDER BY id LIMIT 1
   FOR UPDATE SKIP LOCKED;
```

inside a transaction, then flipped to `assigned`. `SKIP LOCKED` means concurrent peer creation never blocks or collides, and a partial unique index on `(vpn_server_id, ip_address)` for non-released rows makes double-assignment impossible even under a bug. Pools larger than a configurable `/22` (1022 hosts) are rejected at validation time to keep materialisation bounded; that is far above realistic per-node peer counts.

The alternative — computing the next free IP arithmetically — avoids the rows but makes concurrent allocation a race that is genuinely hard to close correctly. Materialising trades a bounded number of rows for a correctness guarantee the database enforces.

**Private keys are encrypted at rest and optional.** `vpn_peers.private_key` uses Laravel's `encrypted` cast (AES-256-GCM under `APP_KEY`). A setting `zylovpn.wireguard.retain_private_keys` controls whether the key is kept at all; when off, the key is returned once at generation time and then discarded, and "download config again" becomes "regenerate keypair". Private keys are never logged, never serialised into API resources, and never exposed in the admin panel — admin sees only the public key.

**Soft deletes** on `users`, `admin_users`, `plans`, `vpn_locations`, `vpn_servers`, `vpn_devices`, `support_tickets` — records referenced by financial or audit history must not vanish. `vpn_peers` are never deleted; they transition to `revoked` and retain history.

**Indexes** on every FK, plus `users.email`, `vpn_peers.public_key` (unique), `vpn_peers.status`, `vpn_servers.status`, `vpn_servers.last_heartbeat_at`, `vpn_ip_addresses(vpn_ip_pool_id, status)`, `subscriptions(user_id, status)`, `subscriptions.ends_at`, `vpn_usage(vpn_peer_id, date)`, `audit_logs(admin_user_id, created_at)`.

---

## 5. Folder structure

```
app/
├── Console/Commands/          # zylo:sync-servers, zylo:expire-subscriptions, …
├── DTOs/                      # PeerCredentials, WireGuardConfig, ServerHealth, …
├── Enums/                     # ServerStatus, PeerStatus, SubscriptionStatus, PaymentStatus, …
├── Events/                    # PeerCreated, PeerRevoked, ServerWentOffline, …
├── Exceptions/                # NoServerAvailable, IpPoolExhausted, DeviceLimitReached, …
├── Http/
│   ├── Controllers/
│   │   ├── Admin/             # Server, Location, Plan, Customer, Settings, Audit
│   │   ├── Api/V1/            # Auth, User, Device, Location, Server, Vpn, Subscription
│   │   ├── Customer/          # Dashboard, Device, Config, Subscription, Ticket
│   │   ├── Node/              # State, Heartbeat, Usage
│   │   └── Site/              # Home, Pricing, Locations, Legal, Contact
│   ├── Middleware/            # EnsureActiveSubscription, NodeAuth, AdminPermission
│   ├── Requests/              # Admin/ Customer/ Api/ Node/
│   └── Resources/V1/          # API resources — never raw models
├── Jobs/                      # AggregateUsage, MarkServersOffline, ExpireSubscriptions
├── Livewire/{Admin,Customer}/ # Interactive tables, dashboards, ticket threads
├── Models/
├── Notifications/
├── Policies/                  # DevicePolicy, SubscriptionPolicy, TicketPolicy
├── Providers/
├── Services/
│   ├── Billing/               # GatewayManager, PaymentGateway contract, drivers
│   ├── Settings/              # SettingsRepository (cached, typed)
│   ├── Support/
│   └── Vpn/                   # ← the core domain
│       ├── WireGuardKeyGenerator.php
│       ├── VpnServerSelector.php
│       ├── IpAllocator.php
│       ├── PeerManager.php
│       ├── ConfigGenerator.php
│       └── NodeStateBuilder.php
└── Support/

resources/views/
├── components/                # Blade components: card, table, modal, badge, …
├── layouts/                   # site, customer, admin
├── site/  customer/  admin/
└── emails/

ansible/                       # node + app-server provisioning
├── playbooks/  roles/  inventory/
docs/                          # this file, DEPLOYMENT.md, API.md, NODE-AGENT.md
tests/{Feature,Unit}/
```

Controllers stay thin: validate via Form Request → call a service → return a Resource or view. All VPN logic lives in `Services/Vpn`, per spec §12.

---

## 6. VPN domain services

| Service | Responsibility |
|---|---|
| `WireGuardKeyGenerator` | Clamped X25519 keypairs via ext-sodium; also preshared keys |
| `IpAllocator` | `allocate(pool)` / `release(address)`; transactional, `SKIP LOCKED` |
| `VpnServerSelector` | Picks the best healthy server for a location + plan |
| `PeerManager` | Orchestrates create / disable / revoke / rotate across the above |
| `ConfigGenerator` | Renders `.conf` text and QR payload from a peer |
| `NodeStateBuilder` | Builds the desired peer set + `state_hash` for a node |

### 6.1 Server selection

`VpnServerSelector::select(VpnLocation $location, Plan $plan): VpnServer`

Filters to servers that are `online`, not disabled/maintenance/provisioning, below capacity, with a fresh heartbeat and a non-exhausted IP pool, in a location the plan permits. Ranks the survivors by load ratio (`current_peers / capacity`), then heartbeat freshness, then lowest reported latency. Throws `NoServerAvailableException` when nothing qualifies — never silently assigns to an offline, full, disabled, or maintenance node.

### 6.2 Peer lifecycle

**Create** — inside one transaction: check subscription is active → check device limit for the plan → select server → allocate IP → generate keypair → persist peer → commit. Then bump the node's `state_version`, so the agent picks the change up on its next poll. Config is available to the customer immediately.

**Revoke** — mark peer `revoked`, release the IP lease, bump `state_version`. The node removes the peer on next sync and the config stops working. Revocation is recorded in the audit log.

Every step is transactional: a failure at IP allocation or key generation rolls back cleanly rather than leaking a half-created peer or a leased-but-unused address.

---

## 7. Node agent protocol (`/api/v1/node`)

Auth: `Authorization: Bearer <node-token>`. Tokens are stored hashed on `vpn_servers`, are per-node, and are rotatable from the admin panel. No SSH access is ever reachable from the customer-facing application.

| Method | Endpoint | Purpose |
|---|---|---|
| `POST` | `/heartbeat` | CPU, RAM, disk, network, WireGuard status, uptime, peer count, applied `state_hash`. Returns current desired `state_hash`. |
| `GET` | `/state` | Full desired peer set + `state_hash` + interface settings. |
| `POST` | `/state/ack` | Agent confirms the hash it has applied. |
| `POST` | `/usage` | Per-peer `rx_bytes`, `tx_bytes`, `last_handshake_at`. |

The heartbeat response carrying the desired hash is what makes this cheap: the agent polls a tiny endpoint and only fetches the full state when the hash differs.

**Offline detection:** a scheduled job flips servers whose `last_heartbeat_at` exceeds a configurable timeout to `offline`, which immediately removes them from selection. Recovery is automatic on the next heartbeat.

---

## 8. Public API (`/api/v1`)

Versioned, Sanctum-authenticated, API-Resource-serialised, correct status codes. Built now so the Android/iOS/Windows/macOS clients in Phase 7 need no backend changes.

```
POST   /auth/register  /auth/login  /auth/logout  /auth/refresh
       /auth/forgot-password  /auth/reset-password
GET    /user                        PATCH /user
GET    /subscription                GET   /subscription/plans
GET    /locations                   GET   /servers?location=
GET    /devices                     POST  /devices
DELETE /devices/{id}                POST  /devices/{id}/regenerate
GET    /vpn/config/{device}         GET   /vpn/status
```

Rate limits are per-route-group; auth endpoints are throttled hardest.

---

## 9. Security posture

HTTPS everywhere · CSRF on all web forms · Blade escaping by default · Eloquent parameter binding · per-group rate limiting · bcrypt/argon2 hashing · TOTP 2FA for customers and admins · Sanctum for API · policies on every customer-owned resource · Form Request validation on every write · signed temporary URLs for config downloads · audit logging of sensitive admin actions · all secrets in `.env` (never committed) · private keys never in logs, frontends, or admin views · UFW on nodes limited to WireGuard UDP + SSH from the management range · least-privilege DB user.

Error handling: customers see friendly messages; stack traces, credentials, secrets, keys, and internal paths never reach a response. Technical detail goes to logs only.

### 9.1 Privacy

Data is separated into required account data, billing records, operational data, security logs, and optional analytics — each with its own admin-configurable retention period. No browsing activity, DNS queries, or destination IPs are collected; the schema has nowhere to put them. Aggregate byte counters and connection timestamps *are* retained, because they are required for capacity planning and abuse handling.

Accordingly the Privacy Policy will describe exactly this, and the marketing site will **not** claim "zero logs" — connection metadata is retained, so that claim would be false. It will use accurate language about what is and is not collected.

---

## 9.2 App-store billing is a separate abstraction from payment gateways

Publishing the clients on the App Store and Google Play changes the billing model, not just the payment method.

`PaymentGateway` has `charge()` — we ask a provider to take money. Store subscriptions invert this: **the store charges the customer on its own schedule and then tells us.** There is no server-side call that initiates a payment. Forcing stores into `PaymentGateway` would mean a `charge()` that cannot charge — an abstraction containing a lie.

So there are two sibling contracts:

| | `PaymentGateway` | `StoreSubscriptionProvider` |
|---|---|---|
| Direction | We initiate a charge | The store notifies us |
| Source of truth | Our database | **The store** |
| Renewal driven by | Our scheduler | The store |
| Cancellation | We can cancel | Only the customer can, in their store account |
| Refund | We issue it | Apple/Google issue it; we react |

**The store is authoritative.** `StoreSubscriptionManager` never decides entitlement; it verifies and mirrors. When our row and the store disagree, the store wins.

Consequences that shaped the code:

- **Idempotency is mandatory, not defensive.** Store notifications are delivered at least once, often out of order, and clients call "restore purchases" freely. A unique index on `(source, store_transaction_id)` makes a duplicate entitlement impossible.
- **Revocation beats expiry.** A refunded purchase still has a future expiry date. Honouring it would make refund-and-keep-using free.
- **Google reissues `purchaseToken`** on upgrade, chaining the old one via `linkedPurchaseToken`. Matching only the new token orphans the original subscription.
- **Our renewal scheduler must not touch store subscriptions**, or the customer is billed twice.
- **Product identifiers are reverse-DNS**, so the product→plan map is fetched then indexed. `config('…products.com.zylovpn.pro.monthly')` reads every dot as nesting and always misses.
- **We cannot cancel a store subscription.** `Subscription::managementUrl()` returns where the customer must go; an in-app cancel button that silently does nothing turns a refund request into a chargeback.

Client-supplied receipts are untrusted claims. `verifyPurchase()` must call the store's API — treating a receipt as proof is the most common way IAP entitlement is pirated.

---

## 10. Development roadmap

| Phase | Contents | Status |
|---|---|---|
| **1** | Laravel foundation, schema, RBAC, customer + admin auth, 2FA, design system, base layouts, public site | **Complete** — 39 tests, 145 assertions passing |
| **2** | Locations, servers, node registration, agent protocol, heartbeat, key generation, IP allocation, peer management, config generation, admin screens, `zylo-agent` + provisioning | **Complete** — 123 panel tests / 425 assertions, 22 agent tests. See docs/DEPLOYMENT.md |
| **3** | Devices, location picker, config download + QR, revoke/regenerate, subscription enforcement | **Complete** — 139 tests, 481 assertions |
| **4** | Plans, subscriptions, gateway abstraction, payments, invoices, expiry jobs, admin plan CRUD | **Complete** — 166 tests, 553 assertions. Customer checkout awaits a real payment provider |
| **5** | Metrics, bandwidth, connection history, offline alerts, admin charts | Planned |
| **6** | Tickets, notifications, editable email templates | Planned |
| **7** | Android / iOS / Windows / macOS clients against the v1 API | API **built and tested** (13 endpoints); the clients themselves are future work |

Each phase ends with an error check, schema and relationship review, security review, and UI/responsive review before the next begins.

---

## 11. Testing strategy

Feature tests for registration, login, authorization boundaries, plan creation, subscription lifecycle, payment state transitions, device limits, API authentication, and admin permission enforcement. Unit and integration tests for the VPN core — peer creation, peer revocation, concurrent IP allocation, pool exhaustion, server selection ranking, refusal to select offline/full/maintenance/disabled servers, and expired-subscription enforcement. The VPN services are where correctness bugs become security bugs, so they carry the heaviest coverage.