spidy-look-at-it/SYSTEM_DESIGN.md

# Contract Analysis Platform - System Design

## High-Level Architecture

```
┌─────────────────────────────────────────────────────────────────────────┐
│                           INGESTION LAYER                               │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐ │
│  │ Email Inbox  │  │ Web Upload   │  │ Bulk Import  │  │ Integrations│ │
│  │ (Graph API)  │  │              │  │ (CSV/ZIP)    │  │ (ERP/CRM)   │ │
│  └──────────────┘  └──────────────┘  └──────────────┘  └─────────────┘ │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         PROCESSING LAYER                                │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────────┐   │
│  │ Document Parser  │  │ Contract Classifier││ Chunking Engine      │   │
│  │ (PDF/DOCX/Image) │  │ (Type Detection)   ││ (Clause Extraction)  │   │
│  └──────────────────┘  └──────────────────┘  └──────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                          ANALYSIS LAYER                                 │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────────┐   │
│  │ Key Term Analysis│  │ Playbook Engine  │  │ Checklist Engine     │   │
│  │ (AI Extraction)  │  │ (Risk/Deviation) │  │ (Compliance Check)   │   │
│  └──────────────────┘  └──────────────────┘  └──────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                         OUTPUT LAYER                                    │
│  ┌──────────────────┐  ┌──────────────────┐  ┌──────────────────────┐   │
│  │ Dashboard Views  │  │ Reports/Exports  │  │ Notifications        │   │
│  │ (Type-specific)  │  │ (PDF/Excel)      │  │ (Email/Webhook)      │   │
│  └──────────────────┘  └──────────────────┘  └──────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────┘
```

---

## Core Entities

### 1. Contract Type Hierarchy

```
ContractType
├── MSA (Master Service Agreement)
├── NDA (Non-Disclosure Agreement)  
├── SOW (Statement of Work)
├── SLA (Service Level Agreement)
└── COI (Certificate of Insurance)
    ├── Commercial (Retail, Office)
    ├── Residential (Apartment, Condo)
    ├── Contractor (General, Sub)
    └── Vendor
```

### 2. Entity Relationship Diagram

```
Organization (1)
    │
    ├── Team (N) ─────────────────┐
    │       │                     │
    │       └── User (N)          │
    │                             │
    ├── Contract (N) ◄────────────┤
    │       │                     │
    │       ├── ContractType      │
    │       │       └── SubType   │
    │       │                     │
    │       ├── AnalysisResult (N)│
    │       │       ├── KeyTerms  │
    │       │       ├── Risks     │
    │       │       └── Scores    │
    │       │                     │
    │       └── Obligation (N)    │
    │               ├── DueDate   │
    │               └── Status    │
    │                             │
    ├── Playbook (N) ─────────────┤
    │       └── PlaybookClause (N)│
    │                             │
    ├── Checklist (N) ────────────┤
    │       └── ChecklistItem (N) │
    │                             │
    ├── KeyTermTemplate (N)       │
    │       └── by ContractType   │
    │                             │
    └── Counterparty (N)
            └── Contract (N)
```

---

## Entity Definitions

### Organization
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| name | Organization name |
| settings | JSON config (integrations, email templates, reminder settings) |
| subscription_tier | Plan level (affects limits & features) |

### Contract
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| organization_id | Tenant isolation |
| type | MSA/NDA/SOW/COI/SLA |
| subtype | Type-specific subtype (e.g., Commercial COI) |
| status | Draft/Active/Expired/Terminated |
| counterparty_id | Link to counterparty |
| effective_date | Start date |
| expiry_date | End date |
| source | Upload/Email/Integration |
| metadata | Type-specific extracted data (JSON) |

### Counterparty
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| organization_id | Tenant isolation |
| name | Company/Individual name |
| type | Vendor/Tenant/Partner/Client |
| contact_info | Primary contact details |
| compliance_status | Overall compliance state |

### KeyTermTemplate
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| contract_type | Which contract type this applies to |
| subtype | Optional subtype filter |
| key | Term name (e.g., "Liability Limit") |
| question | AI extraction prompt |
| category | Grouping (Coverage, Dates, Parties) |
| is_required | Mandatory for compliance |
| validation_rules | JSON rules for checking values |

### Playbook
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| organization_id | Tenant isolation |
| name | Playbook name |
| contract_type | Applicable contract type |
| is_default | System default vs custom |

### PlaybookClause
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| playbook_id | Parent playbook |
| key | Clause/term name |
| standard_language | Expected/acceptable language |
| risk_guidance | What to flag as risky |
| negotiation_tips | Suggested alternatives |

### Checklist (COI-specific)
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| organization_id | Tenant isolation |
| name | Checklist name |
| subtype | COI subtype (Commercial/Residential/Contractor) |
| items | Required coverage items with thresholds |

### Obligation
| Field | Purpose |
|-------|---------|
| id | Primary identifier |
| contract_id | Parent contract |
| type | Renewal/Payment/Deliverable/Review |
| due_date | When due |
| status | Pending/Completed/Overdue |
| assigned_to | User responsible |
| reminder_settings | When to notify |

---

## Contract Type Specifics

### COI Subtypes & Data Points

| Subtype | Key Data Points | Dashboard Focus |
|---------|-----------------|-----------------|
| **Commercial** | GL Limits, Property, Workers Comp, Umbrella, Additional Insured, Lease Requirements | Tenant compliance grid, Expiry calendar |
| **Residential** | Renters Insurance, Personal Property, Liability, Loss of Use | Unit-level tracking, Renewal alerts |
| **Contractor** | GL, Workers Comp, Auto, Umbrella, Waiver of Subrogation, Per-Project endorsements | Project compliance, Active job tracking |
| **Vendor** | GL, Professional Liability, Cyber, Product Liability | Vendor scorecards, Risk tiering |

### MSA/NDA/SOW Key Analysis Points

| Type | Key Terms | Risk Areas |
|------|-----------|------------|
| **MSA** | Payment terms, IP ownership, Liability caps, Termination, Governing law | Unlimited liability, Auto-renewal, Broad indemnity |
| **NDA** | Definition of confidential, Duration, Exclusions, Return/Destroy | Perpetual terms, Broad definitions, Missing carve-outs |
| **SOW** | Scope, Deliverables, Timeline, Acceptance criteria, Change process | Scope creep, Vague acceptance, Missing milestones |

---

## Dashboard Views

### 1. Portfolio Overview (All Users)
- Contract count by type and status
- Upcoming expirations (30/60/90 days)
- Compliance score trend
- Recent activity feed

### 2. COI Dashboard (Insurance-focused)
- Compliance grid: Counterparty × Coverage status
- Expiry heatmap calendar
- Non-compliant items requiring action
- Automated reminder status

### 3. Contract Analysis View
- Side-by-side: Document ↔ Extracted data
- Risk flags with severity
- Deviation from playbook
- Suggested redlines

### 4. Counterparty View
- All contracts with counterparty
- Compliance history
- Communication log
- Document timeline

---

## Analysis Workflows

### 1. COI Analysis Flow
```
Upload/Email → Type Detection → Subtype Classification
    → Checklist Selection (by subtype) 
    → Key Term Extraction → Compliance Check 
    → Generate Report → Notify Counterparty (if non-compliant)
```

### 2. MSA/SOW/NDA Analysis Flow
```
Upload → Type Detection → Clause Chunking 
    → Playbook Selection → Risk Identification 
    → Deviation Scoring → Redline Suggestions
```

---

## Data Model Summary

```sql
-- Core hierarchy
organizations (1) → teams (N) → users (N)

-- Contract management  
organizations (1) → counterparties (N)
organizations (1) → contracts (N) → contract_type + subtype
contracts (1) → analysis_results (N)
contracts (1) → obligations (N)
contracts (1) → activity_log (N)

-- Analysis configuration
organizations (1) → playbooks (N) → playbook_clauses (N)
organizations (1) → checklists (N) → checklist_items (N)
contract_types (1) → key_term_templates (N)

-- Results
contracts (1) → key_term_values (N)  -- extracted values
contracts (1) → risk_flags (N)       -- identified issues
contracts (1) → checklist_results (N) -- compliance status
```

---