automiz_app_full/development/CLAUDE.md

# TPM CMS — Compact Codebase Reference

> **Purpose:** Load this file at the start of any AI session to understand the full stack without reading source files.
> Last updated: 2026-06-05

---

## Architecture Overview

```
Browser (React)  ←→  tpm_cms_backend (Flask :5000)
                         ├── tpm_cms_workflow (Flask :5001)        # email/node workflow engine
                         ├── tpm_cms_asset_manager (Flask :5002)   # S3 file ops
                         └── tpm_asset_manager_app_web_socket      # Node.js WS bridge
                              ├── WebSocket server (:WS_PORT)      # desktop Flutter app connects here
                              └── HTTP bridge (:HTTP_PORT)         # backend POSTs to /send_message
Flutter Desktop App  ←→  WebSocket bridge
```

All Python services share the same MongoDB database (default database from MONGO_URI). No message queue; services call each other via HTTP.

---

## Services

### 1. `tpm_cms_backend` — main API (Flask + JWT)
- **Entry:** `app.py` (~3466 lines, single file)
- **Port:** 5000 (dev), proxied via Nginx in prod
- **Auth:** JWT (`flask_jwt_extended`), bcrypt passwords, optional Azure SSO
- **Key env vars:** `MONGO_URI`, `JWT_SECRET_KEY`, `FLASK_SECRET_KEY`, `AZURE_REDIRECT_URI`, `WS_BRIDGE_HTTP_URL`, `DESKTOP_UNIQUE_ID`, `ASSET_MANAGER_URL`

#### Collections used
| Collection | Purpose |
|---|---|
| `users` | auth, roles (admin/pm/lead/user) |
| `clients` | client companies |
| `projects` | projects with embedded stages[] |
| `tasks` | user task assignments with time tracking |
| `brief_templates` | project brief + workflow templates |
| `sections` | reusable brief sections |
| `histories` | project history log |
| `dashboard_configs` | per-user dashboard widget configs |
| `planxo_auth_keys` | desktop app auth tokens |
| `azure_config` | Azure SSO config (single doc) |
| `settings` | `{type:'desktop_sync'}`, `{type:'folder_sync'}` |
| `planxo_approvals` | desktop access approval records |

#### API routes (all `@jwt_required()` unless noted)
```
POST   /register                        # public
POST   /login                           # public; GET returns HTML login page
GET    /me
GET    /health                          # public

# Clients
GET/POST         /clients
GET/PUT/DELETE   /clients/<id>

# Projects
GET/POST         /projects
GET/PUT/DELETE   /projects/<id>         # id = MongoDB ObjectId
GET              /projects/<int:id>     # id = numeric project_id
GET              /projects/search
GET/POST         /projects/<id>/history
GET              /projects/<id>/brief_template
POST             /projects/<id>/brief
GET              /projects/<id>/stages
POST             /projects/<id>/stages/<key>       # set status
POST             /projects/<id>/stages/<key>/start
POST             /projects/<id>/stages/<key>/pause
POST             /projects/<id>/stages/<key>/complete
POST             /projects/<id>/stages/<key>/assign
POST             /projects/<id>/desktop-download   # triggers WS bridge

# Brief Templates / Sections
GET/POST         /brief_templates
GET/PUT/DELETE   /brief_templates/<id>
GET              /clients/<id>/brief_templates
GET/POST         /sections
PUT/DELETE       /sections/<id>

# Tasks
GET/POST         /tasks
POST             /tasks/<id>/assign|start|pause|resume|complete|reject

# Users (admin only for write)
GET/POST         /users
PUT              /users/<id>
POST             /users/<id>/activate|deactivate

# Dashboard
GET              /dashboard-stats
GET/POST         /dashboard-configs
GET/PUT/DELETE   /dashboard-configs/<id>
GET              /dashboard-configs/default
POST             /dashboard-data         # widget data with filters
POST             /dashboard-export       # CSV

# Settings
GET/POST         /settings/desktop-sync
GET/POST         /settings/folder-sync

# PlanXO Desktop Auth
GET              /planxo-auth/<auth_key>            # HTML approval page
POST             /planxo/app/auth/register-key      # public
POST             /api/planxo/app/auth/<key>/approve
POST             /planxo/app/auth/<key>/approve     # alias
POST             /api/planxo-auth/<key>/approve     # alias
GET              /planxo/app/auth/check
POST             /planxo/app/auth/revoke
GET              /planxo/app/auth/keys
POST             /planxo/app/auth/validate-key      # public (called by WS server)
GET              /planxo/folders/filtered           # role-filtered S3 folder list

# Azure SSO
GET              /azure-login-url
POST             /azure-callback
POST             /azure-config  (admin)

# Server admin
GET              /server-status
GET              /server-status/logs/<service>
POST             /server-status/restart/<service>   # admin only
GET/POST         /desktop-bridge/health|clients     # proxy to WS bridge

# Files (GridFS — legacy)
POST             /files/upload
GET              /files/<id>

# Color (prepress)
POST             /compute                           # Pantone→ECG via transicc
```

#### Role system
- `role_required('admin')` — admin only
- `role_required('pm','admin','lead','user')` — most write ops
- `jwt_required()` — any authenticated user
- `get_authenticated_user()` — reads token from URL param `?token=` OR `Authorization: Bearer`

---

### 2. `tpm_cms_asset_manager` — S3 file service (Flask :5002)
- **Entry:** `asset_manager_app.py`
- **No auth** — internal calls only from backend
- **Key env vars:** `S3_BUCKET`, `AWS_REGION`, `MONGO_URI`

#### S3 path convention
```
clients/{client_name}/{numeric_project_id}/       ← if client exists
projects/{numeric_project_id}/                    ← no client
```
Folders created by uploading `{path}/.keep` placeholder.

#### Routes
```
POST   /clients/<id>/create-folder
POST   /projects/<id>/create-folder
POST   /folders/create              # generic, validates root prefix
POST   /folders/batch-create        # {base_folder, subfolders:[]}
GET    /assets                      # ?folder=&q=&owner_id=&include_folders=
DELETE /assets/<id>
POST   /assets/presign-upload       # returns S3 presigned URL
POST   /assets/confirm-upload       # save asset record to MongoDB
GET    /assets/<id>/download        # presigned GET URL
POST   /assets/batch-delete
GET    /assets/folder-contents      # flat list with S3 metadata
POST   /zip-download                # zip multiple keys
```

Assets stored in MongoDB `assets` collection with `s3_key`, `folder`, `filename`, `content_type`, `size`, `owner_id`, `created_at`.

---

### 3. `tpm_cms_workflow` — workflow engine (Flask :5001)
- **Entry:** `workflow_app.py`
- **No auth** — internal calls only from backend
- **Key env vars:** `MONGO_URI`, `AWS_REGION`, `SENDER_EMAIL`, `SENDER_NAME`
- Uses `ThreadPoolExecutor(max_workers=5)` for async node processing

#### Collections
- `workflows` — `{project_id, nodes:[], edges:[]}`
- `workflow_logs` — execution log per node

#### Node types
- `project` — pass-through start node
- `sendEmail` — sends via AWS SES; resolves `{{variable}}` placeholders
- `router` — conditional branching
- `stage` — maps to project stage key
- `approval` — waits for human approval action

#### Routes
```
POST   /workflow                    # create workflow for project
POST   /workflow/<id>/process       # {node_id, variables:{}} — triggers async execution
GET    /workflow/<id>
GET    /workflow/project/<project_id>
POST   /workflow/<id>/nodes/<node_id>/approve|reject
```

---

### 4. `tpm_asset_manager_app_web_socket` — WS bridge (Node.js)
- **Entry:** `server.js`
- **Ports:** `WS_PORT` (WebSocket), `HTTP_PORT` (Express HTTP)
- **Auth:** `?apiKey=<key>` on WebSocket connect + `unique_id` per client
- Stores connected clients in `Map<unique_id, ws>`

#### HTTP endpoints (called by backend)
```
POST   /send_message        # {unique_id, message:{type,data}}  → forward to WS client
POST   /request_download    # {unique_id, folder, projectId}    → trigger download on desktop
GET    /health              # returns {clients:[...unique_ids]}
GET    /clients
```

#### WebSocket message types (server→client)
- `auth_approved` — desktop auth approved; includes user, expires_at, sync_interval
- `auth_revoked` — user logged in from web; desktop must logout
- `download_request` — trigger file download

#### WebSocket message types (client→server)
- `register_auth_key` — desktop registers key before browser approval
- `file_sync_complete`, `file_upload_complete` — status callbacks

---

## Data Models (key fields)

### Project
```json
{
  "_id": ObjectId,
  "project_id": 1000007,          // numeric, auto-increment from last+1
  "projectName": "string",        // indexed unique per clientId
  "clientId": ObjectId|null,
  "project_name": "string",       // duplicate of projectName (legacy)
  "client_id": "string"|null,     // duplicate of clientId (legacy)
  "brand": "", "product": "", "variant": "",
  "brief": {},
  "template_id": "string",
  "stages": [
    {
      "key": "stage_key",
      "title": "Stage Title",
      "status": "pending|in_progress|paused|completed",
      "time": {"total_seconds": 0},
      "start_time": 1234567890,    // unix timestamp, present only when in_progress
      "assigned_to": "user_id",
      "history": [{"action":"start","timestamp":0,"user_id":""}]
    }
  ],
  "version": 1,
  "created_at": ISODate,
  "created_by": "user_id",
  "project_update_history": [{"user_id":"","user_name":"","changed_fields":[],"timestamp":ms}]
}
```

### User
```json
{
  "_id": ObjectId,
  "username": "string",
  "password": "bcrypt_hash",
  "role": "admin|pm|lead|user",
  "email": "", "name": "", "fullName": "",  // name+fullName kept in sync
  "mobile": "",
  "isActive": true, "active": true,         // both kept in sync
  "last_active": ISODate                     // updated on /me call
}
```

### PlanXO Auth Key
```json
{
  "auth_key": "string",
  "unique_id": "desktop_client_id",
  "status": "pending|approved|revoked",
  "is_active": true,
  "user_id": ObjectId,
  "role": "string",
  "expires_at": ISODate,           // admin: +100 years; others: +8 hours
  "approved_at": ISODate
}
```

---

## Code Review Findings

### 🔴 Critical Issues

**1. Password logged in plaintext (backend/app.py:300)**
```python
print(f"Login attempt for user: {username}, password: {password}")  # REMOVE THIS
```

**2. `get_current_user_id()` referenced but never defined (app.py:3402)**
`/planxo/folders/filtered` calls `get_current_user_id()` — this will raise `NameError` at runtime. Replace with `get_jwt_identity()` and add `@jwt_required()`.

**3. `safe_text()` referenced but never defined (app.py:3142)**
`/projects/<id>/desktop-download` calls `safe_text(resp)` which doesn't exist — will crash.

**4. `pymongo` imported twice; `pymongo.errors.DuplicateKeyError` used without top-level import**
`app.py:7` imports `secrets` twice. `pymongo` is used at line 785 but only imported via `from pymongo import ...` — `pymongo.errors` won't resolve. Add `import pymongo` at top.

**5. `/compute` endpoint has no auth**
The Pantone/ECG color conversion endpoint is publicly accessible. Add `@jwt_required()`.

**6. Asset manager has no authentication**
All routes in `asset_manager_app.py` are unauthenticated. If the port is ever exposed, anyone can read/write S3. Add at minimum an internal shared secret header check.

**7. Unreachable code in `start_stage` (app.py:1232)**
```python
return jsonify({'success': True, 'stages': stages, 'workflow_result': workflow_result})
return jsonify({'success': True, 'stages': stages})  # ← dead code, never reached
```

### 🟡 Important Issues

**8. `list_projects` loads ALL projects with no pagination**
`projects_col.find().sort(...)` returns everything. With hundreds of projects this will be slow. Add `limit`/`skip` or cursor-based pagination.

**9. Duplicate field names in MongoDB documents**
`project_name`/`projectName`, `client_id`/`clientId`, `name`/`fullName`, `active`/`isActive`, `created_at`/`createdAt` are all stored twice. This doubles write load and creates inconsistency risk. Pick one canonical name per field.

**10. Workflow service URL hardcoded**
`app.py:878,885,894` uses `http://localhost:5001` directly instead of an env var like `WORKFLOW_URL`. Inconsistent with other services.

**11. Stage parsing handles both JSON string and list**
Multiple endpoints do:
```python
if isinstance(project['stages'], str):
    stages = json.loads(project['stages'])
else:
    stages = project['stages']
```
Stages should always be stored as a list. This defensive code masks a schema inconsistency.

**12. `before_request` logs every endpoint — noisy in production**
```python
@app.before_request
def log_request():
    print(f">>> Endpoint called: {request.endpoint}")
```
Replace with proper logging or remove for production.

**13. DEBUG print statements left in approval flows (app.py:2394–2512)**
Many `print(f"DEBUG: ...")` lines in the PlanXO approval endpoints should be removed or converted to `logging.debug()`.

**14. `get_azure_config()` is a function, not a route**
`app.py:2937` defines `get_azure_config()` without a `@app.route` decorator, so it's never reachable.

**15. WS bridge API key is in env with a weak default**
`server.js:22`: `process.env.API_KEY || "aasdf345scwe"` — the fallback default is weak. Fail hard if `API_KEY` not set in production.

**16. `delete_asset` in asset manager uses string `_id`**
`assets.find_one({"_id": asset_id})` — MongoDB stores `_id` as ObjectId, not string. This will never find anything. Use `ObjectId(asset_id)`.

### 🟢 Suggestions

**17. Extract stage manipulation into a helper**
`start_stage`, `pause_stage`, `complete_stage`, and `assign_stage` share ~40 lines of identical stage-parsing boilerplate. Extract to `_get_stages(project)` and `_save_stages(project_id, stages)`.

**18. `get_project_folder_path` is duplicated logic**
The folder path logic in `app.py:3082` and `asset_manager_app.py` should live in one place (asset manager) and backend should call it.

**19. JWT token expiry is 1 hour; comment says 12h/1d/30d**
`JWT_ACCESS_TOKEN_EXPIRES = timedelta(hours=1)` — the comment is stale. Clarify intended expiry and consider refresh tokens.

**20. No index on `planxo_auth_keys.auth_key`**
This field is queried on every desktop connect. Add a unique index.

---

## Environment Variables Reference

### Backend (`.env` / `.env.prod`)
```
MONGO_URI=
JWT_SECRET_KEY=
FLASK_SECRET_KEY=
FLASK_ENV=production
AZURE_REDIRECT_URI=https://cms.techpremedia.com/azure-callback
AZURE_CLIENT_SECRET=
WS_BRIDGE_HTTP_URL=http://localhost:5003
DESKTOP_UNIQUE_ID=
ASSET_MANAGER_URL=http://localhost:5002
```

### Asset Manager
```
MONGO_URI=
S3_BUCKET=
AWS_REGION=
```

### Workflow
```
MONGO_URI=
AWS_REGION=
SENDER_EMAIL=
SENDER_NAME=
FLASK_SECRET_KEY=
```

### WebSocket Bridge
```
WS_PORT=
HTTP_PORT=
API_KEY=
ASSET_MANAGER_URL=http://localhost:5002
CMS_BACKEND_URL=https://cms.techpremedia.com
NODE_ENV=production
```

---

## Key Patterns

- **ObjectId serialization:** every route converts `_id` via `str(doc['_id'])` before jsonify
- **`now()`:** all services define their own `now()` returning `datetime.utcnow()`
- **S3 "folders":** implemented as empty `.keep` placeholder objects
- **Project numeric ID:** auto-incremented by querying `projects_col.find_one(sort=[("project_id",-1)])` — not atomic, could duplicate under concurrent creates
- **Desktop auth flow:** desktop → register-key → browser opens `/planxo-auth/<key>` → user approves → WS bridge notified → desktop receives `auth_approved`