mirror of
https://github.com/complexcaresolutions/cms.c2sgmbh.git
synced 2026-03-17 18:34:13 +00:00
docs: add BullMQ queue system documentation
- Document queue architecture (email/PDF workers) - Add API examples for PDF generation endpoint - Update PM2 commands for queue-worker process - Add programmatic queue usage examples - Update server RAM info (now 8GB) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
ce4962e74b
commit
9e589b895c
1 changed files with 91 additions and 1 deletions
92
CLAUDE.md
92
CLAUDE.md
|
|
@ -19,6 +19,7 @@ Multi-Tenant CMS für 4 Websites unter einer Payload CMS 3.x Instanz:
|
||||||
- **Process Manager:** PM2
|
- **Process Manager:** PM2
|
||||||
- **Package Manager:** pnpm
|
- **Package Manager:** pnpm
|
||||||
- **Cache:** Redis (optional, mit In-Memory-Fallback)
|
- **Cache:** Redis (optional, mit In-Memory-Fallback)
|
||||||
|
- **Job Queue:** BullMQ (Redis-basiert)
|
||||||
|
|
||||||
## Architektur
|
## Architektur
|
||||||
|
|
||||||
|
|
@ -66,6 +67,14 @@ Internet → 37.24.237.181 → Caddy (443) → Payload (3000)
|
||||||
│ │ │ ├── csrf.ts
|
│ │ │ ├── csrf.ts
|
||||||
│ │ │ ├── ip-allowlist.ts
|
│ │ │ ├── ip-allowlist.ts
|
||||||
│ │ │ └── data-masking.ts
|
│ │ │ └── data-masking.ts
|
||||||
|
│ │ ├── queue/ # BullMQ Job Queue
|
||||||
|
│ │ │ ├── queue-service.ts
|
||||||
|
│ │ │ ├── jobs/email-job.ts
|
||||||
|
│ │ │ ├── jobs/pdf-job.ts
|
||||||
|
│ │ │ ├── workers/email-worker.ts
|
||||||
|
│ │ │ └── workers/pdf-worker.ts
|
||||||
|
│ │ ├── pdf/ # PDF-Generierung
|
||||||
|
│ │ │ └── pdf-service.ts
|
||||||
│ │ ├── search.ts # Volltextsuche
|
│ │ ├── search.ts # Volltextsuche
|
||||||
│ │ └── redis.ts # Redis Cache Client
|
│ │ └── redis.ts # Redis Cache Client
|
||||||
│ └── hooks/ # Collection Hooks
|
│ └── hooks/ # Collection Hooks
|
||||||
|
|
@ -75,6 +84,8 @@ Internet → 37.24.237.181 → Caddy (443) → Payload (3000)
|
||||||
├── tests/ # Test Suite
|
├── tests/ # Test Suite
|
||||||
│ ├── unit/security/ # Security Unit Tests
|
│ ├── unit/security/ # Security Unit Tests
|
||||||
│ └── int/ # Integration Tests
|
│ └── int/ # Integration Tests
|
||||||
|
├── scripts/
|
||||||
|
│ └── run-queue-worker.ts # Queue Worker Starter
|
||||||
├── .env # Umgebungsvariablen
|
├── .env # Umgebungsvariablen
|
||||||
├── ecosystem.config.cjs # PM2 Config
|
├── ecosystem.config.cjs # PM2 Config
|
||||||
└── .next/ # Build Output
|
└── .next/ # Build Output
|
||||||
|
|
@ -140,7 +151,9 @@ pnpm payload generate:importmap
|
||||||
# PM2
|
# PM2
|
||||||
pm2 status
|
pm2 status
|
||||||
pm2 logs payload
|
pm2 logs payload
|
||||||
|
pm2 logs queue-worker
|
||||||
pm2 restart payload
|
pm2 restart payload
|
||||||
|
pm2 restart queue-worker
|
||||||
|
|
||||||
# Tests
|
# Tests
|
||||||
pnpm test # Alle Tests
|
pnpm test # Alle Tests
|
||||||
|
|
@ -172,7 +185,7 @@ Der Build ist für speichereffizientes Kompilieren optimiert:
|
||||||
- `package.json`: `--max-old-space-size=2048` (2GB Heap-Limit)
|
- `package.json`: `--max-old-space-size=2048` (2GB Heap-Limit)
|
||||||
- `next.config.mjs`: `experimental.cpus: 1`, `workerThreads: false`
|
- `next.config.mjs`: `experimental.cpus: 1`, `workerThreads: false`
|
||||||
|
|
||||||
**WICHTIG:** Der Server hat nur 4GB RAM ohne Swap. Bei laufendem VS Code Server muss der Build mit reduziertem Memory ausgeführt werden:
|
**WICHTIG:** Der Server hat 8GB RAM ohne Swap. Bei laufendem VS Code Server kann der Build mit reduziertem Memory ausgeführt werden:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
pm2 stop payload # Speicher freigeben
|
pm2 stop payload # Speicher freigeben
|
||||||
|
|
@ -204,6 +217,7 @@ PGPASSWORD=Finden55 psql -h 10.10.181.101 -U payload -d payload_db -c "\dt *_loc
|
||||||
- **E-Mail API:** https://pl.c2sgmbh.de/api/send-email (POST, Auth erforderlich)
|
- **E-Mail API:** https://pl.c2sgmbh.de/api/send-email (POST, Auth erforderlich)
|
||||||
- **Test-E-Mail:** https://pl.c2sgmbh.de/api/test-email (POST, Admin erforderlich)
|
- **Test-E-Mail:** https://pl.c2sgmbh.de/api/test-email (POST, Admin erforderlich)
|
||||||
- **E-Mail Stats:** https://pl.c2sgmbh.de/api/email-logs/stats (GET, Auth erforderlich)
|
- **E-Mail Stats:** https://pl.c2sgmbh.de/api/email-logs/stats (GET, Auth erforderlich)
|
||||||
|
- **PDF-Generierung:** https://pl.c2sgmbh.de/api/generate-pdf (POST/GET, Auth erforderlich)
|
||||||
|
|
||||||
## Security-Features
|
## Security-Features
|
||||||
|
|
||||||
|
|
@ -267,6 +281,82 @@ curl -X POST https://pl.c2sgmbh.de/api/send-email \
|
||||||
- Rate-Limiting: 10 E-Mails/Minute pro User
|
- Rate-Limiting: 10 E-Mails/Minute pro User
|
||||||
- SMTP-Passwort nie in API-Responses
|
- SMTP-Passwort nie in API-Responses
|
||||||
|
|
||||||
|
## BullMQ Job Queue
|
||||||
|
|
||||||
|
Das System verwendet BullMQ für asynchrone Job-Verarbeitung mit Redis als Backend.
|
||||||
|
|
||||||
|
### PM2 Worker
|
||||||
|
|
||||||
|
Der Queue-Worker läuft als separater PM2-Prozess:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# Worker-Status prüfen
|
||||||
|
pm2 status
|
||||||
|
pm2 logs queue-worker
|
||||||
|
|
||||||
|
# Worker neustarten
|
||||||
|
pm2 restart queue-worker
|
||||||
|
```
|
||||||
|
|
||||||
|
### Email Queue
|
||||||
|
|
||||||
|
E-Mails können asynchron über die Queue versendet werden:
|
||||||
|
|
||||||
|
```typescript
|
||||||
|
import { queueEmail } from '@/lib/queue'
|
||||||
|
|
||||||
|
await queueEmail({
|
||||||
|
tenantId: 1,
|
||||||
|
to: 'empfaenger@example.com',
|
||||||
|
subject: 'Betreff',
|
||||||
|
html: '<p>Inhalt</p>',
|
||||||
|
source: 'form-submission'
|
||||||
|
})
|
||||||
|
```
|
||||||
|
|
||||||
|
### PDF Queue
|
||||||
|
|
||||||
|
PDF-Generierung erfolgt über Playwright (HTML/URL zu PDF):
|
||||||
|
|
||||||
|
**API-Endpoint `/api/generate-pdf`:**
|
||||||
|
```bash
|
||||||
|
# PDF aus HTML generieren
|
||||||
|
curl -X POST https://pl.c2sgmbh.de/api/generate-pdf \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Cookie: payload-token=..." \
|
||||||
|
-d '{
|
||||||
|
"source": "html",
|
||||||
|
"html": "<h1>Test</h1><p>Inhalt</p>",
|
||||||
|
"filename": "test.pdf"
|
||||||
|
}'
|
||||||
|
|
||||||
|
# Job-Status abfragen
|
||||||
|
curl "https://pl.c2sgmbh.de/api/generate-pdf?jobId=abc123" \
|
||||||
|
-H "Cookie: payload-token=..."
|
||||||
|
```
|
||||||
|
|
||||||
|
**Programmatisch:**
|
||||||
|
```typescript
|
||||||
|
import { queuePdfFromHtml, queuePdfFromUrl, getPdfJobStatus } from '@/lib/queue'
|
||||||
|
|
||||||
|
// HTML zu PDF
|
||||||
|
const job = await queuePdfFromHtml('<h1>Test</h1>', { filename: 'test.pdf' })
|
||||||
|
|
||||||
|
// URL zu PDF
|
||||||
|
const job2 = await queuePdfFromUrl('https://example.com', { format: 'A4' })
|
||||||
|
|
||||||
|
// Status abfragen
|
||||||
|
const status = await getPdfJobStatus(job.id)
|
||||||
|
```
|
||||||
|
|
||||||
|
### Worker-Konfiguration
|
||||||
|
|
||||||
|
Über `ecosystem.config.cjs`:
|
||||||
|
- `QUEUE_EMAIL_CONCURRENCY`: Parallele E-Mail-Jobs (default: 3)
|
||||||
|
- `QUEUE_PDF_CONCURRENCY`: Parallele PDF-Jobs (default: 2)
|
||||||
|
- `QUEUE_DEFAULT_RETRY`: Retry-Versuche (default: 3)
|
||||||
|
- `QUEUE_REDIS_DB`: Redis-Datenbank für Queue (default: 1)
|
||||||
|
|
||||||
## Redis Caching
|
## Redis Caching
|
||||||
|
|
||||||
Redis wird für API-Response-Caching und E-Mail-Transporter-Caching verwendet:
|
Redis wird für API-Response-Caching und E-Mail-Transporter-Caching verwendet:
|
||||||
|
|
|
||||||
Loading…
Reference in a new issue