chore: remove outdated testing guide for scheduled tasks
This commit is contained in:
Andras Bacsai
parent
1580c0d3ad
commit
372ea26810
1 changed files with 0 additions and 235 deletions
235
TESTING_GUIDE.md
235
TESTING_GUIDE.md
|
|
@ -1,235 +0,0 @@
|
|||
# Testing Guide: Scheduled Tasks Improvements
|
||||
|
||||
## Overview
|
||||
This guide covers testing all the improvements made to the scheduled tasks system, including retry logic, timeout handling, and error logging.
|
||||
|
||||
## Jobs Modified
|
||||
|
||||
1. **CoolifyTask** - Infrastructure job for SSH operations (3 retries, 600s timeout)
|
||||
2. **ScheduledTaskJob** - Scheduled container commands (3 retries, configurable timeout)
|
||||
3. **DatabaseBackupJob** - Database backups (2 retries, existing timeout)
|
||||
|
||||
---
|
||||
|
||||
## Quick Test Commands
|
||||
|
||||
### Run Unit Tests (No Database Required)
|
||||
```bash
|
||||
./vendor/bin/pest tests/Unit/ScheduledJobsRetryConfigTest.php
|
||||
```
|
||||
|
||||
### Run Feature Tests (Requires Database - Run in Docker)
|
||||
```bash
|
||||
docker exec coolify php artisan test --filter=CoolifyTaskRetryTest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Manual Testing
|
||||
|
||||
### 1. Test ScheduledTaskJob ✅ (You tested this)
|
||||
|
||||
**How to test:**
|
||||
1. Create a scheduled task in the UI
|
||||
2. Set a short frequency (every minute)
|
||||
3. Monitor execution in the UI
|
||||
4. Check logs: `storage/logs/scheduled-errors-2025-11-09.log`
|
||||
|
||||
**What to verify:**
|
||||
- Task executes successfully
|
||||
- Duration is recorded (in seconds with 2 decimal places)
|
||||
- Retry count is tracked
|
||||
- Timeout configuration is respected
|
||||
|
||||
---
|
||||
|
||||
### 2. Test DatabaseBackupJob ✅ (You tested this)
|
||||
|
||||
**How to test:**
|
||||
1. Create a scheduled database backup
|
||||
2. Set frequency to manual or very short interval
|
||||
3. Trigger backup manually or wait for schedule
|
||||
4. Check logs for any errors
|
||||
|
||||
**What to verify:**
|
||||
- Backup completes successfully
|
||||
- Retry logic works if there's a transient failure
|
||||
- Error logging is consistent
|
||||
- Backoff timing is correct (60s, 300s)
|
||||
|
||||
---
|
||||
|
||||
### 3. Test CoolifyTask ⚠️ (IMPORTANT - Not tested yet)
|
||||
|
||||
CoolifyTask is used throughout the application for ALL SSH operations. Here are multiple ways to test it:
|
||||
|
||||
#### **Option A: Server Validation** (Easiest)
|
||||
1. Go to **Servers** in Coolify UI
|
||||
2. Select any server
|
||||
3. Click **"Validate Server"** or **"Check Connection"**
|
||||
4. This triggers CoolifyTask jobs
|
||||
5. Check Horizon dashboard for job processing
|
||||
6. Check logs: `storage/logs/scheduled-errors-2025-11-09.log`
|
||||
|
||||
#### **Option B: Container Operations**
|
||||
1. Go to any **Application** or **Service**
|
||||
2. Try these actions (each triggers CoolifyTask):
|
||||
- Restart container
|
||||
- View logs
|
||||
- Execute command in container
|
||||
3. Monitor Horizon for job processing
|
||||
4. Check logs for errors
|
||||
|
||||
#### **Option C: Application Deployment**
|
||||
1. Deploy or redeploy any application
|
||||
2. This triggers MANY CoolifyTask jobs
|
||||
3. Watch Horizon dashboard - you should see:
|
||||
- Jobs being dispatched
|
||||
- Jobs completing successfully
|
||||
- If any fail, they should retry (check "Failed Jobs")
|
||||
4. Check logs for retry attempts
|
||||
|
||||
#### **Option D: Docker Cleanup**
|
||||
1. Wait for or trigger Docker cleanup (runs on schedule)
|
||||
2. This uses CoolifyTask for cleanup commands
|
||||
3. Check logs: `storage/logs/scheduled-errors-2025-11-09.log`
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Verification
|
||||
|
||||
### Horizon Dashboard
|
||||
1. Open Horizon: `/horizon`
|
||||
2. Watch these sections:
|
||||
- **Recent Jobs** - See jobs being processed
|
||||
- **Failed Jobs** - Jobs that failed permanently after retries
|
||||
- **Monitoring** - Job throughput and wait times
|
||||
|
||||
### Log Monitoring
|
||||
```bash
|
||||
# Watch scheduled errors in real-time
|
||||
tail -f storage/logs/scheduled-errors-2025-11-09.log
|
||||
|
||||
# Check for specific job errors
|
||||
grep "CoolifyTask" storage/logs/scheduled-errors-2025-11-09.log
|
||||
grep "ScheduledTaskJob" storage/logs/scheduled-errors-2025-11-09.log
|
||||
grep "DatabaseBackupJob" storage/logs/scheduled-errors-2025-11-09.log
|
||||
```
|
||||
|
||||
### Database Verification
|
||||
```sql
|
||||
-- Check execution tracking
|
||||
SELECT * FROM scheduled_task_executions
|
||||
ORDER BY created_at DESC
|
||||
LIMIT 10;
|
||||
|
||||
-- Verify duration is decimal (not throwing errors)
|
||||
SELECT id, duration, retry_count, started_at, finished_at
|
||||
FROM scheduled_task_executions
|
||||
WHERE duration IS NOT NULL;
|
||||
|
||||
-- Check for tasks with retries
|
||||
SELECT * FROM scheduled_task_executions
|
||||
WHERE retry_count > 0;
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Expected Behavior
|
||||
|
||||
### ✅ Success Indicators
|
||||
|
||||
1. **Jobs Complete Successfully**
|
||||
- Horizon shows completed jobs
|
||||
- No errors in scheduled-errors log
|
||||
- Execution records in database
|
||||
|
||||
2. **Retry Logic Works**
|
||||
- Failed jobs retry automatically
|
||||
- Backoff timing is respected (30s, 60s, etc.)
|
||||
- Jobs marked failed only after all retries exhausted
|
||||
|
||||
3. **Timeout Enforcement**
|
||||
- Long-running jobs terminate at timeout
|
||||
- Timeout is configurable per task
|
||||
- No hanging jobs
|
||||
|
||||
4. **Error Logging**
|
||||
- All errors logged to `storage/logs/scheduled-errors-2025-11-09.log`
|
||||
- Consistent format with job name, attempt count, error details
|
||||
- Trace included for debugging
|
||||
|
||||
5. **Execution Tracking**
|
||||
- Duration recorded correctly (decimal with 2 places)
|
||||
- Retry count incremented on failures
|
||||
- Started/finished timestamps accurate
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Jobs fail immediately without retrying
|
||||
**Check:**
|
||||
- Verify `$tries` property is set on the job
|
||||
- Check if exception is being caught and re-thrown correctly
|
||||
- Look for `maxExceptions` being reached
|
||||
|
||||
### Issue: "Invalid text representation" errors
|
||||
**Fix Applied:**
|
||||
- Duration field changed from integer to decimal(10,2)
|
||||
- If you see this, run migrations again
|
||||
|
||||
### Issue: Jobs not appearing in Horizon
|
||||
**Check:**
|
||||
- Horizon is running (`php artisan horizon`)
|
||||
- Queue workers are active
|
||||
- Job is dispatched to correct queue ('high' for these jobs)
|
||||
|
||||
### Issue: Timeout not working
|
||||
**Check:**
|
||||
- Timeout is set on job (CoolifyTask: 600s, ScheduledTask: configurable)
|
||||
- PHP `max_execution_time` allows job timeout
|
||||
- Queue worker timeout is higher than job timeout
|
||||
|
||||
---
|
||||
|
||||
## Test Checklist
|
||||
|
||||
- [ ] Unit tests pass: `./vendor/bin/pest tests/Unit/ScheduledJobsRetryConfigTest.php`
|
||||
- [ ] ScheduledTaskJob tested manually ✅
|
||||
- [ ] DatabaseBackupJob tested manually ✅
|
||||
- [ ] CoolifyTask tested manually (server validation, container ops, or deployment)
|
||||
- [ ] Retry logic verified (force a failure, watch retry attempts)
|
||||
- [ ] Timeout enforcement tested (create long-running task with short timeout)
|
||||
- [ ] Error logs checked: `storage/logs/scheduled-errors-2025-11-09.log`
|
||||
- [ ] Horizon dashboard shows jobs processing correctly
|
||||
- [ ] Database execution records show duration as decimal
|
||||
- [ ] UI shows timeout configuration field for scheduled tasks
|
||||
|
||||
---
|
||||
|
||||
## Next Steps After Testing
|
||||
|
||||
1. If all tests pass, run migrations on production/staging:
|
||||
```bash
|
||||
php artisan migrate
|
||||
```
|
||||
|
||||
2. Monitor logs for the first 24 hours:
|
||||
```bash
|
||||
tail -f storage/logs/scheduled-errors-2025-11-09.log
|
||||
```
|
||||
|
||||
3. Check Horizon for any failed jobs needing attention
|
||||
|
||||
4. Verify existing scheduled tasks now have retry capability
|
||||
|
||||
---
|
||||
|
||||
## Questions?
|
||||
|
||||
If you encounter issues:
|
||||
1. Check `storage/logs/scheduled-errors-2025-11-09.log` first
|
||||
2. Check `storage/logs/laravel.log` for general errors
|
||||
3. Look at Horizon "Failed Jobs" for detailed error info
|
||||
4. Review database execution records for patterns
|
||||
Loading…
Reference in a new issue