coolify/app/Jobs
Andras Bacsai d2d9c1b2bc debug: add comprehensive status change logging
Added detailed debug logging to all status update paths to help
diagnose why "unhealthy" status appears in the UI.

## Logging Added

### 1. PushServerUpdateJob (Sentinel updates)
**Location**: Lines 303-315
**Logs**: Status changes from Sentinel push updates
**Data tracked**:
- Old vs new status
- Container statuses that led to aggregation
- Status flags (hasRunning, hasUnhealthy, hasUnknown)

### 2. GetContainersStatus (SSH updates)
**Location**: Lines 441-449, 346-354, 358-365
**Logs**: Status changes from SSH-based checks
**Scenarios**:
- Normal status aggregation
- Recently restarted containers (kept as degraded)
- Applications not running (set to exited)
**Data tracked**:
- Old vs new status
- Container statuses
- Restart count and timing
- Whether containers exist

### 3. Application Model Status Accessor
**Location**: Lines 706-712, 726-732
**Logs**: When status is set without explicit health information
**Issue**: Highlights cases where health defaults to "unhealthy"
**Data tracked**:
- Raw value passed to setter
- Final result after default applied

## How to Use

### Enable Debug Logging
Edit `.env` or `config/logging.php` to set log level to debug:
```
LOG_LEVEL=debug
```

### Monitor Logs
```bash
tail -f storage/logs/laravel.log | grep STATUS-DEBUG
```

### Log Format
All logs use `[STATUS-DEBUG]` prefix for easy filtering:
```
[2025-11-19 13:00:00] local.DEBUG: [STATUS-DEBUG] Sentinel status change
{
  "source": "PushServerUpdateJob",
  "app_id": 123,
  "app_name": "my-app",
  "old_status": "running:unknown",
  "new_status": "running:healthy",
  "container_statuses": [...],
  "flags": {...}
}
```

## What to Look For

1. **Default to unhealthy**: Check Application model accessor logs
2. **Status flipping**: Compare timestamps between Sentinel and SSH updates
3. **Incorrect aggregation**: Check flags and container_statuses
4. **Stale database values**: Check if old_status persists across multiple logs

## Next Steps

After gathering logs, we can:
1. Identify the exact source of "unhealthy" status
2. Determine if it's a default issue, aggregation bug, or timing problem
3. Apply targeted fix based on evidence

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-19 13:52:08 +01:00
..
ApplicationDeploymentJob.php fix: normalize preview paths and use BUILD_TIME_ENV_PATH constant 2025-11-18 13:48:06 +01:00
ApplicationPullRequestUpdateJob.php Merge branch 'next' into feat/update-applicationpullrequestupdatejob-documentation 2025-10-15 00:59:02 +02:00
CheckAndStartSentinelJob.php fix: add timeout to sentinel process checks for improved reliability 2025-01-10 13:03:47 +01:00
CheckForUpdatesJob.php refactor(proxy): implement centralized caching for versions.json and improve UX 2025-11-17 14:53:28 +01:00
CheckHelperImageJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
CheckTraefikVersionForServerJob.php refactor: send immediate Traefik version notifications instead of delayed aggregation 2025-11-18 12:30:50 +01:00
CheckTraefikVersionJob.php refactor: send immediate Traefik version notifications instead of delayed aggregation 2025-11-18 12:30:50 +01:00
CleanupHelperContainersJob.php fix(deployment): prevent base deployments from being killed when PRs close (#7113) 2025-11-09 14:41:35 +01:00
CleanupInstanceStuffsJob.php feat(email): implement email change request and verification process 2025-08-18 14:54:08 +02:00
CleanupStaleMultiplexedConnections.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
CoolifyTask.php feat(jobs): improve scheduled tasks with retry logic and queue cleanup 2025-11-10 11:11:18 +01:00
DatabaseBackupJob.php refactor(DatabaseBackupJob): remove retry attempts and backoff logic for job execution 2025-11-11 15:39:01 +01:00
DeleteResourceJob.php Update app/Jobs/DeleteResourceJob.php 2025-11-11 11:35:30 +01:00
DockerCleanupJob.php fix(docker): volumes get delete when stopping a service if Delete Unused Volumes is activated (#6317) 2025-08-04 21:15:56 +02:00
GithubAppPermissionJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
PullChangelog.php refactor(jobs): pull github changelogs from cdn instead of github 2025-09-01 16:13:55 +02:00
PullTemplatesFromCDN.php refactor(templates): update service template file handling to use dynamic file name from constants 2025-08-12 10:06:19 +02:00
PushServerUpdateJob.php debug: add comprehensive status change logging 2025-11-19 13:52:08 +01:00
RegenerateSslCertJob.php refactor: replace direct SslCertificate queries with server relationship methods for consistency 2025-10-09 17:00:05 +02:00
RestartProxyJob.php feat(proxy): add Traefik version tracking with notifications and dismissible UI warnings 2025-11-14 11:35:22 +01:00
ScheduledJobManager.php feat(jobs): improve scheduled tasks with retry logic and queue cleanup 2025-11-10 11:11:18 +01:00
ScheduledTaskJob.php fix(ScheduledTaskJob): make server property nullable and update logging to handle null values 2025-11-11 15:38:55 +01:00
SendMessageToDiscordJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
SendMessageToPushoverJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
SendMessageToSlackJob.php feat(slack): Show message title in notification previews (#5063) 2025-02-11 16:31:37 +01:00
SendMessageToTelegramJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
SendWebhookJob.php feat: implement actual webhook delivery with Ray debugging 2025-10-10 18:07:04 +02:00
ServerCheckJob.php fix(jobs): update middleware to use expireAfter for WithoutOverlapping in multiple job classes 2025-07-01 10:50:27 +02:00
ServerCleanupMux.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
ServerConnectionCheckJob.php fix: improve error logging and handling in ServerConnectionCheckJob for Hetzner server status 2025-10-10 10:12:59 +02:00
ServerFilesFromServerJob.php fix: make sure important jobs/actions are running on high prio queue 2024-11-22 11:16:01 +01:00
ServerLimitCheckJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
ServerManagerJob.php fix(deployment): improve error logging with exception types and hidden technical details 2025-11-17 14:44:39 +01:00
ServerPatchCheckJob.php fix(jobs): update middleware to use expireAfter for WithoutOverlapping in multiple job classes 2025-07-01 10:50:27 +02:00
ServerStorageCheckJob.php fix(horizon): add silenced jobs 2025-07-12 14:44:32 +02:00
ServerStorageSaveJob.php fix: make sure important jobs/actions are running on high prio queue 2024-11-22 11:16:01 +01:00
StripeProcessJob.php feat(stripe): enhance subscription handling and verification process 2025-09-23 11:00:38 +02:00
SubscriptionInvoiceFailedJob.php refactor(stripe-jobs): comment out internal notification calls and add subscription status verification before sending failure notifications 2025-09-13 19:35:32 +02:00
UpdateCoolifyJob.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
UpdateStripeCustomerEmailJob.php feat(email): implement email change request and verification process 2025-08-18 14:54:08 +02:00
ValidateAndInstallServerJob.php work work on hetzner integration 2025-10-09 16:54:13 +02:00
VerifyStripeSubscriptionStatusJob.php fix(subscription-job): enhance retry logic for VerifyStripeSubscriptionStatusJob 2025-09-23 18:17:52 +02:00
VolumeCloneJob.php feat: remote server volume cloning 2025-01-13 11:13:15 +01:00