coolify/app/Console/Commands
Andras Bacsai c6a2d1fe0a Fix stale lock issue causing scheduled tasks to stop (#4539)
## Problem
Scheduled tasks, backups, and auto-updates stopped working after 1-2 months
with error: MaxAttemptsExceededException: App\Jobs\ScheduledJobManager has
been attempted too many times.

Root cause: ScheduledJobManager used WithoutOverlapping with only
releaseAfter(60), causing locks without expiration (TTL=-1) that persisted
indefinitely when jobs hung or processes crashed.

## Solution

### Part 1: Prevention (Future Locks)
- Added expireAfter(60) to ScheduledJobManager middleware
- Lock now auto-expires after 60 seconds (matches everyMinute schedule)
- Changed from releaseAfter(60) to expireAfter(60)->dontRelease()
- Follows Laravel best practices and matches other Coolify jobs

### Part 2: Recovery (Existing Locks)
- Enhanced cleanup:redis command with --clear-locks flag
- Scans Redis for stale locks (TTL=-1) and removes them
- Called automatically during app:init on startup/upgrade
- Provides immediate recovery for affected instances

## Changes
- app/Jobs/ScheduledJobManager.php: Added expireAfter(60)->dontRelease()
- app/Console/Commands/CleanupRedis.php: Added cleanupCacheLocks() method
- app/Console/Commands/Init.php: Auto-clear locks on startup
- tests/Unit/ScheduledJobManagerLockTest.php: Test to prevent regression
- STALE_LOCK_FIX.md: Complete documentation

## Testing
- Unit tests pass (2 tests, 8 assertions)
- Code formatted with Pint
- Matches pattern used by CleanupInstanceStuffsJob

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-10-23 10:07:33 +02:00
..
Cloud Changes auto-committed by Conductor 2025-10-16 17:23:22 +02:00
Generate feat: generate category for services 2025-08-18 17:28:20 +02:00
AdminDeleteUser.php Changes auto-committed by Conductor 2025-10-16 17:23:22 +02:00
CheckApplicationDeploymentQueue.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
CleanupApplicationDeploymentQueue.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
CleanupDatabase.php chore: remove webhooks table cleanup 2025-09-10 19:56:15 +02:00
CleanupNames.php feat(validation): centralize validation patterns for names and descriptions 2025-08-19 12:14:48 +02:00
CleanupRedis.php Fix stale lock issue causing scheduled tasks to stop (#4539) 2025-10-23 10:07:33 +02:00
CleanupStuckedResources.php feat(cleanup): add force deletion of stuck servers and orphaned SSL certificates 2025-10-09 16:37:56 +02:00
CleanupUnreachableServers.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
ClearGlobalSearchCache.php feat: add artisan command to clear global search cache 2025-10-11 13:36:14 +02:00
Dev.php feat(dev-command): dispatch CheckHelperImageJob during instance initialization to enhance setup process 2025-09-09 16:46:24 +02:00
Emails.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
Horizon.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
HorizonManage.php refactor: streamline job status retrieval and clean up repository interface 2025-01-10 19:53:13 +01:00
Init.php Fix stale lock issue causing scheduled tasks to stop (#4539) 2025-10-23 10:07:33 +02:00
Migration.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
NotifyDemo.php reorder more stuff 2024-12-11 18:44:19 +01:00
RootChangeEmail.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
RootResetPassword.php fix(console): handle missing root user in password reset command 2025-03-24 18:00:31 +01:00
RunScheduledJobsManually.php feat(scheduling): add frequency filter option for manual execution of scheduled jobs 2025-07-11 15:10:43 +02:00
Scheduler.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
Seeder.php Revert "rector: arrrrr" 2025-01-07 15:31:43 +01:00
ServicesDelete.php refactor(command): improve database collection logic for deletion command by using unique identifiers and enhancing user experience 2025-09-05 17:43:19 +02:00
SyncBunny.php feat(command): add option to sync GitHub releases to BunnyCDN and refactor sync logic 2025-09-01 16:14:02 +02:00
ViewScheduledLogs.php feat(logging): add frequency filters for scheduled logs command to support hourly, daily, weekly, and monthly job views 2025-07-11 19:26:42 +02:00