Commit graph

14 commits

Author SHA1 Message Date
Andras Bacsai
470cc15e62 feat(jobs): implement encrypted queue jobs
- Add ShouldBeEncrypted interface to all queue jobs to encrypt sensitive
  job payloads
- Configure explicit retry policies for messaging jobs (5 attempts,
  10-second backoff)
2026-03-10 14:05:05 +01:00
Andras Bacsai
a0c177f6f2 feat(jobs): add queue delay resilience to scheduled job execution
Implement dedup key-based cron tracking to make scheduled jobs resilient to queue
delays. Even if a job is delayed by minutes, it will catch the missed cron window
by tracking previousRunDate in cache instead of relying on isDue() alone.

- Add dedupKey parameter to shouldRunNow() in ScheduledJobManager
  - When provided, uses getPreviousRunDate() + cache tracking for resilience
  - Falls back to isDue() for docker cleanups without dedup key
  - Prevents double-dispatch within same cron window

- Optimize ServerConnectionCheckJob dispatch
  - Skip SSH checks if Sentinel is healthy (enabled and live)
  - Reduces redundant checks when Sentinel heartbeat proves connectivity

- Remove hourly Sentinel update checks
  - Consolidate to daily CheckAndStartSentinelJob dispatch
  - Crash recovery handled by sentinelOutOfSync → ServerCheckJob flow

- Add logging for skipped database backups with context (backup_id, database_id, status)

- Refactor skip reason methods to accept server parameter, avoiding redundant queries

- Add comprehensive test suite for scheduling with various delay scenarios and timezones
2026-02-28 15:06:25 +01:00
Andras Bacsai
3962f1a5b9 perf: Remove dead server filtering code from Kernel scheduler
Remove unused server filtering logic in Kernel.php that was querying servers
but never using the results. Simplify Sentinel update checks in ServerManagerJob
by reusing the $isSentinelEnabled variable and removing unnecessary timezone
parameter for hourly cron execution.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-11 10:33:55 +01:00
Andras Bacsai
70ff73e954 Merge branch 'next' into macau-v1
Resolved conflicts in ServerManagerJob.php by:
- Keeping sentinel update check code from macau-v1
- Preserving sentinel restart code from next branch
- Ensuring no duplicate code blocks
2025-12-04 15:07:36 +01:00
Andras Bacsai
4002044877 Refactor: Move sentinel update checks to ServerManagerJob and add tests for hourly dispatch 2025-12-04 14:58:18 +01:00
Andras Bacsai
56a0143a25 Fix: Prevent ServerStorageCheckJob duplication when Sentinel is active
When Sentinel is enabled and in sync, ServerStorageCheckJob was being
dispatched from two locations causing unnecessary duplication:
1. PushServerUpdateJob (every ~30s with real-time filesystem data)
2. ServerManagerJob (scheduled cron check via SSH)

This commit modifies ServerManagerJob to only dispatch ServerStorageCheckJob
when Sentinel is out of sync or disabled. When Sentinel is active and in sync,
PushServerUpdateJob provides real-time storage data, making the scheduled SSH
check redundant.

Benefits:
- Eliminates duplicate storage checks when Sentinel is working
- Reduces unnecessary SSH overhead
- Storage checks still run as fallback when Sentinel fails
- Maintains scheduled checks for servers without Sentinel

Updated tests to reflect new behavior:
- Storage check NOT dispatched when Sentinel is in sync
- Storage check dispatched when Sentinel is out of sync or disabled
- All timezone and frequency tests updated accordingly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-03 10:05:10 +01:00
Andras Bacsai
c982d58eee Refactor: Move Sentinel restart logic into processServerTasks method 2025-12-03 09:22:00 +01:00
Andras Bacsai
8ff83cc3d6 Fix: Pass $serverTimezone to shouldRunNow() in ServerCheckJob dispatch
Pass the server timezone parameter to shouldRunNow() call at line 127,
ensuring ServerCheckJob dispatch respects the server's local timezone
instead of falling back to the instance default.

This aligns the behavior with other scheduled tasks in the same method:
- ServerStorageCheckJob (line 137)
- ServerPatchCheckJob (line 144)
- Sentinel restart (line 152)

All scheduled tasks in processServerTasks() now consistently use the
server's configured timezone for cron evaluation.

Added unit test to verify timezone-aware cron schedule evaluation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 16:58:43 +01:00
Andras Bacsai
ed5796739f Fix: Prevent ServerManagerJob executionTime mutation across server loop
Fixed a critical bug where $this->executionTime was being mutated during
the server processing loop, causing incorrect scheduling calculations for
subsequent servers.

The issue occurred at line 123 where subSeconds() was called directly on
the shared executionTime instance. This caused the baseline time to shift
by waitTime seconds with each server iteration, resulting in compounding
scheduling errors (e.g., 1680 seconds drift over 5 servers).

Changed:
- app/Jobs/ServerManagerJob.php:123
  Added .copy() before .subSeconds() to prevent mutation

Added comprehensive unit tests that verify:
- Immutability when using .copy()
- Demonstration of the bug without .copy()
- Correct behavior across multiple iterations

This follows the existing pattern in shouldRunNow() (line 167) and aligns
with other jobs in the codebase.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 15:27:17 +01:00
Andras Bacsai
b47181c790 Decouple ServerStorageCheckJob from Sentinel sync status
Server disk usage checks now run on their configured schedule regardless of Sentinel status, eliminating monitoring blind spots when Sentinel is offline, out of sync, or disabled. Storage checks now respect server timezone settings, consistent with patch checks.

Changes:
- Moved server timezone calculation to top of processServerTasks()
- Extracted ServerStorageCheckJob dispatch from Sentinel conditional
- Fixed default frequency to '0 23 * * *' (11 PM daily)
- Added timezone parameter to storage check scheduling

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-02 13:36:25 +01:00
Andras Bacsai
b602fef4db fix(deployment): improve error logging with exception types and hidden technical details
- Add exception class names to error messages for better debugging
- Mark technical details (error type, code, location, stack trace) as hidden in logs
- Preserve original exception types when wrapping in DeploymentException
- Update ServerManagerJob to include exception class in log messages
- Enhance unit tests to verify hidden log entry behavior

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-11-17 14:44:39 +01:00
Andras Bacsai
ed93031a39 feat(docker): implement Docker cleanup processing in ScheduledJobManager; refactor server task scheduling to streamline cleanup job dispatching 2025-08-26 14:43:57 +02:00
Andras Bacsai
13397b62cf refactor: update check frequency logic for cloud and self-hosted environments; streamline server task scheduling and timezone handling 2025-08-26 13:51:42 +02:00
Andras Bacsai
7d10711a65 fix(servercheck): Properly check server statuses with and without Sentinel 2025-08-22 11:50:56 +02:00