feat: add validation for YAML parsing, integer parameters, and Docker Compose custom fields
This commit adds comprehensive validation improvements and DRY principles for handling Coolify's custom Docker Compose extensions. ## Changes ### 1. Created Reusable stripCoolifyCustomFields() Function - Added shared helper in bootstrap/helpers/docker.php - Removes all Coolify custom fields (exclude_from_hc, content, isDirectory, is_directory) - Handles both long syntax (arrays) and short syntax (strings) for volumes - Well-documented with comprehensive docblock - Follows DRY principle for consistent field stripping ### 2. Fixed Docker Compose Modal Validation - Updated validateComposeFile() to use stripCoolifyCustomFields() - Now removes ALL custom fields before Docker validation (previously only removed content) - Fixes validation errors when using templates with custom fields (e.g., traccar.yaml) - Users can now validate compose files with Coolify extensions in UI ### 3. Enhanced YAML Validation in CalculatesExcludedStatus - Added proper exception handling with ParseException vs generic Exception - Added structure validation (checks if parsed result and services are arrays) - Comprehensive logging with context (error message, line number, snippet) - Maintains safe fallback behavior (returns empty collection on error) ### 4. Added Integer Validation to ContainerStatusAggregator - Validates maxRestartCount parameter in both aggregateFromStrings() and aggregateFromContainers() - Corrects negative values to 0 with warning log - Logs warnings for suspiciously high values (> 1000) - Prevents logic errors in crash loop detection ### 5. Comprehensive Unit Tests - tests/Unit/StripCoolifyCustomFieldsTest.php (NEW) - 9 tests, 43 assertions - tests/Unit/ContainerStatusAggregatorTest.php - Added 6 tests for integer validation - tests/Unit/ExcludeFromHealthCheckTest.php - Added 4 tests for YAML validation - All tests passing with proper Log facade mocking ### 6. Documentation - Added comprehensive Docker Compose extensions documentation to .ai/core/deployment-architecture.md - Documents all custom fields: exclude_from_hc, content, isDirectory/is_directory - Includes examples, use cases, implementation details, and test references - Updated .ai/README.md with navigation links to new documentation ## Benefits - Better UX: Users can validate compose files with custom fields - Better Debugging: Comprehensive logging for errors - Better Code Quality: DRY principle with reusable validation - Better Reliability: Prevents logic errors from invalid parameters - Better Maintainability: Easy to add new custom fields in future 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
This commit is contained in:
parent
ae6eef3cdb
commit
7ceb124e9b
8 changed files with 755 additions and 12 deletions
|
|
@ -16,7 +16,7 @@ ### 📚 Core Documentation
|
||||||
- **[Technology Stack](core/technology-stack.md)** - All versions, packages, and dependencies (Laravel 12.4.1, PHP 8.4.7, etc.)
|
- **[Technology Stack](core/technology-stack.md)** - All versions, packages, and dependencies (Laravel 12.4.1, PHP 8.4.7, etc.)
|
||||||
- **[Project Overview](core/project-overview.md)** - What Coolify is and how it works
|
- **[Project Overview](core/project-overview.md)** - What Coolify is and how it works
|
||||||
- **[Application Architecture](core/application-architecture.md)** - System design and component relationships
|
- **[Application Architecture](core/application-architecture.md)** - System design and component relationships
|
||||||
- **[Deployment Architecture](core/deployment-architecture.md)** - How deployments work end-to-end
|
- **[Deployment Architecture](core/deployment-architecture.md)** - How deployments work end-to-end, including Coolify Docker Compose extensions (custom fields)
|
||||||
|
|
||||||
### 💻 Development
|
### 💻 Development
|
||||||
Day-to-day development practices:
|
Day-to-day development practices:
|
||||||
|
|
@ -85,6 +85,13 @@ ### Laravel-Specific Questions
|
||||||
- Pest testing patterns
|
- Pest testing patterns
|
||||||
- Laravel conventions
|
- Laravel conventions
|
||||||
|
|
||||||
|
### Docker Compose Extensions
|
||||||
|
→ [core/deployment-architecture.md](core/deployment-architecture.md#coolify-docker-compose-extensions)
|
||||||
|
- Custom fields: `exclude_from_hc`, `content`, `isDirectory`
|
||||||
|
- How to use inline file content
|
||||||
|
- Health check exclusion patterns
|
||||||
|
- Volume creation control
|
||||||
|
|
||||||
### Version Numbers
|
### Version Numbers
|
||||||
→ [core/technology-stack.md](core/technology-stack.md)
|
→ [core/technology-stack.md](core/technology-stack.md)
|
||||||
- **Single source of truth** for all version numbers
|
- **Single source of truth** for all version numbers
|
||||||
|
|
|
||||||
|
|
@ -303,3 +303,286 @@ ### External Services
|
||||||
- **External database** connections
|
- **External database** connections
|
||||||
- **Third-party monitoring** tools
|
- **Third-party monitoring** tools
|
||||||
- **Custom notification** channels
|
- **Custom notification** channels
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Coolify Docker Compose Extensions
|
||||||
|
|
||||||
|
Coolify extends standard Docker Compose with custom fields (often called "magic fields") that provide Coolify-specific functionality. These extensions are processed during deployment and stripped before sending the final compose file to Docker, maintaining full compatibility with Docker's compose specification.
|
||||||
|
|
||||||
|
### Overview
|
||||||
|
|
||||||
|
**Why Custom Fields?**
|
||||||
|
- Enable Coolify-specific features without breaking Docker Compose compatibility
|
||||||
|
- Simplify configuration by embedding content directly in compose files
|
||||||
|
- Allow fine-grained control over health check monitoring
|
||||||
|
- Reduce external file dependencies
|
||||||
|
|
||||||
|
**Processing Flow:**
|
||||||
|
1. User defines compose file with custom fields
|
||||||
|
2. Coolify parses and processes custom fields (creates files, stores settings)
|
||||||
|
3. Custom fields are stripped from final compose sent to Docker
|
||||||
|
4. Docker receives standard, valid compose file
|
||||||
|
|
||||||
|
### Service-Level Extensions
|
||||||
|
|
||||||
|
#### `exclude_from_hc`
|
||||||
|
|
||||||
|
**Type:** Boolean
|
||||||
|
**Default:** `false`
|
||||||
|
**Purpose:** Exclude specific services from health check monitoring while still showing their status
|
||||||
|
|
||||||
|
**Example Usage:**
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
watchtower:
|
||||||
|
image: containrrr/watchtower
|
||||||
|
exclude_from_hc: true # Don't monitor this service's health
|
||||||
|
|
||||||
|
backup:
|
||||||
|
image: postgres:16
|
||||||
|
exclude_from_hc: true # Backup containers don't need monitoring
|
||||||
|
restart: always
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior:**
|
||||||
|
- Container status is still calculated from Docker state (running, exited, etc.)
|
||||||
|
- Status displays with `:excluded` suffix (e.g., `running:healthy:excluded`)
|
||||||
|
- UI shows "Monitoring Disabled" indicator
|
||||||
|
- Functionally equivalent to `restart: no` for health check purposes
|
||||||
|
- See [Container Status with All Excluded](application-architecture.md#container-status-when-all-containers-excluded) for detailed status handling
|
||||||
|
|
||||||
|
**Use Cases:**
|
||||||
|
- Sidecar containers (watchtower, log collectors)
|
||||||
|
- Backup/maintenance containers
|
||||||
|
- One-time initialization containers
|
||||||
|
- Containers that intentionally restart frequently
|
||||||
|
|
||||||
|
**Implementation:**
|
||||||
|
- Parsed: `bootstrap/helpers/parsers.php`
|
||||||
|
- Status logic: `app/Traits/CalculatesExcludedStatus.php`
|
||||||
|
- Validation: `tests/Unit/ExcludeFromHealthCheckTest.php`
|
||||||
|
|
||||||
|
### Volume-Level Extensions
|
||||||
|
|
||||||
|
Volume extensions only work with **long syntax** (array/object format), not short syntax (string format).
|
||||||
|
|
||||||
|
#### `content`
|
||||||
|
|
||||||
|
**Type:** String (supports multiline with `|` or `>`)
|
||||||
|
**Purpose:** Embed file content directly in compose file for automatic creation during deployment
|
||||||
|
|
||||||
|
**Example Usage:**
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
image: node:20
|
||||||
|
volumes:
|
||||||
|
# Inline entrypoint script
|
||||||
|
- type: bind
|
||||||
|
source: ./entrypoint.sh
|
||||||
|
target: /app/entrypoint.sh
|
||||||
|
content: |
|
||||||
|
#!/bin/sh
|
||||||
|
set -e
|
||||||
|
echo "Starting application..."
|
||||||
|
npm run migrate
|
||||||
|
exec "$@"
|
||||||
|
|
||||||
|
# Configuration file with environment variables
|
||||||
|
- type: bind
|
||||||
|
source: ./config.xml
|
||||||
|
target: /etc/app/config.xml
|
||||||
|
content: |
|
||||||
|
<?xml version='1.0' encoding='UTF-8'?>
|
||||||
|
<config>
|
||||||
|
<database>
|
||||||
|
<host>${DB_HOST}</host>
|
||||||
|
<port>${DB_PORT}</port>
|
||||||
|
</database>
|
||||||
|
</config>
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior:**
|
||||||
|
- Content is written to the host at `source` path before container starts
|
||||||
|
- File is created with mode `644` (readable by all, writable by owner)
|
||||||
|
- Environment variables in content are interpolated at deployment time
|
||||||
|
- Content is stored in `LocalFileVolume` model (encrypted at rest)
|
||||||
|
- Original `docker_compose_raw` retains content for editing
|
||||||
|
|
||||||
|
**Use Cases:**
|
||||||
|
- Entrypoint scripts
|
||||||
|
- Configuration files
|
||||||
|
- Environment-specific settings
|
||||||
|
- Small initialization scripts
|
||||||
|
- Templates that require dynamic content
|
||||||
|
|
||||||
|
**Limitations:**
|
||||||
|
- Not suitable for large files (use git repo or external storage instead)
|
||||||
|
- Binary files not supported
|
||||||
|
- Changes require redeployment
|
||||||
|
|
||||||
|
**Real-World Examples:**
|
||||||
|
- `templates/compose/traccar.yaml` - XML configuration file
|
||||||
|
- `templates/compose/supabase.yaml` - Multiple config files
|
||||||
|
- `templates/compose/chaskiq.yaml` - Entrypoint script
|
||||||
|
|
||||||
|
**Implementation:**
|
||||||
|
- Parsed: `bootstrap/helpers/parsers.php` (line 717)
|
||||||
|
- Storage: `app/Models/LocalFileVolume.php`
|
||||||
|
- Validation: `tests/Unit/StripCoolifyCustomFieldsTest.php`
|
||||||
|
|
||||||
|
#### `is_directory` / `isDirectory`
|
||||||
|
|
||||||
|
**Type:** Boolean
|
||||||
|
**Default:** `true` (if neither `content` nor explicit flag provided)
|
||||||
|
**Purpose:** Indicate whether bind mount source should be created as directory or file
|
||||||
|
|
||||||
|
**Example Usage:**
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
app:
|
||||||
|
volumes:
|
||||||
|
# Explicit file
|
||||||
|
- type: bind
|
||||||
|
source: ./config.json
|
||||||
|
target: /app/config.json
|
||||||
|
is_directory: false # Create as file
|
||||||
|
|
||||||
|
# Explicit directory
|
||||||
|
- type: bind
|
||||||
|
source: ./logs
|
||||||
|
target: /var/log/app
|
||||||
|
is_directory: true # Create as directory
|
||||||
|
|
||||||
|
# Auto-detected as file (has content)
|
||||||
|
- type: bind
|
||||||
|
source: ./script.sh
|
||||||
|
target: /entrypoint.sh
|
||||||
|
content: |
|
||||||
|
#!/bin/sh
|
||||||
|
echo "Hello"
|
||||||
|
# is_directory: false implied by content presence
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior:**
|
||||||
|
- If `is_directory: true` → Creates directory with `mkdir -p`
|
||||||
|
- If `is_directory: false` → Creates empty file with `touch`
|
||||||
|
- If `content` provided → Implies `is_directory: false`
|
||||||
|
- If neither specified → Defaults to `true` (directory)
|
||||||
|
|
||||||
|
**Naming Conventions:**
|
||||||
|
- `is_directory` (snake_case) - **Preferred**, consistent with PHP/Laravel conventions
|
||||||
|
- `isDirectory` (camelCase) - **Legacy support**, both work identically
|
||||||
|
|
||||||
|
**Use Cases:**
|
||||||
|
- Disambiguating files vs directories when no content provided
|
||||||
|
- Ensuring correct bind mount type for Docker
|
||||||
|
- Pre-creating mount points before container starts
|
||||||
|
|
||||||
|
**Implementation:**
|
||||||
|
- Parsed: `bootstrap/helpers/parsers.php` (line 718)
|
||||||
|
- Storage: `app/Models/LocalFileVolume.php` (`is_directory` column)
|
||||||
|
- Validation: `tests/Unit/StripCoolifyCustomFieldsTest.php`
|
||||||
|
|
||||||
|
### Custom Field Stripping
|
||||||
|
|
||||||
|
**Function:** `stripCoolifyCustomFields()` in `bootstrap/helpers/docker.php`
|
||||||
|
|
||||||
|
All custom fields are removed before the compose file is sent to Docker. This happens in two contexts:
|
||||||
|
|
||||||
|
**1. Validation (User-Triggered)**
|
||||||
|
```php
|
||||||
|
// In validateComposeFile() - Edit Docker Compose modal
|
||||||
|
$yaml_compose = Yaml::parse($compose);
|
||||||
|
$yaml_compose = stripCoolifyCustomFields($yaml_compose); // Strip custom fields
|
||||||
|
// Send to docker compose config for validation
|
||||||
|
```
|
||||||
|
|
||||||
|
**2. Deployment (Automatic)**
|
||||||
|
```php
|
||||||
|
// In Service::parse() - During deployment
|
||||||
|
$docker_compose = parseCompose($docker_compose_raw);
|
||||||
|
// Custom fields are processed and then stripped
|
||||||
|
// Final compose sent to Docker has no custom fields
|
||||||
|
```
|
||||||
|
|
||||||
|
**What Gets Stripped:**
|
||||||
|
- Service-level: `exclude_from_hc`
|
||||||
|
- Volume-level: `content`, `isDirectory`, `is_directory`
|
||||||
|
|
||||||
|
**What's Preserved:**
|
||||||
|
- All standard Docker Compose fields
|
||||||
|
- Environment variables
|
||||||
|
- Standard volume definitions (after custom fields removed)
|
||||||
|
|
||||||
|
### Important Notes
|
||||||
|
|
||||||
|
#### Long vs Short Volume Syntax
|
||||||
|
|
||||||
|
**✅ Long Syntax (Works with Custom Fields):**
|
||||||
|
```yaml
|
||||||
|
volumes:
|
||||||
|
- type: bind
|
||||||
|
source: ./data
|
||||||
|
target: /app/data
|
||||||
|
content: "Hello" # ✅ Custom fields work here
|
||||||
|
```
|
||||||
|
|
||||||
|
**❌ Short Syntax (Custom Fields Ignored):**
|
||||||
|
```yaml
|
||||||
|
volumes:
|
||||||
|
- "./data:/app/data" # ❌ Cannot add custom fields to strings
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Docker Compose Compatibility
|
||||||
|
|
||||||
|
Custom fields are **Coolify-specific** and won't work with standalone `docker compose` CLI:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# ❌ Won't work - Docker doesn't recognize custom fields
|
||||||
|
docker compose -f compose.yaml up
|
||||||
|
|
||||||
|
# ✅ Works - Use Coolify's deployment (strips custom fields first)
|
||||||
|
# Deploy through Coolify UI or API
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Editing Custom Fields
|
||||||
|
|
||||||
|
When editing in "Edit Docker Compose" modal:
|
||||||
|
- Custom fields are preserved in the editor
|
||||||
|
- "Validate" button strips them temporarily for Docker validation
|
||||||
|
- "Save" button preserves them in `docker_compose_raw`
|
||||||
|
- They're processed again on next deployment
|
||||||
|
|
||||||
|
### Template Examples
|
||||||
|
|
||||||
|
See these templates for real-world usage:
|
||||||
|
|
||||||
|
**Service Exclusions:**
|
||||||
|
- `templates/compose/budibase.yaml` - Excludes watchtower from monitoring
|
||||||
|
- `templates/compose/pgbackweb.yaml` - Excludes backup service
|
||||||
|
- `templates/compose/elasticsearch-with-kibana.yaml` - Excludes elasticsearch
|
||||||
|
|
||||||
|
**Inline Content:**
|
||||||
|
- `templates/compose/traccar.yaml` - XML configuration (multiline)
|
||||||
|
- `templates/compose/supabase.yaml` - Multiple config files
|
||||||
|
- `templates/compose/searxng.yaml` - Settings file
|
||||||
|
- `templates/compose/invoice-ninja.yaml` - Nginx config
|
||||||
|
|
||||||
|
**Directory Flags:**
|
||||||
|
- `templates/compose/paperless.yaml` - Explicit directory creation
|
||||||
|
|
||||||
|
### Testing
|
||||||
|
|
||||||
|
**Unit Tests:**
|
||||||
|
- `tests/Unit/StripCoolifyCustomFieldsTest.php` - Custom field stripping logic
|
||||||
|
- `tests/Unit/ExcludeFromHealthCheckTest.php` - Health check exclusion behavior
|
||||||
|
- `tests/Unit/ContainerStatusAggregatorTest.php` - Status aggregation with exclusions
|
||||||
|
|
||||||
|
**Test Coverage:**
|
||||||
|
- ✅ All custom fields (exclude_from_hc, content, isDirectory, is_directory)
|
||||||
|
- ✅ Multiline content (YAML `|` syntax)
|
||||||
|
- ✅ Short vs long volume syntax
|
||||||
|
- ✅ Field stripping without data loss
|
||||||
|
- ✅ Standard Docker Compose field preservation
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,7 @@
|
||||||
namespace App\Services;
|
namespace App\Services;
|
||||||
|
|
||||||
use Illuminate\Support\Collection;
|
use Illuminate\Support\Collection;
|
||||||
|
use Illuminate\Support\Facades\Log;
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Container Status Aggregator Service
|
* Container Status Aggregator Service
|
||||||
|
|
@ -35,6 +36,21 @@ class ContainerStatusAggregator
|
||||||
*/
|
*/
|
||||||
public function aggregateFromStrings(Collection $containerStatuses, int $maxRestartCount = 0): string
|
public function aggregateFromStrings(Collection $containerStatuses, int $maxRestartCount = 0): string
|
||||||
{
|
{
|
||||||
|
// Validate maxRestartCount parameter
|
||||||
|
if ($maxRestartCount < 0) {
|
||||||
|
Log::warning('Negative maxRestartCount corrected to 0', [
|
||||||
|
'original_value' => $maxRestartCount,
|
||||||
|
]);
|
||||||
|
$maxRestartCount = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($maxRestartCount > 1000) {
|
||||||
|
Log::warning('High maxRestartCount detected', [
|
||||||
|
'maxRestartCount' => $maxRestartCount,
|
||||||
|
'containers' => $containerStatuses->count(),
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
if ($containerStatuses->isEmpty()) {
|
if ($containerStatuses->isEmpty()) {
|
||||||
return 'exited:unhealthy';
|
return 'exited:unhealthy';
|
||||||
}
|
}
|
||||||
|
|
@ -96,6 +112,21 @@ public function aggregateFromStrings(Collection $containerStatuses, int $maxRest
|
||||||
*/
|
*/
|
||||||
public function aggregateFromContainers(Collection $containers, int $maxRestartCount = 0): string
|
public function aggregateFromContainers(Collection $containers, int $maxRestartCount = 0): string
|
||||||
{
|
{
|
||||||
|
// Validate maxRestartCount parameter
|
||||||
|
if ($maxRestartCount < 0) {
|
||||||
|
Log::warning('Negative maxRestartCount corrected to 0', [
|
||||||
|
'original_value' => $maxRestartCount,
|
||||||
|
]);
|
||||||
|
$maxRestartCount = 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($maxRestartCount > 1000) {
|
||||||
|
Log::warning('High maxRestartCount detected', [
|
||||||
|
'maxRestartCount' => $maxRestartCount,
|
||||||
|
'containers' => $containers->count(),
|
||||||
|
]);
|
||||||
|
}
|
||||||
|
|
||||||
if ($containers->isEmpty()) {
|
if ($containers->isEmpty()) {
|
||||||
return 'exited:unhealthy';
|
return 'exited:unhealthy';
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -4,6 +4,8 @@
|
||||||
|
|
||||||
use App\Services\ContainerStatusAggregator;
|
use App\Services\ContainerStatusAggregator;
|
||||||
use Illuminate\Support\Collection;
|
use Illuminate\Support\Collection;
|
||||||
|
use Illuminate\Support\Facades\Log;
|
||||||
|
use Symfony\Component\Yaml\Exception\ParseException;
|
||||||
|
|
||||||
trait CalculatesExcludedStatus
|
trait CalculatesExcludedStatus
|
||||||
{
|
{
|
||||||
|
|
@ -111,8 +113,27 @@ protected function getExcludedContainersFromDockerCompose(?string $dockerCompose
|
||||||
|
|
||||||
try {
|
try {
|
||||||
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
$dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw);
|
||||||
|
|
||||||
|
// Validate structure
|
||||||
|
if (! is_array($dockerCompose)) {
|
||||||
|
Log::warning('Docker Compose YAML did not parse to array', [
|
||||||
|
'yaml_length' => strlen($dockerComposeRaw),
|
||||||
|
'parsed_type' => gettype($dockerCompose),
|
||||||
|
]);
|
||||||
|
|
||||||
|
return $excludedContainers;
|
||||||
|
}
|
||||||
|
|
||||||
$services = data_get($dockerCompose, 'services', []);
|
$services = data_get($dockerCompose, 'services', []);
|
||||||
|
|
||||||
|
if (! is_array($services)) {
|
||||||
|
Log::warning('Docker Compose services is not an array', [
|
||||||
|
'services_type' => gettype($services),
|
||||||
|
]);
|
||||||
|
|
||||||
|
return $excludedContainers;
|
||||||
|
}
|
||||||
|
|
||||||
foreach ($services as $serviceName => $serviceConfig) {
|
foreach ($services as $serviceName => $serviceConfig) {
|
||||||
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
$excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false);
|
||||||
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
$restartPolicy = data_get($serviceConfig, 'restart', 'always');
|
||||||
|
|
@ -121,8 +142,23 @@ protected function getExcludedContainersFromDockerCompose(?string $dockerCompose
|
||||||
$excludedContainers->push($serviceName);
|
$excludedContainers->push($serviceName);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
} catch (ParseException $e) {
|
||||||
|
// Specific YAML parsing errors
|
||||||
|
Log::warning('Failed to parse Docker Compose YAML for health check exclusions', [
|
||||||
|
'error' => $e->getMessage(),
|
||||||
|
'line' => $e->getParsedLine(),
|
||||||
|
'snippet' => $e->getSnippet(),
|
||||||
|
]);
|
||||||
|
|
||||||
|
return $excludedContainers;
|
||||||
} catch (\Exception $e) {
|
} catch (\Exception $e) {
|
||||||
// If we can't parse, treat all containers as included
|
// Unexpected errors
|
||||||
|
Log::error('Unexpected error parsing Docker Compose YAML', [
|
||||||
|
'error' => $e->getMessage(),
|
||||||
|
'trace' => $e->getTraceAsString(),
|
||||||
|
]);
|
||||||
|
|
||||||
|
return $excludedContainers;
|
||||||
}
|
}
|
||||||
|
|
||||||
return $excludedContainers;
|
return $excludedContainers;
|
||||||
|
|
|
||||||
|
|
@ -1083,6 +1083,44 @@ function generateCustomDockerRunOptionsForDatabases($docker_run_options, $docker
|
||||||
return $docker_compose;
|
return $docker_compose;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Remove Coolify's custom Docker Compose fields from parsed YAML array
|
||||||
|
*
|
||||||
|
* Coolify extends Docker Compose with custom fields that are processed during
|
||||||
|
* parsing and deployment but must be removed before sending to Docker.
|
||||||
|
*
|
||||||
|
* Custom fields:
|
||||||
|
* - exclude_from_hc (service-level): Exclude service from health check monitoring
|
||||||
|
* - content (volume-level): Auto-create file with specified content during init
|
||||||
|
* - isDirectory / is_directory (volume-level): Mark bind mount as directory
|
||||||
|
*
|
||||||
|
* @param array $yamlCompose Parsed Docker Compose array
|
||||||
|
* @return array Cleaned Docker Compose array with custom fields removed
|
||||||
|
*/
|
||||||
|
function stripCoolifyCustomFields(array $yamlCompose): array
|
||||||
|
{
|
||||||
|
foreach ($yamlCompose['services'] ?? [] as $serviceName => $service) {
|
||||||
|
// Remove service-level custom fields
|
||||||
|
unset($yamlCompose['services'][$serviceName]['exclude_from_hc']);
|
||||||
|
|
||||||
|
// Remove volume-level custom fields (only for long syntax - arrays)
|
||||||
|
if (isset($service['volumes'])) {
|
||||||
|
foreach ($service['volumes'] as $volumeName => $volume) {
|
||||||
|
// Skip if volume is string (short syntax like 'db-data:/var/lib/postgresql/data')
|
||||||
|
if (! is_array($volume)) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
unset($yamlCompose['services'][$serviceName]['volumes'][$volumeName]['content']);
|
||||||
|
unset($yamlCompose['services'][$serviceName]['volumes'][$volumeName]['isDirectory']);
|
||||||
|
unset($yamlCompose['services'][$serviceName]['volumes'][$volumeName]['is_directory']);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return $yamlCompose;
|
||||||
|
}
|
||||||
|
|
||||||
function validateComposeFile(string $compose, int $server_id): string|Throwable
|
function validateComposeFile(string $compose, int $server_id): string|Throwable
|
||||||
{
|
{
|
||||||
$uuid = Str::random(18);
|
$uuid = Str::random(18);
|
||||||
|
|
@ -1092,16 +1130,10 @@ function validateComposeFile(string $compose, int $server_id): string|Throwable
|
||||||
throw new \Exception('Server not found');
|
throw new \Exception('Server not found');
|
||||||
}
|
}
|
||||||
$yaml_compose = Yaml::parse($compose);
|
$yaml_compose = Yaml::parse($compose);
|
||||||
foreach ($yaml_compose['services'] as $service_name => $service) {
|
|
||||||
if (! isset($service['volumes'])) {
|
// Remove Coolify's custom fields before Docker validation
|
||||||
continue;
|
$yaml_compose = stripCoolifyCustomFields($yaml_compose);
|
||||||
}
|
|
||||||
foreach ($service['volumes'] as $volume_name => $volume) {
|
|
||||||
if (data_get($volume, 'type') === 'bind' && data_get($volume, 'content')) {
|
|
||||||
unset($yaml_compose['services'][$service_name]['volumes'][$volume_name]['content']);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
$base64_compose = base64_encode(Yaml::dump($yaml_compose));
|
$base64_compose = base64_encode(Yaml::dump($yaml_compose));
|
||||||
instant_remote_process([
|
instant_remote_process([
|
||||||
"echo {$base64_compose} | base64 -d | tee /tmp/{$uuid}.yml > /dev/null",
|
"echo {$base64_compose} | base64 -d | tee /tmp/{$uuid}.yml > /dev/null",
|
||||||
|
|
|
||||||
|
|
@ -1,6 +1,7 @@
|
||||||
<?php
|
<?php
|
||||||
|
|
||||||
use App\Services\ContainerStatusAggregator;
|
use App\Services\ContainerStatusAggregator;
|
||||||
|
use Illuminate\Support\Facades\Log;
|
||||||
|
|
||||||
beforeEach(function () {
|
beforeEach(function () {
|
||||||
$this->aggregator = new ContainerStatusAggregator;
|
$this->aggregator = new ContainerStatusAggregator;
|
||||||
|
|
@ -461,3 +462,79 @@
|
||||||
expect($result)->toBe('running:unknown');
|
expect($result)->toBe('running:unknown');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
describe('maxRestartCount validation', function () {
|
||||||
|
test('negative maxRestartCount is corrected to 0 in aggregateFromStrings', function () {
|
||||||
|
// Mock the Log facade to avoid "facade root not set" error in unit tests
|
||||||
|
Log::shouldReceive('warning')->once();
|
||||||
|
|
||||||
|
$statuses = collect(['exited']);
|
||||||
|
|
||||||
|
// With negative value, should be treated as 0 (no restarts)
|
||||||
|
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: -5);
|
||||||
|
|
||||||
|
// Should return exited:unhealthy (not degraded) since corrected to 0
|
||||||
|
expect($result)->toBe('exited:unhealthy');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('negative maxRestartCount is corrected to 0 in aggregateFromContainers', function () {
|
||||||
|
// Mock the Log facade to avoid "facade root not set" error in unit tests
|
||||||
|
Log::shouldReceive('warning')->once();
|
||||||
|
|
||||||
|
$containers = collect([
|
||||||
|
[
|
||||||
|
'State' => [
|
||||||
|
'Status' => 'exited',
|
||||||
|
'ExitCode' => 1,
|
||||||
|
],
|
||||||
|
],
|
||||||
|
]);
|
||||||
|
|
||||||
|
// With negative value, should be treated as 0 (no restarts)
|
||||||
|
$result = $this->aggregator->aggregateFromContainers($containers, maxRestartCount: -10);
|
||||||
|
|
||||||
|
// Should return exited:unhealthy (not degraded) since corrected to 0
|
||||||
|
expect($result)->toBe('exited:unhealthy');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('zero maxRestartCount works correctly', function () {
|
||||||
|
$statuses = collect(['exited']);
|
||||||
|
|
||||||
|
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 0);
|
||||||
|
|
||||||
|
// Zero is valid default - no crash loop detection
|
||||||
|
expect($result)->toBe('exited:unhealthy');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('positive maxRestartCount works correctly', function () {
|
||||||
|
$statuses = collect(['exited']);
|
||||||
|
|
||||||
|
$result = $this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 5);
|
||||||
|
|
||||||
|
// Positive value enables crash loop detection
|
||||||
|
expect($result)->toBe('degraded:unhealthy');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('crash loop detection still functions after validation', function () {
|
||||||
|
$statuses = collect(['exited']);
|
||||||
|
|
||||||
|
// Test with various positive restart counts
|
||||||
|
expect($this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 1))
|
||||||
|
->toBe('degraded:unhealthy');
|
||||||
|
|
||||||
|
expect($this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 100))
|
||||||
|
->toBe('degraded:unhealthy');
|
||||||
|
|
||||||
|
expect($this->aggregator->aggregateFromStrings($statuses, maxRestartCount: 999))
|
||||||
|
->toBe('degraded:unhealthy');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('default maxRestartCount parameter works', function () {
|
||||||
|
$statuses = collect(['exited']);
|
||||||
|
|
||||||
|
// Call without specifying maxRestartCount (should default to 0)
|
||||||
|
$result = $this->aggregator->aggregateFromStrings($statuses);
|
||||||
|
|
||||||
|
expect($result)->toBe('exited:unhealthy');
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
|
||||||
|
|
@ -104,3 +104,51 @@
|
||||||
->toContain('str($service->status)->contains(\'degraded\')')
|
->toContain('str($service->status)->contains(\'degraded\')')
|
||||||
->toContain('str($service->status)->contains(\'exited\')');
|
->toContain('str($service->status)->contains(\'exited\')');
|
||||||
});
|
});
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unit tests for YAML validation in CalculatesExcludedStatus trait
|
||||||
|
*/
|
||||||
|
it('ensures YAML validation has proper exception handling for parse errors', function () {
|
||||||
|
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||||
|
|
||||||
|
// Verify that ParseException is imported and caught separately from generic Exception
|
||||||
|
expect($traitFile)
|
||||||
|
->toContain('use Symfony\Component\Yaml\Exception\ParseException')
|
||||||
|
->toContain('use Illuminate\Support\Facades\Log')
|
||||||
|
->toContain('} catch (ParseException $e) {')
|
||||||
|
->toContain('} catch (\Exception $e) {');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ensures YAML validation logs parse errors with context', function () {
|
||||||
|
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||||
|
|
||||||
|
// Verify that parse errors are logged with useful context (error message, line, snippet)
|
||||||
|
expect($traitFile)
|
||||||
|
->toContain('Log::warning(\'Failed to parse Docker Compose YAML for health check exclusions\'')
|
||||||
|
->toContain('\'error\' => $e->getMessage()')
|
||||||
|
->toContain('\'line\' => $e->getParsedLine()')
|
||||||
|
->toContain('\'snippet\' => $e->getSnippet()');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ensures YAML validation logs unexpected errors', function () {
|
||||||
|
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||||
|
|
||||||
|
// Verify that unexpected errors are logged with error level
|
||||||
|
expect($traitFile)
|
||||||
|
->toContain('Log::error(\'Unexpected error parsing Docker Compose YAML\'')
|
||||||
|
->toContain('\'trace\' => $e->getTraceAsString()');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('ensures YAML validation checks structure after parsing', function () {
|
||||||
|
$traitFile = file_get_contents(__DIR__.'/../../app/Traits/CalculatesExcludedStatus.php');
|
||||||
|
|
||||||
|
// Verify that parsed result is validated to be an array
|
||||||
|
expect($traitFile)
|
||||||
|
->toContain('if (! is_array($dockerCompose)) {')
|
||||||
|
->toContain('Log::warning(\'Docker Compose YAML did not parse to array\'');
|
||||||
|
|
||||||
|
// Verify that services is validated to be an array
|
||||||
|
expect($traitFile)
|
||||||
|
->toContain('if (! is_array($services)) {')
|
||||||
|
->toContain('Log::warning(\'Docker Compose services is not an array\'');
|
||||||
|
});
|
||||||
|
|
|
||||||
229
tests/Unit/StripCoolifyCustomFieldsTest.php
Normal file
229
tests/Unit/StripCoolifyCustomFieldsTest.php
Normal file
|
|
@ -0,0 +1,229 @@
|
||||||
|
<?php
|
||||||
|
|
||||||
|
use function PHPUnit\Framework\assertEquals;
|
||||||
|
|
||||||
|
test('removes exclude_from_hc from service level', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'web' => [
|
||||||
|
'image' => 'nginx:latest',
|
||||||
|
'exclude_from_hc' => true,
|
||||||
|
'ports' => ['80:80'],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
assertEquals('nginx:latest', $result['services']['web']['image']);
|
||||||
|
assertEquals(['80:80'], $result['services']['web']['ports']);
|
||||||
|
expect($result['services']['web'])->not->toHaveKey('exclude_from_hc');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('removes content from volume level', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'app' => [
|
||||||
|
'image' => 'php:8.4',
|
||||||
|
'volumes' => [
|
||||||
|
[
|
||||||
|
'type' => 'bind',
|
||||||
|
'source' => './config.xml',
|
||||||
|
'target' => '/app/config.xml',
|
||||||
|
'content' => '<?xml version="1.0"?><config></config>',
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
expect($result['services']['app']['volumes'][0])->toHaveKeys(['type', 'source', 'target']);
|
||||||
|
expect($result['services']['app']['volumes'][0])->not->toHaveKey('content');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('removes isDirectory from volume level', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'app' => [
|
||||||
|
'image' => 'node:20',
|
||||||
|
'volumes' => [
|
||||||
|
[
|
||||||
|
'type' => 'bind',
|
||||||
|
'source' => './data',
|
||||||
|
'target' => '/app/data',
|
||||||
|
'isDirectory' => true,
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
expect($result['services']['app']['volumes'][0])->toHaveKeys(['type', 'source', 'target']);
|
||||||
|
expect($result['services']['app']['volumes'][0])->not->toHaveKey('isDirectory');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('removes is_directory from volume level', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'app' => [
|
||||||
|
'image' => 'python:3.12',
|
||||||
|
'volumes' => [
|
||||||
|
[
|
||||||
|
'type' => 'bind',
|
||||||
|
'source' => './logs',
|
||||||
|
'target' => '/var/log/app',
|
||||||
|
'is_directory' => true,
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
expect($result['services']['app']['volumes'][0])->toHaveKeys(['type', 'source', 'target']);
|
||||||
|
expect($result['services']['app']['volumes'][0])->not->toHaveKey('is_directory');
|
||||||
|
});
|
||||||
|
|
||||||
|
test('removes all custom fields together', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'web' => [
|
||||||
|
'image' => 'nginx:latest',
|
||||||
|
'exclude_from_hc' => true,
|
||||||
|
'volumes' => [
|
||||||
|
[
|
||||||
|
'type' => 'bind',
|
||||||
|
'source' => './config.xml',
|
||||||
|
'target' => '/etc/nginx/config.xml',
|
||||||
|
'content' => '<config></config>',
|
||||||
|
'isDirectory' => false,
|
||||||
|
],
|
||||||
|
[
|
||||||
|
'type' => 'bind',
|
||||||
|
'source' => './data',
|
||||||
|
'target' => '/var/www/data',
|
||||||
|
'is_directory' => true,
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
'worker' => [
|
||||||
|
'image' => 'worker:latest',
|
||||||
|
'exclude_from_hc' => true,
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
// Verify service-level custom fields removed
|
||||||
|
expect($result['services']['web'])->not->toHaveKey('exclude_from_hc');
|
||||||
|
expect($result['services']['worker'])->not->toHaveKey('exclude_from_hc');
|
||||||
|
|
||||||
|
// Verify volume-level custom fields removed
|
||||||
|
expect($result['services']['web']['volumes'][0])->not->toHaveKey('content');
|
||||||
|
expect($result['services']['web']['volumes'][0])->not->toHaveKey('isDirectory');
|
||||||
|
expect($result['services']['web']['volumes'][1])->not->toHaveKey('is_directory');
|
||||||
|
|
||||||
|
// Verify standard fields preserved
|
||||||
|
assertEquals('nginx:latest', $result['services']['web']['image']);
|
||||||
|
assertEquals('worker:latest', $result['services']['worker']['image']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('preserves standard Docker Compose fields', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'db' => [
|
||||||
|
'image' => 'postgres:16',
|
||||||
|
'environment' => [
|
||||||
|
'POSTGRES_DB' => 'mydb',
|
||||||
|
'POSTGRES_USER' => 'user',
|
||||||
|
],
|
||||||
|
'ports' => ['5432:5432'],
|
||||||
|
'volumes' => [
|
||||||
|
'db-data:/var/lib/postgresql/data',
|
||||||
|
],
|
||||||
|
'healthcheck' => [
|
||||||
|
'test' => ['CMD', 'pg_isready'],
|
||||||
|
'interval' => '5s',
|
||||||
|
],
|
||||||
|
'restart' => 'unless-stopped',
|
||||||
|
'networks' => ['backend'],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
'networks' => [
|
||||||
|
'backend' => [
|
||||||
|
'driver' => 'bridge',
|
||||||
|
],
|
||||||
|
],
|
||||||
|
'volumes' => [
|
||||||
|
'db-data' => null,
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
// All standard fields should be preserved
|
||||||
|
expect($result)->toHaveKeys(['services', 'networks', 'volumes']);
|
||||||
|
expect($result['services']['db'])->toHaveKeys([
|
||||||
|
'image', 'environment', 'ports', 'volumes',
|
||||||
|
'healthcheck', 'restart', 'networks',
|
||||||
|
]);
|
||||||
|
assertEquals('postgres:16', $result['services']['db']['image']);
|
||||||
|
assertEquals(['5432:5432'], $result['services']['db']['ports']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('handles missing services gracefully', function () {
|
||||||
|
$yaml = [
|
||||||
|
'version' => '3.8',
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
expect($result)->toBe($yaml);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('handles missing volumes in service gracefully', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'app' => [
|
||||||
|
'image' => 'nginx:latest',
|
||||||
|
'exclude_from_hc' => true,
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
expect($result['services']['app'])->not->toHaveKey('exclude_from_hc');
|
||||||
|
expect($result['services']['app'])->not->toHaveKey('volumes');
|
||||||
|
assertEquals('nginx:latest', $result['services']['app']['image']);
|
||||||
|
});
|
||||||
|
|
||||||
|
test('handles traccar.yaml example with multiline content', function () {
|
||||||
|
$yaml = [
|
||||||
|
'services' => [
|
||||||
|
'traccar' => [
|
||||||
|
'image' => 'traccar/traccar:latest',
|
||||||
|
'volumes' => [
|
||||||
|
[
|
||||||
|
'type' => 'bind',
|
||||||
|
'source' => './srv/traccar/conf/traccar.xml',
|
||||||
|
'target' => '/opt/traccar/conf/traccar.xml',
|
||||||
|
'content' => "<?xml version='1.0' encoding='UTF-8'?>\n<!DOCTYPE properties SYSTEM 'http://java.sun.com/dtd/properties.dtd'>\n<properties>\n <entry key='config.default'>./conf/default.xml</entry>\n</properties>",
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
],
|
||||||
|
];
|
||||||
|
|
||||||
|
$result = stripCoolifyCustomFields($yaml);
|
||||||
|
|
||||||
|
expect($result['services']['traccar']['volumes'][0])->toHaveKeys(['type', 'source', 'target']);
|
||||||
|
expect($result['services']['traccar']['volumes'][0])->not->toHaveKey('content');
|
||||||
|
assertEquals('./srv/traccar/conf/traccar.xml', $result['services']['traccar']['volumes'][0]['source']);
|
||||||
|
});
|
||||||
Loading…
Reference in a new issue