diff --git a/.claude/agents/project-manager-backlog.md b/.claude/agents/project-manager-backlog.md deleted file mode 100644 index 1cc6ad612..000000000 --- a/.claude/agents/project-manager-backlog.md +++ /dev/null @@ -1,193 +0,0 @@ ---- -name: project-manager-backlog -description: Use this agent when you need to manage project tasks using the backlog.md CLI tool. This includes creating new tasks, editing tasks, ensuring tasks follow the proper format and guidelines, breaking down large tasks into atomic units, and maintaining the project's task management workflow. Examples: Context: User wants to create a new task for adding a feature. user: "I need to add a new authentication system to the project" assistant: "I'll use the project-manager-backlog agent that will use backlog cli to create a properly structured task for this feature." Since the user needs to create a task for the project, use the Task tool to launch the project-manager-backlog agent to ensure the task follows backlog.md guidelines. Context: User has multiple related features to implement. user: "We need to implement user profiles, settings page, and notification preferences" assistant: "Let me use the project-manager-backlog agent to break these down into atomic, independent tasks." The user has a complex set of features that need to be broken down into proper atomic tasks following backlog.md structure. Context: User wants to review if their task description is properly formatted. user: "Can you check if this task follows our guidelines: 'task-123 - Implement user login'" assistant: "I'll use the project-manager-backlog agent to review this task against our backlog.md standards." The user needs task review, so use the project-manager-backlog agent to ensure compliance with project guidelines. -color: blue ---- - -You are an expert project manager specializing in the backlog.md task management system. You have deep expertise in creating well-structured, atomic, and testable tasks that follow software development best practices. - -## Backlog.md CLI Tool - -**IMPORTANT: Backlog.md uses standard CLI commands, NOT slash commands.** - -You use the `backlog` CLI tool to manage project tasks. This tool allows you to create, edit, and manage tasks in a structured way using Markdown files. You will never create tasks manually; instead, you will use the CLI commands to ensure all tasks are properly formatted and adhere to the project's guidelines. - -The backlog CLI is installed globally and available in the PATH. Here are the exact commands you should use: - -### Creating Tasks -```bash -backlog task create "Task title" -d "Description" --ac "First criteria,Second criteria" -l label1,label2 -``` - -### Editing Tasks -```bash -backlog task edit 123 -s "In Progress" -a @claude -``` - -### Listing Tasks -```bash -backlog task list --plain -``` - -**NEVER use slash commands like `/create-task` or `/edit`. These do not exist in Backlog.md.** -**ALWAYS use the standard CLI format: `backlog task create` (without any slash prefix).** - -### Example Usage - -When a user asks you to create a task, here's exactly what you should do: - -**User**: "Create a task to add user authentication" -**You should run**: -```bash -backlog task create "Add user authentication system" -d "Implement a secure authentication system to allow users to register and login" --ac "Users can register with email and password,Users can login with valid credentials,Invalid login attempts show appropriate error messages" -l authentication,backend -``` - -**NOT**: `/create-task "Add user authentication"` ❌ (This is wrong - slash commands don't exist) - -## Your Core Responsibilities - -1. **Task Creation**: You create tasks that strictly adhere to the backlog.md cli commands. Never create tasks manually. Use available task create parameters to ensure tasks are properly structured and follow the guidelines. -2. **Task Review**: You ensure all tasks meet the quality standards for atomicity, testability, and independence and task anatomy from below. -3. **Task Breakdown**: You expertly decompose large features into smaller, manageable tasks -4. **Context understanding**: You analyze user requests against the project codebase and existing tasks to ensure relevance and accuracy -5. **Handling ambiguity**: You clarify vague or ambiguous requests by asking targeted questions to the user to gather necessary details - -## Task Creation Guidelines - -### **Title (one liner)** - -Use a clear brief title that summarizes the task. - -### **Description**: (The **"why"**) - -Provide a concise summary of the task purpose and its goal. Do not add implementation details here. It -should explain the purpose, the scope and context of the task. Code snippets should be avoided. - -### **Acceptance Criteria**: (The **"what"**) - -List specific, measurable outcomes that define what means to reach the goal from the description. Use checkboxes (`- [ ]`) for tracking. -When defining `## Acceptance Criteria` for a task, focus on **outcomes, behaviors, and verifiable requirements** rather -than step-by-step implementation details. -Acceptance Criteria (AC) define *what* conditions must be met for the task to be considered complete. -They should be testable and confirm that the core purpose of the task is achieved. -**Key Principles for Good ACs:** - -- **Outcome-Oriented:** Focus on the result, not the method. -- **Testable/Verifiable:** Each criterion should be something that can be objectively tested or verified. -- **Clear and Concise:** Unambiguous language. -- **Complete:** Collectively, ACs should cover the scope of the task. -- **User-Focused (where applicable):** Frame ACs from the perspective of the end-user or the system's external behavior. - - - *Good Example:* "- [ ] User can successfully log in with valid credentials." - - *Good Example:* "- [ ] System processes 1000 requests per second without errors." - - *Bad Example (Implementation Step):* "- [ ] Add a new function `handleLogin()` in `auth.ts`." - -### Task file - -Once a task is created using backlog cli, it will be stored in `backlog/tasks/` directory as a Markdown file with the format -`task- - .md` (e.g. `task-42 - Add GraphQL resolver.md`). - -## Task Breakdown Strategy - -When breaking down features: -1. Identify the foundational components first -2. Create tasks in dependency order (foundations before features) -3. Ensure each task delivers value independently -4. Avoid creating tasks that block each other - -### Additional task requirements - -- Tasks must be **atomic** and **testable**. If a task is too large, break it down into smaller subtasks. - Each task should represent a single unit of work that can be completed in a single PR. - -- **Never** reference tasks that are to be done in the future or that are not yet created. You can only reference - previous tasks (id < current task id). - -- When creating multiple tasks, ensure they are **independent** and they do not depend on future tasks. - Example of correct tasks splitting: task 1: "Add system for handling API requests", task 2: "Add user model and DB - schema", task 3: "Add API endpoint for user data". - Example of wrong tasks splitting: task 1: "Add API endpoint for user data", task 2: "Define the user model and DB - schema". - -## Recommended Task Anatomy - -```markdown -# task‑42 - Add GraphQL resolver - -## Description (the why) - -Short, imperative explanation of the goal of the task and why it is needed. - -## Acceptance Criteria (the what) - -- [ ] Resolver returns correct data for happy path -- [ ] Error response matches REST -- [ ] P95 latency ≤ 50 ms under 100 RPS - -## Implementation Plan (the how) (added after putting the task in progress but before implementing any code change) - -1. Research existing GraphQL resolver patterns -2. Implement basic resolver with error handling -3. Add performance monitoring -4. Write unit and integration tests -5. Benchmark performance under load - -## Implementation Notes (for reviewers) (only added after finishing the code implementation of a task) - -- Approach taken -- Features implemented or modified -- Technical decisions and trade-offs -- Modified or added files -``` - -## Quality Checks - -Before finalizing any task creation, verify: -- [ ] Title is clear and brief -- [ ] Description explains WHY without HOW -- [ ] Each AC is outcome-focused and testable -- [ ] Task is atomic (single PR scope) -- [ ] No dependencies on future tasks - -You are meticulous about these standards and will guide users to create high-quality tasks that enhance project productivity and maintainability. - -## Self reflection -When creating a task, always think from the perspective of an AI Agent that will have to work with this task in the future. -Ensure that the task is structured in a way that it can be easily understood and processed by AI coding agents. - -## Handy CLI Commands - -| Action | Example | -|-------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------| -| Create task | `backlog task create "Add OAuth System"` | -| Create with description | `backlog task create "Feature" -d "Add authentication system"` | -| Create with assignee | `backlog task create "Feature" -a @sara` | -| Create with status | `backlog task create "Feature" -s "In Progress"` | -| Create with labels | `backlog task create "Feature" -l auth,backend` | -| Create with priority | `backlog task create "Feature" --priority high` | -| Create with plan | `backlog task create "Feature" --plan "1. Research\n2. Implement"` | -| Create with AC | `backlog task create "Feature" --ac "Must work,Must be tested"` | -| Create with notes | `backlog task create "Feature" --notes "Started initial research"` | -| Create with deps | `backlog task create "Feature" --dep task-1,task-2` | -| Create sub task | `backlog task create -p 14 "Add Login with Google"` | -| Create (all options) | `backlog task create "Feature" -d "Description" -a @sara -s "To Do" -l auth --priority high --ac "Must work" --notes "Initial setup done" --dep task-1 -p 14` | -| List tasks | `backlog task list [-s <status>] [-a <assignee>] [-p <parent>]` | -| List by parent | `backlog task list --parent 42` or `backlog task list -p task-42` | -| View detail | `backlog task 7` (interactive UI, press 'E' to edit in editor) | -| View (AI mode) | `backlog task 7 --plain` | -| Edit | `backlog task edit 7 -a @sara -l auth,backend` | -| Add plan | `backlog task edit 7 --plan "Implementation approach"` | -| Add AC | `backlog task edit 7 --ac "New criterion,Another one"` | -| Add notes | `backlog task edit 7 --notes "Completed X, working on Y"` | -| Add deps | `backlog task edit 7 --dep task-1 --dep task-2` | -| Archive | `backlog task archive 7` | -| Create draft | `backlog task create "Feature" --draft` | -| Draft flow | `backlog draft create "Spike GraphQL"` → `backlog draft promote 3.1` | -| Demote to draft | `backlog task demote <id>` | - -Full help: `backlog --help` - -## Tips for AI Agents - -- **Always use `--plain` flag** when listing or viewing tasks for AI-friendly text output instead of using Backlog.md - interactive UI. diff --git a/.cursor/mcp.json b/.cursor/mcp.json new file mode 100644 index 000000000..8c6715a15 --- /dev/null +++ b/.cursor/mcp.json @@ -0,0 +1,11 @@ +{ + "mcpServers": { + "laravel-boost": { + "command": "php", + "args": [ + "artisan", + "boost:mcp" + ] + } + } +} \ No newline at end of file diff --git a/.cursor/rules/backlog-guildlines.md b/.cursor/rules/backlog-guildlines.md deleted file mode 100644 index ea95eb0b5..000000000 --- a/.cursor/rules/backlog-guildlines.md +++ /dev/null @@ -1,398 +0,0 @@ - -# === BACKLOG.MD GUIDELINES START === -# Instructions for the usage of Backlog.md CLI Tool - -## What is Backlog.md? - -**Backlog.md is the complete project management system for this codebase.** It provides everything needed to manage tasks, track progress, and collaborate on development - all through a powerful CLI that operates on markdown files. - -### Core Capabilities - -✅ **Task Management**: Create, edit, assign, prioritize, and track tasks with full metadata -✅ **Acceptance Criteria**: Granular control with add/remove/check/uncheck by index -✅ **Board Visualization**: Terminal-based Kanban board (`backlog board`) and web UI (`backlog browser`) -✅ **Git Integration**: Automatic tracking of task states across branches -✅ **Dependencies**: Task relationships and subtask hierarchies -✅ **Documentation & Decisions**: Structured docs and architectural decision records -✅ **Export & Reporting**: Generate markdown reports and board snapshots -✅ **AI-Optimized**: `--plain` flag provides clean text output for AI processing - -### Why This Matters to You (AI Agent) - -1. **Comprehensive system** - Full project management capabilities through CLI -2. **The CLI is the interface** - All operations go through `backlog` commands -3. **Unified interaction model** - You can use CLI for both reading (`backlog task 1 --plain`) and writing (`backlog task edit 1`) -4. **Metadata stays synchronized** - The CLI handles all the complex relationships - -### Key Understanding - -- **Tasks** live in `backlog/tasks/` as `task-<id> - <title>.md` files -- **You interact via CLI only**: `backlog task create`, `backlog task edit`, etc. -- **Use `--plain` flag** for AI-friendly output when viewing/listing -- **Never bypass the CLI** - It handles Git, metadata, file naming, and relationships - ---- - -# ⚠️ CRITICAL: NEVER EDIT TASK FILES DIRECTLY - -**ALL task operations MUST use the Backlog.md CLI commands** -- ✅ **DO**: Use `backlog task edit` and other CLI commands -- ✅ **DO**: Use `backlog task create` to create new tasks -- ✅ **DO**: Use `backlog task edit <id> --check-ac <index>` to mark acceptance criteria -- ❌ **DON'T**: Edit markdown files directly -- ❌ **DON'T**: Manually change checkboxes in files -- ❌ **DON'T**: Add or modify text in task files without using CLI - -**Why?** Direct file editing breaks metadata synchronization, Git tracking, and task relationships. - ---- - -## 1. Source of Truth & File Structure - -### 📖 **UNDERSTANDING** (What you'll see when reading) -- Markdown task files live under **`backlog/tasks/`** (drafts under **`backlog/drafts/`**) -- Files are named: `task-<id> - <title>.md` (e.g., `task-42 - Add GraphQL resolver.md`) -- Project documentation is in **`backlog/docs/`** -- Project decisions are in **`backlog/decisions/`** - -### 🔧 **ACTING** (How to change things) -- **All task operations MUST use the Backlog.md CLI tool** -- This ensures metadata is correctly updated and the project stays in sync -- **Always use `--plain` flag** when listing or viewing tasks for AI-friendly text output - ---- - -## 2. Common Mistakes to Avoid - -### ❌ **WRONG: Direct File Editing** -```markdown -# DON'T DO THIS: -1. Open backlog/tasks/task-7 - Feature.md in editor -2. Change "- [ ]" to "- [x]" manually -3. Add notes directly to the file -4. Save the file -``` - -### ✅ **CORRECT: Using CLI Commands** -```bash -# DO THIS INSTEAD: -backlog task edit 7 --check-ac 1 # Mark AC #1 as complete -backlog task edit 7 --notes "Implementation complete" # Add notes -backlog task edit 7 -s "In Progress" -a @agent-k # Multiple commands: change status and assign the task -``` - ---- - -## 3. Understanding Task Format (Read-Only Reference) - -⚠️ **FORMAT REFERENCE ONLY** - The following sections show what you'll SEE in task files. -**Never edit these directly! Use CLI commands to make changes.** - -### Task Structure You'll See - -```markdown ---- -id: task-42 -title: Add GraphQL resolver -status: To Do -assignee: [@sara] -labels: [backend, api] ---- - -## Description -Brief explanation of the task purpose. - -## Acceptance Criteria -<!-- AC:BEGIN --> -- [ ] #1 First criterion -- [x] #2 Second criterion (completed) -- [ ] #3 Third criterion -<!-- AC:END --> - -## Implementation Plan -1. Research approach -2. Implement solution - -## Implementation Notes -Summary of what was done. -``` - -### How to Modify Each Section - -| What You Want to Change | CLI Command to Use | -|------------------------|-------------------| -| Title | `backlog task edit 42 -t "New Title"` | -| Status | `backlog task edit 42 -s "In Progress"` | -| Assignee | `backlog task edit 42 -a @sara` | -| Labels | `backlog task edit 42 -l backend,api` | -| Description | `backlog task edit 42 -d "New description"` | -| Add AC | `backlog task edit 42 --ac "New criterion"` | -| Check AC #1 | `backlog task edit 42 --check-ac 1` | -| Uncheck AC #2 | `backlog task edit 42 --uncheck-ac 2` | -| Remove AC #3 | `backlog task edit 42 --remove-ac 3` | -| Add Plan | `backlog task edit 42 --plan "1. Step one\n2. Step two"` | -| Add Notes | `backlog task edit 42 --notes "What I did"` | - ---- - -## 4. Defining Tasks - -### Creating New Tasks - -**Always use CLI to create tasks:** -```bash -backlog task create "Task title" -d "Description" --ac "First criterion" --ac "Second criterion" -``` - -### Title (one liner) -Use a clear brief title that summarizes the task. - -### Description (The "why") -Provide a concise summary of the task purpose and its goal. Explains the context without implementation details. - -### Acceptance Criteria (The "what") - -**Understanding the Format:** -- Acceptance criteria appear as numbered checkboxes in the markdown files -- Format: `- [ ] #1 Criterion text` (unchecked) or `- [x] #1 Criterion text` (checked) - -**Managing Acceptance Criteria via CLI:** - -⚠️ **IMPORTANT: How AC Commands Work** -- **Adding criteria (`--ac`)** accepts multiple flags: `--ac "First" --ac "Second"` ✅ -- **Checking/unchecking/removing** accept multiple flags too: `--check-ac 1 --check-ac 2` ✅ -- **Mixed operations** work in a single command: `--check-ac 1 --uncheck-ac 2 --remove-ac 3` ✅ - -```bash -# Add new criteria (MULTIPLE values allowed) -backlog task edit 42 --ac "User can login" --ac "Session persists" - -# Check specific criteria by index (MULTIPLE values supported) -backlog task edit 42 --check-ac 1 --check-ac 2 --check-ac 3 # Check multiple ACs -# Or check them individually if you prefer: -backlog task edit 42 --check-ac 1 # Mark #1 as complete -backlog task edit 42 --check-ac 2 # Mark #2 as complete - -# Mixed operations in single command -backlog task edit 42 --check-ac 1 --uncheck-ac 2 --remove-ac 3 - -# ❌ STILL WRONG - These formats don't work: -# backlog task edit 42 --check-ac 1,2,3 # No comma-separated values -# backlog task edit 42 --check-ac 1-3 # No ranges -# backlog task edit 42 --check 1 # Wrong flag name - -# Multiple operations of same type -backlog task edit 42 --uncheck-ac 1 --uncheck-ac 2 # Uncheck multiple ACs -backlog task edit 42 --remove-ac 2 --remove-ac 4 # Remove multiple ACs (processed high-to-low) -``` - -**Key Principles for Good ACs:** -- **Outcome-Oriented:** Focus on the result, not the method -- **Testable/Verifiable:** Each criterion should be objectively testable -- **Clear and Concise:** Unambiguous language -- **Complete:** Collectively cover the task scope -- **User-Focused:** Frame from end-user or system behavior perspective - -Good Examples: -- "User can successfully log in with valid credentials" -- "System processes 1000 requests per second without errors" - -Bad Example (Implementation Step): -- "Add a new function handleLogin() in auth.ts" - -### Task Breakdown Strategy - -1. Identify foundational components first -2. Create tasks in dependency order (foundations before features) -3. Ensure each task delivers value independently -4. Avoid creating tasks that block each other - -### Task Requirements - -- Tasks must be **atomic** and **testable** or **verifiable** -- Each task should represent a single unit of work for one PR -- **Never** reference future tasks (only tasks with id < current task id) -- Ensure tasks are **independent** and don't depend on future work - ---- - -## 5. Implementing Tasks - -### Implementation Plan (The "how") (only after starting work) -```bash -backlog task edit 42 -s "In Progress" -a @{myself} -backlog task edit 42 --plan "1. Research patterns\n2. Implement\n3. Test" -``` - -### Implementation Notes (Imagine you need to copy paste this into a PR description) -```bash -backlog task edit 42 --notes "Implemented using pattern X, modified files Y and Z" -``` - -**IMPORTANT**: Do NOT include an Implementation Plan when creating a task. The plan is added only after you start implementation. -- Creation phase: provide Title, Description, Acceptance Criteria, and optionally labels/priority/assignee. -- When you begin work, switch to edit and add the plan: `backlog task edit <id> --plan "..."`. -- Add Implementation Notes only after completing the work: `backlog task edit <id> --notes "..."`. - -Phase discipline: What goes where -- Creation: Title, Description, Acceptance Criteria, labels/priority/assignee. -- Implementation: Implementation Plan (after moving to In Progress). -- Wrap-up: Implementation Notes, AC and Definition of Done checks. - -**IMPORTANT**: Only implement what's in the Acceptance Criteria. If you need to do more, either: -1. Update the AC first: `backlog task edit 42 --ac "New requirement"` -2. Or create a new task: `backlog task create "Additional feature"` - ---- - -## 6. Typical Workflow - -```bash -# 1. Identify work -backlog task list -s "To Do" --plain - -# 2. Read task details -backlog task 42 --plain - -# 3. Start work: assign yourself & change status -backlog task edit 42 -a @myself -s "In Progress" - -# 4. Add implementation plan -backlog task edit 42 --plan "1. Analyze\n2. Refactor\n3. Test" - -# 5. Work on the task (write code, test, etc.) - -# 6. Mark acceptance criteria as complete (supports multiple in one command) -backlog task edit 42 --check-ac 1 --check-ac 2 --check-ac 3 # Check all at once -# Or check them individually if preferred: -# backlog task edit 42 --check-ac 1 -# backlog task edit 42 --check-ac 2 -# backlog task edit 42 --check-ac 3 - -# 7. Add implementation notes -backlog task edit 42 --notes "Refactored using strategy pattern, updated tests" - -# 8. Mark task as done -backlog task edit 42 -s Done -``` - ---- - -## 7. Definition of Done (DoD) - -A task is **Done** only when **ALL** of the following are complete: - -### ✅ Via CLI Commands: -1. **All acceptance criteria checked**: Use `backlog task edit <id> --check-ac <index>` for each -2. **Implementation notes added**: Use `backlog task edit <id> --notes "..."` -3. **Status set to Done**: Use `backlog task edit <id> -s Done` - -### ✅ Via Code/Testing: -4. **Tests pass**: Run test suite and linting -5. **Documentation updated**: Update relevant docs if needed -6. **Code reviewed**: Self-review your changes -7. **No regressions**: Performance, security checks pass - -⚠️ **NEVER mark a task as Done without completing ALL items above** - ---- - -## 8. Quick Reference: DO vs DON'T - -### Viewing Tasks -| Task | ✅ DO | ❌ DON'T | -|------|-------|----------| -| View task | `backlog task 42 --plain` | Open and read .md file directly | -| List tasks | `backlog task list --plain` | Browse backlog/tasks folder | -| Check status | `backlog task 42 --plain` | Look at file content | - -### Modifying Tasks -| Task | ✅ DO | ❌ DON'T | -|------|-------|----------| -| Check AC | `backlog task edit 42 --check-ac 1` | Change `- [ ]` to `- [x]` in file | -| Add notes | `backlog task edit 42 --notes "..."` | Type notes into .md file | -| Change status | `backlog task edit 42 -s Done` | Edit status in frontmatter | -| Add AC | `backlog task edit 42 --ac "New"` | Add `- [ ] New` to file | - ---- - -## 9. Complete CLI Command Reference - -### Task Creation -| Action | Command | -|--------|---------| -| Create task | `backlog task create "Title"` | -| With description | `backlog task create "Title" -d "Description"` | -| With AC | `backlog task create "Title" --ac "Criterion 1" --ac "Criterion 2"` | -| With all options | `backlog task create "Title" -d "Desc" -a @sara -s "To Do" -l auth --priority high` | -| Create draft | `backlog task create "Title" --draft` | -| Create subtask | `backlog task create "Title" -p 42` | - -### Task Modification -| Action | Command | -|--------|---------| -| Edit title | `backlog task edit 42 -t "New Title"` | -| Edit description | `backlog task edit 42 -d "New description"` | -| Change status | `backlog task edit 42 -s "In Progress"` | -| Assign | `backlog task edit 42 -a @sara` | -| Add labels | `backlog task edit 42 -l backend,api` | -| Set priority | `backlog task edit 42 --priority high` | - -### Acceptance Criteria Management -| Action | Command | -|--------|---------| -| Add AC | `backlog task edit 42 --ac "New criterion" --ac "Another"` | -| Remove AC #2 | `backlog task edit 42 --remove-ac 2` | -| Remove multiple ACs | `backlog task edit 42 --remove-ac 2 --remove-ac 4` | -| Check AC #1 | `backlog task edit 42 --check-ac 1` | -| Check multiple ACs | `backlog task edit 42 --check-ac 1 --check-ac 3` | -| Uncheck AC #3 | `backlog task edit 42 --uncheck-ac 3` | -| Mixed operations | `backlog task edit 42 --check-ac 1 --uncheck-ac 2 --remove-ac 3 --ac "New"` | - -### Task Content -| Action | Command | -|--------|---------| -| Add plan | `backlog task edit 42 --plan "1. Step one\n2. Step two"` | -| Add notes | `backlog task edit 42 --notes "Implementation details"` | -| Add dependencies | `backlog task edit 42 --dep task-1 --dep task-2` | - -### Task Operations -| Action | Command | -|--------|---------| -| View task | `backlog task 42 --plain` | -| List tasks | `backlog task list --plain` | -| Filter by status | `backlog task list -s "In Progress" --plain` | -| Filter by assignee | `backlog task list -a @sara --plain` | -| Archive task | `backlog task archive 42` | -| Demote to draft | `backlog task demote 42` | - ---- - -## 10. Troubleshooting - -### If You Accidentally Edited a File Directly - -1. **DON'T PANIC** - But don't save or commit -2. Revert the changes -3. Make changes properly via CLI -4. If already saved, the metadata might be out of sync - use `backlog task edit` to fix - -### Common Issues - -| Problem | Solution | -|---------|----------| -| "Task not found" | Check task ID with `backlog task list --plain` | -| AC won't check | Use correct index: `backlog task 42 --plain` to see AC numbers | -| Changes not saving | Ensure you're using CLI, not editing files | -| Metadata out of sync | Re-edit via CLI to fix: `backlog task edit 42 -s <current-status>` | - ---- - -## Remember: The Golden Rule - -**🎯 If you want to change ANYTHING in a task, use the `backlog task edit` command.** -**📖 Only READ task files directly, never WRITE to them.** - -Full help available: `backlog --help` - -# === BACKLOG.MD GUIDELINES END === diff --git a/.cursor/rules/laravel-boost.mdc b/.cursor/rules/laravel-boost.mdc new file mode 100644 index 000000000..005ede849 --- /dev/null +++ b/.cursor/rules/laravel-boost.mdc @@ -0,0 +1,405 @@ +--- +alwaysApply: true +--- +<laravel-boost-guidelines> +=== foundation rules === + +# Laravel Boost Guidelines + +The Laravel Boost guidelines are specifically curated by Laravel maintainers for this application. These guidelines should be followed closely to enhance the user's satisfaction building Laravel applications. + +## Foundational Context +This application is a Laravel application and its main Laravel ecosystems package & versions are below. You are an expert with them all. Ensure you abide by these specific packages & versions. + +- php - 8.4.7 +- laravel/fortify (FORTIFY) - v1 +- laravel/framework (LARAVEL) - v12 +- laravel/horizon (HORIZON) - v5 +- laravel/prompts (PROMPTS) - v0 +- laravel/sanctum (SANCTUM) - v4 +- laravel/socialite (SOCIALITE) - v5 +- livewire/livewire (LIVEWIRE) - v3 +- laravel/dusk (DUSK) - v8 +- laravel/pint (PINT) - v1 +- laravel/telescope (TELESCOPE) - v5 +- pestphp/pest (PEST) - v3 +- phpunit/phpunit (PHPUNIT) - v11 +- rector/rector (RECTOR) - v2 +- laravel-echo (ECHO) - v2 +- tailwindcss (TAILWINDCSS) - v4 +- vue (VUE) - v3 + + +## Conventions +- You must follow all existing code conventions used in this application. When creating or editing a file, check sibling files for the correct structure, approach, naming. +- Use descriptive names for variables and methods. For example, `isRegisteredForDiscounts`, not `discount()`. +- Check for existing components to reuse before writing a new one. + +## Verification Scripts +- Do not create verification scripts or tinker when tests cover that functionality and prove it works. Unit and feature tests are more important. + +## Application Structure & Architecture +- Stick to existing directory structure - don't create new base folders without approval. +- Do not change the application's dependencies without approval. + +## Frontend Bundling +- If the user doesn't see a frontend change reflected in the UI, it could mean they need to run `npm run build`, `npm run dev`, or `composer run dev`. Ask them. + +## Replies +- Be concise in your explanations - focus on what's important rather than explaining obvious details. + +## Documentation Files +- You must only create documentation files if explicitly requested by the user. + + +=== boost rules === + +## Laravel Boost +- Laravel Boost is an MCP server that comes with powerful tools designed specifically for this application. Use them. + +## Artisan +- Use the `list-artisan-commands` tool when you need to call an Artisan command to double check the available parameters. + +## URLs +- Whenever you share a project URL with the user you should use the `get-absolute-url` tool to ensure you're using the correct scheme, domain / IP, and port. + +## Tinker / Debugging +- You should use the `tinker` tool when you need to execute PHP to debug code or query Eloquent models directly. +- Use the `database-query` tool when you only need to read from the database. + +## Reading Browser Logs With the `browser-logs` Tool +- You can read browser logs, errors, and exceptions using the `browser-logs` tool from Boost. +- Only recent browser logs will be useful - ignore old logs. + +## Searching Documentation (Critically Important) +- Boost comes with a powerful `search-docs` tool you should use before any other approaches. This tool automatically passes a list of installed packages and their versions to the remote Boost API, so it returns only version-specific documentation specific for the user's circumstance. You should pass an array of packages to filter on if you know you need docs for particular packages. +- The 'search-docs' tool is perfect for all Laravel related packages, including Laravel, Inertia, Livewire, Filament, Tailwind, Pest, Nova, Nightwatch, etc. +- You must use this tool to search for Laravel-ecosystem documentation before falling back to other approaches. +- Search the documentation before making code changes to ensure we are taking the correct approach. +- Use multiple, broad, simple, topic based queries to start. For example: `['rate limiting', 'routing rate limiting', 'routing']`. +- Do not add package names to queries - package information is already shared. For example, use `test resource table`, not `filament 4 test resource table`. + +### Available Search Syntax +- You can and should pass multiple queries at once. The most relevant results will be returned first. + +1. Simple Word Searches with auto-stemming - query=authentication - finds 'authenticate' and 'auth' +2. Multiple Words (AND Logic) - query=rate limit - finds knowledge containing both "rate" AND "limit" +3. Quoted Phrases (Exact Position) - query="infinite scroll" - Words must be adjacent and in that order +4. Mixed Queries - query=middleware "rate limit" - "middleware" AND exact phrase "rate limit" +5. Multiple Queries - queries=["authentication", "middleware"] - ANY of these terms + + +=== php rules === + +## PHP + +- Always use curly braces for control structures, even if it has one line. + +### Constructors +- Use PHP 8 constructor property promotion in `__construct()`. + - <code-snippet>public function __construct(public GitHub $github) { }</code-snippet> +- Do not allow empty `__construct()` methods with zero parameters. + +### Type Declarations +- Always use explicit return type declarations for methods and functions. +- Use appropriate PHP type hints for method parameters. + +<code-snippet name="Explicit Return Types and Method Params" lang="php"> +protected function isAccessible(User $user, ?string $path = null): bool +{ + ... +} +</code-snippet> + +## Comments +- Prefer PHPDoc blocks over comments. Never use comments within the code itself unless there is something _very_ complex going on. + +## PHPDoc Blocks +- Add useful array shape type definitions for arrays when appropriate. + +## Enums +- Typically, keys in an Enum should be TitleCase. For example: `FavoritePerson`, `BestLake`, `Monthly`. + + +=== laravel/core rules === + +## Do Things the Laravel Way + +- Use `php artisan make:` commands to create new files (i.e. migrations, controllers, models, etc.). You can list available Artisan commands using the `list-artisan-commands` tool. +- If you're creating a generic PHP class, use `artisan make:class`. +- Pass `--no-interaction` to all Artisan commands to ensure they work without user input. You should also pass the correct `--options` to ensure correct behavior. + +### Database +- Always use proper Eloquent relationship methods with return type hints. Prefer relationship methods over raw queries or manual joins. +- Use Eloquent models and relationships before suggesting raw database queries +- Avoid `DB::`; prefer `Model::query()`. Generate code that leverages Laravel's ORM capabilities rather than bypassing them. +- Generate code that prevents N+1 query problems by using eager loading. +- Use Laravel's query builder for very complex database operations. + +### Model Creation +- When creating new models, create useful factories and seeders for them too. Ask the user if they need any other things, using `list-artisan-commands` to check the available options to `php artisan make:model`. + +### APIs & Eloquent Resources +- For APIs, default to using Eloquent API Resources and API versioning unless existing API routes do not, then you should follow existing application convention. + +### Controllers & Validation +- Always create Form Request classes for validation rather than inline validation in controllers. Include both validation rules and custom error messages. +- Check sibling Form Requests to see if the application uses array or string based validation rules. + +### Queues +- Use queued jobs for time-consuming operations with the `ShouldQueue` interface. + +### Authentication & Authorization +- Use Laravel's built-in authentication and authorization features (gates, policies, Sanctum, etc.). + +### URL Generation +- When generating links to other pages, prefer named routes and the `route()` function. + +### Configuration +- Use environment variables only in configuration files - never use the `env()` function directly outside of config files. Always use `config('app.name')`, not `env('APP_NAME')`. + +### Testing +- When creating models for tests, use the factories for the models. Check if the factory has custom states that can be used before manually setting up the model. +- Faker: Use methods such as `$this->faker->word()` or `fake()->randomDigit()`. Follow existing conventions whether to use `$this->faker` or `fake()`. +- When creating tests, make use of `php artisan make:test [options] <name>` to create a feature test, and pass `--unit` to create a unit test. Most tests should be feature tests. + +### Vite Error +- If you receive an "Illuminate\Foundation\ViteException: Unable to locate file in Vite manifest" error, you can run `npm run build` or ask the user to run `npm run dev` or `composer run dev`. + + +=== laravel/v12 rules === + +## Laravel 12 + +- Use the `search-docs` tool to get version specific documentation. +- This project upgraded from Laravel 10 without migrating to the new streamlined Laravel file structure. +- This is **perfectly fine** and recommended by Laravel. Follow the existing structure from Laravel 10. We do not to need migrate to the new Laravel structure unless the user explicitly requests that. + +### Laravel 10 Structure +- Middleware typically lives in `app/Http/Middleware/` and service providers in `app/Providers/`. +- There is no `bootstrap/app.php` application configuration in a Laravel 10 structure: + - Middleware registration happens in `app/Http/Kernel.php` + - Exception handling is in `app/Exceptions/Handler.php` + - Console commands and schedule register in `app/Console/Kernel.php` + - Rate limits likely exist in `RouteServiceProvider` or `app/Http/Kernel.php` + +### Database +- When modifying a column, the migration must include all of the attributes that were previously defined on the column. Otherwise, they will be dropped and lost. +- Laravel 11 allows limiting eagerly loaded records natively, without external packages: `$query->latest()->limit(10);`. + +### Models +- Casts can and likely should be set in a `casts()` method on a model rather than the `$casts` property. Follow existing conventions from other models. + + +=== livewire/core rules === + +## Livewire Core +- Use the `search-docs` tool to find exact version specific documentation for how to write Livewire & Livewire tests. +- Use the `php artisan make:livewire [Posts\\CreatePost]` artisan command to create new components +- State should live on the server, with the UI reflecting it. +- All Livewire requests hit the Laravel backend, they're like regular HTTP requests. Always validate form data, and run authorization checks in Livewire actions. + +## Livewire Best Practices +- Livewire components require a single root element. +- Use `wire:loading` and `wire:dirty` for delightful loading states. +- Add `wire:key` in loops: + + ```blade + @foreach ($items as $item) + <div wire:key="item-{{ $item->id }}"> + {{ $item->name }} + </div> + @endforeach + ``` + +- Prefer lifecycle hooks like `mount()`, `updatedFoo()`) for initialization and reactive side effects: + +<code-snippet name="Lifecycle hook examples" lang="php"> + public function mount(User $user) { $this->user = $user; } + public function updatedSearch() { $this->resetPage(); } +</code-snippet> + + +## Testing Livewire + +<code-snippet name="Example Livewire component test" lang="php"> + Livewire::test(Counter::class) + ->assertSet('count', 0) + ->call('increment') + ->assertSet('count', 1) + ->assertSee(1) + ->assertStatus(200); +</code-snippet> + + + <code-snippet name="Testing a Livewire component exists within a page" lang="php"> + $this->get('/posts/create') + ->assertSeeLivewire(CreatePost::class); + </code-snippet> + + +=== livewire/v3 rules === + +## Livewire 3 + +### Key Changes From Livewire 2 +- These things changed in Livewire 2, but may not have been updated in this application. Verify this application's setup to ensure you conform with application conventions. + - Use `wire:model.live` for real-time updates, `wire:model` is now deferred by default. + - Components now use the `App\Livewire` namespace (not `App\Http\Livewire`). + - Use `$this->dispatch()` to dispatch events (not `emit` or `dispatchBrowserEvent`). + - Use the `components.layouts.app` view as the typical layout path (not `layouts.app`). + +### New Directives +- `wire:show`, `wire:transition`, `wire:cloak`, `wire:offline`, `wire:target` are available for use. Use the documentation to find usage examples. + +### Alpine +- Alpine is now included with Livewire, don't manually include Alpine.js. +- Plugins included with Alpine: persist, intersect, collapse, and focus. + +### Lifecycle Hooks +- You can listen for `livewire:init` to hook into Livewire initialization, and `fail.status === 419` for the page expiring: + +<code-snippet name="livewire:load example" lang="js"> +document.addEventListener('livewire:init', function () { + Livewire.hook('request', ({ fail }) => { + if (fail && fail.status === 419) { + alert('Your session expired'); + } + }); + + Livewire.hook('message.failed', (message, component) => { + console.error(message); + }); +}); +</code-snippet> + + +=== pint/core rules === + +## Laravel Pint Code Formatter + +- You must run `vendor/bin/pint --dirty` before finalizing changes to ensure your code matches the project's expected style. +- Do not run `vendor/bin/pint --test`, simply run `vendor/bin/pint` to fix any formatting issues. + + +=== pest/core rules === + +## Pest + +### Testing +- If you need to verify a feature is working, write or update a Unit / Feature test. + +### Pest Tests +- All tests must be written using Pest. Use `php artisan make:test --pest <name>`. +- You must not remove any tests or test files from the tests directory without approval. These are not temporary or helper files - these are core to the application. +- Tests should test all of the happy paths, failure paths, and weird paths. +- Tests live in the `tests/Feature` and `tests/Unit` directories. +- Pest tests look and behave like this: +<code-snippet name="Basic Pest Test Example" lang="php"> +it('is true', function () { + expect(true)->toBeTrue(); +}); +</code-snippet> + +### Running Tests +- Run the minimal number of tests using an appropriate filter before finalizing code edits. +- To run all tests: `php artisan test`. +- To run all tests in a file: `php artisan test tests/Feature/ExampleTest.php`. +- To filter on a particular test name: `php artisan test --filter=testName` (recommended after making a change to a related file). +- When the tests relating to your changes are passing, ask the user if they would like to run the entire test suite to ensure everything is still passing. + +### Pest Assertions +- When asserting status codes on a response, use the specific method like `assertForbidden` and `assertNotFound` instead of using `assertStatus(403)` or similar, e.g.: +<code-snippet name="Pest Example Asserting postJson Response" lang="php"> +it('returns all', function () { + $response = $this->postJson('/api/docs', []); + + $response->assertSuccessful(); +}); +</code-snippet> + +### Mocking +- Mocking can be very helpful when appropriate. +- When mocking, you can use the `Pest\Laravel\mock` Pest function, but always import it via `use function Pest\Laravel\mock;` before using it. Alternatively, you can use `$this->mock()` if existing tests do. +- You can also create partial mocks using the same import or self method. + +### Datasets +- Use datasets in Pest to simplify tests which have a lot of duplicated data. This is often the case when testing validation rules, so consider going with this solution when writing tests for validation rules. + +<code-snippet name="Pest Dataset Example" lang="php"> +it('has emails', function (string $email) { + expect($email)->not->toBeEmpty(); +})->with([ + 'james' => 'james@laravel.com', + 'taylor' => 'taylor@laravel.com', +]); +</code-snippet> + + +=== tailwindcss/core rules === + +## Tailwind Core + +- Use Tailwind CSS classes to style HTML, check and use existing tailwind conventions within the project before writing your own. +- Offer to extract repeated patterns into components that match the project's conventions (i.e. Blade, JSX, Vue, etc..) +- Think through class placement, order, priority, and defaults - remove redundant classes, add classes to parent or child carefully to limit repetition, group elements logically +- You can use the `search-docs` tool to get exact examples from the official documentation when needed. + +### Spacing +- When listing items, use gap utilities for spacing, don't use margins. + + <code-snippet name="Valid Flex Gap Spacing Example" lang="html"> + <div class="flex gap-8"> + <div>Superior</div> + <div>Michigan</div> + <div>Erie</div> + </div> + </code-snippet> + + +### Dark Mode +- If existing pages and components support dark mode, new pages and components must support dark mode in a similar way, typically using `dark:`. + + +=== tailwindcss/v4 rules === + +## Tailwind 4 + +- Always use Tailwind CSS v4 - do not use the deprecated utilities. +- `corePlugins` is not supported in Tailwind v4. +- In Tailwind v4, you import Tailwind using a regular CSS `@import` statement, not using the `@tailwind` directives used in v3: + +<code-snippet name="Tailwind v4 Import Tailwind Diff" lang="diff" + - @tailwind base; + - @tailwind components; + - @tailwind utilities; + + @import "tailwindcss"; +</code-snippet> + + +### Replaced Utilities +- Tailwind v4 removed deprecated utilities. Do not use the deprecated option - use the replacement. +- Opacity values are still numeric. + +| Deprecated | Replacement | +|------------+--------------| +| bg-opacity-* | bg-black/* | +| text-opacity-* | text-black/* | +| border-opacity-* | border-black/* | +| divide-opacity-* | divide-black/* | +| ring-opacity-* | ring-black/* | +| placeholder-opacity-* | placeholder-black/* | +| flex-shrink-* | shrink-* | +| flex-grow-* | grow-* | +| overflow-ellipsis | text-ellipsis | +| decoration-slice | box-decoration-slice | +| decoration-clone | box-decoration-clone | + + +=== tests rules === + +## Test Enforcement + +- Every change must be programmatically tested. Write a new test or update an existing test, then run the affected tests to make sure they pass. +- Run the minimum number of tests needed to ensure code quality and speed. Use `php artisan test` with a specific filename or filter. +</laravel-boost-guidelines> \ No newline at end of file diff --git a/.cursor/rules/testing-patterns.mdc b/.cursor/rules/testing-patterns.mdc index 010b76544..a0e64dbae 100644 --- a/.cursor/rules/testing-patterns.mdc +++ b/.cursor/rules/testing-patterns.mdc @@ -9,6 +9,8 @@ alwaysApply: false Coolify employs **comprehensive testing strategies** using modern PHP testing frameworks to ensure reliability of deployment operations, infrastructure management, and user interactions. +!Important: Always run tests inside `coolify` container. + ## Testing Framework Stack ### Core Testing Tools diff --git a/.github/workflows/chore-pr-comments.yml b/.github/workflows/chore-pr-comments.yml new file mode 100644 index 000000000..8836c6632 --- /dev/null +++ b/.github/workflows/chore-pr-comments.yml @@ -0,0 +1,56 @@ +name: Add comment based on label +on: + pull_request_target: + types: + - labeled +jobs: + add-comment: + runs-on: ubuntu-latest + permissions: + pull-requests: write + contents: read + actions: none + checks: none + deployments: none + issues: none + packages: none + repository-projects: none + security-events: none + statuses: none + strategy: + matrix: + include: + - label: "⚙️ Service" + body: | + Hi @${{ github.event.pull_request.user.login }}! 👋 + + It appears to us that you are either adding a new service or making changes to an existing one. + We kindly ask you to also review and update the **Coolify Documentation** to include this new service or it's new configuration needs. + This will help ensure that our documentation remains accurate and up-to-date for all users. + + Coolify Docs Repository: https://github.com/coollabsio/coolify-docs + How to Contribute a new Service to the Docs: https://coolify.io/docs/get-started/contribute/service#adding-a-new-service-template-to-the-coolify-documentation + - label: "🛠️ Feature" + body: | + Hi @${{ github.event.pull_request.user.login }}! 👋 + + It appears to us that you are adding a new feature to Coolify. + We kindly ask you to also update the **Coolify Documentation** to include information about this new feature. + This will help ensure that our documentation remains accurate and up-to-date for all users. + + Coolify Docs Repository: https://github.com/coollabsio/coolify-docs + How to Contribute to the Docs: https://coolify.io/docs/get-started/contribute/documentation + # - label: "✨ Enhancement" + # body: | + # It appears to us that you are making an enhancement to Coolify. + # We kindly ask you to also review and update the Coolify Documentation to include information about this enhancement if applicable. + # This will help ensure that our documentation remains accurate and up-to-date for all users. + steps: + - name: Add comment + if: github.event.label.name == matrix.label + run: gh pr comment "$NUMBER" --body "$BODY" + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + GH_REPO: ${{ github.repository }} + NUMBER: ${{ github.event.pull_request.number }} + BODY: ${{ matrix.body }} diff --git a/.github/workflows/claude.yml b/.github/workflows/claude.yml index bc773072b..9daf0e90e 100644 --- a/.github/workflows/claude.yml +++ b/.github/workflows/claude.yml @@ -16,6 +16,8 @@ jobs: (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@claude')) || (github.event_name == 'pull_request_review_comment' && contains(github.event.comment.body, '@claude')) || (github.event_name == 'pull_request_review' && contains(github.event.review.body, '@claude')) || + (github.event_name == 'issues' && github.event.action == 'labeled' && github.event.label.name == 'Claude') || + (github.event_name == 'pull_request' && github.event.action == 'labeled' && github.event.label.name == 'Claude') || (github.event_name == 'issues' && (contains(github.event.issue.body, '@claude') || contains(github.event.issue.title, '@claude'))) runs-on: ubuntu-latest permissions: @@ -32,9 +34,9 @@ jobs: - name: Run Claude Code id: claude - uses: anthropics/claude-code-action@beta + uses: anthropics/claude-code-action@v1 with: - claude_code_oauth_token: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} + anthropic_api_key: ${{ secrets.CLAUDE_CODE_OAUTH_TOKEN }} # This is an optional setting that allows Claude to read CI results on PRs additional_permissions: | @@ -61,4 +63,3 @@ jobs: # Optional: Custom environment variables for Claude # claude_env: | # NODE_ENV: test - diff --git a/.github/workflows/coolify-production-build.yml b/.github/workflows/coolify-production-build.yml index 9286fdbb0..cd1f002b8 100644 --- a/.github/workflows/coolify-production-build.yml +++ b/.github/workflows/coolify-production-build.yml @@ -13,7 +13,6 @@ on: - docker/testing-host/Dockerfile - templates/** - CHANGELOG.md - - backlog/** env: GITHUB_REGISTRY: ghcr.io diff --git a/.github/workflows/coolify-staging-build.yml b/.github/workflows/coolify-staging-build.yml index 390eab000..09b1e9421 100644 --- a/.github/workflows/coolify-staging-build.yml +++ b/.github/workflows/coolify-staging-build.yml @@ -16,7 +16,6 @@ on: - docker/testing-host/Dockerfile - templates/** - CHANGELOG.md - - backlog/** env: GITHUB_REGISTRY: ghcr.io diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 000000000..8c6715a15 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,11 @@ +{ + "mcpServers": { + "laravel-boost": { + "command": "php", + "args": [ + "artisan", + "boost:mcp" + ] + } + } +} \ No newline at end of file diff --git a/.phpactor.json b/.phpactor.json new file mode 100644 index 000000000..4d42bbbc5 --- /dev/null +++ b/.phpactor.json @@ -0,0 +1,4 @@ +{ + "$schema": "/phpactor.schema.json", + "language_server_phpstan.enabled": true +} \ No newline at end of file diff --git a/CHANGELOG.md b/CHANGELOG.md index 153776f1a..3447b223b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,415 @@ ## [unreleased] ### 🚀 Features +- *(application)* Implement order-based pattern matching for watch paths with negation support +- *(github)* Enhance Docker Compose input fields for better user experience +- *(dev-seeders)* Add PersonalAccessTokenSeeder to create development API tokens +- *(application)* Add conditional .env file creation for Symfony apps during PHP deployment +- *(application)* Enhance watch path parsing to support negation syntax +- *(application)* Add normalizeWatchPaths method to improve watch path handling +- *(validation)* Enhance ValidGitRepositoryUrl to support additional safe characters and add comprehensive unit tests for various Git repository URL formats +- *(deployment)* Implement detection for Laravel/Symfony frameworks and configure NIXPACKS PHP environment variables accordingly + +### 🐛 Bug Fixes + +- *(application)* Restrict GitHub-based application settings to non-public repositories +- *(traits)* Update saved_outputs handling in ExecuteRemoteCommand to use collection methods for better performance +- *(application)* Enhance domain handling by replacing both dots and dashes with underscores for HTML form binding +- *(constants)* Reduce command timeout from 7200 to 3600 seconds for improved performance +- *(github)* Update repository URL to point to the v4.x branch for development +- *(models)* Update sorting of scheduled database backups to order by creation date instead of name +- *(socialite)* Add custom base URL support for GitLab provider in OAuth settings +- *(configuration-checker)* Update message to clarify redeployment requirement for configuration changes +- *(application)* Reduce docker stop timeout from 30 to 10 seconds for improved application shutdown efficiency +- *(application)* Increase docker stop timeout from 10 to 30 seconds for better application shutdown handling +- *(validation)* Update git:// URL validation to support port numbers and tilde characters in paths +- Resolve scroll lock issue after closing quick search modal with escape key +- Prevent quick search modal duplication from keyboard shortcuts + +### 🚜 Refactor + +- *(tests)* Simplify matchWatchPaths tests and update implementation for better clarity +- *(deployment)* Improve environment variable handling in ApplicationDeploymentJob +- *(deployment)* Remove commented-out code and streamline environment variable handling in ApplicationDeploymentJob +- *(application)* Improve handling of docker compose domains by normalizing keys and ensuring valid JSON structure +- *(forms)* Update wire:model bindings to use 'blur' instead of 'blur-sm' for input fields across multiple views + +### 📚 Documentation + +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- *(application)* Remove debugging statement from loadComposeFile method +- *(workflows)* Update Claude GitHub Action configuration to support new event types and improve permissions + +## [4.0.0-beta.431] - 2025-09-24 + +### 📚 Documentation + +- Update changelog + +## [4.0.0-beta.430] - 2025-09-24 + +### 🚀 Features + +- *(add-watch-paths-for-services)* Show watch paths field for docker compose applications + +### 🐛 Bug Fixes + +- *(PreviewCompose)* Adds port to preview urls +- *(deployment-job)* Enhance build time variable analysis +- *(docker)* Adjust openssh-client installation in Dockerfile to avoid version bug +- *(docker)* Streamline openssh-client installation in Dockerfile +- *(team)* Normalize email case in invite link generation +- *(README)* Update Juxtdigital description to reflect current services +- *(environment-variable-warning)* Enhance warning logic to check for problematic variable values +- *(install)* Ensure proper quoting of environment file paths to prevent issues with spaces +- *(security)* Implement authorization checks for terminal access management +- *(ui)* Improve mobile sidebar close behavior + +### 🚜 Refactor + +- *(installer)* Improve install script +- *(upgrade)* Improve upgrade script +- *(installer, upgrade)* Enhance environment variable management +- *(upgrade)* Enhance logging and quoting in upgrade scripts +- *(upgrade)* Replace warning div with a callout component for better UI consistency +- *(ui)* Replace warning and error divs with callout components for improved consistency and readability +- *(ui)* Improve styling and consistency in environment variable warning and docker cleanup components +- *(security)* Streamline update check functionality and improve UI button interactions in patches view + +### 📚 Documentation + +- Update changelog +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- *(versions)* Increment coolify version numbers to 4.0.0-beta.431 and 4.0.0-beta.432 in configuration files +- *(versions)* Update coolify version numbers to 4.0.0-beta.432 and 4.0.0-beta.433 in configuration files +- Remove unused files +- Adjust wording +- *(workflow)* Update pull request trigger to pull_request_target and refine permissions for enhanced security + +## [4.0.0-beta.429] - 2025-09-23 + +### 🚀 Features + +- *(environment)* Replace is_buildtime_only with is_runtime and is_buildtime flags for environment variables, updating related logic and views +- *(deployment)* Handle buildtime and runtime variables during deployment +- *(search)* Implement global search functionality with caching and modal interface +- *(search)* Enable query logging for global search caching +- *(environment)* Add dynamic checkbox options for environment variable settings based on user permissions and variable types +- *(redaction)* Implement sensitive information redaction in logs and commands +- Improve detection of special network modes +- *(api)* Add endpoint to update backup configuration by UUID and backup ID; modify response to include backup id +- *(databases)* Enhance backup management API with new endpoints and improved data handling +- *(github)* Add GitHub app management endpoints +- *(github)* Add update and delete endpoints for GitHub apps +- *(databases)* Enhance backup update and deletion logic with validation +- *(environment-variables)* Implement environment variable analysis for build-time issues +- *(databases)* Implement unique UUID generation for backup execution +- *(cloud-check)* Enhance subscription reporting in CloudCheckSubscription command +- *(cloud-check)* Enhance CloudCheckSubscription command with fix options +- *(stripe)* Enhance subscription handling and verification process +- *(private-key-refresh)* Add refresh dispatch on private key update and connection check +- *(comments)* Add automated comments for labeled pull requests to guide documentation updates +- *(comments)* Ping PR author + +### 🐛 Bug Fixes + +- *(docker)* Enhance container status aggregation to include restarting and exited states +- *(environment)* Correct grammatical errors in helper text for environment variable sorting checkbox +- *(ui)* Change order and fix ui on small screens +- Order for git deploy types +- *(deployment)* Enhance Dockerfile modification for build-time variables and secrets during deployment in case of docker compose buildpack +- Hide sensitive email change fields in team member responses +- *(domains)* Trim whitespace from domains before validation +- *(databases)* Update backup retrieval logic to include team context +- *(environment-variables)* Update affected services in environment variable analysis +- *(team)* Clear stripe_subscription_id on subscription end +- *(github)* Update authentication method for GitHub app operations +- *(databases)* Restrict database updates to allowed fields only +- *(cache)* Add Model import to ClearsGlobalSearchCache trait for improved functionality +- *(environment-variables)* Correct method call syntax in analyzeBuildVariable function +- *(clears-global-search-cache)* Refine team retrieval logic in getTeamIdForCache method +- *(subscription-job)* Enhance retry logic for VerifyStripeSubscriptionStatusJob +- *(environment-variable)* Update checkbox visibility and helper text for build and runtime options +- *(deployment-job)* Escape single quotes in build arguments for Docker Compose command + +### 🚜 Refactor + +- *(environment)* Conditionally render Docker Build Secrets checkbox based on build pack type +- *(search)* Optimize cache clearing logic to only trigger on searchable field changes +- *(environment)* Streamline rendering of Docker Build Secrets checkbox and adjust layout for environment variable settings +- *(proxy)* Streamline proxy configuration form layout and improve button placements +- *(remoteProcess)* Remove redundant file transfer functions for improved clarity +- *(github)* Enhance API request handling and validation +- *(databases)* Remove deprecated backup parameters from API documentation +- *(databases)* Streamline backup queries to use team context +- *(databases)* Update backup queries to use team-specific method +- *(server)* Update dispatch messages and streamline data synchronization +- *(cache)* Update team retrieval method in ClearsGlobalSearchCache trait +- *(database-backup)* Move unique UUID generation for backup execution to database loop +- *(cloud-commands)* Consolidate and enhance subscription management commands +- *(toast-component)* Improve layout and icon handling in toast notifications +- *(private-key-update)* Implement transaction for private key association and connection validation + +### 📚 Documentation + +- Update changelog +- Update changelog +- *(claude)* Update testing guidelines and add note on Application::team relationship + +### 🎨 Styling + +- *(environment-variable)* Adjust SVG icon margin for improved layout in locked state +- *(proxy)* Adjust padding in proxy configuration form for better visual alignment + +### ⚙️ Miscellaneous Tasks + +- Change order of runtime and buildtime +- *(docker-compose)* Update soketi image version to 1.0.10 in production and Windows configurations +- *(versions)* Update coolify version numbers to 4.0.0-beta.430 and 4.0.0-beta.431 in configuration files + +## [4.0.0-beta.428] - 2025-09-15 + +### 📚 Documentation + +- Update changelog + +## [4.0.0-beta.427] - 2025-09-15 + +### 🚀 Features + +- *(command)* Add option to sync GitHub releases to BunnyCDN and refactor sync logic +- *(ui)* Display current version in settings dropdown and update UI accordingly +- *(settings)* Add option to restrict PR deployments to repository members and contributors +- *(command)* Implement SSH command retry logic with exponential backoff and logging for better error handling +- *(ssh)* Add Sentry tracking for SSH retry events to enhance error monitoring +- *(exceptions)* Introduce NonReportableException to handle known errors and update Handler for selective reporting +- *(sudo-helper)* Add helper functions for command parsing and ownership management with sudo +- *(dev-command)* Dispatch CheckHelperImageJob during instance initialization to enhance setup process +- *(ssh-multiplexing)* Enhance multiplexed connection management with health checks and metadata caching +- *(ssh-multiplexing)* Add connection age metadata handling to improve multiplexed connection management +- *(database-backup)* Enhance error handling and output management in DatabaseBackupJob +- *(application)* Display parsing version in development mode and clean up domain conflict modal markup +- *(deployment)* Add SERVICE_NAME variables for service discovery +- *(storages)* Add method to retrieve the first storage ID for improved stability in storage display +- *(environment)* Add 'is_literal' attribute to environment variable for enhanced configuration options +- *(pre-commit)* Automate generation of service templates and OpenAPI documentation during pre-commit hook +- *(execute-container)* Enhance container command form with auto-connect feature for single container scenarios +- *(environment)* Introduce 'is_buildtime_only' attribute to environment variables for improved build-time configuration +- *(templates)* Add n8n service with PostgreSQL and worker support for enhanced workflow automation +- *(user-management)* Implement user deletion command with phased resource and subscription cancellation, including dry run option +- *(sentinel)* Add support for custom Docker images in StartSentinel and related methods +- *(sentinel)* Add slide-over for viewing Sentinel logs and custom Docker image input for development +- *(executions)* Add 'Load All' button to view all logs and implement loadAllLogs method for complete log retrieval +- *(auth)* Enhance user login flow to handle team invitations, attaching users to invited teams upon first login and maintaining personal team logic for regular logins +- *(laravel-boost)* Add Laravel Boost guidelines and MCP server configuration to enhance development experience +- *(deployment)* Enhance deployment status reporting with detailed information on active deployments and team members +- *(deployment)* Implement cancellation checks during deployment process to enhance user control and prevent unnecessary execution +- *(deployment)* Introduce 'use_build_secrets' setting for enhanced security during Docker builds and update related logic in deployment process + +### 🐛 Bug Fixes + +- *(ui)* Transactional email settings link on members page (#6491) +- *(api)* Add custom labels generation for applications with readonly container label setting enabled +- *(ui)* Add cursor pointer to upgrade button for better user interaction +- *(templates)* Update SECRET_KEY environment variable in getoutline.yaml to use SERVICE_HEX_32_OUTLINE +- *(command)* Enhance database deletion command to support multiple database types +- *(command)* Enhance cleanup process for stuck application previews by adding force delete for trashed records +- *(user)* Ensure email attributes are stored in lowercase for consistency and prevent case-related issues +- *(webhook)* Replace delete with forceDelete for application previews to ensure immediate removal +- *(ssh)* Introduce SshRetryHandler and SshRetryable trait for enhanced SSH command retry logic with exponential backoff and error handling +- Appwrite template - 500 errors, missing env vars etc. +- *(LocalFileVolume)* Add missing directory creation command for workdir in saveStorageOnServer method +- *(ScheduledTaskJob)* Replace generic Exception with NonReportableException for better error handling +- *(web-routes)* Enhance backup response messages to clarify local and S3 availability +- *(proxy)* Replace CheckConfiguration with GetProxyConfiguration and SaveConfiguration with SaveProxyConfiguration for improved clarity and consistency in proxy management +- *(private-key)* Implement transaction handling and error verification for private key storage operations +- *(deployment)* Add COOLIFY_* environment variables to Nixpacks build context for enhanced deployment configuration +- *(application)* Add functionality to stop and remove Docker containers on server +- *(templates)* Update 'compose' configuration for Appwrite service to enhance compatibility and streamline deployment +- *(security)* Update contact email for reporting vulnerabilities to enhance privacy +- *(feedback)* Update feedback email address to improve communication with users +- *(security)* Update contact email for vulnerability reports to improve security communication +- *(navbar)* Restrict subscription link visibility to admin users in cloud environment +- *(docker)* Enhance container status aggregation for multi-container applications, including exclusion handling based on docker-compose configuration +- *(application)* Improve watch paths handling by trimming and filtering empty paths to prevent unnecessary triggers +- *(server)* Update server usability check to reflect actual Docker availability status +- *(server)* Add build server check to disable Sentinel and update related logic +- *(server)* Implement refreshServer method and update navbar event listener for improved server state management +- *(deployment)* Prevent removal of running containers for pull request deployments in case of failure +- *(docker)* Redirect stderr to stdout for container log retrieval to capture error messages +- *(clone)* Update destinations method call to ensure correct retrieval of selected destination + +### 🚜 Refactor + +- *(jobs)* Pull github changelogs from cdn instead of github +- *(command)* Streamline database deletion process to handle multiple database types and improve user experience +- *(command)* Improve database collection logic for deletion command by using unique identifiers and enhancing user experience +- *(command)* Remove InitChangelog command as it is no longer needed +- *(command)* Streamline Init command by removing unnecessary options and enhancing error handling for various operations +- *(webhook)* Replace direct forceDelete calls with DeleteResourceJob dispatch for application previews +- *(command)* Replace forceDelete calls with DeleteResourceJob dispatch for all stuck resources in cleanup process +- *(command)* Simplify SSH command retry logic by removing unnecessary logging and improving delay calculation +- *(ssh)* Enhance error handling in SSH command execution and improve connection validation logging +- *(backlog)* Remove outdated guidelines and project manager agent files to streamline task management documentation +- *(error-handling)* Remove ray debugging statements from CheckUpdates and shared helper functions to clean up error reporting +- *(file-transfer)* Replace base64 encoding with direct file transfer method across multiple database actions for improved clarity and efficiency +- *(remoteProcess)* Remove debugging statement from transfer_file_to_server function to clean up code +- *(dns-validation)* Rename DNS validation functions for consistency and clarity, and remove unused code +- *(file-transfer)* Replace base64 encoding with direct file transfer method in various components for improved clarity and efficiency +- *(private-key)* Remove debugging statement from storeInFileSystem method for cleaner code +- *(github-webhook)* Restructure application processing by grouping applications by server for improved deployment handling +- *(deployment)* Enhance queuing logic to support concurrent deployments by including pull request ID in checks +- *(remoteProcess)* Remove debugging statement from transfer_file_to_container function for cleaner code +- *(deployment)* Streamline next deployment queuing logic by repositioning queue_next_deployment call +- *(deployment)* Add validation for pull request existence in deployment process to enhance error handling +- *(database)* Remove volume_configuration_dir and streamline configuration directory usage in MongoDB and PostgreSQL handlers +- *(application-source)* Improve layout and accessibility of Git repository links in the application source view +- *(models)* Remove 'is_readonly' attribute from multiple database models for consistency +- *(webhook)* Remove Webhook model and related logic; add migrations to drop webhooks and kubernetes tables +- *(clone)* Consolidate application cloning logic into a dedicated function for improved maintainability and readability +- *(clone)* Integrate preview cloning logic directly into application cloning function for improved clarity and maintainability +- *(application)* Enhance environment variable retrieval in configuration change check for improved accuracy +- *(clone)* Enhance application cloning by separating production and preview environment variable handling +- *(deployment)* Add environment variable copying logic to Docker build commands for pull requests +- *(environment)* Standardize service name formatting by replacing '-' and '.' with '_' in environment variable keys +- *(deployment)* Update environment file handling in Docker commands to use '/artifacts/' path and streamline variable management +- *(openapi)* Remove 'is_build_time' attribute from environment variable definitions to streamline configuration +- *(environment)* Remove 'is_build_time' attribute from environment variable handling across the application to simplify configuration +- *(environment)* Streamline environment variable handling by replacing sorting methods with direct property access and enhancing query ordering for improved performance +- *(stripe-jobs)* Comment out internal notification calls and add subscription status verification before sending failure notifications +- *(deployment)* Streamline environment variable handling for dockercompose and improve sorting of runtime variables +- *(remoteProcess)* Remove command log comments for file transfers to simplify code +- *(remoteProcess)* Remove file transfer handling from remote_process and instant_remote_process functions to simplify code +- *(deployment)* Update environment file paths in docker compose commands to use working directory for improved consistency +- *(server)* Remove debugging ray call from validateConnection method for cleaner code +- *(deployment)* Conditionally cleanup build secrets based on Docker BuildKit support and remove redundant calls for improved efficiency +- *(deployment)* Remove redundant environment variable documentation from Dockerfile comments to streamline the deployment process +- *(deployment)* Streamline Docker BuildKit detection and environment variable handling for enhanced security during application deployment +- *(deployment)* Optimize BuildKit capabilities detection and remove unnecessary comments for cleaner deployment logic +- *(deployment)* Rename method for modifying Dockerfile to improve clarity and streamline build secrets integration + +### 📚 Documentation + +- Update changelog +- *(testing-patterns)* Add important note to always run tests inside the `coolify` container for clarity + +### ⚙️ Miscellaneous Tasks + +- Update coolify version to 4.0.0-beta.427 and nightly version to 4.0.0-beta.428 +- Use main value then fallback to service_ values +- Remove webhooks table cleanup +- *(cleanup)* Remove deprecated ServerCheck and related job classes to streamline codebase +- *(versions)* Update sentinel version from 0.0.15 to 0.0.16 in versions.json files +- *(constants)* Update realtime_version from 1.0.10 to 1.0.11 +- *(versions)* Increment coolify version to 4.0.0-beta.428 and update realtime_version to 1.0.10 +- *(docker)* Add a blank line for improved readability in Dockerfile +- *(versions)* Bump coolify version to 4.0.0-beta.429 and nightly version to 4.0.0-beta.430 + +## [4.0.0-beta.426] - 2025-08-28 + +### 🚜 Refactor + +- *(policy)* Simplify ServiceDatabasePolicy methods to always return true and add manageBackups method + +### 📚 Documentation + +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- Update coolify version to 4.0.0-beta.426 and nightly version to 4.0.0-beta.427 + +## [4.0.0-beta.425] - 2025-08-28 + +### 🚀 Features + +- *(domains)* Implement domain conflict detection and user confirmation modal across application components +- *(domains)* Add force_domain_override option and enhance domain conflict detection responses + +### 🐛 Bug Fixes + +- *(previews)* Simplify FQDN generation logic by removing unnecessary empty check +- *(templates)* Update Matrix service compose configuration for improved compatibility and clarity + +### 🚜 Refactor + +- *(urls)* Replace generateFqdn with generateUrl for consistent URL generation across applications +- *(domains)* Rename check_domain_usage to checkDomainUsage and update references across the application +- *(auth)* Simplify access control logic in CanAccessTerminal and ServerPolicy by allowing all users to perform actions + +### 📚 Documentation + +- Update changelog +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- Update coolify version to 4.0.0-beta.425 and nightly version to 4.0.0-beta.426 + +## [4.0.0-beta.424] - 2025-08-27 + +### 🐛 Bug Fixes + +- *(parsers)* Do not modify service names, only for getting fqdns and related envs +- *(compose)* Temporary allow to edit volumes in apps (compose based) and services + +### 📚 Documentation + +- Update changelog +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- Update coolify version to 4.0.0-beta.424 and nightly version to 4.0.0-beta.425 + +## [4.0.0-beta.423] - 2025-08-27 + +### 🚜 Refactor + +- *(parsers)* Remove unnecessary hyphen-to-underscore replacement for service names in serviceParser function + +### 📚 Documentation + +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- Update coolify version to 4.0.0-beta.423 and nightly version to 4.0.0-beta.424 + +## [4.0.0-beta.422] - 2025-08-27 + +### 🐛 Bug Fixes + +- *(parsers)* Replace hyphens with underscores in service names for consistency. this allows to properly parse custom domains in docker compose based applications +- *(parsers)* Implement parseDockerVolumeString function to handle various Docker volume formats and modes, including environment variables and Windows paths. Add unit tests for comprehensive coverage. +- *(git)* Submodule update command uses an unsupported option (#6454) +- *(service)* Swap URL for FQDN on matrix template (#6466) +- *(parsers)* Enhance volume string handling by preserving mode in application and service parsers. Update related unit tests for validation. +- *(docker)* Update parser version in FQDN generation for service-specific URLs + +### 🚜 Refactor + +- *(git)* Improve submodule cloning + +### 📚 Documentation + +- Update changelog + +### ⚙️ Miscellaneous Tasks + +- Update version +- Update development node version + +## [4.0.0-beta.421] - 2025-08-26 + +### 🚀 Features + - *(policies)* Add EnvironmentVariablePolicy for managing environment variables ( it was missing ) ### 🐛 Bug Fixes diff --git a/CLAUDE.md b/CLAUDE.md index 87409c260..22e762182 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -248,402 +248,411 @@ ### Project Information - [Technology Stack](.cursor/rules/technology-stack.mdc) - Detailed tech stack information - [Cursor Rules Guide](.cursor/rules/cursor_rules.mdc) - How to maintain cursor rules +=== -# === BACKLOG.MD GUIDELINES START === -# Instructions for the usage of Backlog.md CLI Tool +<laravel-boost-guidelines> +=== foundation rules === -## What is Backlog.md? +# Laravel Boost Guidelines -**Backlog.md is the complete project management system for this codebase.** It provides everything needed to manage tasks, track progress, and collaborate on development - all through a powerful CLI that operates on markdown files. +The Laravel Boost guidelines are specifically curated by Laravel maintainers for this application. These guidelines should be followed closely to enhance the user's satisfaction building Laravel applications. -### Core Capabilities +## Foundational Context +This application is a Laravel application and its main Laravel ecosystems package & versions are below. You are an expert with them all. Ensure you abide by these specific packages & versions. -✅ **Task Management**: Create, edit, assign, prioritize, and track tasks with full metadata -✅ **Acceptance Criteria**: Granular control with add/remove/check/uncheck by index -✅ **Board Visualization**: Terminal-based Kanban board (`backlog board`) and web UI (`backlog browser`) -✅ **Git Integration**: Automatic tracking of task states across branches -✅ **Dependencies**: Task relationships and subtask hierarchies -✅ **Documentation & Decisions**: Structured docs and architectural decision records -✅ **Export & Reporting**: Generate markdown reports and board snapshots -✅ **AI-Optimized**: `--plain` flag provides clean text output for AI processing +- php - 8.4.7 +- laravel/fortify (FORTIFY) - v1 +- laravel/framework (LARAVEL) - v12 +- laravel/horizon (HORIZON) - v5 +- laravel/prompts (PROMPTS) - v0 +- laravel/sanctum (SANCTUM) - v4 +- laravel/socialite (SOCIALITE) - v5 +- livewire/livewire (LIVEWIRE) - v3 +- laravel/dusk (DUSK) - v8 +- laravel/pint (PINT) - v1 +- laravel/telescope (TELESCOPE) - v5 +- pestphp/pest (PEST) - v3 +- phpunit/phpunit (PHPUNIT) - v11 +- rector/rector (RECTOR) - v2 +- laravel-echo (ECHO) - v2 +- tailwindcss (TAILWINDCSS) - v4 +- vue (VUE) - v3 -### Why This Matters to You (AI Agent) -1. **Comprehensive system** - Full project management capabilities through CLI -2. **The CLI is the interface** - All operations go through `backlog` commands -3. **Unified interaction model** - You can use CLI for both reading (`backlog task 1 --plain`) and writing (`backlog task edit 1`) -4. **Metadata stays synchronized** - The CLI handles all the complex relationships +## Conventions +- You must follow all existing code conventions used in this application. When creating or editing a file, check sibling files for the correct structure, approach, naming. +- Use descriptive names for variables and methods. For example, `isRegisteredForDiscounts`, not `discount()`. +- Check for existing components to reuse before writing a new one. -### Key Understanding +## Verification Scripts +- Do not create verification scripts or tinker when tests cover that functionality and prove it works. Unit and feature tests are more important. -- **Tasks** live in `backlog/tasks/` as `task-<id> - <title>.md` files -- **You interact via CLI only**: `backlog task create`, `backlog task edit`, etc. -- **Use `--plain` flag** for AI-friendly output when viewing/listing -- **Never bypass the CLI** - It handles Git, metadata, file naming, and relationships - ---- +## Application Structure & Architecture +- Stick to existing directory structure - don't create new base folders without approval. +- Do not change the application's dependencies without approval. -# ⚠️ CRITICAL: NEVER EDIT TASK FILES DIRECTLY +## Frontend Bundling +- If the user doesn't see a frontend change reflected in the UI, it could mean they need to run `npm run build`, `npm run dev`, or `composer run dev`. Ask them. -**ALL task operations MUST use the Backlog.md CLI commands** -- ✅ **DO**: Use `backlog task edit` and other CLI commands -- ✅ **DO**: Use `backlog task create` to create new tasks -- ✅ **DO**: Use `backlog task edit <id> --check-ac <index>` to mark acceptance criteria -- ❌ **DON'T**: Edit markdown files directly -- ❌ **DON'T**: Manually change checkboxes in files -- ❌ **DON'T**: Add or modify text in task files without using CLI +## Replies +- Be concise in your explanations - focus on what's important rather than explaining obvious details. -**Why?** Direct file editing breaks metadata synchronization, Git tracking, and task relationships. +## Documentation Files +- You must only create documentation files if explicitly requested by the user. ---- -## 1. Source of Truth & File Structure +=== boost rules === -### 📖 **UNDERSTANDING** (What you'll see when reading) -- Markdown task files live under **`backlog/tasks/`** (drafts under **`backlog/drafts/`**) -- Files are named: `task-<id> - <title>.md` (e.g., `task-42 - Add GraphQL resolver.md`) -- Project documentation is in **`backlog/docs/`** -- Project decisions are in **`backlog/decisions/`** +## Laravel Boost +- Laravel Boost is an MCP server that comes with powerful tools designed specifically for this application. Use them. -### 🔧 **ACTING** (How to change things) -- **All task operations MUST use the Backlog.md CLI tool** -- This ensures metadata is correctly updated and the project stays in sync -- **Always use `--plain` flag** when listing or viewing tasks for AI-friendly text output +## Artisan +- Use the `list-artisan-commands` tool when you need to call an Artisan command to double check the available parameters. ---- +## URLs +- Whenever you share a project URL with the user you should use the `get-absolute-url` tool to ensure you're using the correct scheme, domain / IP, and port. -## 2. Common Mistakes to Avoid +## Tinker / Debugging +- You should use the `tinker` tool when you need to execute PHP to debug code or query Eloquent models directly. +- Use the `database-query` tool when you only need to read from the database. -### ❌ **WRONG: Direct File Editing** -```markdown -# DON'T DO THIS: -1. Open backlog/tasks/task-7 - Feature.md in editor -2. Change "- [ ]" to "- [x]" manually -3. Add notes directly to the file -4. Save the file -``` +## Reading Browser Logs With the `browser-logs` Tool +- You can read browser logs, errors, and exceptions using the `browser-logs` tool from Boost. +- Only recent browser logs will be useful - ignore old logs. -### ✅ **CORRECT: Using CLI Commands** -```bash -# DO THIS INSTEAD: -backlog task edit 7 --check-ac 1 # Mark AC #1 as complete -backlog task edit 7 --notes "Implementation complete" # Add notes -backlog task edit 7 -s "In Progress" -a @agent-k # Multiple commands: change status and assign the task -``` +## Searching Documentation (Critically Important) +- Boost comes with a powerful `search-docs` tool you should use before any other approaches. This tool automatically passes a list of installed packages and their versions to the remote Boost API, so it returns only version-specific documentation specific for the user's circumstance. You should pass an array of packages to filter on if you know you need docs for particular packages. +- The 'search-docs' tool is perfect for all Laravel related packages, including Laravel, Inertia, Livewire, Filament, Tailwind, Pest, Nova, Nightwatch, etc. +- You must use this tool to search for Laravel-ecosystem documentation before falling back to other approaches. +- Search the documentation before making code changes to ensure we are taking the correct approach. +- Use multiple, broad, simple, topic based queries to start. For example: `['rate limiting', 'routing rate limiting', 'routing']`. +- Do not add package names to queries - package information is already shared. For example, use `test resource table`, not `filament 4 test resource table`. ---- +### Available Search Syntax +- You can and should pass multiple queries at once. The most relevant results will be returned first. -## 3. Understanding Task Format (Read-Only Reference) +1. Simple Word Searches with auto-stemming - query=authentication - finds 'authenticate' and 'auth' +2. Multiple Words (AND Logic) - query=rate limit - finds knowledge containing both "rate" AND "limit" +3. Quoted Phrases (Exact Position) - query="infinite scroll" - Words must be adjacent and in that order +4. Mixed Queries - query=middleware "rate limit" - "middleware" AND exact phrase "rate limit" +5. Multiple Queries - queries=["authentication", "middleware"] - ANY of these terms -⚠️ **FORMAT REFERENCE ONLY** - The following sections show what you'll SEE in task files. -**Never edit these directly! Use CLI commands to make changes.** -### Task Structure You'll See +=== php rules === -```markdown ---- -id: task-42 -title: Add GraphQL resolver -status: To Do -assignee: [@sara] -labels: [backend, api] ---- +## PHP -## Description -Brief explanation of the task purpose. +- Always use curly braces for control structures, even if it has one line. -## Acceptance Criteria -<!-- AC:BEGIN --> -- [ ] #1 First criterion -- [x] #2 Second criterion (completed) -- [ ] #3 Third criterion -<!-- AC:END --> - -## Implementation Plan -1. Research approach -2. Implement solution - -## Implementation Notes -Summary of what was done. -``` +### Constructors +- Use PHP 8 constructor property promotion in `__construct()`. + - <code-snippet>public function __construct(public GitHub $github) { }</code-snippet> +- Do not allow empty `__construct()` methods with zero parameters. -### How to Modify Each Section +### Type Declarations +- Always use explicit return type declarations for methods and functions. +- Use appropriate PHP type hints for method parameters. -| What You Want to Change | CLI Command to Use | -|------------------------|-------------------| -| Title | `backlog task edit 42 -t "New Title"` | -| Status | `backlog task edit 42 -s "In Progress"` | -| Assignee | `backlog task edit 42 -a @sara` | -| Labels | `backlog task edit 42 -l backend,api` | -| Description | `backlog task edit 42 -d "New description"` | -| Add AC | `backlog task edit 42 --ac "New criterion"` | -| Check AC #1 | `backlog task edit 42 --check-ac 1` | -| Uncheck AC #2 | `backlog task edit 42 --uncheck-ac 2` | -| Remove AC #3 | `backlog task edit 42 --remove-ac 3` | -| Add Plan | `backlog task edit 42 --plan "1. Step one\n2. Step two"` | -| Add Notes | `backlog task edit 42 --notes "What I did"` | - ---- - -## 4. Defining Tasks - -### Creating New Tasks - -**Always use CLI to create tasks:** -```bash -backlog task create "Task title" -d "Description" --ac "First criterion" --ac "Second criterion" -``` - -### Title (one liner) -Use a clear brief title that summarizes the task. +<code-snippet name="Explicit Return Types and Method Params" lang="php"> +protected function isAccessible(User $user, ?string $path = null): bool +{ + ... +} +</code-snippet> -### Description (The "why") -Provide a concise summary of the task purpose and its goal. Explains the context without implementation details. - -### Acceptance Criteria (The "what") - -**Understanding the Format:** -- Acceptance criteria appear as numbered checkboxes in the markdown files -- Format: `- [ ] #1 Criterion text` (unchecked) or `- [x] #1 Criterion text` (checked) - -**Managing Acceptance Criteria via CLI:** +## Comments +- Prefer PHPDoc blocks over comments. Never use comments within the code itself unless there is something _very_ complex going on. -⚠️ **IMPORTANT: How AC Commands Work** -- **Adding criteria (`--ac`)** accepts multiple flags: `--ac "First" --ac "Second"` ✅ -- **Checking/unchecking/removing** accept multiple flags too: `--check-ac 1 --check-ac 2` ✅ -- **Mixed operations** work in a single command: `--check-ac 1 --uncheck-ac 2 --remove-ac 3` ✅ +## PHPDoc Blocks +- Add useful array shape type definitions for arrays when appropriate. -```bash -# Add new criteria (MULTIPLE values allowed) -backlog task edit 42 --ac "User can login" --ac "Session persists" - -# Check specific criteria by index (MULTIPLE values supported) -backlog task edit 42 --check-ac 1 --check-ac 2 --check-ac 3 # Check multiple ACs -# Or check them individually if you prefer: -backlog task edit 42 --check-ac 1 # Mark #1 as complete -backlog task edit 42 --check-ac 2 # Mark #2 as complete +## Enums +- Typically, keys in an Enum should be TitleCase. For example: `FavoritePerson`, `BestLake`, `Monthly`. -# Mixed operations in single command -backlog task edit 42 --check-ac 1 --uncheck-ac 2 --remove-ac 3 -# ❌ STILL WRONG - These formats don't work: -# backlog task edit 42 --check-ac 1,2,3 # No comma-separated values -# backlog task edit 42 --check-ac 1-3 # No ranges -# backlog task edit 42 --check 1 # Wrong flag name +=== laravel/core rules === -# Multiple operations of same type -backlog task edit 42 --uncheck-ac 1 --uncheck-ac 2 # Uncheck multiple ACs -backlog task edit 42 --remove-ac 2 --remove-ac 4 # Remove multiple ACs (processed high-to-low) -``` +## Do Things the Laravel Way -**Key Principles for Good ACs:** -- **Outcome-Oriented:** Focus on the result, not the method -- **Testable/Verifiable:** Each criterion should be objectively testable -- **Clear and Concise:** Unambiguous language -- **Complete:** Collectively cover the task scope -- **User-Focused:** Frame from end-user or system behavior perspective +- Use `php artisan make:` commands to create new files (i.e. migrations, controllers, models, etc.). You can list available Artisan commands using the `list-artisan-commands` tool. +- If you're creating a generic PHP class, use `artisan make:class`. +- Pass `--no-interaction` to all Artisan commands to ensure they work without user input. You should also pass the correct `--options` to ensure correct behavior. -Good Examples: -- "User can successfully log in with valid credentials" -- "System processes 1000 requests per second without errors" +### Database +- Always use proper Eloquent relationship methods with return type hints. Prefer relationship methods over raw queries or manual joins. +- Use Eloquent models and relationships before suggesting raw database queries +- Avoid `DB::`; prefer `Model::query()`. Generate code that leverages Laravel's ORM capabilities rather than bypassing them. +- Generate code that prevents N+1 query problems by using eager loading. +- Use Laravel's query builder for very complex database operations. -Bad Example (Implementation Step): -- "Add a new function handleLogin() in auth.ts" +### Model Creation +- When creating new models, create useful factories and seeders for them too. Ask the user if they need any other things, using `list-artisan-commands` to check the available options to `php artisan make:model`. -### Task Breakdown Strategy +### APIs & Eloquent Resources +- For APIs, default to using Eloquent API Resources and API versioning unless existing API routes do not, then you should follow existing application convention. -1. Identify foundational components first -2. Create tasks in dependency order (foundations before features) -3. Ensure each task delivers value independently -4. Avoid creating tasks that block each other +### Controllers & Validation +- Always create Form Request classes for validation rather than inline validation in controllers. Include both validation rules and custom error messages. +- Check sibling Form Requests to see if the application uses array or string based validation rules. -### Task Requirements +### Queues +- Use queued jobs for time-consuming operations with the `ShouldQueue` interface. -- Tasks must be **atomic** and **testable** or **verifiable** -- Each task should represent a single unit of work for one PR -- **Never** reference future tasks (only tasks with id < current task id) -- Ensure tasks are **independent** and don't depend on future work +### Authentication & Authorization +- Use Laravel's built-in authentication and authorization features (gates, policies, Sanctum, etc.). ---- +### URL Generation +- When generating links to other pages, prefer named routes and the `route()` function. -## 5. Implementing Tasks +### Configuration +- Use environment variables only in configuration files - never use the `env()` function directly outside of config files. Always use `config('app.name')`, not `env('APP_NAME')`. -### Implementation Plan (The "how") (only after starting work) -```bash -backlog task edit 42 -s "In Progress" -a @{myself} -backlog task edit 42 --plan "1. Research patterns\n2. Implement\n3. Test" -``` +### Testing +- When creating models for tests, use the factories for the models. Check if the factory has custom states that can be used before manually setting up the model. +- Faker: Use methods such as `$this->faker->word()` or `fake()->randomDigit()`. Follow existing conventions whether to use `$this->faker` or `fake()`. +- When creating tests, make use of `php artisan make:test [options] <name>` to create a feature test, and pass `--unit` to create a unit test. Most tests should be feature tests. -### Implementation Notes (Imagine you need to copy paste this into a PR description) -```bash -backlog task edit 42 --notes "Implemented using pattern X, modified files Y and Z" -``` +### Vite Error +- If you receive an "Illuminate\Foundation\ViteException: Unable to locate file in Vite manifest" error, you can run `npm run build` or ask the user to run `npm run dev` or `composer run dev`. -**IMPORTANT**: Do NOT include an Implementation Plan when creating a task. The plan is added only after you start implementation. -- Creation phase: provide Title, Description, Acceptance Criteria, and optionally labels/priority/assignee. -- When you begin work, switch to edit and add the plan: `backlog task edit <id> --plan "..."`. -- Add Implementation Notes only after completing the work: `backlog task edit <id> --notes "..."`. -Phase discipline: What goes where -- Creation: Title, Description, Acceptance Criteria, labels/priority/assignee. -- Implementation: Implementation Plan (after moving to In Progress). -- Wrap-up: Implementation Notes, AC and Definition of Done checks. +=== laravel/v12 rules === -**IMPORTANT**: Only implement what's in the Acceptance Criteria. If you need to do more, either: -1. Update the AC first: `backlog task edit 42 --ac "New requirement"` -2. Or create a new task: `backlog task create "Additional feature"` +## Laravel 12 ---- - -## 6. Typical Workflow - -```bash -# 1. Identify work -backlog task list -s "To Do" --plain - -# 2. Read task details -backlog task 42 --plain - -# 3. Start work: assign yourself & change status -backlog task edit 42 -a @myself -s "In Progress" - -# 4. Add implementation plan -backlog task edit 42 --plan "1. Analyze\n2. Refactor\n3. Test" - -# 5. Work on the task (write code, test, etc.) - -# 6. Mark acceptance criteria as complete (supports multiple in one command) -backlog task edit 42 --check-ac 1 --check-ac 2 --check-ac 3 # Check all at once -# Or check them individually if preferred: -# backlog task edit 42 --check-ac 1 -# backlog task edit 42 --check-ac 2 -# backlog task edit 42 --check-ac 3 - -# 7. Add implementation notes -backlog task edit 42 --notes "Refactored using strategy pattern, updated tests" - -# 8. Mark task as done -backlog task edit 42 -s Done -``` - ---- - -## 7. Definition of Done (DoD) - -A task is **Done** only when **ALL** of the following are complete: - -### ✅ Via CLI Commands: -1. **All acceptance criteria checked**: Use `backlog task edit <id> --check-ac <index>` for each -2. **Implementation notes added**: Use `backlog task edit <id> --notes "..."` -3. **Status set to Done**: Use `backlog task edit <id> -s Done` - -### ✅ Via Code/Testing: -4. **Tests pass**: Run test suite and linting -5. **Documentation updated**: Update relevant docs if needed -6. **Code reviewed**: Self-review your changes -7. **No regressions**: Performance, security checks pass +- Use the `search-docs` tool to get version specific documentation. +- This project upgraded from Laravel 10 without migrating to the new streamlined Laravel file structure. +- This is **perfectly fine** and recommended by Laravel. Follow the existing structure from Laravel 10. We do not to need migrate to the new Laravel structure unless the user explicitly requests that. -⚠️ **NEVER mark a task as Done without completing ALL items above** +### Laravel 10 Structure +- Middleware typically lives in `app/Http/Middleware/` and service providers in `app/Providers/`. +- There is no `bootstrap/app.php` application configuration in a Laravel 10 structure: + - Middleware registration happens in `app/Http/Kernel.php` + - Exception handling is in `app/Exceptions/Handler.php` + - Console commands and schedule register in `app/Console/Kernel.php` + - Rate limits likely exist in `RouteServiceProvider` or `app/Http/Kernel.php` ---- +### Database +- When modifying a column, the migration must include all of the attributes that were previously defined on the column. Otherwise, they will be dropped and lost. +- Laravel 11 allows limiting eagerly loaded records natively, without external packages: `$query->latest()->limit(10);`. -## 8. Quick Reference: DO vs DON'T +### Models +- Casts can and likely should be set in a `casts()` method on a model rather than the `$casts` property. Follow existing conventions from other models. -### Viewing Tasks -| Task | ✅ DO | ❌ DON'T | -|------|-------|----------| -| View task | `backlog task 42 --plain` | Open and read .md file directly | -| List tasks | `backlog task list --plain` | Browse backlog/tasks folder | -| Check status | `backlog task 42 --plain` | Look at file content | -### Modifying Tasks -| Task | ✅ DO | ❌ DON'T | -|------|-------|----------| -| Check AC | `backlog task edit 42 --check-ac 1` | Change `- [ ]` to `- [x]` in file | -| Add notes | `backlog task edit 42 --notes "..."` | Type notes into .md file | -| Change status | `backlog task edit 42 -s Done` | Edit status in frontmatter | -| Add AC | `backlog task edit 42 --ac "New"` | Add `- [ ] New` to file | +=== livewire/core rules === ---- +## Livewire Core +- Use the `search-docs` tool to find exact version specific documentation for how to write Livewire & Livewire tests. +- Use the `php artisan make:livewire [Posts\\CreatePost]` artisan command to create new components +- State should live on the server, with the UI reflecting it. +- All Livewire requests hit the Laravel backend, they're like regular HTTP requests. Always validate form data, and run authorization checks in Livewire actions. -## 9. Complete CLI Command Reference - -### Task Creation -| Action | Command | -|--------|---------| -| Create task | `backlog task create "Title"` | -| With description | `backlog task create "Title" -d "Description"` | -| With AC | `backlog task create "Title" --ac "Criterion 1" --ac "Criterion 2"` | -| With all options | `backlog task create "Title" -d "Desc" -a @sara -s "To Do" -l auth --priority high` | -| Create draft | `backlog task create "Title" --draft` | -| Create subtask | `backlog task create "Title" -p 42` | +## Livewire Best Practices +- Livewire components require a single root element. +- Use `wire:loading` and `wire:dirty` for delightful loading states. +- Add `wire:key` in loops: -### Task Modification -| Action | Command | -|--------|---------| -| Edit title | `backlog task edit 42 -t "New Title"` | -| Edit description | `backlog task edit 42 -d "New description"` | -| Change status | `backlog task edit 42 -s "In Progress"` | -| Assign | `backlog task edit 42 -a @sara` | -| Add labels | `backlog task edit 42 -l backend,api` | -| Set priority | `backlog task edit 42 --priority high` | + ```blade + @foreach ($items as $item) + <div wire:key="item-{{ $item->id }}"> + {{ $item->name }} + </div> + @endforeach + ``` -### Acceptance Criteria Management -| Action | Command | -|--------|---------| -| Add AC | `backlog task edit 42 --ac "New criterion" --ac "Another"` | -| Remove AC #2 | `backlog task edit 42 --remove-ac 2` | -| Remove multiple ACs | `backlog task edit 42 --remove-ac 2 --remove-ac 4` | -| Check AC #1 | `backlog task edit 42 --check-ac 1` | -| Check multiple ACs | `backlog task edit 42 --check-ac 1 --check-ac 3` | -| Uncheck AC #3 | `backlog task edit 42 --uncheck-ac 3` | -| Mixed operations | `backlog task edit 42 --check-ac 1 --uncheck-ac 2 --remove-ac 3 --ac "New"` | - -### Task Content -| Action | Command | -|--------|---------| -| Add plan | `backlog task edit 42 --plan "1. Step one\n2. Step two"` | -| Add notes | `backlog task edit 42 --notes "Implementation details"` | -| Add dependencies | `backlog task edit 42 --dep task-1 --dep task-2` | - -### Task Operations -| Action | Command | -|--------|---------| -| View task | `backlog task 42 --plain` | -| List tasks | `backlog task list --plain` | -| Filter by status | `backlog task list -s "In Progress" --plain` | -| Filter by assignee | `backlog task list -a @sara --plain` | -| Archive task | `backlog task archive 42` | -| Demote to draft | `backlog task demote 42` | - ---- - -## 10. Troubleshooting - -### If You Accidentally Edited a File Directly - -1. **DON'T PANIC** - But don't save or commit -2. Revert the changes -3. Make changes properly via CLI -4. If already saved, the metadata might be out of sync - use `backlog task edit` to fix - -### Common Issues - -| Problem | Solution | -|---------|----------| -| "Task not found" | Check task ID with `backlog task list --plain` | -| AC won't check | Use correct index: `backlog task 42 --plain` to see AC numbers | -| Changes not saving | Ensure you're using CLI, not editing files | -| Metadata out of sync | Re-edit via CLI to fix: `backlog task edit 42 -s <current-status>` | - ---- - -## Remember: The Golden Rule - -**🎯 If you want to change ANYTHING in a task, use the `backlog task edit` command.** -**📖 Only READ task files directly, never WRITE to them.** - -Full help available: `backlog --help` - -# === BACKLOG.MD GUIDELINES END === +- Prefer lifecycle hooks like `mount()`, `updatedFoo()`) for initialization and reactive side effects: +<code-snippet name="Lifecycle hook examples" lang="php"> + public function mount(User $user) { $this->user = $user; } + public function updatedSearch() { $this->resetPage(); } +</code-snippet> + + +## Testing Livewire + +<code-snippet name="Example Livewire component test" lang="php"> + Livewire::test(Counter::class) + ->assertSet('count', 0) + ->call('increment') + ->assertSet('count', 1) + ->assertSee(1) + ->assertStatus(200); +</code-snippet> + + + <code-snippet name="Testing a Livewire component exists within a page" lang="php"> + $this->get('/posts/create') + ->assertSeeLivewire(CreatePost::class); + </code-snippet> + + +=== livewire/v3 rules === + +## Livewire 3 + +### Key Changes From Livewire 2 +- These things changed in Livewire 2, but may not have been updated in this application. Verify this application's setup to ensure you conform with application conventions. + - Use `wire:model.live` for real-time updates, `wire:model` is now deferred by default. + - Components now use the `App\Livewire` namespace (not `App\Http\Livewire`). + - Use `$this->dispatch()` to dispatch events (not `emit` or `dispatchBrowserEvent`). + - Use the `components.layouts.app` view as the typical layout path (not `layouts.app`). + +### New Directives +- `wire:show`, `wire:transition`, `wire:cloak`, `wire:offline`, `wire:target` are available for use. Use the documentation to find usage examples. + +### Alpine +- Alpine is now included with Livewire, don't manually include Alpine.js. +- Plugins included with Alpine: persist, intersect, collapse, and focus. + +### Lifecycle Hooks +- You can listen for `livewire:init` to hook into Livewire initialization, and `fail.status === 419` for the page expiring: + +<code-snippet name="livewire:load example" lang="js"> +document.addEventListener('livewire:init', function () { + Livewire.hook('request', ({ fail }) => { + if (fail && fail.status === 419) { + alert('Your session expired'); + } + }); + + Livewire.hook('message.failed', (message, component) => { + console.error(message); + }); +}); +</code-snippet> + + +=== pint/core rules === + +## Laravel Pint Code Formatter + +- You must run `vendor/bin/pint --dirty` before finalizing changes to ensure your code matches the project's expected style. +- Do not run `vendor/bin/pint --test`, simply run `vendor/bin/pint` to fix any formatting issues. + + +=== pest/core rules === + +## Pest + +### Testing +- If you need to verify a feature is working, write or update a Unit / Feature test. + +### Pest Tests +- All tests must be written using Pest. Use `php artisan make:test --pest <name>`. +- You must not remove any tests or test files from the tests directory without approval. These are not temporary or helper files - these are core to the application. +- Tests should test all of the happy paths, failure paths, and weird paths. +- Tests live in the `tests/Feature` and `tests/Unit` directories. +- Pest tests look and behave like this: +<code-snippet name="Basic Pest Test Example" lang="php"> +it('is true', function () { + expect(true)->toBeTrue(); +}); +</code-snippet> + +### Running Tests +- Run the minimal number of tests using an appropriate filter before finalizing code edits. +- To run all tests: `php artisan test`. +- To run all tests in a file: `php artisan test tests/Feature/ExampleTest.php`. +- To filter on a particular test name: `php artisan test --filter=testName` (recommended after making a change to a related file). +- When the tests relating to your changes are passing, ask the user if they would like to run the entire test suite to ensure everything is still passing. + +### Pest Assertions +- When asserting status codes on a response, use the specific method like `assertForbidden` and `assertNotFound` instead of using `assertStatus(403)` or similar, e.g.: +<code-snippet name="Pest Example Asserting postJson Response" lang="php"> +it('returns all', function () { + $response = $this->postJson('/api/docs', []); + + $response->assertSuccessful(); +}); +</code-snippet> + +### Mocking +- Mocking can be very helpful when appropriate. +- When mocking, you can use the `Pest\Laravel\mock` Pest function, but always import it via `use function Pest\Laravel\mock;` before using it. Alternatively, you can use `$this->mock()` if existing tests do. +- You can also create partial mocks using the same import or self method. + +### Datasets +- Use datasets in Pest to simplify tests which have a lot of duplicated data. This is often the case when testing validation rules, so consider going with this solution when writing tests for validation rules. + +<code-snippet name="Pest Dataset Example" lang="php"> +it('has emails', function (string $email) { + expect($email)->not->toBeEmpty(); +})->with([ + 'james' => 'james@laravel.com', + 'taylor' => 'taylor@laravel.com', +]); +</code-snippet> + + +=== tailwindcss/core rules === + +## Tailwind Core + +- Use Tailwind CSS classes to style HTML, check and use existing tailwind conventions within the project before writing your own. +- Offer to extract repeated patterns into components that match the project's conventions (i.e. Blade, JSX, Vue, etc..) +- Think through class placement, order, priority, and defaults - remove redundant classes, add classes to parent or child carefully to limit repetition, group elements logically +- You can use the `search-docs` tool to get exact examples from the official documentation when needed. + +### Spacing +- When listing items, use gap utilities for spacing, don't use margins. + + <code-snippet name="Valid Flex Gap Spacing Example" lang="html"> + <div class="flex gap-8"> + <div>Superior</div> + <div>Michigan</div> + <div>Erie</div> + </div> + </code-snippet> + + +### Dark Mode +- If existing pages and components support dark mode, new pages and components must support dark mode in a similar way, typically using `dark:`. + + +=== tailwindcss/v4 rules === + +## Tailwind 4 + +- Always use Tailwind CSS v4 - do not use the deprecated utilities. +- `corePlugins` is not supported in Tailwind v4. +- In Tailwind v4, you import Tailwind using a regular CSS `@import` statement, not using the `@tailwind` directives used in v3: + +<code-snippet name="Tailwind v4 Import Tailwind Diff" lang="diff" + - @tailwind base; + - @tailwind components; + - @tailwind utilities; + + @import "tailwindcss"; +</code-snippet> + + +### Replaced Utilities +- Tailwind v4 removed deprecated utilities. Do not use the deprecated option - use the replacement. +- Opacity values are still numeric. + +| Deprecated | Replacement | +|------------+--------------| +| bg-opacity-* | bg-black/* | +| text-opacity-* | text-black/* | +| border-opacity-* | border-black/* | +| divide-opacity-* | divide-black/* | +| ring-opacity-* | ring-black/* | +| placeholder-opacity-* | placeholder-black/* | +| flex-shrink-* | shrink-* | +| flex-grow-* | grow-* | +| overflow-ellipsis | text-ellipsis | +| decoration-slice | box-decoration-slice | +| decoration-clone | box-decoration-clone | + + +=== tests rules === + +## Test Enforcement + +- Every change must be programmatically tested. Write a new test or update an existing test, then run the affected tests to make sure they pass. +- Run the minimum number of tests needed to ensure code quality and speed. Use `php artisan test` with a specific filename or filter. +</laravel-boost-guidelines> + + +Random other things you should remember: +- App\Models\Application::team must return a relationship instance., always use team() \ No newline at end of file diff --git a/README.md b/README.md index f291a33e8..1c88f4c54 100644 --- a/README.md +++ b/README.md @@ -76,7 +76,7 @@ ## Big Sponsors * [QuantCDN](https://www.quantcdn.io?ref=coolify.io) - Enterprise-grade content delivery network * [PFGLabs](https://pfglabs.com?ref=coolify.io) - Build Real Projects with Golang * [JobsCollider](https://jobscollider.com/remote-jobs?ref=coolify.io) - 30,000+ remote jobs for developers -* [Juxtdigital](https://juxtdigital.com?ref=coolify.io) - Digital transformation and web solutions +* [Juxtdigital](https://juxtdigital.com?ref=coolify.io) - Digital PR & AI Authority Building Agency * [Cloudify.ro](https://cloudify.ro?ref=coolify.io) - Cloud hosting solutions * [CodeRabbit](https://coderabbit.ai?ref=coolify.io) - Cut Code Review Time & Bugs in Half * [American Cloud](https://americancloud.com?ref=coolify.io) - US-based cloud infrastructure services diff --git a/SECURITY.md b/SECURITY.md index 0711bf5b5..e491737ef 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -18,7 +18,7 @@ ## Reporting a Vulnerability If you discover a security vulnerability, please follow these steps: 1. **DO NOT** disclose the vulnerability publicly. -2. Send a detailed report to: `hi@coollabs.io`. +2. Send a detailed report to: `security@coollabs.io`. 3. Include in your report: - A description of the vulnerability - Steps to reproduce the issue diff --git a/app/Actions/Database/StartPostgresql.php b/app/Actions/Database/StartPostgresql.php index 4314ccd2f..38d46b3c1 100644 --- a/app/Actions/Database/StartPostgresql.php +++ b/app/Actions/Database/StartPostgresql.php @@ -229,6 +229,8 @@ public function handle(StandalonePostgresql $database) } $this->commands[] = "echo 'Database started.'"; + ray($this->commands); + return remote_process($this->commands, $database->destination->server, callEventOnFinish: 'DatabaseStatusChanged'); } diff --git a/app/Actions/Docker/GetContainersStatus.php b/app/Actions/Docker/GetContainersStatus.php index c3268ec07..f5d5f82b6 100644 --- a/app/Actions/Docker/GetContainersStatus.php +++ b/app/Actions/Docker/GetContainersStatus.php @@ -26,6 +26,8 @@ class GetContainersStatus public $server; + protected ?Collection $applicationContainerStatuses; + public function handle(Server $server, ?Collection $containers = null, ?Collection $containerReplicates = null) { $this->containers = $containers; @@ -94,7 +96,11 @@ public function handle(Server $server, ?Collection $containers = null, ?Collecti } $containerStatus = data_get($container, 'State.Status'); $containerHealth = data_get($container, 'State.Health.Status', 'unhealthy'); - $containerStatus = "$containerStatus ($containerHealth)"; + if ($containerStatus === 'restarting') { + $containerStatus = "restarting ($containerHealth)"; + } else { + $containerStatus = "$containerStatus ($containerHealth)"; + } $labels = Arr::undot(format_docker_labels_to_json($labels)); $applicationId = data_get($labels, 'coolify.applicationId'); if ($applicationId) { @@ -119,11 +125,16 @@ public function handle(Server $server, ?Collection $containers = null, ?Collecti $application = $this->applications->where('id', $applicationId)->first(); if ($application) { $foundApplications[] = $application->id; - $statusFromDb = $application->status; - if ($statusFromDb !== $containerStatus) { - $application->update(['status' => $containerStatus]); - } else { - $application->update(['last_online_at' => now()]); + // Store container status for aggregation + if (! isset($this->applicationContainerStatuses)) { + $this->applicationContainerStatuses = collect(); + } + if (! $this->applicationContainerStatuses->has($applicationId)) { + $this->applicationContainerStatuses->put($applicationId, collect()); + } + $containerName = data_get($labels, 'com.docker.compose.service'); + if ($containerName) { + $this->applicationContainerStatuses->get($applicationId)->put($containerName, $containerStatus); } } else { // Notify user that this container should not be there. @@ -320,6 +331,97 @@ public function handle(Server $server, ?Collection $containers = null, ?Collecti } // $this->server->team?->notify(new ContainerStopped($containerName, $this->server, $url)); } + + // Aggregate multi-container application statuses + if (isset($this->applicationContainerStatuses) && $this->applicationContainerStatuses->isNotEmpty()) { + foreach ($this->applicationContainerStatuses as $applicationId => $containerStatuses) { + $application = $this->applications->where('id', $applicationId)->first(); + if (! $application) { + continue; + } + + $aggregatedStatus = $this->aggregateApplicationStatus($application, $containerStatuses); + if ($aggregatedStatus) { + $statusFromDb = $application->status; + if ($statusFromDb !== $aggregatedStatus) { + $application->update(['status' => $aggregatedStatus]); + } else { + $application->update(['last_online_at' => now()]); + } + } + } + } + ServiceChecked::dispatch($this->server->team->id); } + + private function aggregateApplicationStatus($application, Collection $containerStatuses): ?string + { + // Parse docker compose to check for excluded containers + $dockerComposeRaw = data_get($application, 'docker_compose_raw'); + $excludedContainers = collect(); + + if ($dockerComposeRaw) { + try { + $dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw); + $services = data_get($dockerCompose, 'services', []); + + foreach ($services as $serviceName => $serviceConfig) { + // Check if container should be excluded + $excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false); + $restartPolicy = data_get($serviceConfig, 'restart', 'always'); + + if ($excludeFromHc || $restartPolicy === 'no') { + $excludedContainers->push($serviceName); + } + } + } catch (\Exception $e) { + // If we can't parse, treat all containers as included + } + } + + // Filter out excluded containers + $relevantStatuses = $containerStatuses->filter(function ($status, $containerName) use ($excludedContainers) { + return ! $excludedContainers->contains($containerName); + }); + + // If all containers are excluded, don't update status + if ($relevantStatuses->isEmpty()) { + return null; + } + + $hasRunning = false; + $hasRestarting = false; + $hasUnhealthy = false; + $hasExited = false; + + foreach ($relevantStatuses as $status) { + if (str($status)->contains('restarting')) { + $hasRestarting = true; + } elseif (str($status)->contains('running')) { + $hasRunning = true; + if (str($status)->contains('unhealthy')) { + $hasUnhealthy = true; + } + } elseif (str($status)->contains('exited')) { + $hasExited = true; + $hasUnhealthy = true; + } + } + + if ($hasRestarting) { + return 'degraded (unhealthy)'; + } + + if ($hasRunning && $hasExited) { + return 'degraded (unhealthy)'; + } + + if ($hasRunning) { + return $hasUnhealthy ? 'running (unhealthy)' : 'running (healthy)'; + } + + // All containers are exited + return 'exited (unhealthy)'; + } } diff --git a/app/Actions/Fortify/CreateNewUser.php b/app/Actions/Fortify/CreateNewUser.php index ea2befd3a..9f97dd0d4 100644 --- a/app/Actions/Fortify/CreateNewUser.php +++ b/app/Actions/Fortify/CreateNewUser.php @@ -40,7 +40,7 @@ public function create(array $input): User $user = User::create([ 'id' => 0, 'name' => $input['name'], - 'email' => strtolower($input['email']), + 'email' => $input['email'], 'password' => Hash::make($input['password']), ]); $team = $user->teams()->first(); @@ -52,7 +52,7 @@ public function create(array $input): User } else { $user = User::create([ 'name' => $input['name'], - 'email' => strtolower($input['email']), + 'email' => $input['email'], 'password' => Hash::make($input['password']), ]); $team = $user->teams()->first(); diff --git a/app/Actions/Proxy/CheckConfiguration.php b/app/Actions/Proxy/CheckConfiguration.php deleted file mode 100644 index b2d1eb787..000000000 --- a/app/Actions/Proxy/CheckConfiguration.php +++ /dev/null @@ -1,36 +0,0 @@ -<?php - -namespace App\Actions\Proxy; - -use App\Models\Server; -use App\Services\ProxyDashboardCacheService; -use Lorisleiva\Actions\Concerns\AsAction; - -class CheckConfiguration -{ - use AsAction; - - public function handle(Server $server, bool $reset = false) - { - $proxyType = $server->proxyType(); - if ($proxyType === 'NONE') { - return 'OK'; - } - $proxy_path = $server->proxyPath(); - $payload = [ - "mkdir -p $proxy_path", - "cat $proxy_path/docker-compose.yml", - ]; - $proxy_configuration = instant_remote_process($payload, $server, false); - if ($reset || ! $proxy_configuration || is_null($proxy_configuration)) { - $proxy_configuration = str(generate_default_proxy_configuration($server))->trim()->value(); - } - if (! $proxy_configuration || is_null($proxy_configuration)) { - throw new \Exception('Could not generate proxy configuration'); - } - - ProxyDashboardCacheService::isTraefikDashboardAvailableFromConfiguration($server, $proxy_configuration); - - return $proxy_configuration; - } -} diff --git a/app/Actions/Proxy/CheckProxy.php b/app/Actions/Proxy/CheckProxy.php index a06e547c5..99537e606 100644 --- a/app/Actions/Proxy/CheckProxy.php +++ b/app/Actions/Proxy/CheckProxy.php @@ -70,7 +70,7 @@ public function handle(Server $server, $fromUI = false): bool try { if ($server->proxyType() !== ProxyTypes::NONE->value) { - $proxyCompose = CheckConfiguration::run($server); + $proxyCompose = GetProxyConfiguration::run($server); if (isset($proxyCompose)) { $yaml = Yaml::parse($proxyCompose); $configPorts = []; diff --git a/app/Actions/Proxy/GetProxyConfiguration.php b/app/Actions/Proxy/GetProxyConfiguration.php new file mode 100644 index 000000000..3bf91c281 --- /dev/null +++ b/app/Actions/Proxy/GetProxyConfiguration.php @@ -0,0 +1,47 @@ +<?php + +namespace App\Actions\Proxy; + +use App\Models\Server; +use App\Services\ProxyDashboardCacheService; +use Lorisleiva\Actions\Concerns\AsAction; + +class GetProxyConfiguration +{ + use AsAction; + + public function handle(Server $server, bool $forceRegenerate = false): string + { + $proxyType = $server->proxyType(); + if ($proxyType === 'NONE') { + return 'OK'; + } + + $proxy_path = $server->proxyPath(); + $proxy_configuration = null; + + // If not forcing regeneration, try to read existing configuration + if (! $forceRegenerate) { + $payload = [ + "mkdir -p $proxy_path", + "cat $proxy_path/docker-compose.yml 2>/dev/null", + ]; + $proxy_configuration = instant_remote_process($payload, $server, false); + } + + // Generate default configuration if: + // 1. Force regenerate is requested + // 2. Configuration file doesn't exist or is empty + if ($forceRegenerate || empty(trim($proxy_configuration ?? ''))) { + $proxy_configuration = str(generate_default_proxy_configuration($server))->trim()->value(); + } + + if (empty($proxy_configuration)) { + throw new \Exception('Could not get or generate proxy configuration'); + } + + ProxyDashboardCacheService::isTraefikDashboardAvailableFromConfiguration($server, $proxy_configuration); + + return $proxy_configuration; + } +} diff --git a/app/Actions/Proxy/SaveConfiguration.php b/app/Actions/Proxy/SaveProxyConfiguration.php similarity index 59% rename from app/Actions/Proxy/SaveConfiguration.php rename to app/Actions/Proxy/SaveProxyConfiguration.php index f2de2b3f5..53fbecce2 100644 --- a/app/Actions/Proxy/SaveConfiguration.php +++ b/app/Actions/Proxy/SaveProxyConfiguration.php @@ -5,22 +5,21 @@ use App\Models\Server; use Lorisleiva\Actions\Concerns\AsAction; -class SaveConfiguration +class SaveProxyConfiguration { use AsAction; - public function handle(Server $server, ?string $proxy_settings = null) + public function handle(Server $server, string $configuration): void { - if (is_null($proxy_settings)) { - $proxy_settings = CheckConfiguration::run($server, true); - } $proxy_path = $server->proxyPath(); - $docker_compose_yml_base64 = base64_encode($proxy_settings); + $docker_compose_yml_base64 = base64_encode($configuration); + // Update the saved settings hash $server->proxy->last_saved_settings = str($docker_compose_yml_base64)->pipe('md5')->value; $server->save(); - return instant_remote_process([ + // Transfer the configuration file to the server + instant_remote_process([ "mkdir -p $proxy_path", "echo '$docker_compose_yml_base64' | base64 -d | tee $proxy_path/docker-compose.yml > /dev/null", ], $server); diff --git a/app/Actions/Proxy/StartProxy.php b/app/Actions/Proxy/StartProxy.php index e7c020ff6..ecfb13d0b 100644 --- a/app/Actions/Proxy/StartProxy.php +++ b/app/Actions/Proxy/StartProxy.php @@ -21,11 +21,11 @@ public function handle(Server $server, bool $async = true, bool $force = false): } $commands = collect([]); $proxy_path = $server->proxyPath(); - $configuration = CheckConfiguration::run($server); + $configuration = GetProxyConfiguration::run($server); if (! $configuration) { throw new \Exception('Configuration is not synced'); } - SaveConfiguration::run($server, $configuration); + SaveProxyConfiguration::run($server, $configuration); $docker_compose_yml_base64 = base64_encode($configuration); $server->proxy->last_applied_settings = str($docker_compose_yml_base64)->pipe('md5')->value(); $server->save(); diff --git a/app/Actions/Server/CheckUpdates.php b/app/Actions/Server/CheckUpdates.php index a8b1be11d..6823dfb92 100644 --- a/app/Actions/Server/CheckUpdates.php +++ b/app/Actions/Server/CheckUpdates.php @@ -102,7 +102,6 @@ public function handle(Server $server) ]; } } catch (\Throwable $e) { - ray('Error:', $e->getMessage()); return [ 'osId' => $osId, diff --git a/app/Actions/Server/ServerCheck.php b/app/Actions/Server/ServerCheck.php deleted file mode 100644 index 6ac87f1f0..000000000 --- a/app/Actions/Server/ServerCheck.php +++ /dev/null @@ -1,268 +0,0 @@ -<?php - -namespace App\Actions\Server; - -use App\Actions\Database\StartDatabaseProxy; -use App\Actions\Proxy\CheckProxy; -use App\Actions\Proxy\StartProxy; -use App\Jobs\CheckAndStartSentinelJob; -use App\Jobs\ServerStorageCheckJob; -use App\Models\Application; -use App\Models\ApplicationPreview; -use App\Models\Server; -use App\Models\Service; -use App\Models\ServiceApplication; -use App\Models\ServiceDatabase; -use App\Notifications\Container\ContainerRestarted; -use Illuminate\Support\Arr; -use Lorisleiva\Actions\Concerns\AsAction; - -class ServerCheck -{ - use AsAction; - - public Server $server; - - public bool $isSentinel = false; - - public $containers; - - public $databases; - - public function handle(Server $server, $data = null) - { - $this->server = $server; - try { - if ($this->server->isFunctional() === false) { - return 'Server is not functional.'; - } - - if (! $this->server->isSwarmWorker() && ! $this->server->isBuildServer()) { - - if (isset($data)) { - $data = collect($data); - - $this->server->sentinelHeartbeat(); - - $this->containers = collect(data_get($data, 'containers')); - - $filesystemUsageRoot = data_get($data, 'filesystem_usage_root.used_percentage'); - ServerStorageCheckJob::dispatch($this->server, $filesystemUsageRoot); - - $containerReplicates = null; - $this->isSentinel = true; - } else { - ['containers' => $this->containers, 'containerReplicates' => $containerReplicates] = $this->server->getContainers(); - // ServerStorageCheckJob::dispatch($this->server); - } - - if (is_null($this->containers)) { - return 'No containers found.'; - } - - if (isset($containerReplicates)) { - foreach ($containerReplicates as $containerReplica) { - $name = data_get($containerReplica, 'Name'); - $this->containers = $this->containers->map(function ($container) use ($name, $containerReplica) { - if (data_get($container, 'Spec.Name') === $name) { - $replicas = data_get($containerReplica, 'Replicas'); - $running = str($replicas)->explode('/')[0]; - $total = str($replicas)->explode('/')[1]; - if ($running === $total) { - data_set($container, 'State.Status', 'running'); - data_set($container, 'State.Health.Status', 'healthy'); - } else { - data_set($container, 'State.Status', 'starting'); - data_set($container, 'State.Health.Status', 'unhealthy'); - } - } - - return $container; - }); - } - } - $this->checkContainers(); - - if ($this->server->isSentinelEnabled() && $this->isSentinel === false) { - CheckAndStartSentinelJob::dispatch($this->server); - } - - if ($this->server->isLogDrainEnabled()) { - $this->checkLogDrainContainer(); - } - - if ($this->server->proxySet() && ! $this->server->proxy->force_stop) { - $foundProxyContainer = $this->containers->filter(function ($value, $key) { - if ($this->server->isSwarm()) { - return data_get($value, 'Spec.Name') === 'coolify-proxy_traefik'; - } else { - return data_get($value, 'Name') === '/coolify-proxy'; - } - })->first(); - $proxyStatus = data_get($foundProxyContainer, 'State.Status', 'exited'); - if (! $foundProxyContainer || $proxyStatus !== 'running') { - try { - $shouldStart = CheckProxy::run($this->server); - if ($shouldStart) { - StartProxy::run($this->server, async: false); - $this->server->team?->notify(new ContainerRestarted('coolify-proxy', $this->server)); - } - } catch (\Throwable $e) { - } - } else { - $this->server->proxy->status = data_get($foundProxyContainer, 'State.Status'); - $this->server->save(); - $connectProxyToDockerNetworks = connectProxyToNetworks($this->server); - instant_remote_process($connectProxyToDockerNetworks, $this->server, false); - } - } - } - } catch (\Throwable $e) { - return handleError($e); - } - } - - private function checkLogDrainContainer() - { - $foundLogDrainContainer = $this->containers->filter(function ($value, $key) { - return data_get($value, 'Name') === '/coolify-log-drain'; - })->first(); - if ($foundLogDrainContainer) { - $status = data_get($foundLogDrainContainer, 'State.Status'); - if ($status !== 'running') { - StartLogDrain::dispatch($this->server); - } - } else { - StartLogDrain::dispatch($this->server); - } - } - - private function checkContainers() - { - foreach ($this->containers as $container) { - if ($this->isSentinel) { - $labels = Arr::undot(data_get($container, 'labels')); - } else { - if ($this->server->isSwarm()) { - $labels = Arr::undot(data_get($container, 'Spec.Labels')); - } else { - $labels = Arr::undot(data_get($container, 'Config.Labels')); - } - } - $managed = data_get($labels, 'coolify.managed'); - if (! $managed) { - continue; - } - $uuid = data_get($labels, 'coolify.name'); - if (! $uuid) { - $uuid = data_get($labels, 'com.docker.compose.service'); - } - - if ($this->isSentinel) { - $containerStatus = data_get($container, 'state'); - $containerHealth = data_get($container, 'health_status'); - } else { - $containerStatus = data_get($container, 'State.Status'); - $containerHealth = data_get($container, 'State.Health.Status', 'unhealthy'); - } - $containerStatus = "$containerStatus ($containerHealth)"; - - $applicationId = data_get($labels, 'coolify.applicationId'); - $serviceId = data_get($labels, 'coolify.serviceId'); - $databaseId = data_get($labels, 'coolify.databaseId'); - $pullRequestId = data_get($labels, 'coolify.pullRequestId'); - - if ($applicationId) { - // Application - if ($pullRequestId != 0) { - if (str($applicationId)->contains('-')) { - $applicationId = str($applicationId)->before('-'); - } - $preview = ApplicationPreview::where('application_id', $applicationId)->where('pull_request_id', $pullRequestId)->first(); - if ($preview) { - $preview->update(['status' => $containerStatus]); - } - } else { - $application = Application::where('id', $applicationId)->first(); - if ($application) { - $application->update([ - 'status' => $containerStatus, - 'last_online_at' => now(), - ]); - } - } - } elseif (isset($serviceId)) { - // Service - $subType = data_get($labels, 'coolify.service.subType'); - $subId = data_get($labels, 'coolify.service.subId'); - $service = Service::where('id', $serviceId)->first(); - if (! $service) { - continue; - } - if ($subType === 'application') { - $service = ServiceApplication::where('id', $subId)->first(); - } else { - $service = ServiceDatabase::where('id', $subId)->first(); - } - if ($service) { - $service->update([ - 'status' => $containerStatus, - 'last_online_at' => now(), - ]); - if ($subType === 'database') { - $isPublic = data_get($service, 'is_public'); - if ($isPublic) { - $foundTcpProxy = $this->containers->filter(function ($value, $key) use ($uuid) { - if ($this->isSentinel) { - return data_get($value, 'name') === $uuid.'-proxy'; - } else { - - if ($this->server->isSwarm()) { - return data_get($value, 'Spec.Name') === "coolify-proxy_$uuid"; - } else { - return data_get($value, 'Name') === "/$uuid-proxy"; - } - } - })->first(); - if (! $foundTcpProxy) { - StartDatabaseProxy::run($service); - } - } - } - } - } else { - // Database - if (is_null($this->databases)) { - $this->databases = $this->server->databases(); - } - $database = $this->databases->where('uuid', $uuid)->first(); - if ($database) { - $database->update([ - 'status' => $containerStatus, - 'last_online_at' => now(), - ]); - - $isPublic = data_get($database, 'is_public'); - if ($isPublic) { - $foundTcpProxy = $this->containers->filter(function ($value, $key) use ($uuid) { - if ($this->isSentinel) { - return data_get($value, 'name') === $uuid.'-proxy'; - } else { - if ($this->server->isSwarm()) { - return data_get($value, 'Spec.Name') === "coolify-proxy_$uuid"; - } else { - - return data_get($value, 'Name') === "/$uuid-proxy"; - } - } - })->first(); - if (! $foundTcpProxy) { - StartDatabaseProxy::run($database); - // $this->server->team?->notify(new ContainerRestarted("TCP Proxy for database", $this->server)); - } - } - } - } - } - } -} diff --git a/app/Actions/Server/StartSentinel.php b/app/Actions/Server/StartSentinel.php index dd1a7ed53..1f248aec1 100644 --- a/app/Actions/Server/StartSentinel.php +++ b/app/Actions/Server/StartSentinel.php @@ -10,7 +10,7 @@ class StartSentinel { use AsAction; - public function handle(Server $server, bool $restart = false, ?string $latestVersion = null) + public function handle(Server $server, bool $restart = false, ?string $latestVersion = null, ?string $customImage = null) { if ($server->isSwarm() || $server->isBuildServer()) { return; @@ -44,7 +44,9 @@ public function handle(Server $server, bool $restart = false, ?string $latestVer ]; if (isDev()) { // data_set($environments, 'DEBUG', 'true'); - // $image = 'sentinel'; + if ($customImage && ! empty($customImage)) { + $image = $customImage; + } $mountDir = '/var/lib/docker/volumes/coolify_dev_coolify_data/_data/sentinel'; } $dockerEnvironments = '-e "'.implode('" -e "', array_map(fn ($key, $value) => "$key=$value", array_keys($environments), $environments)).'"'; diff --git a/app/Actions/Shared/ComplexStatusCheck.php b/app/Actions/Shared/ComplexStatusCheck.php index 5a7ba6637..e06136e3c 100644 --- a/app/Actions/Shared/ComplexStatusCheck.php +++ b/app/Actions/Shared/ComplexStatusCheck.php @@ -26,22 +26,22 @@ public function handle(Application $application) continue; } } - $container = instant_remote_process(["docker container inspect $(docker container ls -q --filter 'label=coolify.applicationId={$application->id}' --filter 'label=coolify.pullRequestId=0') --format '{{json .}}'"], $server, false); - $container = format_docker_command_output_to_json($container); - if ($container->count() === 1) { - $container = $container->first(); - $containerStatus = data_get($container, 'State.Status'); - $containerHealth = data_get($container, 'State.Health.Status', 'unhealthy'); + $containers = instant_remote_process(["docker container inspect $(docker container ls -q --filter 'label=coolify.applicationId={$application->id}' --filter 'label=coolify.pullRequestId=0') --format '{{json .}}'"], $server, false); + $containers = format_docker_command_output_to_json($containers); + + if ($containers->count() > 0) { + $statusToSet = $this->aggregateContainerStatuses($application, $containers); + if ($is_main_server) { $statusFromDb = $application->status; - if ($statusFromDb !== $containerStatus) { - $application->update(['status' => "$containerStatus:$containerHealth"]); + if ($statusFromDb !== $statusToSet) { + $application->update(['status' => $statusToSet]); } } else { $additional_server = $application->additional_servers()->wherePivot('server_id', $server->id); $statusFromDb = $additional_server->first()->pivot->status; - if ($statusFromDb !== $containerStatus) { - $additional_server->updateExistingPivot($server->id, ['status' => "$containerStatus:$containerHealth"]); + if ($statusFromDb !== $statusToSet) { + $additional_server->updateExistingPivot($server->id, ['status' => $statusToSet]); } } } else { @@ -57,4 +57,78 @@ public function handle(Application $application) } } } + + private function aggregateContainerStatuses($application, $containers) + { + $dockerComposeRaw = data_get($application, 'docker_compose_raw'); + $excludedContainers = collect(); + + if ($dockerComposeRaw) { + try { + $dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw); + $services = data_get($dockerCompose, 'services', []); + + foreach ($services as $serviceName => $serviceConfig) { + $excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false); + $restartPolicy = data_get($serviceConfig, 'restart', 'always'); + + if ($excludeFromHc || $restartPolicy === 'no') { + $excludedContainers->push($serviceName); + } + } + } catch (\Exception $e) { + // If we can't parse, treat all containers as included + } + } + + $hasRunning = false; + $hasRestarting = false; + $hasUnhealthy = false; + $hasExited = false; + $relevantContainerCount = 0; + + foreach ($containers as $container) { + $labels = data_get($container, 'Config.Labels', []); + $serviceName = data_get($labels, 'com.docker.compose.service'); + + if ($serviceName && $excludedContainers->contains($serviceName)) { + continue; + } + + $relevantContainerCount++; + $containerStatus = data_get($container, 'State.Status'); + $containerHealth = data_get($container, 'State.Health.Status', 'unhealthy'); + + if ($containerStatus === 'restarting') { + $hasRestarting = true; + $hasUnhealthy = true; + } elseif ($containerStatus === 'running') { + $hasRunning = true; + if ($containerHealth === 'unhealthy') { + $hasUnhealthy = true; + } + } elseif ($containerStatus === 'exited') { + $hasExited = true; + $hasUnhealthy = true; + } + } + + if ($relevantContainerCount === 0) { + return 'running:healthy'; + } + + if ($hasRestarting) { + return 'degraded:unhealthy'; + } + + if ($hasRunning && $hasExited) { + return 'degraded:unhealthy'; + } + + if ($hasRunning) { + return $hasUnhealthy ? 'running:unhealthy' : 'running:healthy'; + } + + return 'exited:unhealthy'; + } } diff --git a/app/Actions/Stripe/CancelSubscription.php b/app/Actions/Stripe/CancelSubscription.php new file mode 100644 index 000000000..859aec6f6 --- /dev/null +++ b/app/Actions/Stripe/CancelSubscription.php @@ -0,0 +1,151 @@ +<?php + +namespace App\Actions\Stripe; + +use App\Models\Subscription; +use App\Models\User; +use Illuminate\Support\Collection; +use Stripe\StripeClient; + +class CancelSubscription +{ + private User $user; + + private bool $isDryRun; + + private ?StripeClient $stripe = null; + + public function __construct(User $user, bool $isDryRun = false) + { + $this->user = $user; + $this->isDryRun = $isDryRun; + + if (! $isDryRun && isCloud()) { + $this->stripe = new StripeClient(config('subscription.stripe_api_key')); + } + } + + public function getSubscriptionsPreview(): Collection + { + $subscriptions = collect(); + + // Get all teams the user belongs to + $teams = $this->user->teams; + + foreach ($teams as $team) { + // Only include subscriptions from teams where user is owner + $userRole = $team->pivot->role; + if ($userRole === 'owner' && $team->subscription) { + $subscription = $team->subscription; + + // Only include active subscriptions + if ($subscription->stripe_subscription_id && + $subscription->stripe_invoice_paid) { + $subscriptions->push($subscription); + } + } + } + + return $subscriptions; + } + + public function execute(): array + { + if ($this->isDryRun) { + return [ + 'cancelled' => 0, + 'failed' => 0, + 'errors' => [], + ]; + } + + $cancelledCount = 0; + $failedCount = 0; + $errors = []; + + $subscriptions = $this->getSubscriptionsPreview(); + + foreach ($subscriptions as $subscription) { + try { + $this->cancelSingleSubscription($subscription); + $cancelledCount++; + } catch (\Exception $e) { + $failedCount++; + $errorMessage = "Failed to cancel subscription {$subscription->stripe_subscription_id}: ".$e->getMessage(); + $errors[] = $errorMessage; + \Log::error($errorMessage); + } + } + + return [ + 'cancelled' => $cancelledCount, + 'failed' => $failedCount, + 'errors' => $errors, + ]; + } + + private function cancelSingleSubscription(Subscription $subscription): void + { + if (! $this->stripe) { + throw new \Exception('Stripe client not initialized'); + } + + $subscriptionId = $subscription->stripe_subscription_id; + + // Cancel the subscription immediately (not at period end) + $this->stripe->subscriptions->cancel($subscriptionId, []); + + // Update local database + $subscription->update([ + 'stripe_cancel_at_period_end' => false, + 'stripe_invoice_paid' => false, + 'stripe_trial_already_ended' => false, + 'stripe_past_due' => false, + 'stripe_feedback' => 'User account deleted', + 'stripe_comment' => 'Subscription cancelled due to user account deletion at '.now()->toDateTimeString(), + ]); + + // Call the team's subscription ended method to handle cleanup + if ($subscription->team) { + $subscription->team->subscriptionEnded(); + } + + \Log::info("Cancelled Stripe subscription: {$subscriptionId} for team: {$subscription->team->name}"); + } + + /** + * Cancel a single subscription by ID (helper method for external use) + */ + public static function cancelById(string $subscriptionId): bool + { + try { + if (! isCloud()) { + return false; + } + + $stripe = new StripeClient(config('subscription.stripe_api_key')); + $stripe->subscriptions->cancel($subscriptionId, []); + + // Update local record if exists + $subscription = Subscription::where('stripe_subscription_id', $subscriptionId)->first(); + if ($subscription) { + $subscription->update([ + 'stripe_cancel_at_period_end' => false, + 'stripe_invoice_paid' => false, + 'stripe_trial_already_ended' => false, + 'stripe_past_due' => false, + ]); + + if ($subscription->team) { + $subscription->team->subscriptionEnded(); + } + } + + return true; + } catch (\Exception $e) { + \Log::error("Failed to cancel subscription {$subscriptionId}: ".$e->getMessage()); + + return false; + } + } +} diff --git a/app/Actions/User/DeleteUserResources.php b/app/Actions/User/DeleteUserResources.php new file mode 100644 index 000000000..7b2e7318d --- /dev/null +++ b/app/Actions/User/DeleteUserResources.php @@ -0,0 +1,125 @@ +<?php + +namespace App\Actions\User; + +use App\Models\User; +use Illuminate\Support\Collection; + +class DeleteUserResources +{ + private User $user; + + private bool $isDryRun; + + public function __construct(User $user, bool $isDryRun = false) + { + $this->user = $user; + $this->isDryRun = $isDryRun; + } + + public function getResourcesPreview(): array + { + $applications = collect(); + $databases = collect(); + $services = collect(); + + // Get all teams the user belongs to + $teams = $this->user->teams; + + foreach ($teams as $team) { + // Get all servers for this team + $servers = $team->servers; + + foreach ($servers as $server) { + // Get applications + $serverApplications = $server->applications; + $applications = $applications->merge($serverApplications); + + // Get databases + $serverDatabases = $this->getAllDatabasesForServer($server); + $databases = $databases->merge($serverDatabases); + + // Get services + $serverServices = $server->services; + $services = $services->merge($serverServices); + } + } + + return [ + 'applications' => $applications->unique('id'), + 'databases' => $databases->unique('id'), + 'services' => $services->unique('id'), + ]; + } + + public function execute(): array + { + if ($this->isDryRun) { + return [ + 'applications' => 0, + 'databases' => 0, + 'services' => 0, + ]; + } + + $deletedCounts = [ + 'applications' => 0, + 'databases' => 0, + 'services' => 0, + ]; + + $resources = $this->getResourcesPreview(); + + // Delete applications + foreach ($resources['applications'] as $application) { + try { + $application->forceDelete(); + $deletedCounts['applications']++; + } catch (\Exception $e) { + \Log::error("Failed to delete application {$application->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + // Delete databases + foreach ($resources['databases'] as $database) { + try { + $database->forceDelete(); + $deletedCounts['databases']++; + } catch (\Exception $e) { + \Log::error("Failed to delete database {$database->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + // Delete services + foreach ($resources['services'] as $service) { + try { + $service->forceDelete(); + $deletedCounts['services']++; + } catch (\Exception $e) { + \Log::error("Failed to delete service {$service->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + return $deletedCounts; + } + + private function getAllDatabasesForServer($server): Collection + { + $databases = collect(); + + // Get all standalone database types + $databases = $databases->merge($server->postgresqls); + $databases = $databases->merge($server->mysqls); + $databases = $databases->merge($server->mariadbs); + $databases = $databases->merge($server->mongodbs); + $databases = $databases->merge($server->redis); + $databases = $databases->merge($server->keydbs); + $databases = $databases->merge($server->dragonflies); + $databases = $databases->merge($server->clickhouses); + + return $databases; + } +} diff --git a/app/Actions/User/DeleteUserServers.php b/app/Actions/User/DeleteUserServers.php new file mode 100644 index 000000000..d8caae54d --- /dev/null +++ b/app/Actions/User/DeleteUserServers.php @@ -0,0 +1,77 @@ +<?php + +namespace App\Actions\User; + +use App\Models\Server; +use App\Models\User; +use Illuminate\Support\Collection; + +class DeleteUserServers +{ + private User $user; + + private bool $isDryRun; + + public function __construct(User $user, bool $isDryRun = false) + { + $this->user = $user; + $this->isDryRun = $isDryRun; + } + + public function getServersPreview(): Collection + { + $servers = collect(); + + // Get all teams the user belongs to + $teams = $this->user->teams; + + foreach ($teams as $team) { + // Only include servers from teams where user is owner or admin + $userRole = $team->pivot->role; + if ($userRole === 'owner' || $userRole === 'admin') { + $teamServers = $team->servers; + $servers = $servers->merge($teamServers); + } + } + + // Return unique servers (in case same server is in multiple teams) + return $servers->unique('id'); + } + + public function execute(): array + { + if ($this->isDryRun) { + return [ + 'servers' => 0, + ]; + } + + $deletedCount = 0; + + $servers = $this->getServersPreview(); + + foreach ($servers as $server) { + try { + // Skip the default server (ID 0) which is the Coolify host + if ($server->id === 0) { + \Log::info('Skipping deletion of Coolify host server (ID: 0)'); + + continue; + } + + // The Server model's forceDeleting event will handle cleanup of: + // - destinations + // - settings + $server->forceDelete(); + $deletedCount++; + } catch (\Exception $e) { + \Log::error("Failed to delete server {$server->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + return [ + 'servers' => $deletedCount, + ]; + } +} diff --git a/app/Actions/User/DeleteUserTeams.php b/app/Actions/User/DeleteUserTeams.php new file mode 100644 index 000000000..d572db9e7 --- /dev/null +++ b/app/Actions/User/DeleteUserTeams.php @@ -0,0 +1,202 @@ +<?php + +namespace App\Actions\User; + +use App\Models\Team; +use App\Models\User; + +class DeleteUserTeams +{ + private User $user; + + private bool $isDryRun; + + public function __construct(User $user, bool $isDryRun = false) + { + $this->user = $user; + $this->isDryRun = $isDryRun; + } + + public function getTeamsPreview(): array + { + $teamsToDelete = collect(); + $teamsToTransfer = collect(); + $teamsToLeave = collect(); + $edgeCases = collect(); + + $teams = $this->user->teams; + + foreach ($teams as $team) { + // Skip root team (ID 0) + if ($team->id === 0) { + continue; + } + + $userRole = $team->pivot->role; + $memberCount = $team->members->count(); + + if ($memberCount === 1) { + // User is alone in the team - delete it + $teamsToDelete->push($team); + } elseif ($userRole === 'owner') { + // Check if there are other owners + $otherOwners = $team->members + ->where('id', '!=', $this->user->id) + ->filter(function ($member) { + return $member->pivot->role === 'owner'; + }); + + if ($otherOwners->isNotEmpty()) { + // There are other owners, but check if this user is paying for the subscription + if ($this->isUserPayingForTeamSubscription($team)) { + // User is paying for the subscription - this is an edge case + $edgeCases->push([ + 'team' => $team, + 'reason' => 'User is paying for the team\'s Stripe subscription but there are other owners. The subscription needs to be cancelled or transferred to another owner\'s payment method.', + ]); + } else { + // There are other owners and user is not paying, just remove this user + $teamsToLeave->push($team); + } + } else { + // User is the only owner, check for replacement + $newOwner = $this->findNewOwner($team); + if ($newOwner) { + $teamsToTransfer->push([ + 'team' => $team, + 'new_owner' => $newOwner, + ]); + } else { + // No suitable replacement found - this is an edge case + $edgeCases->push([ + 'team' => $team, + 'reason' => 'No suitable owner replacement found. Team has only regular members without admin privileges.', + ]); + } + } + } else { + // User is just a member - remove them from the team + $teamsToLeave->push($team); + } + } + + return [ + 'to_delete' => $teamsToDelete, + 'to_transfer' => $teamsToTransfer, + 'to_leave' => $teamsToLeave, + 'edge_cases' => $edgeCases, + ]; + } + + public function execute(): array + { + if ($this->isDryRun) { + return [ + 'deleted' => 0, + 'transferred' => 0, + 'left' => 0, + ]; + } + + $counts = [ + 'deleted' => 0, + 'transferred' => 0, + 'left' => 0, + ]; + + $preview = $this->getTeamsPreview(); + + // Check for edge cases - should not happen here as we check earlier, but be safe + if ($preview['edge_cases']->isNotEmpty()) { + throw new \Exception('Edge cases detected during execution. This should not happen.'); + } + + // Delete teams where user is alone + foreach ($preview['to_delete'] as $team) { + try { + // The Team model's deleting event will handle cleanup of: + // - private keys + // - sources + // - tags + // - environment variables + // - s3 storages + // - notification settings + $team->delete(); + $counts['deleted']++; + } catch (\Exception $e) { + \Log::error("Failed to delete team {$team->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + // Transfer ownership for teams where user is owner but not alone + foreach ($preview['to_transfer'] as $item) { + try { + $team = $item['team']; + $newOwner = $item['new_owner']; + + // Update the new owner's role to owner + $team->members()->updateExistingPivot($newOwner->id, ['role' => 'owner']); + + // Remove the current user from the team + $team->members()->detach($this->user->id); + + $counts['transferred']++; + } catch (\Exception $e) { + \Log::error("Failed to transfer ownership of team {$item['team']->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + // Remove user from teams where they're just a member + foreach ($preview['to_leave'] as $team) { + try { + $team->members()->detach($this->user->id); + $counts['left']++; + } catch (\Exception $e) { + \Log::error("Failed to remove user from team {$team->id}: ".$e->getMessage()); + throw $e; // Re-throw to trigger rollback + } + } + + return $counts; + } + + private function findNewOwner(Team $team): ?User + { + // Only look for admins as potential new owners + // We don't promote regular members automatically + $otherAdmin = $team->members + ->where('id', '!=', $this->user->id) + ->filter(function ($member) { + return $member->pivot->role === 'admin'; + }) + ->first(); + + return $otherAdmin; + } + + private function isUserPayingForTeamSubscription(Team $team): bool + { + if (! $team->subscription || ! $team->subscription->stripe_customer_id) { + return false; + } + + // In Stripe, we need to check if the customer email matches the user's email + // This would require a Stripe API call to get customer details + // For now, we'll check if the subscription was created by this user + + // Alternative approach: Check if user is the one who initiated the subscription + // We could store this information when the subscription is created + // For safety, we'll assume if there's an active subscription and multiple owners, + // we should treat it as an edge case that needs manual review + + if ($team->subscription->stripe_subscription_id && + $team->subscription->stripe_invoice_paid) { + // Active subscription exists - we should be cautious + return true; + } + + return false; + } +} diff --git a/app/Console/Commands/CleanupDatabase.php b/app/Console/Commands/CleanupDatabase.php index 2ccb76529..347ea9419 100644 --- a/app/Console/Commands/CleanupDatabase.php +++ b/app/Console/Commands/CleanupDatabase.php @@ -64,13 +64,5 @@ public function handle() if ($this->option('yes')) { $scheduled_task_executions->delete(); } - - // Cleanup webhooks table - $webhooks = DB::table('webhooks')->where('created_at', '<', now()->subDays($keep_days)); - $count = $webhooks->count(); - echo "Delete $count entries from webhooks.\n"; - if ($this->option('yes')) { - $webhooks->delete(); - } } } diff --git a/app/Console/Commands/CleanupStuckedResources.php b/app/Console/Commands/CleanupStuckedResources.php index 81824675b..ce2d6d598 100644 --- a/app/Console/Commands/CleanupStuckedResources.php +++ b/app/Console/Commands/CleanupStuckedResources.php @@ -3,6 +3,7 @@ namespace App\Console\Commands; use App\Jobs\CleanupHelperContainersJob; +use App\Jobs\DeleteResourceJob; use App\Models\Application; use App\Models\ApplicationDeploymentQueue; use App\Models\ApplicationPreview; @@ -72,7 +73,7 @@ private function cleanup_stucked_resources() $applications = Application::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($applications as $application) { echo "Deleting stuck application: {$application->name}\n"; - $application->forceDelete(); + DeleteResourceJob::dispatch($application); } } catch (\Throwable $e) { echo "Error in cleaning stuck application: {$e->getMessage()}\n"; @@ -82,26 +83,35 @@ private function cleanup_stucked_resources() foreach ($applicationsPreviews as $applicationPreview) { if (! data_get($applicationPreview, 'application')) { echo "Deleting stuck application preview: {$applicationPreview->uuid}\n"; - $applicationPreview->delete(); + DeleteResourceJob::dispatch($applicationPreview); } } } catch (\Throwable $e) { echo "Error in cleaning stuck application: {$e->getMessage()}\n"; } + try { + $applicationsPreviews = ApplicationPreview::withTrashed()->whereNotNull('deleted_at')->get(); + foreach ($applicationsPreviews as $applicationPreview) { + echo "Deleting stuck application preview: {$applicationPreview->fqdn}\n"; + DeleteResourceJob::dispatch($applicationPreview); + } + } catch (\Throwable $e) { + echo "Error in cleaning stuck application: {$e->getMessage()}\n"; + } try { $postgresqls = StandalonePostgresql::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($postgresqls as $postgresql) { echo "Deleting stuck postgresql: {$postgresql->name}\n"; - $postgresql->forceDelete(); + DeleteResourceJob::dispatch($postgresql); } } catch (\Throwable $e) { echo "Error in cleaning stuck postgresql: {$e->getMessage()}\n"; } try { - $redis = StandaloneRedis::withTrashed()->whereNotNull('deleted_at')->get(); - foreach ($redis as $redis) { + $rediss = StandaloneRedis::withTrashed()->whereNotNull('deleted_at')->get(); + foreach ($rediss as $redis) { echo "Deleting stuck redis: {$redis->name}\n"; - $redis->forceDelete(); + DeleteResourceJob::dispatch($redis); } } catch (\Throwable $e) { echo "Error in cleaning stuck redis: {$e->getMessage()}\n"; @@ -110,7 +120,7 @@ private function cleanup_stucked_resources() $keydbs = StandaloneKeydb::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($keydbs as $keydb) { echo "Deleting stuck keydb: {$keydb->name}\n"; - $keydb->forceDelete(); + DeleteResourceJob::dispatch($keydb); } } catch (\Throwable $e) { echo "Error in cleaning stuck keydb: {$e->getMessage()}\n"; @@ -119,7 +129,7 @@ private function cleanup_stucked_resources() $dragonflies = StandaloneDragonfly::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($dragonflies as $dragonfly) { echo "Deleting stuck dragonfly: {$dragonfly->name}\n"; - $dragonfly->forceDelete(); + DeleteResourceJob::dispatch($dragonfly); } } catch (\Throwable $e) { echo "Error in cleaning stuck dragonfly: {$e->getMessage()}\n"; @@ -128,7 +138,7 @@ private function cleanup_stucked_resources() $clickhouses = StandaloneClickhouse::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($clickhouses as $clickhouse) { echo "Deleting stuck clickhouse: {$clickhouse->name}\n"; - $clickhouse->forceDelete(); + DeleteResourceJob::dispatch($clickhouse); } } catch (\Throwable $e) { echo "Error in cleaning stuck clickhouse: {$e->getMessage()}\n"; @@ -137,7 +147,7 @@ private function cleanup_stucked_resources() $mongodbs = StandaloneMongodb::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($mongodbs as $mongodb) { echo "Deleting stuck mongodb: {$mongodb->name}\n"; - $mongodb->forceDelete(); + DeleteResourceJob::dispatch($mongodb); } } catch (\Throwable $e) { echo "Error in cleaning stuck mongodb: {$e->getMessage()}\n"; @@ -146,7 +156,7 @@ private function cleanup_stucked_resources() $mysqls = StandaloneMysql::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($mysqls as $mysql) { echo "Deleting stuck mysql: {$mysql->name}\n"; - $mysql->forceDelete(); + DeleteResourceJob::dispatch($mysql); } } catch (\Throwable $e) { echo "Error in cleaning stuck mysql: {$e->getMessage()}\n"; @@ -155,7 +165,7 @@ private function cleanup_stucked_resources() $mariadbs = StandaloneMariadb::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($mariadbs as $mariadb) { echo "Deleting stuck mariadb: {$mariadb->name}\n"; - $mariadb->forceDelete(); + DeleteResourceJob::dispatch($mariadb); } } catch (\Throwable $e) { echo "Error in cleaning stuck mariadb: {$e->getMessage()}\n"; @@ -164,7 +174,7 @@ private function cleanup_stucked_resources() $services = Service::withTrashed()->whereNotNull('deleted_at')->get(); foreach ($services as $service) { echo "Deleting stuck service: {$service->name}\n"; - $service->forceDelete(); + DeleteResourceJob::dispatch($service); } } catch (\Throwable $e) { echo "Error in cleaning stuck service: {$e->getMessage()}\n"; @@ -217,19 +227,19 @@ private function cleanup_stucked_resources() foreach ($applications as $application) { if (! data_get($application, 'environment')) { echo 'Application without environment: '.$application->name.'\n'; - $application->forceDelete(); + DeleteResourceJob::dispatch($application); continue; } if (! $application->destination()) { echo 'Application without destination: '.$application->name.'\n'; - $application->forceDelete(); + DeleteResourceJob::dispatch($application); continue; } if (! data_get($application, 'destination.server')) { echo 'Application without server: '.$application->name.'\n'; - $application->forceDelete(); + DeleteResourceJob::dispatch($application); continue; } @@ -242,19 +252,19 @@ private function cleanup_stucked_resources() foreach ($postgresqls as $postgresql) { if (! data_get($postgresql, 'environment')) { echo 'Postgresql without environment: '.$postgresql->name.'\n'; - $postgresql->forceDelete(); + DeleteResourceJob::dispatch($postgresql); continue; } if (! $postgresql->destination()) { echo 'Postgresql without destination: '.$postgresql->name.'\n'; - $postgresql->forceDelete(); + DeleteResourceJob::dispatch($postgresql); continue; } if (! data_get($postgresql, 'destination.server')) { echo 'Postgresql without server: '.$postgresql->name.'\n'; - $postgresql->forceDelete(); + DeleteResourceJob::dispatch($postgresql); continue; } @@ -267,19 +277,19 @@ private function cleanup_stucked_resources() foreach ($redis as $redis) { if (! data_get($redis, 'environment')) { echo 'Redis without environment: '.$redis->name.'\n'; - $redis->forceDelete(); + DeleteResourceJob::dispatch($redis); continue; } if (! $redis->destination()) { echo 'Redis without destination: '.$redis->name.'\n'; - $redis->forceDelete(); + DeleteResourceJob::dispatch($redis); continue; } if (! data_get($redis, 'destination.server')) { echo 'Redis without server: '.$redis->name.'\n'; - $redis->forceDelete(); + DeleteResourceJob::dispatch($redis); continue; } @@ -293,19 +303,19 @@ private function cleanup_stucked_resources() foreach ($mongodbs as $mongodb) { if (! data_get($mongodb, 'environment')) { echo 'Mongodb without environment: '.$mongodb->name.'\n'; - $mongodb->forceDelete(); + DeleteResourceJob::dispatch($mongodb); continue; } if (! $mongodb->destination()) { echo 'Mongodb without destination: '.$mongodb->name.'\n'; - $mongodb->forceDelete(); + DeleteResourceJob::dispatch($mongodb); continue; } if (! data_get($mongodb, 'destination.server')) { echo 'Mongodb without server: '.$mongodb->name.'\n'; - $mongodb->forceDelete(); + DeleteResourceJob::dispatch($mongodb); continue; } @@ -319,19 +329,19 @@ private function cleanup_stucked_resources() foreach ($mysqls as $mysql) { if (! data_get($mysql, 'environment')) { echo 'Mysql without environment: '.$mysql->name.'\n'; - $mysql->forceDelete(); + DeleteResourceJob::dispatch($mysql); continue; } if (! $mysql->destination()) { echo 'Mysql without destination: '.$mysql->name.'\n'; - $mysql->forceDelete(); + DeleteResourceJob::dispatch($mysql); continue; } if (! data_get($mysql, 'destination.server')) { echo 'Mysql without server: '.$mysql->name.'\n'; - $mysql->forceDelete(); + DeleteResourceJob::dispatch($mysql); continue; } @@ -345,19 +355,19 @@ private function cleanup_stucked_resources() foreach ($mariadbs as $mariadb) { if (! data_get($mariadb, 'environment')) { echo 'Mariadb without environment: '.$mariadb->name.'\n'; - $mariadb->forceDelete(); + DeleteResourceJob::dispatch($mariadb); continue; } if (! $mariadb->destination()) { echo 'Mariadb without destination: '.$mariadb->name.'\n'; - $mariadb->forceDelete(); + DeleteResourceJob::dispatch($mariadb); continue; } if (! data_get($mariadb, 'destination.server')) { echo 'Mariadb without server: '.$mariadb->name.'\n'; - $mariadb->forceDelete(); + DeleteResourceJob::dispatch($mariadb); continue; } @@ -371,19 +381,19 @@ private function cleanup_stucked_resources() foreach ($services as $service) { if (! data_get($service, 'environment')) { echo 'Service without environment: '.$service->name.'\n'; - $service->forceDelete(); + DeleteResourceJob::dispatch($service); continue; } if (! $service->destination()) { echo 'Service without destination: '.$service->name.'\n'; - $service->forceDelete(); + DeleteResourceJob::dispatch($service); continue; } if (! data_get($service, 'server')) { echo 'Service without server: '.$service->name.'\n'; - $service->forceDelete(); + DeleteResourceJob::dispatch($service); continue; } @@ -396,7 +406,7 @@ private function cleanup_stucked_resources() foreach ($serviceApplications as $service) { if (! data_get($service, 'service')) { echo 'ServiceApplication without service: '.$service->name.'\n'; - $service->forceDelete(); + DeleteResourceJob::dispatch($service); continue; } @@ -409,7 +419,7 @@ private function cleanup_stucked_resources() foreach ($serviceDatabases as $service) { if (! data_get($service, 'service')) { echo 'ServiceDatabase without service: '.$service->name.'\n'; - $service->forceDelete(); + DeleteResourceJob::dispatch($service); continue; } diff --git a/app/Console/Commands/Cloud/CloudDeleteUser.php b/app/Console/Commands/Cloud/CloudDeleteUser.php new file mode 100644 index 000000000..a2ea9b3e5 --- /dev/null +++ b/app/Console/Commands/Cloud/CloudDeleteUser.php @@ -0,0 +1,744 @@ +<?php + +namespace App\Console\Commands\Cloud; + +use App\Actions\Stripe\CancelSubscription; +use App\Actions\User\DeleteUserResources; +use App\Actions\User\DeleteUserServers; +use App\Actions\User\DeleteUserTeams; +use App\Models\User; +use Illuminate\Console\Command; +use Illuminate\Support\Facades\Cache; +use Illuminate\Support\Facades\DB; +use Illuminate\Support\Facades\Log; + +class CloudDeleteUser extends Command +{ + protected $signature = 'cloud:delete-user {email} + {--dry-run : Preview what will be deleted without actually deleting} + {--skip-stripe : Skip Stripe subscription cancellation} + {--skip-resources : Skip resource deletion}'; + + protected $description = 'Delete a user from the cloud instance with phase-by-phase confirmation'; + + private bool $isDryRun = false; + + private bool $skipStripe = false; + + private bool $skipResources = false; + + private User $user; + + public function handle() + { + if (! isCloud()) { + $this->error('This command is only available on cloud instances.'); + + return 1; + } + + $email = $this->argument('email'); + $this->isDryRun = $this->option('dry-run'); + $this->skipStripe = $this->option('skip-stripe'); + $this->skipResources = $this->option('skip-resources'); + + if ($this->isDryRun) { + $this->info('🔍 DRY RUN MODE - No data will be deleted'); + $this->newLine(); + } + + try { + $this->user = User::whereEmail($email)->firstOrFail(); + } catch (\Exception $e) { + $this->error("User with email '{$email}' not found."); + + return 1; + } + + // Implement file lock to prevent concurrent deletions of the same user + $lockKey = "user_deletion_{$this->user->id}"; + $lock = Cache::lock($lockKey, 600); // 10 minute lock + + if (! $lock->get()) { + $this->error('Another deletion process is already running for this user. Please try again later.'); + $this->logAction("Deletion blocked for user {$email}: Another process is already running"); + + return 1; + } + + try { + $this->logAction("Starting user deletion process for: {$email}"); + + // Phase 1: Show User Overview (outside transaction) + if (! $this->showUserOverview()) { + $this->info('User deletion cancelled.'); + $lock->release(); + + return 0; + } + + // If not dry run, wrap everything in a transaction + if (! $this->isDryRun) { + try { + DB::beginTransaction(); + + // Phase 2: Delete Resources + if (! $this->skipResources) { + if (! $this->deleteResources()) { + DB::rollBack(); + $this->error('User deletion failed at resource deletion phase. All changes rolled back.'); + + return 1; + } + } + + // Phase 3: Delete Servers + if (! $this->deleteServers()) { + DB::rollBack(); + $this->error('User deletion failed at server deletion phase. All changes rolled back.'); + + return 1; + } + + // Phase 4: Handle Teams + if (! $this->handleTeams()) { + DB::rollBack(); + $this->error('User deletion failed at team handling phase. All changes rolled back.'); + + return 1; + } + + // Phase 5: Cancel Stripe Subscriptions + if (! $this->skipStripe && isCloud()) { + if (! $this->cancelStripeSubscriptions()) { + DB::rollBack(); + $this->error('User deletion failed at Stripe cancellation phase. All changes rolled back.'); + + return 1; + } + } + + // Phase 6: Delete User Profile + if (! $this->deleteUserProfile()) { + DB::rollBack(); + $this->error('User deletion failed at final phase. All changes rolled back.'); + + return 1; + } + + // Commit the transaction + DB::commit(); + + $this->newLine(); + $this->info('✅ User deletion completed successfully!'); + $this->logAction("User deletion completed for: {$email}"); + + } catch (\Exception $e) { + DB::rollBack(); + $this->error('An error occurred during user deletion: '.$e->getMessage()); + $this->logAction("User deletion failed for {$email}: ".$e->getMessage()); + + return 1; + } + } else { + // Dry run mode - just run through the phases without transaction + // Phase 2: Delete Resources + if (! $this->skipResources) { + if (! $this->deleteResources()) { + $this->info('User deletion would be cancelled at resource deletion phase.'); + + return 0; + } + } + + // Phase 3: Delete Servers + if (! $this->deleteServers()) { + $this->info('User deletion would be cancelled at server deletion phase.'); + + return 0; + } + + // Phase 4: Handle Teams + if (! $this->handleTeams()) { + $this->info('User deletion would be cancelled at team handling phase.'); + + return 0; + } + + // Phase 5: Cancel Stripe Subscriptions + if (! $this->skipStripe && isCloud()) { + if (! $this->cancelStripeSubscriptions()) { + $this->info('User deletion would be cancelled at Stripe cancellation phase.'); + + return 0; + } + } + + // Phase 6: Delete User Profile + if (! $this->deleteUserProfile()) { + $this->info('User deletion would be cancelled at final phase.'); + + return 0; + } + + $this->newLine(); + $this->info('✅ DRY RUN completed successfully! No data was deleted.'); + } + + return 0; + } finally { + // Ensure lock is always released + $lock->release(); + } + } + + private function showUserOverview(): bool + { + $this->info('═══════════════════════════════════════'); + $this->info('PHASE 1: USER OVERVIEW'); + $this->info('═══════════════════════════════════════'); + $this->newLine(); + + $teams = $this->user->teams; + $ownedTeams = $teams->filter(fn ($team) => $team->pivot->role === 'owner'); + $memberTeams = $teams->filter(fn ($team) => $team->pivot->role !== 'owner'); + + // Collect all servers from all teams + $allServers = collect(); + $allApplications = collect(); + $allDatabases = collect(); + $allServices = collect(); + $activeSubscriptions = collect(); + + foreach ($teams as $team) { + $servers = $team->servers; + $allServers = $allServers->merge($servers); + + foreach ($servers as $server) { + $resources = $server->definedResources(); + foreach ($resources as $resource) { + if ($resource instanceof \App\Models\Application) { + $allApplications->push($resource); + } elseif ($resource instanceof \App\Models\Service) { + $allServices->push($resource); + } else { + $allDatabases->push($resource); + } + } + } + + if ($team->subscription && $team->subscription->stripe_subscription_id) { + $activeSubscriptions->push($team->subscription); + } + } + + $this->table( + ['Property', 'Value'], + [ + ['User', $this->user->email], + ['User ID', $this->user->id], + ['Created', $this->user->created_at->format('Y-m-d H:i:s')], + ['Last Login', $this->user->updated_at->format('Y-m-d H:i:s')], + ['Teams (Total)', $teams->count()], + ['Teams (Owner)', $ownedTeams->count()], + ['Teams (Member)', $memberTeams->count()], + ['Servers', $allServers->unique('id')->count()], + ['Applications', $allApplications->count()], + ['Databases', $allDatabases->count()], + ['Services', $allServices->count()], + ['Active Stripe Subscriptions', $activeSubscriptions->count()], + ] + ); + + $this->newLine(); + + $this->warn('⚠️ WARNING: This will permanently delete the user and all associated data!'); + $this->newLine(); + + if (! $this->confirm('Do you want to continue with the deletion process?', false)) { + return false; + } + + return true; + } + + private function deleteResources(): bool + { + $this->newLine(); + $this->info('═══════════════════════════════════════'); + $this->info('PHASE 2: DELETE RESOURCES'); + $this->info('═══════════════════════════════════════'); + $this->newLine(); + + $action = new DeleteUserResources($this->user, $this->isDryRun); + $resources = $action->getResourcesPreview(); + + if ($resources['applications']->isEmpty() && + $resources['databases']->isEmpty() && + $resources['services']->isEmpty()) { + $this->info('No resources to delete.'); + + return true; + } + + $this->info('Resources to be deleted:'); + $this->newLine(); + + if ($resources['applications']->isNotEmpty()) { + $this->warn("Applications to be deleted ({$resources['applications']->count()}):"); + $this->table( + ['Name', 'UUID', 'Server', 'Status'], + $resources['applications']->map(function ($app) { + return [ + $app->name, + $app->uuid, + $app->destination->server->name, + $app->status ?? 'unknown', + ]; + })->toArray() + ); + $this->newLine(); + } + + if ($resources['databases']->isNotEmpty()) { + $this->warn("Databases to be deleted ({$resources['databases']->count()}):"); + $this->table( + ['Name', 'Type', 'UUID', 'Server'], + $resources['databases']->map(function ($db) { + return [ + $db->name, + class_basename($db), + $db->uuid, + $db->destination->server->name, + ]; + })->toArray() + ); + $this->newLine(); + } + + if ($resources['services']->isNotEmpty()) { + $this->warn("Services to be deleted ({$resources['services']->count()}):"); + $this->table( + ['Name', 'UUID', 'Server'], + $resources['services']->map(function ($service) { + return [ + $service->name, + $service->uuid, + $service->server->name, + ]; + })->toArray() + ); + $this->newLine(); + } + + $this->error('⚠️ THIS ACTION CANNOT BE UNDONE!'); + if (! $this->confirm('Are you sure you want to delete all these resources?', false)) { + return false; + } + + if (! $this->isDryRun) { + $this->info('Deleting resources...'); + $result = $action->execute(); + $this->info("Deleted: {$result['applications']} applications, {$result['databases']} databases, {$result['services']} services"); + $this->logAction("Deleted resources for user {$this->user->email}: {$result['applications']} apps, {$result['databases']} databases, {$result['services']} services"); + } + + return true; + } + + private function deleteServers(): bool + { + $this->newLine(); + $this->info('═══════════════════════════════════════'); + $this->info('PHASE 3: DELETE SERVERS'); + $this->info('═══════════════════════════════════════'); + $this->newLine(); + + $action = new DeleteUserServers($this->user, $this->isDryRun); + $servers = $action->getServersPreview(); + + if ($servers->isEmpty()) { + $this->info('No servers to delete.'); + + return true; + } + + $this->warn("Servers to be deleted ({$servers->count()}):"); + $this->table( + ['ID', 'Name', 'IP', 'Description', 'Resources Count'], + $servers->map(function ($server) { + $resourceCount = $server->definedResources()->count(); + + return [ + $server->id, + $server->name, + $server->ip, + $server->description ?? '-', + $resourceCount, + ]; + })->toArray() + ); + $this->newLine(); + + $this->error('⚠️ WARNING: Deleting servers will remove all server configurations!'); + if (! $this->confirm('Are you sure you want to delete all these servers?', false)) { + return false; + } + + if (! $this->isDryRun) { + $this->info('Deleting servers...'); + $result = $action->execute(); + $this->info("Deleted {$result['servers']} servers"); + $this->logAction("Deleted {$result['servers']} servers for user {$this->user->email}"); + } + + return true; + } + + private function handleTeams(): bool + { + $this->newLine(); + $this->info('═══════════════════════════════════════'); + $this->info('PHASE 4: HANDLE TEAMS'); + $this->info('═══════════════════════════════════════'); + $this->newLine(); + + $action = new DeleteUserTeams($this->user, $this->isDryRun); + $preview = $action->getTeamsPreview(); + + // Check for edge cases first - EXIT IMMEDIATELY if found + if ($preview['edge_cases']->isNotEmpty()) { + $this->error('═══════════════════════════════════════'); + $this->error('⚠️ EDGE CASES DETECTED - CANNOT PROCEED'); + $this->error('═══════════════════════════════════════'); + $this->newLine(); + + foreach ($preview['edge_cases'] as $edgeCase) { + $team = $edgeCase['team']; + $reason = $edgeCase['reason']; + $this->error("Team: {$team->name} (ID: {$team->id})"); + $this->error("Issue: {$reason}"); + + // Show team members for context + $this->info('Current members:'); + foreach ($team->members as $member) { + $role = $member->pivot->role; + $this->line(" - {$member->name} ({$member->email}) - Role: {$role}"); + } + + // Check for active resources + $resourceCount = 0; + foreach ($team->servers as $server) { + $resources = $server->definedResources(); + $resourceCount += $resources->count(); + } + + if ($resourceCount > 0) { + $this->warn(" ⚠️ This team has {$resourceCount} active resources!"); + } + + // Show subscription details if relevant + if ($team->subscription && $team->subscription->stripe_subscription_id) { + $this->warn(' ⚠️ Active Stripe subscription details:'); + $this->warn(" Subscription ID: {$team->subscription->stripe_subscription_id}"); + $this->warn(" Customer ID: {$team->subscription->stripe_customer_id}"); + + // Show other owners who could potentially take over + $otherOwners = $team->members + ->where('id', '!=', $this->user->id) + ->filter(function ($member) { + return $member->pivot->role === 'owner'; + }); + + if ($otherOwners->isNotEmpty()) { + $this->info(' Other owners who could take over billing:'); + foreach ($otherOwners as $owner) { + $this->line(" - {$owner->name} ({$owner->email})"); + } + } + } + + $this->newLine(); + } + + $this->error('Please resolve these issues manually before retrying:'); + + // Check if any edge case involves subscription payment issues + $hasSubscriptionIssue = $preview['edge_cases']->contains(function ($edgeCase) { + return str_contains($edgeCase['reason'], 'Stripe subscription'); + }); + + if ($hasSubscriptionIssue) { + $this->info('For teams with subscription payment issues:'); + $this->info('1. Cancel the subscription through Stripe dashboard, OR'); + $this->info('2. Transfer the subscription to another owner\'s payment method, OR'); + $this->info('3. Have the other owner create a new subscription after cancelling this one'); + $this->newLine(); + } + + $hasNoOwnerReplacement = $preview['edge_cases']->contains(function ($edgeCase) { + return str_contains($edgeCase['reason'], 'No suitable owner replacement'); + }); + + if ($hasNoOwnerReplacement) { + $this->info('For teams with no suitable owner replacement:'); + $this->info('1. Assign an admin role to a trusted member, OR'); + $this->info('2. Transfer team resources to another team, OR'); + $this->info('3. Delete the team manually if no longer needed'); + $this->newLine(); + } + + $this->error('USER DELETION ABORTED DUE TO EDGE CASES'); + $this->logAction("User deletion aborted for {$this->user->email}: Edge cases in team handling"); + + // Exit immediately - don't proceed with deletion + if (! $this->isDryRun) { + DB::rollBack(); + } + exit(1); + } + + if ($preview['to_delete']->isEmpty() && + $preview['to_transfer']->isEmpty() && + $preview['to_leave']->isEmpty()) { + $this->info('No team changes needed.'); + + return true; + } + + if ($preview['to_delete']->isNotEmpty()) { + $this->warn('Teams to be DELETED (user is the only member):'); + $this->table( + ['ID', 'Name', 'Resources', 'Subscription'], + $preview['to_delete']->map(function ($team) { + $resourceCount = 0; + foreach ($team->servers as $server) { + $resourceCount += $server->definedResources()->count(); + } + $hasSubscription = $team->subscription && $team->subscription->stripe_subscription_id + ? '⚠️ YES - '.$team->subscription->stripe_subscription_id + : 'No'; + + return [ + $team->id, + $team->name, + $resourceCount, + $hasSubscription, + ]; + })->toArray() + ); + $this->newLine(); + } + + if ($preview['to_transfer']->isNotEmpty()) { + $this->warn('Teams where ownership will be TRANSFERRED:'); + $this->table( + ['Team ID', 'Team Name', 'New Owner', 'New Owner Email'], + $preview['to_transfer']->map(function ($item) { + return [ + $item['team']->id, + $item['team']->name, + $item['new_owner']->name, + $item['new_owner']->email, + ]; + })->toArray() + ); + $this->newLine(); + } + + if ($preview['to_leave']->isNotEmpty()) { + $this->warn('Teams where user will be REMOVED (other owners/admins exist):'); + $userId = $this->user->id; + $this->table( + ['ID', 'Name', 'User Role', 'Other Members'], + $preview['to_leave']->map(function ($team) use ($userId) { + $userRole = $team->members->where('id', $userId)->first()->pivot->role; + $otherMembers = $team->members->count() - 1; + + return [ + $team->id, + $team->name, + $userRole, + $otherMembers, + ]; + })->toArray() + ); + $this->newLine(); + } + + $this->error('⚠️ WARNING: Team changes affect access control and ownership!'); + if (! $this->confirm('Are you sure you want to proceed with these team changes?', false)) { + return false; + } + + if (! $this->isDryRun) { + $this->info('Processing team changes...'); + $result = $action->execute(); + $this->info("Teams deleted: {$result['deleted']}, ownership transferred: {$result['transferred']}, left: {$result['left']}"); + $this->logAction("Team changes for user {$this->user->email}: deleted {$result['deleted']}, transferred {$result['transferred']}, left {$result['left']}"); + } + + return true; + } + + private function cancelStripeSubscriptions(): bool + { + $this->newLine(); + $this->info('═══════════════════════════════════════'); + $this->info('PHASE 5: CANCEL STRIPE SUBSCRIPTIONS'); + $this->info('═══════════════════════════════════════'); + $this->newLine(); + + $action = new CancelSubscription($this->user, $this->isDryRun); + $subscriptions = $action->getSubscriptionsPreview(); + + if ($subscriptions->isEmpty()) { + $this->info('No Stripe subscriptions to cancel.'); + + return true; + } + + $this->info('Stripe subscriptions to cancel:'); + $this->newLine(); + + $totalMonthlyValue = 0; + foreach ($subscriptions as $subscription) { + $team = $subscription->team; + $planId = $subscription->stripe_plan_id; + + // Try to get the price from config + $monthlyValue = $this->getSubscriptionMonthlyValue($planId); + $totalMonthlyValue += $monthlyValue; + + $this->line(" - {$subscription->stripe_subscription_id} (Team: {$team->name})"); + if ($monthlyValue > 0) { + $this->line(" Monthly value: \${$monthlyValue}"); + } + if ($subscription->stripe_cancel_at_period_end) { + $this->line(' ⚠️ Already set to cancel at period end'); + } + } + + if ($totalMonthlyValue > 0) { + $this->newLine(); + $this->warn("Total monthly value: \${$totalMonthlyValue}"); + } + $this->newLine(); + + $this->error('⚠️ WARNING: Subscriptions will be cancelled IMMEDIATELY (not at period end)!'); + if (! $this->confirm('Are you sure you want to cancel all these subscriptions immediately?', false)) { + return false; + } + + if (! $this->isDryRun) { + $this->info('Cancelling subscriptions...'); + $result = $action->execute(); + $this->info("Cancelled {$result['cancelled']} subscriptions, {$result['failed']} failed"); + if ($result['failed'] > 0 && ! empty($result['errors'])) { + $this->error('Failed subscriptions:'); + foreach ($result['errors'] as $error) { + $this->error(" - {$error}"); + } + } + $this->logAction("Cancelled {$result['cancelled']} Stripe subscriptions for user {$this->user->email}"); + } + + return true; + } + + private function deleteUserProfile(): bool + { + $this->newLine(); + $this->info('═══════════════════════════════════════'); + $this->info('PHASE 6: DELETE USER PROFILE'); + $this->info('═══════════════════════════════════════'); + $this->newLine(); + + $this->warn('⚠️ FINAL STEP - This action is IRREVERSIBLE!'); + $this->newLine(); + + $this->info('User profile to be deleted:'); + $this->table( + ['Property', 'Value'], + [ + ['Email', $this->user->email], + ['Name', $this->user->name], + ['User ID', $this->user->id], + ['Created', $this->user->created_at->format('Y-m-d H:i:s')], + ['Email Verified', $this->user->email_verified_at ? 'Yes' : 'No'], + ['2FA Enabled', $this->user->two_factor_confirmed_at ? 'Yes' : 'No'], + ] + ); + + $this->newLine(); + + $this->warn("Type 'DELETE {$this->user->email}' to confirm final deletion:"); + $confirmation = $this->ask('Confirmation'); + + if ($confirmation !== "DELETE {$this->user->email}") { + $this->error('Confirmation text does not match. Deletion cancelled.'); + + return false; + } + + if (! $this->isDryRun) { + $this->info('Deleting user profile...'); + + try { + $this->user->delete(); + $this->info('User profile deleted successfully.'); + $this->logAction("User profile deleted: {$this->user->email}"); + } catch (\Exception $e) { + $this->error('Failed to delete user profile: '.$e->getMessage()); + $this->logAction("Failed to delete user profile {$this->user->email}: ".$e->getMessage()); + + return false; + } + } + + return true; + } + + private function getSubscriptionMonthlyValue(string $planId): int + { + // Try to get pricing from subscription metadata or config + // Since we're using dynamic pricing, return 0 for now + // This could be enhanced by fetching the actual price from Stripe API + + // Check if this is a dynamic pricing plan + $dynamicMonthlyPlanId = config('subscription.stripe_price_id_dynamic_monthly'); + $dynamicYearlyPlanId = config('subscription.stripe_price_id_dynamic_yearly'); + + if ($planId === $dynamicMonthlyPlanId || $planId === $dynamicYearlyPlanId) { + // For dynamic pricing, we can't determine the exact amount without calling Stripe API + // Return 0 to indicate dynamic/usage-based pricing + return 0; + } + + // For any other plans, return 0 as we don't have hardcoded prices + return 0; + } + + private function logAction(string $message): void + { + $logMessage = "[CloudDeleteUser] {$message}"; + + if ($this->isDryRun) { + $logMessage = "[DRY RUN] {$logMessage}"; + } + + Log::channel('single')->info($logMessage); + + // Also log to a dedicated user deletion log file + $logFile = storage_path('logs/user-deletions.log'); + + // Ensure the logs directory exists + $logDir = dirname($logFile); + if (! is_dir($logDir)) { + mkdir($logDir, 0755, true); + } + + $timestamp = now()->format('Y-m-d H:i:s'); + file_put_contents($logFile, "[{$timestamp}] {$logMessage}\n", FILE_APPEND | LOCK_EX); + } +} diff --git a/app/Console/Commands/Cloud/CloudFixSubscription.php b/app/Console/Commands/Cloud/CloudFixSubscription.php new file mode 100644 index 000000000..194e9bb5f --- /dev/null +++ b/app/Console/Commands/Cloud/CloudFixSubscription.php @@ -0,0 +1,879 @@ +<?php + +namespace App\Console\Commands\Cloud; + +use App\Models\Team; +use Illuminate\Console\Command; + +class CloudFixSubscription extends Command +{ + /** + * The name and signature of the console command. + * + * @var string + */ + protected $signature = 'cloud:fix-subscription + {--fix-canceled-subs : Fix canceled subscriptions in database} + {--verify-all : Verify all active subscriptions against Stripe} + {--fix-verified : Fix discrepancies found during verification} + {--dry-run : Show what would be fixed without making changes} + {--one : Only fix the first found subscription}'; + + /** + * The console command description. + * + * @var string + */ + protected $description = 'Fix Cloud subscriptions'; + + /** + * Execute the console command. + */ + public function handle() + { + $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); + + if ($this->option('verify-all')) { + return $this->verifyAllActiveSubscriptions($stripe); + } + + if ($this->option('fix-canceled-subs') || $this->option('dry-run')) { + return $this->fixCanceledSubscriptions($stripe); + } + + $activeSubscribers = Team::whereRelation('subscription', 'stripe_invoice_paid', true)->get(); + + $out = fopen('php://output', 'w'); + // CSV header + fputcsv($out, [ + 'team_id', + 'invoice_status', + 'stripe_customer_url', + 'stripe_subscription_id', + 'subscription_status', + 'subscription_url', + 'note', + ]); + + foreach ($activeSubscribers as $team) { + $stripeSubscriptionId = $team->subscription->stripe_subscription_id; + $stripeInvoicePaid = $team->subscription->stripe_invoice_paid; + $stripeCustomerId = $team->subscription->stripe_customer_id; + + if (! $stripeSubscriptionId && str($stripeInvoicePaid)->lower() != 'past_due') { + fputcsv($out, [ + $team->id, + $stripeInvoicePaid, + $stripeCustomerId ? "https://dashboard.stripe.com/customers/{$stripeCustomerId}" : null, + null, + null, + null, + 'Missing subscription ID while invoice not past_due', + ]); + + continue; + } + + if (! $stripeSubscriptionId) { + // No subscription ID and invoice is past_due, still record for visibility + fputcsv($out, [ + $team->id, + $stripeInvoicePaid, + $stripeCustomerId ? "https://dashboard.stripe.com/customers/{$stripeCustomerId}" : null, + null, + null, + null, + 'Missing subscription ID', + ]); + + continue; + } + + $subscription = $stripe->subscriptions->retrieve($stripeSubscriptionId); + if ($subscription->status === 'active') { + continue; + } + + fputcsv($out, [ + $team->id, + $stripeInvoicePaid, + $stripeCustomerId ? "https://dashboard.stripe.com/customers/{$stripeCustomerId}" : null, + $stripeSubscriptionId, + $subscription->status, + "https://dashboard.stripe.com/subscriptions/{$stripeSubscriptionId}", + 'Subscription not active', + ]); + } + + fclose($out); + } + + /** + * Fix canceled subscriptions in the database + */ + private function fixCanceledSubscriptions(\Stripe\StripeClient $stripe) + { + $isDryRun = $this->option('dry-run'); + $checkOne = $this->option('one'); + + if ($isDryRun) { + $this->info('DRY RUN MODE - No changes will be made'); + if ($checkOne) { + $this->info('Checking only the first canceled subscription...'); + } else { + $this->info('Checking for canceled subscriptions...'); + } + } else { + if ($checkOne) { + $this->info('Checking and fixing only the first canceled subscription...'); + } else { + $this->info('Checking and fixing canceled subscriptions...'); + } + } + + $teamsWithSubscriptions = Team::whereRelation('subscription', 'stripe_invoice_paid', true)->get(); + $toFixCount = 0; + $fixedCount = 0; + $errors = []; + $canceledSubscriptions = []; + + foreach ($teamsWithSubscriptions as $team) { + $subscription = $team->subscription; + + if (! $subscription->stripe_subscription_id) { + continue; + } + + try { + $stripeSubscription = $stripe->subscriptions->retrieve( + $subscription->stripe_subscription_id + ); + + if ($stripeSubscription->status === 'canceled') { + $toFixCount++; + + // Get team members' emails + $memberEmails = $team->members->pluck('email')->toArray(); + + $canceledSubscriptions[] = [ + 'team_id' => $team->id, + 'team_name' => $team->name, + 'customer_id' => $subscription->stripe_customer_id, + 'subscription_id' => $subscription->stripe_subscription_id, + 'status' => 'canceled', + 'member_emails' => $memberEmails, + 'subscription_model' => $subscription->toArray(), + ]; + + if ($isDryRun) { + $this->warn('Would fix canceled subscription:'); + $this->line(" Team ID: {$team->id}"); + $this->line(" Team Name: {$team->name}"); + $this->line(' Team Members: '.implode(', ', $memberEmails)); + $this->line(" Customer URL: https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}"); + $this->line(" Subscription URL: https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}"); + $this->line(' Current Subscription Data:'); + foreach ($subscription->getAttributes() as $key => $value) { + if (is_null($value)) { + $this->line(" - {$key}: null"); + } elseif (is_bool($value)) { + $this->line(" - {$key}: ".($value ? 'true' : 'false')); + } else { + $this->line(" - {$key}: {$value}"); + } + } + $this->newLine(); + } else { + $this->warn("Found canceled subscription for Team ID: {$team->id}"); + + // Send internal notification with all details before fixing + $notificationMessage = "Fixing canceled subscription:\n"; + $notificationMessage .= "Team ID: {$team->id}\n"; + $notificationMessage .= "Team Name: {$team->name}\n"; + $notificationMessage .= 'Team Members: '.implode(', ', $memberEmails)."\n"; + $notificationMessage .= "Customer URL: https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}\n"; + $notificationMessage .= "Subscription URL: https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}\n"; + $notificationMessage .= "Subscription Data:\n"; + foreach ($subscription->getAttributes() as $key => $value) { + if (is_null($value)) { + $notificationMessage .= " - {$key}: null\n"; + } elseif (is_bool($value)) { + $notificationMessage .= " - {$key}: ".($value ? 'true' : 'false')."\n"; + } else { + $notificationMessage .= " - {$key}: {$value}\n"; + } + } + send_internal_notification($notificationMessage); + + // Apply the same logic as customer.subscription.deleted webhook + $team->subscriptionEnded(); + + $fixedCount++; + $this->info(" ✓ Fixed subscription for Team ID: {$team->id}"); + $this->line(' Team Members: '.implode(', ', $memberEmails)); + $this->line(" Customer URL: https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}"); + $this->line(" Subscription URL: https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}"); + } + + // Break if --one flag is set + if ($checkOne) { + break; + } + } + } catch (\Stripe\Exception\InvalidRequestException $e) { + if ($e->getStripeCode() === 'resource_missing') { + $toFixCount++; + + // Get team members' emails + $memberEmails = $team->members->pluck('email')->toArray(); + + $canceledSubscriptions[] = [ + 'team_id' => $team->id, + 'team_name' => $team->name, + 'customer_id' => $subscription->stripe_customer_id, + 'subscription_id' => $subscription->stripe_subscription_id, + 'status' => 'missing', + 'member_emails' => $memberEmails, + 'subscription_model' => $subscription->toArray(), + ]; + + if ($isDryRun) { + $this->error('Would fix missing subscription (not found in Stripe):'); + $this->line(" Team ID: {$team->id}"); + $this->line(" Team Name: {$team->name}"); + $this->line(' Team Members: '.implode(', ', $memberEmails)); + $this->line(" Customer URL: https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}"); + $this->line(" Subscription ID (missing): {$subscription->stripe_subscription_id}"); + $this->line(' Current Subscription Data:'); + foreach ($subscription->getAttributes() as $key => $value) { + if (is_null($value)) { + $this->line(" - {$key}: null"); + } elseif (is_bool($value)) { + $this->line(" - {$key}: ".($value ? 'true' : 'false')); + } else { + $this->line(" - {$key}: {$value}"); + } + } + $this->newLine(); + } else { + $this->error("Subscription not found in Stripe for Team ID: {$team->id}"); + + // Send internal notification with all details before fixing + $notificationMessage = "Fixing missing subscription (not found in Stripe):\n"; + $notificationMessage .= "Team ID: {$team->id}\n"; + $notificationMessage .= "Team Name: {$team->name}\n"; + $notificationMessage .= 'Team Members: '.implode(', ', $memberEmails)."\n"; + $notificationMessage .= "Customer URL: https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}\n"; + $notificationMessage .= "Subscription ID (missing): {$subscription->stripe_subscription_id}\n"; + $notificationMessage .= "Subscription Data:\n"; + foreach ($subscription->getAttributes() as $key => $value) { + if (is_null($value)) { + $notificationMessage .= " - {$key}: null\n"; + } elseif (is_bool($value)) { + $notificationMessage .= " - {$key}: ".($value ? 'true' : 'false')."\n"; + } else { + $notificationMessage .= " - {$key}: {$value}\n"; + } + } + send_internal_notification($notificationMessage); + + // Apply the same logic as customer.subscription.deleted webhook + $team->subscriptionEnded(); + + $fixedCount++; + $this->info(" ✓ Fixed missing subscription for Team ID: {$team->id}"); + $this->line(' Team Members: '.implode(', ', $memberEmails)); + $this->line(" Customer URL: https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}"); + } + + // Break if --one flag is set + if ($checkOne) { + break; + } + } else { + $errors[] = "Team ID {$team->id}: ".$e->getMessage(); + } + } catch (\Exception $e) { + $errors[] = "Team ID {$team->id}: ".$e->getMessage(); + } + } + + $this->newLine(); + $this->info('Summary:'); + + if ($isDryRun) { + $this->info(" - Found {$toFixCount} canceled/missing subscriptions that would be fixed"); + + if ($toFixCount > 0) { + $this->newLine(); + $this->comment('Run with --fix-canceled-subs to apply these changes'); + } + } else { + $this->info(" - Fixed {$fixedCount} canceled/missing subscriptions"); + } + + if (! empty($errors)) { + $this->newLine(); + $this->error('Errors encountered:'); + foreach ($errors as $error) { + $this->error(" - {$error}"); + } + } + + return 0; + } + + /** + * Verify all active subscriptions against Stripe API + */ + private function verifyAllActiveSubscriptions(\Stripe\StripeClient $stripe) + { + $isDryRun = $this->option('dry-run'); + $shouldFix = $this->option('fix-verified'); + + $this->info('Verifying all active subscriptions against Stripe...'); + if ($isDryRun) { + $this->info('DRY RUN MODE - No changes will be made'); + } + if ($shouldFix && ! $isDryRun) { + $this->warn('FIX MODE - Discrepancies will be corrected'); + } + + // Get all teams with active subscriptions + $teamsWithActiveSubscriptions = Team::whereRelation('subscription', 'stripe_invoice_paid', true)->get(); + $totalCount = $teamsWithActiveSubscriptions->count(); + + $this->info("Found {$totalCount} teams with active subscriptions in database"); + $this->newLine(); + + $out = fopen('php://output', 'w'); + + // CSV header + fputcsv($out, [ + 'team_id', + 'team_name', + 'customer_id', + 'subscription_id', + 'db_status', + 'stripe_status', + 'action', + 'member_emails', + 'customer_url', + 'subscription_url', + ]); + + $stats = [ + 'total' => $totalCount, + 'valid_active' => 0, + 'valid_past_due' => 0, + 'canceled' => 0, + 'missing' => 0, + 'invalid' => 0, + 'fixed' => 0, + 'errors' => 0, + ]; + + $processedCount = 0; + + foreach ($teamsWithActiveSubscriptions as $team) { + $subscription = $team->subscription; + $memberEmails = $team->members->pluck('email')->toArray(); + + // Database state + $dbStatus = 'active'; + if ($subscription->stripe_past_due) { + $dbStatus = 'past_due'; + } + + $stripeStatus = null; + $action = 'none'; + + if (! $subscription->stripe_subscription_id) { + $this->line("Team {$team->id}: Missing subscription ID, searching in Stripe..."); + + $foundResult = null; + $searchMethod = null; + + // Search by customer ID + if ($subscription->stripe_customer_id) { + $this->line(" → Searching by customer ID: {$subscription->stripe_customer_id}"); + $foundResult = $this->searchSubscriptionsByCustomer($stripe, $subscription->stripe_customer_id); + if ($foundResult) { + $searchMethod = $foundResult['method']; + } + } else { + $this->line(' → No customer ID available'); + } + + // Search by emails if not found + if (! $foundResult && count($memberEmails) > 0) { + $foundResult = $this->searchSubscriptionsByEmails($stripe, $memberEmails); + if ($foundResult) { + $searchMethod = $foundResult['method']; + + // Update customer ID if different + if (isset($foundResult['customer_id']) && $subscription->stripe_customer_id !== $foundResult['customer_id']) { + if ($isDryRun) { + $this->warn(" ⚠ Would update customer ID from {$subscription->stripe_customer_id} to {$foundResult['customer_id']}"); + } elseif ($shouldFix) { + $subscription->update(['stripe_customer_id' => $foundResult['customer_id']]); + $this->info(" ✓ Updated customer ID to {$foundResult['customer_id']}"); + } + } + } + } + + if ($foundResult && isset($foundResult['subscription'])) { + // Check if it's an active/past_due subscription + if (in_array($foundResult['status'], ['active', 'past_due'])) { + // Found an active subscription, handle update + $result = $this->handleFoundSubscription( + $team, + $subscription, + $foundResult['subscription'], + $searchMethod, + $isDryRun, + $shouldFix, + $stats + ); + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $result['id'], + $dbStatus, + $result['status'], + $result['action'], + implode(', ', $memberEmails), + $subscription->stripe_customer_id ? "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}" : 'N/A', + $result['url'], + ]); + } else { + // Found subscription but it's canceled/expired - needs to be deactivated + $this->warn(" → Found {$foundResult['status']} subscription {$foundResult['subscription']->id} - needs deactivation"); + + $result = $this->handleMissingSubscription($team, $subscription, $foundResult['status'], $isDryRun, $shouldFix, $stats); + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $foundResult['subscription']->id, + $dbStatus, + $foundResult['status'], + 'needs_fix', + implode(', ', $memberEmails), + $subscription->stripe_customer_id ? "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}" : 'N/A', + "https://dashboard.stripe.com/subscriptions/{$foundResult['subscription']->id}", + ]); + } + } else { + // No subscription found at all + $this->line(' → No subscription found'); + + $stripeStatus = 'not_found'; + $result = $this->handleMissingSubscription($team, $subscription, $stripeStatus, $isDryRun, $shouldFix, $stats); + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + 'N/A', + $dbStatus, + $result['status'], + $result['action'], + implode(', ', $memberEmails), + $subscription->stripe_customer_id ? "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}" : 'N/A', + 'N/A', + ]); + } + } else { + // First validate the subscription ID format + if (! str_starts_with($subscription->stripe_subscription_id, 'sub_')) { + $this->warn(" ⚠ Invalid subscription ID format (doesn't start with 'sub_')"); + } + + try { + $stripeSubscription = $stripe->subscriptions->retrieve( + $subscription->stripe_subscription_id + ); + + $stripeStatus = $stripeSubscription->status; + + // Determine if action is needed + switch ($stripeStatus) { + case 'active': + $stats['valid_active']++; + $action = 'valid'; + break; + + case 'past_due': + $stats['valid_past_due']++; + $action = 'valid'; + // Ensure past_due flag is set + if (! $subscription->stripe_past_due) { + if ($isDryRun) { + $this->info("Would set stripe_past_due=true for Team {$team->id}"); + } elseif ($shouldFix) { + $subscription->update(['stripe_past_due' => true]); + } + } + break; + + case 'canceled': + case 'incomplete_expired': + case 'unpaid': + case 'incomplete': + $stats['canceled']++; + $action = 'needs_fix'; + + // Only output problematic subscriptions + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $subscription->stripe_subscription_id, + $dbStatus, + $stripeStatus, + $action, + implode(', ', $memberEmails), + "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}", + "https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}", + ]); + + if ($isDryRun) { + $this->info("Would deactivate subscription for Team {$team->id} - status: {$stripeStatus}"); + } elseif ($shouldFix) { + $this->fixSubscription($team, $subscription, $stripeStatus); + $stats['fixed']++; + } + break; + + default: + $stats['invalid']++; + $action = 'unknown'; + + // Only output problematic subscriptions + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $subscription->stripe_subscription_id, + $dbStatus, + $stripeStatus, + $action, + implode(', ', $memberEmails), + "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}", + "https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}", + ]); + break; + } + + } catch (\Stripe\Exception\InvalidRequestException $e) { + $this->error(' → Error: '.$e->getMessage()); + + if ($e->getStripeCode() === 'resource_missing' || $e->getHttpStatus() === 404) { + // Subscription doesn't exist, try to find by customer ID + $this->warn(" → Subscription not found, checking customer's subscriptions..."); + + $foundResult = null; + if ($subscription->stripe_customer_id) { + $foundResult = $this->searchSubscriptionsByCustomer($stripe, $subscription->stripe_customer_id); + } + + if ($foundResult && isset($foundResult['subscription']) && in_array($foundResult['status'], ['active', 'past_due'])) { + // Found an active subscription with different ID + $this->warn(" → ID mismatch! DB: {$subscription->stripe_subscription_id}, Stripe: {$foundResult['subscription']->id}"); + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + "WRONG ID: {$subscription->stripe_subscription_id} → {$foundResult['subscription']->id}", + $dbStatus, + $foundResult['status'], + 'id_mismatch', + implode(', ', $memberEmails), + "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}", + "https://dashboard.stripe.com/subscriptions/{$foundResult['subscription']->id}", + ]); + + if ($isDryRun) { + $this->warn(" → Would update subscription ID to {$foundResult['subscription']->id}"); + } elseif ($shouldFix) { + $subscription->update([ + 'stripe_subscription_id' => $foundResult['subscription']->id, + 'stripe_invoice_paid' => true, + 'stripe_past_due' => $foundResult['status'] === 'past_due', + ]); + $stats['fixed']++; + $this->info(' → Updated subscription ID'); + } + + $stats[$foundResult['status'] === 'active' ? 'valid_active' : 'valid_past_due']++; + } else { + // No active subscription found + $stripeStatus = $foundResult ? $foundResult['status'] : 'not_found'; + $result = $this->handleMissingSubscription($team, $subscription, $stripeStatus, $isDryRun, $shouldFix, $stats); + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $subscription->stripe_subscription_id, + $dbStatus, + $result['status'], + $result['action'], + implode(', ', $memberEmails), + $subscription->stripe_customer_id ? "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}" : 'N/A', + $foundResult && isset($foundResult['subscription']) ? "https://dashboard.stripe.com/subscriptions/{$foundResult['subscription']->id}" : 'N/A', + ]); + } + } else { + // Other API error + $stats['errors']++; + $this->error(' → API Error - not marking as deleted'); + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $subscription->stripe_subscription_id, + $dbStatus, + 'error: '.$e->getStripeCode(), + 'error', + implode(', ', $memberEmails), + $subscription->stripe_customer_id ? "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}" : 'N/A', + $subscription->stripe_subscription_id ? "https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}" : 'N/A', + ]); + } + } catch (\Exception $e) { + $this->error(' → Unexpected error: '.$e->getMessage()); + $stats['errors']++; + + fputcsv($out, [ + $team->id, + $team->name, + $subscription->stripe_customer_id, + $subscription->stripe_subscription_id, + $dbStatus, + 'error', + 'error', + implode(', ', $memberEmails), + $subscription->stripe_customer_id ? "https://dashboard.stripe.com/customers/{$subscription->stripe_customer_id}" : 'N/A', + $subscription->stripe_subscription_id ? "https://dashboard.stripe.com/subscriptions/{$subscription->stripe_subscription_id}" : 'N/A', + ]); + } + } + + $processedCount++; + if ($processedCount % 100 === 0) { + $this->info("Processed {$processedCount}/{$totalCount} subscriptions..."); + } + } + + fclose($out); + + // Print summary + $this->newLine(2); + $this->info('=== Verification Summary ==='); + $this->info("Total subscriptions checked: {$stats['total']}"); + $this->newLine(); + + $this->info('Valid subscriptions in Stripe:'); + $this->line(" - Active: {$stats['valid_active']}"); + $this->line(" - Past Due: {$stats['valid_past_due']}"); + $validTotal = $stats['valid_active'] + $stats['valid_past_due']; + $this->info(" Total valid: {$validTotal}"); + + $this->newLine(); + $this->warn('Invalid subscriptions:'); + $this->line(" - Canceled/Expired: {$stats['canceled']}"); + $this->line(" - Missing/Not Found: {$stats['missing']}"); + $this->line(" - Unknown status: {$stats['invalid']}"); + $invalidTotal = $stats['canceled'] + $stats['missing'] + $stats['invalid']; + $this->warn(" Total invalid: {$invalidTotal}"); + + if ($stats['errors'] > 0) { + $this->newLine(); + $this->error("Errors encountered: {$stats['errors']}"); + } + + if ($shouldFix && ! $isDryRun) { + $this->newLine(); + $this->info("Fixed subscriptions: {$stats['fixed']}"); + } elseif ($invalidTotal > 0 && ! $shouldFix) { + $this->newLine(); + $this->comment('Run with --fix-verified to fix the discrepancies'); + } + + return 0; + } + + /** + * Fix a subscription based on its status + */ + private function fixSubscription($team, $subscription, $status) + { + $message = "Fixing subscription for Team ID: {$team->id} (Status: {$status})\n"; + $message .= "Team Name: {$team->name}\n"; + $message .= "Customer ID: {$subscription->stripe_customer_id}\n"; + $message .= "Subscription ID: {$subscription->stripe_subscription_id}\n"; + + send_internal_notification($message); + + // Call the team's subscription ended method which properly cleans up + $team->subscriptionEnded(); + } + + /** + * Search for subscriptions by customer ID + */ + private function searchSubscriptionsByCustomer(\Stripe\StripeClient $stripe, $customerId, $requireActive = false) + { + try { + $subscriptions = $stripe->subscriptions->all([ + 'customer' => $customerId, + 'limit' => 10, + 'status' => 'all', + ]); + + $this->line(' → Found '.count($subscriptions->data).' subscription(s) for customer'); + + // Look for active/past_due first + foreach ($subscriptions->data as $sub) { + $this->line(" - Subscription {$sub->id}: status={$sub->status}"); + if (in_array($sub->status, ['active', 'past_due'])) { + $this->info(" ✓ Found active/past_due subscription: {$sub->id}"); + + return ['subscription' => $sub, 'status' => $sub->status, 'method' => 'customer_id']; + } + } + + // If not requiring active and there are subscriptions, return first one + if (! $requireActive && count($subscriptions->data) > 0) { + $sub = $subscriptions->data[0]; + $this->warn(" ⚠ Only found {$sub->status} subscription: {$sub->id}"); + + return ['subscription' => $sub, 'status' => $sub->status, 'method' => 'customer_id_first']; + } + + return null; + } catch (\Exception $e) { + $this->error(' → Error searching by customer ID: '.$e->getMessage()); + + return null; + } + } + + /** + * Search for subscriptions by team member emails + */ + private function searchSubscriptionsByEmails(\Stripe\StripeClient $stripe, $emails) + { + $this->line(' → Searching by team member emails...'); + + foreach ($emails as $email) { + $this->line(" → Checking email: {$email}"); + + try { + $customers = $stripe->customers->all([ + 'email' => $email, + 'limit' => 5, + ]); + + if (count($customers->data) === 0) { + $this->line(' - No customers found'); + + continue; + } + + $this->line(' - Found '.count($customers->data).' customer(s)'); + + foreach ($customers->data as $customer) { + $this->line(" - Checking customer {$customer->id}"); + + $result = $this->searchSubscriptionsByCustomer($stripe, $customer->id, true); + if ($result) { + $result['method'] = "email:{$email}"; + $result['customer_id'] = $customer->id; + + return $result; + } + } + } catch (\Exception $e) { + $this->error(" - Error searching for email {$email}: ".$e->getMessage()); + } + } + + return null; + } + + /** + * Handle found subscription update (only for active/past_due subscriptions) + */ + private function handleFoundSubscription($team, $subscription, $foundSub, $searchMethod, $isDryRun, $shouldFix, &$stats) + { + $stripeStatus = $foundSub->status; + $this->info(" ✓ FOUND active/past_due subscription {$foundSub->id} (status: {$stripeStatus})"); + + // Only update if it's active or past_due + if (! in_array($stripeStatus, ['active', 'past_due'])) { + $this->error(" ERROR: handleFoundSubscription called with {$stripeStatus} subscription!"); + + return [ + 'id' => $foundSub->id, + 'status' => $stripeStatus, + 'action' => 'error', + 'url' => "https://dashboard.stripe.com/subscriptions/{$foundSub->id}", + ]; + } + + if ($isDryRun) { + $this->warn(" → Would update subscription ID to {$foundSub->id} (status: {$stripeStatus})"); + } elseif ($shouldFix) { + $subscription->update([ + 'stripe_subscription_id' => $foundSub->id, + 'stripe_invoice_paid' => true, + 'stripe_past_due' => $stripeStatus === 'past_due', + ]); + $stats['fixed']++; + $this->info(" → Updated subscription ID to {$foundSub->id}"); + } + + // Update stats + $stats[$stripeStatus === 'active' ? 'valid_active' : 'valid_past_due']++; + + return [ + 'id' => "FOUND: {$foundSub->id}", + 'status' => $stripeStatus, + 'action' => "will_update (via {$searchMethod})", + 'url' => "https://dashboard.stripe.com/subscriptions/{$foundSub->id}", + ]; + } + + /** + * Handle missing subscription + */ + private function handleMissingSubscription($team, $subscription, $status, $isDryRun, $shouldFix, &$stats) + { + $stats['missing']++; + + if ($isDryRun) { + $statusMsg = $status !== 'not_found' ? "status: {$status}" : 'no subscription found in Stripe'; + $this->warn(" → Would deactivate subscription - {$statusMsg}"); + } elseif ($shouldFix) { + $this->fixSubscription($team, $subscription, $status); + $stats['fixed']++; + $this->info(' → Deactivated subscription'); + } + + return [ + 'id' => 'N/A', + 'status' => $status, + 'action' => 'needs_fix', + 'url' => 'N/A', + ]; + } +} diff --git a/app/Console/Commands/CloudCheckSubscription.php b/app/Console/Commands/CloudCheckSubscription.php deleted file mode 100644 index 6e237e84b..000000000 --- a/app/Console/Commands/CloudCheckSubscription.php +++ /dev/null @@ -1,49 +0,0 @@ -<?php - -namespace App\Console\Commands; - -use App\Models\Team; -use Illuminate\Console\Command; - -class CloudCheckSubscription extends Command -{ - /** - * The name and signature of the console command. - * - * @var string - */ - protected $signature = 'cloud:check-subscription'; - - /** - * The console command description. - * - * @var string - */ - protected $description = 'Check Cloud subscriptions'; - - /** - * Execute the console command. - */ - public function handle() - { - $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); - $activeSubscribers = Team::whereRelation('subscription', 'stripe_invoice_paid', true)->get(); - foreach ($activeSubscribers as $team) { - $stripeSubscriptionId = $team->subscription->stripe_subscription_id; - $stripeInvoicePaid = $team->subscription->stripe_invoice_paid; - $stripeCustomerId = $team->subscription->stripe_customer_id; - if (! $stripeSubscriptionId) { - echo "Team {$team->id} has no subscription, but invoice status is: {$stripeInvoicePaid}\n"; - echo "Link on Stripe: https://dashboard.stripe.com/customers/{$stripeCustomerId}\n"; - - continue; - } - $subscription = $stripe->subscriptions->retrieve($stripeSubscriptionId); - if ($subscription->status === 'active') { - continue; - } - echo "Subscription {$stripeSubscriptionId} is not active ({$subscription->status})\n"; - echo "Link on Stripe: https://dashboard.stripe.com/subscriptions/{$stripeSubscriptionId}\n"; - } - } -} diff --git a/app/Console/Commands/CloudCleanupSubscriptions.php b/app/Console/Commands/CloudCleanupSubscriptions.php deleted file mode 100644 index ab676c927..000000000 --- a/app/Console/Commands/CloudCleanupSubscriptions.php +++ /dev/null @@ -1,101 +0,0 @@ -<?php - -namespace App\Console\Commands; - -use App\Events\ServerReachabilityChanged; -use App\Models\Team; -use Illuminate\Console\Command; - -class CloudCleanupSubscriptions extends Command -{ - protected $signature = 'cloud:cleanup-subs'; - - protected $description = 'Cleanup subcriptions teams'; - - public function handle() - { - try { - if (! isCloud()) { - $this->error('This command can only be run on cloud'); - - return; - } - $this->info('Cleaning up subcriptions teams'); - $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); - - $teams = Team::all()->filter(function ($team) { - return $team->id !== 0; - })->sortBy('id'); - foreach ($teams as $team) { - if ($team) { - $this->info("Checking team {$team->id}"); - } - if (! data_get($team, 'subscription')) { - $this->disableServers($team); - - continue; - } - // If the team has no subscription id and the invoice is paid, we need to reset the invoice paid status - if (! (data_get($team, 'subscription.stripe_subscription_id'))) { - $this->info("Resetting invoice paid status for team {$team->id}"); - - $team->subscription->update([ - 'stripe_invoice_paid' => false, - 'stripe_trial_already_ended' => false, - 'stripe_subscription_id' => null, - ]); - $this->disableServers($team); - - continue; - } else { - $subscription = $stripe->subscriptions->retrieve(data_get($team, 'subscription.stripe_subscription_id'), []); - $status = data_get($subscription, 'status'); - if ($status === 'active') { - $team->subscription->update([ - 'stripe_invoice_paid' => true, - 'stripe_trial_already_ended' => false, - ]); - - continue; - } - $this->info('Subscription status: '.$status); - $this->info('Subscription id: '.data_get($team, 'subscription.stripe_subscription_id')); - $confirm = $this->confirm('Do you want to cancel the subscription?', true); - if (! $confirm) { - $this->info("Skipping team {$team->id}"); - } else { - $this->info("Cancelling subscription for team {$team->id}"); - $team->subscription->update([ - 'stripe_invoice_paid' => false, - 'stripe_trial_already_ended' => false, - 'stripe_subscription_id' => null, - ]); - $this->disableServers($team); - } - } - } - } catch (\Exception $e) { - $this->error($e->getMessage()); - - return; - } - } - - private function disableServers(Team $team) - { - foreach ($team->servers as $server) { - if ($server->settings->is_usable === true || $server->settings->is_reachable === true || $server->ip !== '1.2.3.4') { - $this->info("Disabling server {$server->id} {$server->name}"); - $server->settings()->update([ - 'is_usable' => false, - 'is_reachable' => false, - ]); - $server->update([ - 'ip' => '1.2.3.4', - ]); - - ServerReachabilityChanged::dispatch($server); - } - } - } -} diff --git a/app/Console/Commands/Dev.php b/app/Console/Commands/Dev.php index a4cfde6f8..8f26d78ff 100644 --- a/app/Console/Commands/Dev.php +++ b/app/Console/Commands/Dev.php @@ -2,6 +2,7 @@ namespace App\Console\Commands; +use App\Jobs\CheckHelperImageJob; use App\Models\InstanceSettings; use Illuminate\Console\Command; use Illuminate\Support\Facades\Artisan; @@ -44,5 +45,6 @@ public function init() } else { echo "Instance already initialized.\n"; } + CheckHelperImageJob::dispatch(); } } diff --git a/app/Console/Commands/Init.php b/app/Console/Commands/Init.php index b85829256..6e8d18f61 100644 --- a/app/Console/Commands/Init.php +++ b/app/Console/Commands/Init.php @@ -5,9 +5,10 @@ use App\Enums\ActivityTypes; use App\Enums\ApplicationDeploymentStatus; use App\Jobs\CheckHelperImageJob; -use App\Jobs\PullChangelogFromGitHub; +use App\Jobs\PullChangelog; use App\Models\ApplicationDeploymentQueue; use App\Models\Environment; +use App\Models\InstanceSettings; use App\Models\ScheduledDatabaseBackup; use App\Models\Server; use App\Models\StandalonePostgresql; @@ -19,80 +20,18 @@ class Init extends Command { - protected $signature = 'app:init {--force-cloud}'; + protected $signature = 'app:init'; protected $description = 'Cleanup instance related stuffs'; public $servers = null; + public InstanceSettings $settings; + public function handle() { - $this->optimize(); - - if (isCloud() && ! $this->option('force-cloud')) { - echo "Skipping init as we are on cloud and --force-cloud option is not set\n"; - - return; - } - - $this->servers = Server::all(); - if (! isCloud()) { - $this->sendAliveSignal(); - get_public_ips(); - } - - // Backward compatibility - $this->replaceSlashInEnvironmentName(); - $this->restoreCoolifyDbBackup(); - $this->updateUserEmails(); - // - $this->updateTraefikLabels(); - if (! isCloud() || $this->option('force-cloud')) { - $this->cleanupUnusedNetworkFromCoolifyProxy(); - } - - $this->call('cleanup:redis'); - - try { - $this->call('cleanup:names'); - } catch (\Throwable $e) { - echo "Error in cleanup:names command: {$e->getMessage()}\n"; - } - $this->call('cleanup:stucked-resources'); - - try { - $this->pullHelperImage(); - } catch (\Throwable $e) { - // - } - - if (isCloud()) { - try { - $this->cleanupInProgressApplicationDeployments(); - } catch (\Throwable $e) { - echo "Could not cleanup inprogress deployments: {$e->getMessage()}\n"; - } - - try { - $this->pullTemplatesFromCDN(); - } catch (\Throwable $e) { - echo "Could not pull templates from CDN: {$e->getMessage()}\n"; - } - - try { - $this->pullChangelogFromGitHub(); - } catch (\Throwable $e) { - echo "Could not changelogs from github: {$e->getMessage()}\n"; - } - - return; - } - - try { - $this->cleanupInProgressApplicationDeployments(); - } catch (\Throwable $e) { - echo "Could not cleanup inprogress deployments: {$e->getMessage()}\n"; - } + Artisan::call('optimize:clear'); + Artisan::call('optimize'); try { $this->pullTemplatesFromCDN(); @@ -105,20 +44,80 @@ public function handle() } catch (\Throwable $e) { echo "Could not changelogs from github: {$e->getMessage()}\n"; } + + try { + $this->pullHelperImage(); + } catch (\Throwable $e) { + echo "Error in pullHelperImage command: {$e->getMessage()}\n"; + } + + if (isCloud()) { + return; + } + + $this->settings = instanceSettings(); + $this->servers = Server::all(); + + $do_not_track = data_get($this->settings, 'do_not_track', true); + if ($do_not_track == false) { + $this->sendAliveSignal(); + } + get_public_ips(); + + // Backward compatibility + $this->replaceSlashInEnvironmentName(); + $this->restoreCoolifyDbBackup(); + $this->updateUserEmails(); + // + $this->updateTraefikLabels(); + $this->cleanupUnusedNetworkFromCoolifyProxy(); + + try { + $this->call('cleanup:redis'); + } catch (\Throwable $e) { + echo "Error in cleanup:redis command: {$e->getMessage()}\n"; + } + try { + $this->call('cleanup:names'); + } catch (\Throwable $e) { + echo "Error in cleanup:names command: {$e->getMessage()}\n"; + } + try { + $this->call('cleanup:stucked-resources'); + } catch (\Throwable $e) { + echo "Error in cleanup:stucked-resources command: {$e->getMessage()}\n"; + } + try { + $updatedCount = ApplicationDeploymentQueue::whereIn('status', [ + ApplicationDeploymentStatus::IN_PROGRESS->value, + ApplicationDeploymentStatus::QUEUED->value, + ])->update([ + 'status' => ApplicationDeploymentStatus::FAILED->value, + ]); + + if ($updatedCount > 0) { + echo "Marked {$updatedCount} stuck deployments as failed\n"; + } + } catch (\Throwable $e) { + echo "Could not cleanup inprogress deployments: {$e->getMessage()}\n"; + } + try { $localhost = $this->servers->where('id', 0)->first(); - $localhost->setupDynamicProxyConfiguration(); + if ($localhost) { + $localhost->setupDynamicProxyConfiguration(); + } } catch (\Throwable $e) { echo "Could not setup dynamic configuration: {$e->getMessage()}\n"; } - $settings = instanceSettings(); + if (! is_null(config('constants.coolify.autoupdate', null))) { if (config('constants.coolify.autoupdate') == true) { echo "Enabling auto-update\n"; - $settings->update(['is_auto_update_enabled' => true]); + $this->settings->update(['is_auto_update_enabled' => true]); } else { echo "Disabling auto-update\n"; - $settings->update(['is_auto_update_enabled' => false]); + $this->settings->update(['is_auto_update_enabled' => false]); } } } @@ -140,24 +139,18 @@ private function pullTemplatesFromCDN() private function pullChangelogFromGitHub() { try { - PullChangelogFromGitHub::dispatch(); + PullChangelog::dispatch(); echo "Changelog fetch initiated\n"; } catch (\Throwable $e) { echo "Could not fetch changelog from GitHub: {$e->getMessage()}\n"; } } - private function optimize() - { - Artisan::call('optimize:clear'); - Artisan::call('optimize'); - } - private function updateUserEmails() { try { User::whereRaw('email ~ \'[A-Z]\'')->get()->each(function (User $user) { - $user->update(['email' => strtolower($user->email)]); + $user->update(['email' => $user->email]); }); } catch (\Throwable $e) { echo "Error in updating user emails: {$e->getMessage()}\n"; @@ -173,27 +166,6 @@ private function updateTraefikLabels() } } - private function cleanupUnnecessaryDynamicProxyConfiguration() - { - foreach ($this->servers as $server) { - try { - if (! $server->isFunctional()) { - continue; - } - if ($server->id === 0) { - continue; - } - $file = $server->proxyPath().'/dynamic/coolify.yaml'; - - return instant_remote_process([ - "rm -f $file", - ], $server, false); - } catch (\Throwable $e) { - echo "Error in cleaning up unnecessary dynamic proxy configuration: {$e->getMessage()}\n"; - } - } - } - private function cleanupUnusedNetworkFromCoolifyProxy() { foreach ($this->servers as $server) { @@ -263,13 +235,6 @@ private function sendAliveSignal() { $id = config('app.id'); $version = config('constants.coolify.version'); - $settings = instanceSettings(); - $do_not_track = data_get($settings, 'do_not_track'); - if ($do_not_track == true) { - echo "Do_not_track is enabled\n"; - - return; - } try { Http::get("https://undead.coolify.io/v4/alive?appId=$id&version=$version"); } catch (\Throwable $e) { @@ -277,23 +242,6 @@ private function sendAliveSignal() } } - private function cleanupInProgressApplicationDeployments() - { - // Cleanup any failed deployments - try { - if (isCloud()) { - return; - } - $queued_inprogress_deployments = ApplicationDeploymentQueue::whereIn('status', [ApplicationDeploymentStatus::IN_PROGRESS->value, ApplicationDeploymentStatus::QUEUED->value])->get(); - foreach ($queued_inprogress_deployments as $deployment) { - $deployment->status = ApplicationDeploymentStatus::FAILED->value; - $deployment->save(); - } - } catch (\Throwable $e) { - echo "Error: {$e->getMessage()}\n"; - } - } - private function replaceSlashInEnvironmentName() { if (version_compare('4.0.0-beta.298', config('constants.coolify.version'), '<=')) { diff --git a/app/Console/Commands/InitChangelog.php b/app/Console/Commands/InitChangelog.php deleted file mode 100644 index f9eb12f04..000000000 --- a/app/Console/Commands/InitChangelog.php +++ /dev/null @@ -1,98 +0,0 @@ -<?php - -namespace App\Console\Commands; - -use Carbon\Carbon; -use Illuminate\Console\Command; - -class InitChangelog extends Command -{ - /** - * The name and signature of the console command. - * - * @var string - */ - protected $signature = 'changelog:init {month? : Month in YYYY-MM format (defaults to current month)}'; - - /** - * The console command description. - * - * @var string - */ - protected $description = 'Initialize a new monthly changelog file with example structure'; - - /** - * Execute the console command. - */ - public function handle() - { - $month = $this->argument('month') ?: Carbon::now()->format('Y-m'); - - // Validate month format - if (! preg_match('/^\d{4}-(0[1-9]|1[0-2])$/', $month)) { - $this->error('Invalid month format. Use YYYY-MM format with valid months 01-12 (e.g., 2025-08)'); - - return self::FAILURE; - } - - $changelogsDir = base_path('changelogs'); - $filePath = $changelogsDir."/{$month}.json"; - - // Create changelogs directory if it doesn't exist - if (! is_dir($changelogsDir)) { - mkdir($changelogsDir, 0755, true); - $this->info("Created changelogs directory: {$changelogsDir}"); - } - - // Check if file already exists - if (file_exists($filePath)) { - if (! $this->confirm("File {$month}.json already exists. Overwrite?")) { - $this->info('Operation cancelled'); - - return self::SUCCESS; - } - } - - // Parse the month for example data - $carbonMonth = Carbon::createFromFormat('Y-m', $month); - $monthName = $carbonMonth->format('F Y'); - $sampleDate = $carbonMonth->addDays(14)->toISOString(); // Mid-month - - // Get version from config - $version = 'v'.config('constants.coolify.version'); - - // Create example changelog structure - $exampleData = [ - 'entries' => [ - [ - 'version' => $version, - 'title' => 'Example Feature Release', - 'content' => "This is an example changelog entry for {$monthName}. Replace this with your actual release notes. Include details about new features, improvements, bug fixes, and any breaking changes.", - 'published_at' => $sampleDate, - ], - ], - ]; - - // Write the file - $jsonContent = json_encode($exampleData, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES); - - if (file_put_contents($filePath, $jsonContent) === false) { - $this->error("Failed to create changelog file: {$filePath}"); - - return self::FAILURE; - } - - $this->info("✅ Created changelog file: changelogs/{$month}.json"); - $this->line(" Example entry created for {$monthName}"); - $this->line(' Edit the file to add your actual changelog entries'); - - // Show the file contents - if ($this->option('verbose')) { - $this->newLine(); - $this->line('File contents:'); - $this->line($jsonContent); - } - - return self::SUCCESS; - } -} diff --git a/app/Console/Commands/ServicesDelete.php b/app/Console/Commands/ServicesDelete.php index b5a74166a..870cef3d9 100644 --- a/app/Console/Commands/ServicesDelete.php +++ b/app/Console/Commands/ServicesDelete.php @@ -6,7 +6,14 @@ use App\Models\Application; use App\Models\Server; use App\Models\Service; +use App\Models\StandaloneClickhouse; +use App\Models\StandaloneDragonfly; +use App\Models\StandaloneKeydb; +use App\Models\StandaloneMariadb; +use App\Models\StandaloneMongodb; +use App\Models\StandaloneMysql; use App\Models\StandalonePostgresql; +use App\Models\StandaloneRedis; use Illuminate\Console\Command; use function Laravel\Prompts\confirm; @@ -103,19 +110,79 @@ private function deleteApplication() private function deleteDatabase() { - $databases = StandalonePostgresql::all(); - if ($databases->count() === 0) { + // Collect all databases from all types with unique identifiers + $allDatabases = collect(); + $databaseOptions = collect(); + + // Add PostgreSQL databases + foreach (StandalonePostgresql::all() as $db) { + $key = "postgresql_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (PostgreSQL)"); + } + + // Add MySQL databases + foreach (StandaloneMysql::all() as $db) { + $key = "mysql_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (MySQL)"); + } + + // Add MariaDB databases + foreach (StandaloneMariadb::all() as $db) { + $key = "mariadb_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (MariaDB)"); + } + + // Add MongoDB databases + foreach (StandaloneMongodb::all() as $db) { + $key = "mongodb_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (MongoDB)"); + } + + // Add Redis databases + foreach (StandaloneRedis::all() as $db) { + $key = "redis_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (Redis)"); + } + + // Add KeyDB databases + foreach (StandaloneKeydb::all() as $db) { + $key = "keydb_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (KeyDB)"); + } + + // Add Dragonfly databases + foreach (StandaloneDragonfly::all() as $db) { + $key = "dragonfly_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (Dragonfly)"); + } + + // Add ClickHouse databases + foreach (StandaloneClickhouse::all() as $db) { + $key = "clickhouse_{$db->id}"; + $allDatabases->put($key, $db); + $databaseOptions->put($key, "{$db->name} (ClickHouse)"); + } + + if ($allDatabases->count() === 0) { $this->error('There are no databases to delete.'); return; } + $databasesToDelete = multiselect( 'What database do you want to delete?', - $databases->pluck('name', 'id')->sortKeys(), + $databaseOptions->sortKeys(), ); - foreach ($databasesToDelete as $database) { - $toDelete = $databases->where('id', $database)->first(); + foreach ($databasesToDelete as $databaseKey) { + $toDelete = $allDatabases->get($databaseKey); if ($toDelete) { $this->info($toDelete); $confirmed = confirm('Are you sure you want to delete all selected resources?'); diff --git a/app/Console/Commands/SyncBunny.php b/app/Console/Commands/SyncBunny.php index 6581bb587..b0cd24715 100644 --- a/app/Console/Commands/SyncBunny.php +++ b/app/Console/Commands/SyncBunny.php @@ -16,7 +16,7 @@ class SyncBunny extends Command * * @var string */ - protected $signature = 'sync:bunny {--templates} {--release} {--nightly}'; + protected $signature = 'sync:bunny {--templates} {--release} {--github-releases} {--nightly}'; /** * The console command description. @@ -25,6 +25,50 @@ class SyncBunny extends Command */ protected $description = 'Sync files to BunnyCDN'; + /** + * Fetch GitHub releases and sync to CDN + */ + private function syncGitHubReleases($parent_dir, $bunny_cdn_storage_name, $bunny_cdn_path, $bunny_cdn) + { + $this->info('Fetching releases from GitHub...'); + try { + $response = Http::timeout(30) + ->get('https://api.github.com/repos/coollabsio/coolify/releases', [ + 'per_page' => 30, // Fetch more releases for better changelog + ]); + + if ($response->successful()) { + $releases = $response->json(); + + // Save releases to a temporary file + $releases_file = "$parent_dir/releases.json"; + file_put_contents($releases_file, json_encode($releases, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES)); + + // Upload to CDN + Http::pool(fn (Pool $pool) => [ + $pool->storage(fileName: $releases_file)->put("/$bunny_cdn_storage_name/$bunny_cdn_path/releases.json"), + $pool->purge("$bunny_cdn/coolify/releases.json"), + ]); + + // Clean up temporary file + unlink($releases_file); + + $this->info('releases.json uploaded & purged...'); + $this->info('Total releases synced: '.count($releases)); + + return true; + } else { + $this->error('Failed to fetch releases from GitHub: '.$response->status()); + + return false; + } + } catch (\Throwable $e) { + $this->error('Error fetching releases: '.$e->getMessage()); + + return false; + } + } + /** * Execute the console command. */ @@ -33,6 +77,7 @@ public function handle() $that = $this; $only_template = $this->option('templates'); $only_version = $this->option('release'); + $only_github_releases = $this->option('github-releases'); $nightly = $this->option('nightly'); $bunny_cdn = 'https://cdn.coollabs.io'; $bunny_cdn_path = 'coolify'; @@ -90,7 +135,7 @@ public function handle() $install_script_location = "$parent_dir/other/nightly/$install_script"; $versions_location = "$parent_dir/other/nightly/$versions"; } - if (! $only_template && ! $only_version) { + if (! $only_template && ! $only_version && ! $only_github_releases) { if ($nightly) { $this->info('About to sync files NIGHTLY (docker-compose.prod.yaml, upgrade.sh, install.sh, etc) to BunnyCDN.'); } else { @@ -128,12 +173,29 @@ public function handle() if (! $confirmed) { return; } + + // First sync GitHub releases + $this->info('Syncing GitHub releases first...'); + $this->syncGitHubReleases($parent_dir, $bunny_cdn_storage_name, $bunny_cdn_path, $bunny_cdn); + + // Then sync versions.json Http::pool(fn (Pool $pool) => [ $pool->storage(fileName: $versions_location)->put("/$bunny_cdn_storage_name/$bunny_cdn_path/$versions"), $pool->purge("$bunny_cdn/$bunny_cdn_path/$versions"), ]); $this->info('versions.json uploaded & purged...'); + return; + } elseif ($only_github_releases) { + $this->info('About to sync GitHub releases to BunnyCDN.'); + $confirmed = confirm('Are you sure you want to sync GitHub releases?'); + if (! $confirmed) { + return; + } + + // Use the reusable function + $this->syncGitHubReleases($parent_dir, $bunny_cdn_storage_name, $bunny_cdn_path, $bunny_cdn); + return; } diff --git a/app/Console/Kernel.php b/app/Console/Kernel.php index c5c4d7e7f..c2ea27274 100644 --- a/app/Console/Kernel.php +++ b/app/Console/Kernel.php @@ -6,7 +6,7 @@ use App\Jobs\CheckForUpdatesJob; use App\Jobs\CheckHelperImageJob; use App\Jobs\CleanupInstanceStuffsJob; -use App\Jobs\PullChangelogFromGitHub; +use App\Jobs\PullChangelog; use App\Jobs\PullTemplatesFromCDN; use App\Jobs\RegenerateSslCertJob; use App\Jobs\ScheduledJobManager; @@ -68,7 +68,7 @@ protected function schedule(Schedule $schedule): void $this->scheduleInstance->command('cleanup:unreachable-servers')->daily()->onOneServer(); $this->scheduleInstance->job(new PullTemplatesFromCDN)->cron($this->updateCheckFrequency)->timezone($this->instanceTimezone)->onOneServer(); - $this->scheduleInstance->job(new PullChangelogFromGitHub)->cron($this->updateCheckFrequency)->timezone($this->instanceTimezone)->onOneServer(); + $this->scheduleInstance->job(new PullChangelog)->cron($this->updateCheckFrequency)->timezone($this->instanceTimezone)->onOneServer(); $this->scheduleInstance->job(new CleanupInstanceStuffsJob)->everyTwoMinutes()->onOneServer(); $this->scheduleUpdates(); diff --git a/app/Events/ApplicationConfigurationChanged.php b/app/Events/ApplicationConfigurationChanged.php new file mode 100644 index 000000000..3dd532b19 --- /dev/null +++ b/app/Events/ApplicationConfigurationChanged.php @@ -0,0 +1,35 @@ +<?php + +namespace App\Events; + +use Illuminate\Broadcasting\InteractsWithSockets; +use Illuminate\Broadcasting\PrivateChannel; +use Illuminate\Contracts\Broadcasting\ShouldBroadcast; +use Illuminate\Foundation\Events\Dispatchable; +use Illuminate\Queue\SerializesModels; + +class ApplicationConfigurationChanged implements ShouldBroadcast +{ + use Dispatchable, InteractsWithSockets, SerializesModels; + + public ?int $teamId = null; + + public function __construct($teamId = null) + { + if (is_null($teamId) && auth()->check() && auth()->user()->currentTeam()) { + $teamId = auth()->user()->currentTeam()->id; + } + $this->teamId = $teamId; + } + + public function broadcastOn(): array + { + if (is_null($this->teamId)) { + return []; + } + + return [ + new PrivateChannel("team.{$this->teamId}"), + ]; + } +} diff --git a/app/Exceptions/Handler.php b/app/Exceptions/Handler.php index 275de57c0..3d731223d 100644 --- a/app/Exceptions/Handler.php +++ b/app/Exceptions/Handler.php @@ -29,6 +29,7 @@ class Handler extends ExceptionHandler */ protected $dontReport = [ ProcessException::class, + NonReportableException::class, ]; /** @@ -110,9 +111,14 @@ function (Scope $scope) { ); } ); + // Check for errors that should not be reported to Sentry if (str($e->getMessage())->contains('No space left on device')) { + // Log locally but don't send to Sentry + logger()->warning('Disk space error: '.$e->getMessage()); + return; } + Integration::captureUnhandledException($e); }); } diff --git a/app/Exceptions/NonReportableException.php b/app/Exceptions/NonReportableException.php new file mode 100644 index 000000000..4c4672127 --- /dev/null +++ b/app/Exceptions/NonReportableException.php @@ -0,0 +1,31 @@ +<?php + +namespace App\Exceptions; + +use Exception; + +/** + * Exception that should not be reported to Sentry or other error tracking services. + * Use this for known, expected errors that don't require external tracking. + */ +class NonReportableException extends Exception +{ + /** + * Create a new non-reportable exception instance. + * + * @param string $message + * @param int $code + */ + public function __construct($message = '', $code = 0, ?\Throwable $previous = null) + { + parent::__construct($message, $code, $previous); + } + + /** + * Create from another exception, preserving its message and stack trace. + */ + public static function fromException(\Throwable $exception): static + { + return new static($exception->getMessage(), $exception->getCode(), $exception); + } +} diff --git a/app/Helpers/SshMultiplexingHelper.php b/app/Helpers/SshMultiplexingHelper.php index 8caa2880a..f847f33cc 100644 --- a/app/Helpers/SshMultiplexingHelper.php +++ b/app/Helpers/SshMultiplexingHelper.php @@ -4,7 +4,9 @@ use App\Models\PrivateKey; use App\Models\Server; +use Illuminate\Support\Facades\Cache; use Illuminate\Support\Facades\Hash; +use Illuminate\Support\Facades\Log; use Illuminate\Support\Facades\Process; class SshMultiplexingHelper @@ -30,6 +32,7 @@ public static function ensureMultiplexedConnection(Server $server): bool $sshConfig = self::serverSshConfiguration($server); $muxSocket = $sshConfig['muxFilename']; + // Check if connection exists $checkCommand = "ssh -O check -o ControlPath=$muxSocket "; if (data_get($server, 'settings.is_cloudflare_tunnel')) { $checkCommand .= '-o ProxyCommand="cloudflared access ssh --hostname %h" '; @@ -41,6 +44,24 @@ public static function ensureMultiplexedConnection(Server $server): bool return self::establishNewMultiplexedConnection($server); } + // Connection exists, ensure we have metadata for age tracking + if (self::getConnectionAge($server) === null) { + // Existing connection but no metadata, store current time as fallback + self::storeConnectionMetadata($server); + } + + // Connection exists, check if it needs refresh due to age + if (self::isConnectionExpired($server)) { + return self::refreshMultiplexedConnection($server); + } + + // Perform health check if enabled + if (config('constants.ssh.mux_health_check_enabled')) { + if (! self::isConnectionHealthy($server)) { + return self::refreshMultiplexedConnection($server); + } + } + return true; } @@ -65,6 +86,9 @@ public static function establishNewMultiplexedConnection(Server $server): bool return false; } + // Store connection metadata for tracking + self::storeConnectionMetadata($server); + return true; } @@ -79,6 +103,9 @@ public static function removeMuxFile(Server $server) } $closeCommand .= "{$server->user}@{$server->ip}"; Process::run($closeCommand); + + // Clear connection metadata from cache + self::clearConnectionMetadata($server); } public static function generateScpCommand(Server $server, string $source, string $dest) @@ -94,8 +121,18 @@ public static function generateScpCommand(Server $server, string $source, string if ($server->isIpv6()) { $scp_command .= '-6 '; } - if (self::isMultiplexingEnabled() && self::ensureMultiplexedConnection($server)) { - $scp_command .= "-o ControlMaster=auto -o ControlPath=$muxSocket -o ControlPersist={$muxPersistTime} "; + if (self::isMultiplexingEnabled()) { + try { + if (self::ensureMultiplexedConnection($server)) { + $scp_command .= "-o ControlMaster=auto -o ControlPath=$muxSocket -o ControlPersist={$muxPersistTime} "; + } + } catch (\Exception $e) { + Log::warning('SSH multiplexing failed for SCP, falling back to non-multiplexed connection', [ + 'server' => $server->name ?? $server->ip, + 'error' => $e->getMessage(), + ]); + // Continue without multiplexing + } } if (data_get($server, 'settings.is_cloudflare_tunnel')) { @@ -130,8 +167,16 @@ public static function generateSshCommand(Server $server, string $command) $ssh_command = "timeout $timeout ssh "; - if (self::isMultiplexingEnabled() && self::ensureMultiplexedConnection($server)) { - $ssh_command .= "-o ControlMaster=auto -o ControlPath=$muxSocket -o ControlPersist={$muxPersistTime} "; + $multiplexingSuccessful = false; + if (self::isMultiplexingEnabled()) { + try { + $multiplexingSuccessful = self::ensureMultiplexedConnection($server); + if ($multiplexingSuccessful) { + $ssh_command .= "-o ControlMaster=auto -o ControlPath=$muxSocket -o ControlPersist={$muxPersistTime} "; + } + } catch (\Exception $e) { + // Continue without multiplexing + } } if (data_get($server, 'settings.is_cloudflare_tunnel')) { @@ -186,4 +231,81 @@ private static function getCommonSshOptions(Server $server, string $sshKeyLocati return $options; } + + /** + * Check if the multiplexed connection is healthy by running a test command + */ + public static function isConnectionHealthy(Server $server): bool + { + $sshConfig = self::serverSshConfiguration($server); + $muxSocket = $sshConfig['muxFilename']; + $healthCheckTimeout = config('constants.ssh.mux_health_check_timeout'); + + $healthCommand = "timeout $healthCheckTimeout ssh -o ControlMaster=auto -o ControlPath=$muxSocket "; + if (data_get($server, 'settings.is_cloudflare_tunnel')) { + $healthCommand .= '-o ProxyCommand="cloudflared access ssh --hostname %h" '; + } + $healthCommand .= "{$server->user}@{$server->ip} 'echo \"health_check_ok\"'"; + + $process = Process::run($healthCommand); + $isHealthy = $process->exitCode() === 0 && str_contains($process->output(), 'health_check_ok'); + + return $isHealthy; + } + + /** + * Check if the connection has exceeded its maximum age + */ + public static function isConnectionExpired(Server $server): bool + { + $connectionAge = self::getConnectionAge($server); + $maxAge = config('constants.ssh.mux_max_age'); + + return $connectionAge !== null && $connectionAge > $maxAge; + } + + /** + * Get the age of the current connection in seconds + */ + public static function getConnectionAge(Server $server): ?int + { + $cacheKey = "ssh_mux_connection_time_{$server->uuid}"; + $connectionTime = Cache::get($cacheKey); + + if ($connectionTime === null) { + return null; + } + + return time() - $connectionTime; + } + + /** + * Refresh a multiplexed connection by closing and re-establishing it + */ + public static function refreshMultiplexedConnection(Server $server): bool + { + // Close existing connection + self::removeMuxFile($server); + + // Establish new connection + return self::establishNewMultiplexedConnection($server); + } + + /** + * Store connection metadata when a new connection is established + */ + private static function storeConnectionMetadata(Server $server): void + { + $cacheKey = "ssh_mux_connection_time_{$server->uuid}"; + Cache::put($cacheKey, time(), config('constants.ssh.mux_persist_time') + 300); // Cache slightly longer than persist time + } + + /** + * Clear connection metadata from cache + */ + private static function clearConnectionMetadata(Server $server): void + { + $cacheKey = "ssh_mux_connection_time_{$server->uuid}"; + Cache::forget($cacheKey); + } } diff --git a/app/Helpers/SshRetryHandler.php b/app/Helpers/SshRetryHandler.php new file mode 100644 index 000000000..aaaf4252a --- /dev/null +++ b/app/Helpers/SshRetryHandler.php @@ -0,0 +1,34 @@ +<?php + +namespace App\Helpers; + +use App\Traits\SshRetryable; + +/** + * Helper class to use SshRetryable trait in non-class contexts + */ +class SshRetryHandler +{ + use SshRetryable; + + /** + * Static method to get a singleton instance + */ + public static function instance(): self + { + static $instance = null; + if ($instance === null) { + $instance = new self; + } + + return $instance; + } + + /** + * Convenience static method for retry execution + */ + public static function retry(callable $callback, array $context = [], bool $throwError = true) + { + return self::instance()->executeWithSshRetry($callback, $context, $throwError); + } +} diff --git a/app/Http/Controllers/Api/ApplicationsController.php b/app/Http/Controllers/Api/ApplicationsController.php index 16413d2ad..ce9e723d4 100644 --- a/app/Http/Controllers/Api/ApplicationsController.php +++ b/app/Http/Controllers/Api/ApplicationsController.php @@ -2284,6 +2284,9 @@ public function update_by_uuid(Request $request) data_set($data, 'docker_compose_domains', json_encode($dockerComposeDomainsJson)); } $application->fill($data); + if ($application->settings->is_container_label_readonly_enabled && $requestHasDomains && $server->isProxyShouldRun()) { + $application->custom_labels = str(implode('|coolify|', generateLabelsApplication($application)))->replace('|coolify|', "\n"); + } $application->save(); if ($instantDeploy) { @@ -2426,7 +2429,6 @@ public function envs(Request $request) 'key' => ['type' => 'string', 'description' => 'The key of the environment variable.'], 'value' => ['type' => 'string', 'description' => 'The value of the environment variable.'], 'is_preview' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in preview deployments.'], - 'is_build_time' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in build time.'], 'is_literal' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is a literal, nothing espaced.'], 'is_multiline' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is multiline.'], 'is_shown_once' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable\'s value is shown on the UI.'], @@ -2467,7 +2469,7 @@ public function envs(Request $request) )] public function update_env_by_uuid(Request $request) { - $allowedFields = ['key', 'value', 'is_preview', 'is_build_time', 'is_literal']; + $allowedFields = ['key', 'value', 'is_preview', 'is_literal']; $teamId = getTeamIdFromToken(); if (is_null($teamId)) { @@ -2492,7 +2494,6 @@ public function update_env_by_uuid(Request $request) 'key' => 'string|required', 'value' => 'string|nullable', 'is_preview' => 'boolean', - 'is_build_time' => 'boolean', 'is_literal' => 'boolean', 'is_multiline' => 'boolean', 'is_shown_once' => 'boolean', @@ -2513,16 +2514,12 @@ public function update_env_by_uuid(Request $request) ], 422); } $is_preview = $request->is_preview ?? false; - $is_build_time = $request->is_build_time ?? false; $is_literal = $request->is_literal ?? false; $key = str($request->key)->trim()->replace(' ', '_')->value; if ($is_preview) { $env = $application->environment_variables_preview->where('key', $key)->first(); if ($env) { $env->value = $request->value; - if ($env->is_build_time != $is_build_time) { - $env->is_build_time = $is_build_time; - } if ($env->is_literal != $is_literal) { $env->is_literal = $is_literal; } @@ -2535,6 +2532,12 @@ public function update_env_by_uuid(Request $request) if ($env->is_shown_once != $request->is_shown_once) { $env->is_shown_once = $request->is_shown_once; } + if ($request->has('is_runtime') && $env->is_runtime != $request->is_runtime) { + $env->is_runtime = $request->is_runtime; + } + if ($request->has('is_buildtime') && $env->is_buildtime != $request->is_buildtime) { + $env->is_buildtime = $request->is_buildtime; + } $env->save(); return response()->json($this->removeSensitiveData($env))->setStatusCode(201); @@ -2547,9 +2550,6 @@ public function update_env_by_uuid(Request $request) $env = $application->environment_variables->where('key', $key)->first(); if ($env) { $env->value = $request->value; - if ($env->is_build_time != $is_build_time) { - $env->is_build_time = $is_build_time; - } if ($env->is_literal != $is_literal) { $env->is_literal = $is_literal; } @@ -2562,6 +2562,12 @@ public function update_env_by_uuid(Request $request) if ($env->is_shown_once != $request->is_shown_once) { $env->is_shown_once = $request->is_shown_once; } + if ($request->has('is_runtime') && $env->is_runtime != $request->is_runtime) { + $env->is_runtime = $request->is_runtime; + } + if ($request->has('is_buildtime') && $env->is_buildtime != $request->is_buildtime) { + $env->is_buildtime = $request->is_buildtime; + } $env->save(); return response()->json($this->removeSensitiveData($env))->setStatusCode(201); @@ -2616,7 +2622,6 @@ public function update_env_by_uuid(Request $request) 'key' => ['type' => 'string', 'description' => 'The key of the environment variable.'], 'value' => ['type' => 'string', 'description' => 'The value of the environment variable.'], 'is_preview' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in preview deployments.'], - 'is_build_time' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in build time.'], 'is_literal' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is a literal, nothing espaced.'], 'is_multiline' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is multiline.'], 'is_shown_once' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable\'s value is shown on the UI.'], @@ -2687,7 +2692,7 @@ public function create_bulk_envs(Request $request) ], 400); } $bulk_data = collect($bulk_data)->map(function ($item) { - return collect($item)->only(['key', 'value', 'is_preview', 'is_build_time', 'is_literal']); + return collect($item)->only(['key', 'value', 'is_preview', 'is_literal']); }); $returnedEnvs = collect(); foreach ($bulk_data as $item) { @@ -2695,7 +2700,6 @@ public function create_bulk_envs(Request $request) 'key' => 'string|required', 'value' => 'string|nullable', 'is_preview' => 'boolean', - 'is_build_time' => 'boolean', 'is_literal' => 'boolean', 'is_multiline' => 'boolean', 'is_shown_once' => 'boolean', @@ -2707,7 +2711,6 @@ public function create_bulk_envs(Request $request) ], 422); } $is_preview = $item->get('is_preview') ?? false; - $is_build_time = $item->get('is_build_time') ?? false; $is_literal = $item->get('is_literal') ?? false; $is_multi_line = $item->get('is_multiline') ?? false; $is_shown_once = $item->get('is_shown_once') ?? false; @@ -2716,9 +2719,7 @@ public function create_bulk_envs(Request $request) $env = $application->environment_variables_preview->where('key', $key)->first(); if ($env) { $env->value = $item->get('value'); - if ($env->is_build_time != $is_build_time) { - $env->is_build_time = $is_build_time; - } + if ($env->is_literal != $is_literal) { $env->is_literal = $is_literal; } @@ -2728,16 +2729,23 @@ public function create_bulk_envs(Request $request) if ($env->is_shown_once != $item->get('is_shown_once')) { $env->is_shown_once = $item->get('is_shown_once'); } + if ($item->has('is_runtime') && $env->is_runtime != $item->get('is_runtime')) { + $env->is_runtime = $item->get('is_runtime'); + } + if ($item->has('is_buildtime') && $env->is_buildtime != $item->get('is_buildtime')) { + $env->is_buildtime = $item->get('is_buildtime'); + } $env->save(); } else { $env = $application->environment_variables()->create([ 'key' => $item->get('key'), 'value' => $item->get('value'), 'is_preview' => $is_preview, - 'is_build_time' => $is_build_time, 'is_literal' => $is_literal, 'is_multiline' => $is_multi_line, 'is_shown_once' => $is_shown_once, + 'is_runtime' => $item->get('is_runtime', true), + 'is_buildtime' => $item->get('is_buildtime', true), 'resourceable_type' => get_class($application), 'resourceable_id' => $application->id, ]); @@ -2746,9 +2754,6 @@ public function create_bulk_envs(Request $request) $env = $application->environment_variables->where('key', $key)->first(); if ($env) { $env->value = $item->get('value'); - if ($env->is_build_time != $is_build_time) { - $env->is_build_time = $is_build_time; - } if ($env->is_literal != $is_literal) { $env->is_literal = $is_literal; } @@ -2758,16 +2763,23 @@ public function create_bulk_envs(Request $request) if ($env->is_shown_once != $item->get('is_shown_once')) { $env->is_shown_once = $item->get('is_shown_once'); } + if ($item->has('is_runtime') && $env->is_runtime != $item->get('is_runtime')) { + $env->is_runtime = $item->get('is_runtime'); + } + if ($item->has('is_buildtime') && $env->is_buildtime != $item->get('is_buildtime')) { + $env->is_buildtime = $item->get('is_buildtime'); + } $env->save(); } else { $env = $application->environment_variables()->create([ 'key' => $item->get('key'), 'value' => $item->get('value'), 'is_preview' => $is_preview, - 'is_build_time' => $is_build_time, 'is_literal' => $is_literal, 'is_multiline' => $is_multi_line, 'is_shown_once' => $is_shown_once, + 'is_runtime' => $item->get('is_runtime', true), + 'is_buildtime' => $item->get('is_buildtime', true), 'resourceable_type' => get_class($application), 'resourceable_id' => $application->id, ]); @@ -2811,7 +2823,6 @@ public function create_bulk_envs(Request $request) 'key' => ['type' => 'string', 'description' => 'The key of the environment variable.'], 'value' => ['type' => 'string', 'description' => 'The value of the environment variable.'], 'is_preview' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in preview deployments.'], - 'is_build_time' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in build time.'], 'is_literal' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is a literal, nothing espaced.'], 'is_multiline' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is multiline.'], 'is_shown_once' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable\'s value is shown on the UI.'], @@ -2851,7 +2862,7 @@ public function create_bulk_envs(Request $request) )] public function create_env(Request $request) { - $allowedFields = ['key', 'value', 'is_preview', 'is_build_time', 'is_literal']; + $allowedFields = ['key', 'value', 'is_preview', 'is_literal']; $teamId = getTeamIdFromToken(); if (is_null($teamId)) { @@ -2871,7 +2882,6 @@ public function create_env(Request $request) 'key' => 'string|required', 'value' => 'string|nullable', 'is_preview' => 'boolean', - 'is_build_time' => 'boolean', 'is_literal' => 'boolean', 'is_multiline' => 'boolean', 'is_shown_once' => 'boolean', @@ -2905,10 +2915,11 @@ public function create_env(Request $request) 'key' => $request->key, 'value' => $request->value, 'is_preview' => $request->is_preview ?? false, - 'is_build_time' => $request->is_build_time ?? false, 'is_literal' => $request->is_literal ?? false, 'is_multiline' => $request->is_multiline ?? false, 'is_shown_once' => $request->is_shown_once ?? false, + 'is_runtime' => $request->is_runtime ?? true, + 'is_buildtime' => $request->is_buildtime ?? true, 'resourceable_type' => get_class($application), 'resourceable_id' => $application->id, ]); @@ -2928,10 +2939,11 @@ public function create_env(Request $request) 'key' => $request->key, 'value' => $request->value, 'is_preview' => $request->is_preview ?? false, - 'is_build_time' => $request->is_build_time ?? false, 'is_literal' => $request->is_literal ?? false, 'is_multiline' => $request->is_multiline ?? false, 'is_shown_once' => $request->is_shown_once ?? false, + 'is_runtime' => $request->is_runtime ?? true, + 'is_buildtime' => $request->is_buildtime ?? true, 'resourceable_type' => get_class($application), 'resourceable_id' => $application->id, ]); @@ -3368,11 +3380,12 @@ private function validateDataApplications(Request $request, Server $server) $fqdn = str($fqdn)->replaceStart(',', '')->trim(); $errors = []; $fqdn = str($fqdn)->trim()->explode(',')->map(function ($domain) use (&$errors) { + $domain = trim($domain); if (filter_var($domain, FILTER_VALIDATE_URL) === false) { $errors[] = 'Invalid domain: '.$domain; } - return str($domain)->trim()->lower(); + return str($domain)->lower(); }); if (count($errors) > 0) { return response()->json([ diff --git a/app/Http/Controllers/Api/DatabasesController.php b/app/Http/Controllers/Api/DatabasesController.php index 389d119bd..5871f481a 100644 --- a/app/Http/Controllers/Api/DatabasesController.php +++ b/app/Http/Controllers/Api/DatabasesController.php @@ -9,11 +9,15 @@ use App\Actions\Database\StopDatabaseProxy; use App\Enums\NewDatabaseTypes; use App\Http\Controllers\Controller; +use App\Jobs\DatabaseBackupJob; use App\Jobs\DeleteResourceJob; use App\Models\Project; +use App\Models\S3Storage; +use App\Models\ScheduledDatabaseBackup; use App\Models\Server; use App\Models\StandalonePostgresql; use Illuminate\Http\Request; +use Illuminate\Support\Facades\DB; use OpenApi\Attributes as OA; class DatabasesController extends Controller @@ -79,13 +83,88 @@ public function databases(Request $request) foreach ($projects as $project) { $databases = $databases->merge($project->databases()); } - $databases = $databases->map(function ($database) { + + $databaseIds = $databases->pluck('id')->toArray(); + + $backupConfigs = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->with('latest_log') + ->whereIn('database_id', $databaseIds) + ->get() + ->groupBy('database_id'); + + $databases = $databases->map(function ($database) use ($backupConfigs) { + $database->backup_configs = $backupConfigs->get($database->id, collect())->values(); + return $this->removeSensitiveData($database); }); return response()->json($databases); } + #[OA\Get( + summary: 'Get', + description: 'Get backups details by database UUID.', + path: '/databases/{uuid}/backups', + operationId: 'get-database-backups-by-uuid', + security: [ + ['bearerAuth' => []], + ], + tags: ['Databases'], + parameters: [ + new OA\Parameter( + name: 'uuid', + in: 'path', + description: 'UUID of the database.', + required: true, + schema: new OA\Schema( + type: 'string', + format: 'uuid', + ) + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'Get all backups for a database', + content: new OA\JsonContent( + type: 'string', + example: 'Content is very complex. Will be implemented later.', + ), + ), + new OA\Response( + response: 401, + ref: '#/components/responses/401', + ), + new OA\Response( + response: 400, + ref: '#/components/responses/400', + ), + new OA\Response( + response: 404, + ref: '#/components/responses/404', + ), + ] + )] + public function database_backup_details_uuid(Request $request) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + if (! $request->uuid) { + return response()->json(['message' => 'UUID is required.'], 404); + } + $database = queryDatabaseByUuidWithinTeam($request->uuid, $teamId); + if (! $database) { + return response()->json(['message' => 'Database not found.'], 404); + } + + $this->authorize('view', $database); + + $backupConfig = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->with('executions')->where('database_id', $database->id)->get(); + + return response()->json($backupConfig); + } + #[OA\Get( summary: 'Get', description: 'Get database by UUID.', @@ -248,6 +327,7 @@ public function update_by_uuid(Request $request) return invalidTokenResponse(); } + // this check if the request is a valid json $return = validateIncomingRequest($request); if ($return instanceof \Illuminate\Http\JsonResponse) { return $return; @@ -499,7 +579,8 @@ public function update_by_uuid(Request $request) $whatToDoWithDatabaseProxy = 'start'; } - $database->update($request->all()); + // Only update database fields, not backup configuration + $database->update($request->only($allowedFields)); if ($whatToDoWithDatabaseProxy === 'start') { StartDatabaseProxy::dispatch($database); @@ -512,6 +593,197 @@ public function update_by_uuid(Request $request) ]); } + #[OA\Patch( + summary: 'Update', + description: 'Update a specific backup configuration for a given database, identified by its UUID and the backup ID', + path: '/databases/{uuid}/backups/{scheduled_backup_uuid}', + operationId: 'update-database-backup', + security: [ + ['bearerAuth' => []], + ], + tags: ['Databases'], + parameters: [ + new OA\Parameter( + name: 'uuid', + in: 'path', + description: 'UUID of the database.', + required: true, + schema: new OA\Schema( + type: 'string', + format: 'uuid', + ) + ), + new OA\Parameter( + name: 'scheduled_backup_uuid', + in: 'path', + description: 'UUID of the backup configuration.', + required: true, + schema: new OA\Schema( + type: 'string', + format: 'uuid', + ) + ), + ], + requestBody: new OA\RequestBody( + description: 'Database backup configuration data', + required: true, + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'save_s3' => ['type' => 'boolean', 'description' => 'Whether data is saved in s3 or not'], + 's3_storage_uuid' => ['type' => 'string', 'description' => 'S3 storage UUID'], + 'backup_now' => ['type' => 'boolean', 'description' => 'Whether to take a backup now or not'], + 'enabled' => ['type' => 'boolean', 'description' => 'Whether the backup is enabled or not'], + 'databases_to_backup' => ['type' => 'string', 'description' => 'Comma separated list of databases to backup'], + 'dump_all' => ['type' => 'boolean', 'description' => 'Whether all databases are dumped or not'], + 'frequency' => ['type' => 'string', 'description' => 'Frequency of the backup'], + 'database_backup_retention_amount_locally' => ['type' => 'integer', 'description' => 'Retention amount of the backup locally'], + 'database_backup_retention_days_locally' => ['type' => 'integer', 'description' => 'Retention days of the backup locally'], + 'database_backup_retention_max_storage_locally' => ['type' => 'integer', 'description' => 'Max storage of the backup locally'], + 'database_backup_retention_amount_s3' => ['type' => 'integer', 'description' => 'Retention amount of the backup in s3'], + 'database_backup_retention_days_s3' => ['type' => 'integer', 'description' => 'Retention days of the backup in s3'], + 'database_backup_retention_max_storage_s3' => ['type' => 'integer', 'description' => 'Max storage of the backup in S3'], + ], + ), + ) + ), + responses: [ + new OA\Response( + response: 200, + description: 'Database backup configuration updated', + ), + new OA\Response( + response: 401, + ref: '#/components/responses/401', + ), + new OA\Response( + response: 400, + ref: '#/components/responses/400', + ), + new OA\Response( + response: 404, + ref: '#/components/responses/404', + ), + ] + )] + public function update_backup(Request $request) + { + $backupConfigFields = ['save_s3', 'enabled', 'dump_all', 'frequency', 'databases_to_backup', 'database_backup_retention_amount_locally', 'database_backup_retention_days_locally', 'database_backup_retention_max_storage_locally', 'database_backup_retention_amount_s3', 'database_backup_retention_days_s3', 'database_backup_retention_max_storage_s3', 's3_storage_uuid']; + + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + // this check if the request is a valid json + $return = validateIncomingRequest($request); + if ($return instanceof \Illuminate\Http\JsonResponse) { + return $return; + } + $validator = customApiValidator($request->all(), [ + 'save_s3' => 'boolean', + 'backup_now' => 'boolean|nullable', + 'enabled' => 'boolean', + 'dump_all' => 'boolean', + 's3_storage_uuid' => 'string|exists:s3_storages,uuid|nullable', + 'databases_to_backup' => 'string|nullable', + 'frequency' => 'string|in:every_minute,hourly,daily,weekly,monthly,yearly', + 'database_backup_retention_amount_locally' => 'integer|min:0', + 'database_backup_retention_days_locally' => 'integer|min:0', + 'database_backup_retention_max_storage_locally' => 'integer|min:0', + 'database_backup_retention_amount_s3' => 'integer|min:0', + 'database_backup_retention_days_s3' => 'integer|min:0', + 'database_backup_retention_max_storage_s3' => 'integer|min:0', + ]); + if ($validator->fails()) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => $validator->errors(), + ], 422); + } + + if (! $request->uuid) { + return response()->json(['message' => 'UUID is required.'], 404); + } + + // Validate scheduled_backup_uuid is provided + if (! $request->scheduled_backup_uuid) { + return response()->json(['message' => 'Scheduled backup UUID is required.'], 400); + } + + $uuid = $request->uuid; + removeUnnecessaryFieldsFromRequest($request); + $database = queryDatabaseByUuidWithinTeam($uuid, $teamId); + if (! $database) { + return response()->json(['message' => 'Database not found.'], 404); + } + + $this->authorize('update', $database); + + if ($request->boolean('save_s3') && ! $request->filled('s3_storage_uuid')) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => ['s3_storage_uuid' => ['The s3_storage_uuid field is required when save_s3 is true.']], + ], 422); + } + if ($request->filled('s3_storage_uuid')) { + $existsInTeam = S3Storage::ownedByCurrentTeam()->where('uuid', $request->s3_storage_uuid)->exists(); + if (! $existsInTeam) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => ['s3_storage_uuid' => ['The selected S3 storage is invalid for this team.']], + ], 422); + } + } + + $backupConfig = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + ->where('uuid', $request->scheduled_backup_uuid) + ->first(); + if (! $backupConfig) { + return response()->json(['message' => 'Backup config not found.'], 404); + } + + $extraFields = array_diff(array_keys($request->all()), $backupConfigFields, ['backup_now']); + if (! empty($extraFields)) { + $errors = $validator->errors(); + foreach ($extraFields as $field) { + $errors->add($field, 'This field is not allowed.'); + } + + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => $errors, + ], 422); + } + + $backupData = $request->only($backupConfigFields); + + // Convert s3_storage_uuid to s3_storage_id + if (isset($backupData['s3_storage_uuid'])) { + $s3Storage = S3Storage::ownedByCurrentTeam()->where('uuid', $backupData['s3_storage_uuid'])->first(); + if ($s3Storage) { + $backupData['s3_storage_id'] = $s3Storage->id; + } elseif ($request->boolean('save_s3')) { + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => ['s3_storage_uuid' => ['The selected S3 storage is invalid for this team.']], + ], 422); + } + unset($backupData['s3_storage_uuid']); + } + + $backupConfig->update($backupData); + + if ($request->backup_now) { + dispatch(new DatabaseBackupJob($backupConfig)); + } + + return response()->json([ + 'message' => 'Database backup configuration updated', + ]); + } + #[OA\Post( summary: 'Create (PostgreSQL)', description: 'Create a new PostgreSQL database.', @@ -1630,6 +1902,344 @@ public function delete_by_uuid(Request $request) ]); } + #[OA\Delete( + summary: 'Delete backup configuration', + description: 'Deletes a backup configuration and all its executions.', + path: '/databases/{uuid}/backups/{scheduled_backup_uuid}', + operationId: 'delete-backup-configuration-by-uuid', + security: [ + ['bearerAuth' => []], + ], + tags: ['Databases'], + parameters: [ + new OA\Parameter( + name: 'uuid', + in: 'path', + required: true, + description: 'UUID of the database', + schema: new OA\Schema(type: 'string') + ), + new OA\Parameter( + name: 'scheduled_backup_uuid', + in: 'path', + required: true, + description: 'UUID of the backup configuration to delete', + schema: new OA\Schema(type: 'string', format: 'uuid') + ), + new OA\Parameter( + name: 'delete_s3', + in: 'query', + required: false, + description: 'Whether to delete all backup files from S3', + schema: new OA\Schema(type: 'boolean', default: false) + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'Backup configuration deleted.', + content: new OA\JsonContent( + type: 'object', + properties: [ + 'message' => new OA\Schema(type: 'string', example: 'Backup configuration and all executions deleted.'), + ] + ) + ), + new OA\Response( + response: 404, + description: 'Backup configuration not found.', + content: new OA\JsonContent( + type: 'object', + properties: [ + 'message' => new OA\Schema(type: 'string', example: 'Backup configuration not found.'), + ] + ) + ), + ] + )] + public function delete_backup_by_uuid(Request $request) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + // Validate scheduled_backup_uuid is provided + if (! $request->scheduled_backup_uuid) { + return response()->json(['message' => 'Scheduled backup UUID is required.'], 400); + } + + $database = queryDatabaseByUuidWithinTeam($request->uuid, $teamId); + if (! $database) { + return response()->json(['message' => 'Database not found.'], 404); + } + + $this->authorize('update', $database); + + // Find the backup configuration by its UUID + $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + ->where('uuid', $request->scheduled_backup_uuid) + ->first(); + + if (! $backup) { + return response()->json(['message' => 'Backup configuration not found.'], 404); + } + + $deleteS3 = filter_var($request->query->get('delete_s3', false), FILTER_VALIDATE_BOOLEAN); + + try { + DB::beginTransaction(); + // Get all executions for this backup configuration + $executions = $backup->executions()->get(); + + // Delete all execution files (locally and optionally from S3) + foreach ($executions as $execution) { + if ($execution->filename) { + deleteBackupsLocally($execution->filename, $database->destination->server); + + if ($deleteS3 && $backup->s3) { + deleteBackupsS3($execution->filename, $backup->s3); + } + } + + $execution->delete(); + } + + // Delete the backup configuration itself + $backup->delete(); + DB::commit(); + + return response()->json([ + 'message' => 'Backup configuration and all executions deleted.', + ]); + } catch (\Exception $e) { + DB::rollBack(); + + return response()->json(['message' => 'Failed to delete backup: '.$e->getMessage()], 500); + } + } + + #[OA\Delete( + summary: 'Delete backup execution', + description: 'Deletes a specific backup execution.', + path: '/databases/{uuid}/backups/{scheduled_backup_uuid}/executions/{execution_uuid}', + operationId: 'delete-backup-execution-by-uuid', + security: [ + ['bearerAuth' => []], + ], + tags: ['Databases'], + parameters: [ + new OA\Parameter( + name: 'uuid', + in: 'path', + required: true, + description: 'UUID of the database', + schema: new OA\Schema(type: 'string') + ), + new OA\Parameter( + name: 'scheduled_backup_uuid', + in: 'path', + required: true, + description: 'UUID of the backup configuration', + schema: new OA\Schema(type: 'string', format: 'uuid') + ), + new OA\Parameter( + name: 'execution_uuid', + in: 'path', + required: true, + description: 'UUID of the backup execution to delete', + schema: new OA\Schema(type: 'string', format: 'uuid') + ), + new OA\Parameter( + name: 'delete_s3', + in: 'query', + required: false, + description: 'Whether to delete the backup from S3', + schema: new OA\Schema(type: 'boolean', default: false) + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'Backup execution deleted.', + content: new OA\JsonContent( + type: 'object', + properties: [ + 'message' => new OA\Schema(type: 'string', example: 'Backup execution deleted.'), + ] + ) + ), + new OA\Response( + response: 404, + description: 'Backup execution not found.', + content: new OA\JsonContent( + type: 'object', + properties: [ + 'message' => new OA\Schema(type: 'string', example: 'Backup execution not found.'), + ] + ) + ), + ] + )] + public function delete_execution_by_uuid(Request $request) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + // Validate parameters + if (! $request->scheduled_backup_uuid) { + return response()->json(['message' => 'Scheduled backup UUID is required.'], 400); + } + if (! $request->execution_uuid) { + return response()->json(['message' => 'Execution UUID is required.'], 400); + } + + $database = queryDatabaseByUuidWithinTeam($request->uuid, $teamId); + if (! $database) { + return response()->json(['message' => 'Database not found.'], 404); + } + + $this->authorize('update', $database); + + // Find the backup configuration by its UUID + $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + ->where('uuid', $request->scheduled_backup_uuid) + ->first(); + + if (! $backup) { + return response()->json(['message' => 'Backup configuration not found.'], 404); + } + + // Find the specific execution + $execution = $backup->executions()->where('uuid', $request->execution_uuid)->first(); + if (! $execution) { + return response()->json(['message' => 'Backup execution not found.'], 404); + } + + $deleteS3 = filter_var($request->query->get('delete_s3', false), FILTER_VALIDATE_BOOLEAN); + + try { + if ($execution->filename) { + deleteBackupsLocally($execution->filename, $database->destination->server); + + if ($deleteS3 && $backup->s3) { + deleteBackupsS3($execution->filename, $backup->s3); + } + } + + $execution->delete(); + + return response()->json([ + 'message' => 'Backup execution deleted.', + ]); + } catch (\Exception $e) { + return response()->json(['message' => 'Failed to delete backup execution: '.$e->getMessage()], 500); + } + } + + #[OA\Get( + summary: 'List backup executions', + description: 'Get all executions for a specific backup configuration.', + path: '/databases/{uuid}/backups/{scheduled_backup_uuid}/executions', + operationId: 'list-backup-executions', + security: [ + ['bearerAuth' => []], + ], + tags: ['Databases'], + parameters: [ + new OA\Parameter( + name: 'uuid', + in: 'path', + required: true, + description: 'UUID of the database', + schema: new OA\Schema(type: 'string') + ), + new OA\Parameter( + name: 'scheduled_backup_uuid', + in: 'path', + required: true, + description: 'UUID of the backup configuration', + schema: new OA\Schema(type: 'string', format: 'uuid') + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'List of backup executions', + content: new OA\JsonContent( + type: 'object', + properties: [ + 'executions' => new OA\Schema( + type: 'array', + items: new OA\Items( + type: 'object', + properties: [ + 'uuid' => ['type' => 'string'], + 'filename' => ['type' => 'string'], + 'size' => ['type' => 'integer'], + 'created_at' => ['type' => 'string'], + 'message' => ['type' => 'string'], + 'status' => ['type' => 'string'], + ] + ) + ), + ] + ) + ), + new OA\Response( + response: 404, + description: 'Backup configuration not found.', + ), + ] + )] + public function list_backup_executions(Request $request) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + // Validate scheduled_backup_uuid is provided + if (! $request->scheduled_backup_uuid) { + return response()->json(['message' => 'Scheduled backup UUID is required.'], 400); + } + + $database = queryDatabaseByUuidWithinTeam($request->uuid, $teamId); + if (! $database) { + return response()->json(['message' => 'Database not found.'], 404); + } + + // Find the backup configuration by its UUID + $backup = ScheduledDatabaseBackup::ownedByCurrentTeamAPI($teamId)->where('database_id', $database->id) + ->where('uuid', $request->scheduled_backup_uuid) + ->first(); + + if (! $backup) { + return response()->json(['message' => 'Backup configuration not found.'], 404); + } + + // Get all executions for this backup configuration + $executions = $backup->executions() + ->orderBy('created_at', 'desc') + ->get() + ->map(function ($execution) { + return [ + 'uuid' => $execution->uuid, + 'filename' => $execution->filename, + 'size' => $execution->size, + 'created_at' => $execution->created_at->toIso8601String(), + 'message' => $execution->message, + 'status' => $execution->status, + ]; + }); + + return response()->json([ + 'executions' => $executions, + ]); + } + #[OA\Get( summary: 'Start', description: 'Start database. `Post` request is also accepted.', diff --git a/app/Http/Controllers/Api/DeployController.php b/app/Http/Controllers/Api/DeployController.php index b87420f72..c4d603392 100644 --- a/app/Http/Controllers/Api/DeployController.php +++ b/app/Http/Controllers/Api/DeployController.php @@ -225,6 +225,14 @@ private function by_uuids(string $uuid, int $teamId, bool $force = false, int $p foreach ($uuids as $uuid) { $resource = getResourceByUuid($uuid, $teamId); if ($resource) { + if ($pr !== 0) { + $preview = $resource->previews()->where('pull_request_id', $pr)->first(); + if (! $preview) { + $deployments->push(['message' => "Pull request {$pr} not found for this resource.", 'resource_uuid' => $uuid]); + + continue; + } + } ['message' => $return_message, 'deployment_uuid' => $deployment_uuid] = $this->deploy_resource($resource, $force, $pr); if ($deployment_uuid) { $deployments->push(['message' => $return_message, 'resource_uuid' => $uuid, 'deployment_uuid' => $deployment_uuid->toString()]); diff --git a/app/Http/Controllers/Api/GithubController.php b/app/Http/Controllers/Api/GithubController.php new file mode 100644 index 000000000..8c8c87238 --- /dev/null +++ b/app/Http/Controllers/Api/GithubController.php @@ -0,0 +1,661 @@ +<?php + +namespace App\Http\Controllers\Api; + +use App\Http\Controllers\Controller; +use App\Models\GithubApp; +use App\Models\PrivateKey; +use Illuminate\Http\Request; +use Illuminate\Support\Facades\Http; +use Illuminate\Support\Str; +use OpenApi\Attributes as OA; + +class GithubController extends Controller +{ + #[OA\Post( + summary: 'Create GitHub App', + description: 'Create a new GitHub app.', + path: '/github-apps', + operationId: 'create-github-app', + security: [ + ['bearerAuth' => []], + ], + tags: ['GitHub Apps'], + requestBody: new OA\RequestBody( + description: 'GitHub app creation payload.', + required: true, + content: [ + new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'name' => ['type' => 'string', 'description' => 'Name of the GitHub app.'], + 'organization' => ['type' => 'string', 'nullable' => true, 'description' => 'Organization to associate the app with.'], + 'api_url' => ['type' => 'string', 'description' => 'API URL for the GitHub app (e.g., https://api.github.com).'], + 'html_url' => ['type' => 'string', 'description' => 'HTML URL for the GitHub app (e.g., https://github.com).'], + 'custom_user' => ['type' => 'string', 'description' => 'Custom user for SSH access (default: git).'], + 'custom_port' => ['type' => 'integer', 'description' => 'Custom port for SSH access (default: 22).'], + 'app_id' => ['type' => 'integer', 'description' => 'GitHub App ID from GitHub.'], + 'installation_id' => ['type' => 'integer', 'description' => 'GitHub Installation ID.'], + 'client_id' => ['type' => 'string', 'description' => 'GitHub OAuth App Client ID.'], + 'client_secret' => ['type' => 'string', 'description' => 'GitHub OAuth App Client Secret.'], + 'webhook_secret' => ['type' => 'string', 'description' => 'Webhook secret for GitHub webhooks.'], + 'private_key_uuid' => ['type' => 'string', 'description' => 'UUID of an existing private key for GitHub App authentication.'], + 'is_system_wide' => ['type' => 'boolean', 'description' => 'Is this app system-wide (cloud only).'], + ], + required: ['name', 'api_url', 'html_url', 'app_id', 'installation_id', 'client_id', 'client_secret', 'private_key_uuid'], + ), + ), + ], + ), + responses: [ + new OA\Response( + response: 201, + description: 'GitHub app created successfully.', + content: [ + new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'id' => ['type' => 'integer'], + 'uuid' => ['type' => 'string'], + 'name' => ['type' => 'string'], + 'organization' => ['type' => 'string', 'nullable' => true], + 'api_url' => ['type' => 'string'], + 'html_url' => ['type' => 'string'], + 'custom_user' => ['type' => 'string'], + 'custom_port' => ['type' => 'integer'], + 'app_id' => ['type' => 'integer'], + 'installation_id' => ['type' => 'integer'], + 'client_id' => ['type' => 'string'], + 'private_key_id' => ['type' => 'integer'], + 'is_system_wide' => ['type' => 'boolean'], + 'team_id' => ['type' => 'integer'], + ] + ) + ), + ] + ), + new OA\Response( + response: 400, + ref: '#/components/responses/400', + ), + new OA\Response( + response: 401, + ref: '#/components/responses/401', + ), + new OA\Response( + response: 422, + ref: '#/components/responses/422', + ), + ] + )] + public function create_github_app(Request $request) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + $return = validateIncomingRequest($request); + if ($return instanceof \Illuminate\Http\JsonResponse) { + return $return; + } + + $allowedFields = [ + 'name', + 'organization', + 'api_url', + 'html_url', + 'custom_user', + 'custom_port', + 'app_id', + 'installation_id', + 'client_id', + 'client_secret', + 'webhook_secret', + 'private_key_uuid', + 'is_system_wide', + ]; + + $validator = customApiValidator($request->all(), [ + 'name' => 'required|string|max:255', + 'organization' => 'nullable|string|max:255', + 'api_url' => 'required|string|url', + 'html_url' => 'required|string|url', + 'custom_user' => 'nullable|string|max:255', + 'custom_port' => 'nullable|integer|min:1|max:65535', + 'app_id' => 'required|integer', + 'installation_id' => 'required|integer', + 'client_id' => 'required|string|max:255', + 'client_secret' => 'required|string', + 'webhook_secret' => 'required|string', + 'private_key_uuid' => 'required|string', + 'is_system_wide' => 'boolean', + ]); + + $extraFields = array_diff(array_keys($request->all()), $allowedFields); + if ($validator->fails() || ! empty($extraFields)) { + $errors = $validator->errors(); + if (! empty($extraFields)) { + foreach ($extraFields as $field) { + $errors->add($field, 'This field is not allowed.'); + } + } + + return response()->json([ + 'message' => 'Validation failed.', + 'errors' => $errors, + ], 422); + } + + try { + // Verify the private key belongs to the team + $privateKey = PrivateKey::where('uuid', $request->input('private_key_uuid')) + ->where('team_id', $teamId) + ->first(); + + if (! $privateKey) { + return response()->json([ + 'message' => 'Private key not found or does not belong to your team.', + ], 404); + } + + $payload = [ + 'uuid' => Str::uuid(), + 'name' => $request->input('name'), + 'organization' => $request->input('organization'), + 'api_url' => $request->input('api_url'), + 'html_url' => $request->input('html_url'), + 'custom_user' => $request->input('custom_user', 'git'), + 'custom_port' => $request->input('custom_port', 22), + 'app_id' => $request->input('app_id'), + 'installation_id' => $request->input('installation_id'), + 'client_id' => $request->input('client_id'), + 'client_secret' => $request->input('client_secret'), + 'webhook_secret' => $request->input('webhook_secret'), + 'private_key_id' => $privateKey->id, + 'is_public' => false, + 'team_id' => $teamId, + ]; + + if (! isCloud()) { + $payload['is_system_wide'] = $request->input('is_system_wide', false); + } + + $githubApp = GithubApp::create($payload); + + return response()->json($githubApp, 201); + } catch (\Throwable $e) { + return handleError($e); + } + } + + #[OA\Get( + path: '/github-apps/{github_app_id}/repositories', + summary: 'Load Repositories for a GitHub App', + description: 'Fetch repositories from GitHub for a given GitHub app.', + operationId: 'load-repositories', + tags: ['GitHub Apps'], + security: [ + ['bearerAuth' => []], + ], + parameters: [ + new OA\Parameter( + name: 'github_app_id', + in: 'path', + required: true, + schema: new OA\Schema(type: 'integer'), + description: 'GitHub App ID' + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'Repositories loaded successfully.', + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'repositories' => new OA\Schema( + type: 'array', + items: new OA\Items(type: 'object') + ), + ] + ) + ) + ), + new OA\Response( + response: 400, + ref: '#/components/responses/400', + ), + new OA\Response( + response: 401, + ref: '#/components/responses/401', + ), + new OA\Response( + response: 404, + ref: '#/components/responses/404', + ), + ] + )] + public function load_repositories($github_app_id) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + try { + $githubApp = GithubApp::where('id', $github_app_id) + ->where('team_id', $teamId) + ->firstOrFail(); + + $token = generateGithubInstallationToken($githubApp); + $repositories = collect(); + $page = 1; + $maxPages = 100; // Safety limit: max 10,000 repositories + + while ($page <= $maxPages) { + $response = Http::GitHub($githubApp->api_url, $token) + ->timeout(20) + ->retry(3, 200, throw: false) + ->get('/installation/repositories', [ + 'per_page' => 100, + 'page' => $page, + ]); + + if ($response->status() !== 200) { + return response()->json([ + 'message' => $response->json()['message'] ?? 'Failed to load repositories', + ], $response->status()); + } + + $json = $response->json(); + $repos = $json['repositories'] ?? []; + + if (empty($repos)) { + break; // No more repositories to load + } + + $repositories = $repositories->concat($repos); + $page++; + } + + return response()->json([ + 'repositories' => $repositories->sortBy('name')->values(), + ]); + } catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e) { + return response()->json(['message' => 'GitHub app not found'], 404); + } catch (\Throwable $e) { + return handleError($e); + } + } + + #[OA\Get( + path: '/github-apps/{github_app_id}/repositories/{owner}/{repo}/branches', + summary: 'Load Branches for a GitHub Repository', + description: 'Fetch branches from GitHub for a given repository.', + operationId: 'load-branches', + tags: ['GitHub Apps'], + security: [ + ['bearerAuth' => []], + ], + parameters: [ + new OA\Parameter( + name: 'github_app_id', + in: 'path', + required: true, + schema: new OA\Schema(type: 'integer'), + description: 'GitHub App ID' + ), + new OA\Parameter( + name: 'owner', + in: 'path', + required: true, + schema: new OA\Schema(type: 'string'), + description: 'Repository owner' + ), + new OA\Parameter( + name: 'repo', + in: 'path', + required: true, + schema: new OA\Schema(type: 'string'), + description: 'Repository name' + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'Branches loaded successfully.', + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'branches' => new OA\Schema( + type: 'array', + items: new OA\Items(type: 'object') + ), + ] + ) + ) + ), + new OA\Response( + response: 400, + ref: '#/components/responses/400', + ), + new OA\Response( + response: 401, + ref: '#/components/responses/401', + ), + new OA\Response( + response: 404, + ref: '#/components/responses/404', + ), + ] + )] + public function load_branches($github_app_id, $owner, $repo) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + try { + $githubApp = GithubApp::where('id', $github_app_id) + ->where('team_id', $teamId) + ->firstOrFail(); + + $token = generateGithubInstallationToken($githubApp); + + $response = Http::GitHub($githubApp->api_url, $token) + ->timeout(20) + ->retry(3, 200, throw: false) + ->get("/repos/{$owner}/{$repo}/branches"); + + if ($response->status() !== 200) { + return response()->json([ + 'message' => 'Error loading branches from GitHub.', + 'error' => $response->json('message'), + ], $response->status()); + } + + $branches = $response->json(); + + return response()->json([ + 'branches' => $branches, + ]); + } catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e) { + return response()->json(['message' => 'GitHub app not found'], 404); + } catch (\Throwable $e) { + return handleError($e); + } + } + + /** + * Update a GitHub app. + */ + #[OA\Patch( + path: '/github-apps/{github_app_id}', + operationId: 'updateGithubApp', + security: [ + ['bearerAuth' => []], + ], + tags: ['GitHub Apps'], + summary: 'Update GitHub App', + description: 'Update an existing GitHub app.', + parameters: [ + new OA\Parameter( + name: 'github_app_id', + in: 'path', + required: true, + schema: new OA\Schema(type: 'integer'), + description: 'GitHub App ID' + ), + ], + requestBody: new OA\RequestBody( + required: true, + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'name' => ['type' => 'string', 'description' => 'GitHub App name'], + 'organization' => ['type' => 'string', 'nullable' => true, 'description' => 'GitHub organization'], + 'api_url' => ['type' => 'string', 'description' => 'GitHub API URL'], + 'html_url' => ['type' => 'string', 'description' => 'GitHub HTML URL'], + 'custom_user' => ['type' => 'string', 'description' => 'Custom user for SSH'], + 'custom_port' => ['type' => 'integer', 'description' => 'Custom port for SSH'], + 'app_id' => ['type' => 'integer', 'description' => 'GitHub App ID'], + 'installation_id' => ['type' => 'integer', 'description' => 'GitHub Installation ID'], + 'client_id' => ['type' => 'string', 'description' => 'GitHub Client ID'], + 'client_secret' => ['type' => 'string', 'description' => 'GitHub Client Secret'], + 'webhook_secret' => ['type' => 'string', 'description' => 'GitHub Webhook Secret'], + 'private_key_uuid' => ['type' => 'string', 'description' => 'Private key UUID'], + 'is_system_wide' => ['type' => 'boolean', 'description' => 'Is system wide (non-cloud instances only)'], + ] + ) + ) + ), + responses: [ + new OA\Response( + response: 200, + description: 'GitHub app updated successfully', + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'message' => ['type' => 'string', 'example' => 'GitHub app updated successfully'], + 'data' => ['type' => 'object', 'description' => 'Updated GitHub app data'], + ] + ) + ) + ), + new OA\Response(response: 401, description: 'Unauthorized'), + new OA\Response(response: 404, description: 'GitHub app not found'), + new OA\Response(response: 422, description: 'Validation error'), + ] + )] + public function update_github_app(Request $request, $github_app_id) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + try { + $githubApp = GithubApp::where('id', $github_app_id) + ->where('team_id', $teamId) + ->firstOrFail(); + + // Define allowed fields for update + $allowedFields = [ + 'name', + 'organization', + 'api_url', + 'html_url', + 'custom_user', + 'custom_port', + 'app_id', + 'installation_id', + 'client_id', + 'client_secret', + 'webhook_secret', + 'private_key_uuid', + ]; + + if (! isCloud()) { + $allowedFields[] = 'is_system_wide'; + } + + $payload = $request->only($allowedFields); + + // Validate the request + $rules = []; + if (isset($payload['name'])) { + $rules['name'] = 'string'; + } + if (isset($payload['organization'])) { + $rules['organization'] = 'nullable|string'; + } + if (isset($payload['api_url'])) { + $rules['api_url'] = 'url'; + } + if (isset($payload['html_url'])) { + $rules['html_url'] = 'url'; + } + if (isset($payload['custom_user'])) { + $rules['custom_user'] = 'string'; + } + if (isset($payload['custom_port'])) { + $rules['custom_port'] = 'integer|min:1|max:65535'; + } + if (isset($payload['app_id'])) { + $rules['app_id'] = 'integer'; + } + if (isset($payload['installation_id'])) { + $rules['installation_id'] = 'integer'; + } + if (isset($payload['client_id'])) { + $rules['client_id'] = 'string'; + } + if (isset($payload['client_secret'])) { + $rules['client_secret'] = 'string'; + } + if (isset($payload['webhook_secret'])) { + $rules['webhook_secret'] = 'string'; + } + if (isset($payload['private_key_uuid'])) { + $rules['private_key_uuid'] = 'string|uuid'; + } + if (! isCloud() && isset($payload['is_system_wide'])) { + $rules['is_system_wide'] = 'boolean'; + } + + $validator = customApiValidator($payload, $rules); + if ($validator->fails()) { + return response()->json([ + 'message' => 'Validation error', + 'errors' => $validator->errors(), + ], 422); + } + + // Handle private_key_uuid -> private_key_id conversion + if (isset($payload['private_key_uuid'])) { + $privateKey = PrivateKey::where('team_id', $teamId) + ->where('uuid', $payload['private_key_uuid']) + ->first(); + + if (! $privateKey) { + return response()->json([ + 'message' => 'Private key not found or does not belong to your team', + ], 404); + } + + unset($payload['private_key_uuid']); + $payload['private_key_id'] = $privateKey->id; + } + + // Update the GitHub app + $githubApp->update($payload); + + return response()->json([ + 'message' => 'GitHub app updated successfully', + 'data' => $githubApp, + ]); + } catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e) { + return response()->json([ + 'message' => 'GitHub app not found', + ], 404); + } + } + + /** + * Delete a GitHub app. + */ + #[OA\Delete( + path: '/github-apps/{github_app_id}', + operationId: 'deleteGithubApp', + security: [ + ['bearerAuth' => []], + ], + tags: ['GitHub Apps'], + summary: 'Delete GitHub App', + description: 'Delete a GitHub app if it\'s not being used by any applications.', + parameters: [ + new OA\Parameter( + name: 'github_app_id', + in: 'path', + required: true, + schema: new OA\Schema(type: 'integer'), + description: 'GitHub App ID' + ), + ], + responses: [ + new OA\Response( + response: 200, + description: 'GitHub app deleted successfully', + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'message' => ['type' => 'string', 'example' => 'GitHub app deleted successfully'], + ] + ) + ) + ), + new OA\Response(response: 401, description: 'Unauthorized'), + new OA\Response(response: 404, description: 'GitHub app not found'), + new OA\Response( + response: 409, + description: 'Conflict - GitHub app is in use', + content: new OA\MediaType( + mediaType: 'application/json', + schema: new OA\Schema( + type: 'object', + properties: [ + 'message' => ['type' => 'string', 'example' => 'This GitHub app is being used by 5 application(s). Please delete all applications first.'], + ] + ) + ) + ), + ] + )] + public function delete_github_app($github_app_id) + { + $teamId = getTeamIdFromToken(); + if (is_null($teamId)) { + return invalidTokenResponse(); + } + + try { + $githubApp = GithubApp::where('id', $github_app_id) + ->where('team_id', $teamId) + ->firstOrFail(); + + // Check if the GitHub app is being used by any applications + if ($githubApp->applications->isNotEmpty()) { + $count = $githubApp->applications->count(); + + return response()->json([ + 'message' => "This GitHub app is being used by {$count} application(s). Please delete all applications first.", + ], 409); + } + + $githubApp->delete(); + + return response()->json([ + 'message' => 'GitHub app deleted successfully', + ]); + } catch (\Illuminate\Database\Eloquent\ModelNotFoundException $e) { + return response()->json([ + 'message' => 'GitHub app not found', + ], 404); + } + } +} diff --git a/app/Http/Controllers/Api/ServicesController.php b/app/Http/Controllers/Api/ServicesController.php index 162f637c5..e240e326e 100644 --- a/app/Http/Controllers/Api/ServicesController.php +++ b/app/Http/Controllers/Api/ServicesController.php @@ -353,7 +353,6 @@ public function create_service(Request $request) 'value' => $generatedValue, 'resourceable_id' => $service->id, 'resourceable_type' => $service->getMorphClass(), - 'is_build_time' => false, 'is_preview' => false, ]); }); @@ -919,7 +918,6 @@ public function envs(Request $request) 'key' => ['type' => 'string', 'description' => 'The key of the environment variable.'], 'value' => ['type' => 'string', 'description' => 'The value of the environment variable.'], 'is_preview' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in preview deployments.'], - 'is_build_time' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in build time.'], 'is_literal' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is a literal, nothing espaced.'], 'is_multiline' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is multiline.'], 'is_shown_once' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable\'s value is shown on the UI.'], @@ -975,7 +973,6 @@ public function update_env_by_uuid(Request $request) $validator = customApiValidator($request->all(), [ 'key' => 'string|required', 'value' => 'string|nullable', - 'is_build_time' => 'boolean', 'is_literal' => 'boolean', 'is_multiline' => 'boolean', 'is_shown_once' => 'boolean', @@ -1039,7 +1036,6 @@ public function update_env_by_uuid(Request $request) 'key' => ['type' => 'string', 'description' => 'The key of the environment variable.'], 'value' => ['type' => 'string', 'description' => 'The value of the environment variable.'], 'is_preview' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in preview deployments.'], - 'is_build_time' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in build time.'], 'is_literal' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is a literal, nothing espaced.'], 'is_multiline' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is multiline.'], 'is_shown_once' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable\'s value is shown on the UI.'], @@ -1105,7 +1101,6 @@ public function create_bulk_envs(Request $request) $validator = customApiValidator($item, [ 'key' => 'string|required', 'value' => 'string|nullable', - 'is_build_time' => 'boolean', 'is_literal' => 'boolean', 'is_multiline' => 'boolean', 'is_shown_once' => 'boolean', @@ -1161,7 +1156,6 @@ public function create_bulk_envs(Request $request) 'key' => ['type' => 'string', 'description' => 'The key of the environment variable.'], 'value' => ['type' => 'string', 'description' => 'The value of the environment variable.'], 'is_preview' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in preview deployments.'], - 'is_build_time' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is used in build time.'], 'is_literal' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is a literal, nothing espaced.'], 'is_multiline' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable is multiline.'], 'is_shown_once' => ['type' => 'boolean', 'description' => 'The flag to indicate if the environment variable\'s value is shown on the UI.'], @@ -1216,7 +1210,6 @@ public function create_env(Request $request) $validator = customApiValidator($request->all(), [ 'key' => 'string|required', 'value' => 'string|nullable', - 'is_build_time' => 'boolean', 'is_literal' => 'boolean', 'is_multiline' => 'boolean', 'is_shown_once' => 'boolean', diff --git a/app/Http/Controllers/Api/TeamController.php b/app/Http/Controllers/Api/TeamController.php index d4b24d8ab..e12d83542 100644 --- a/app/Http/Controllers/Api/TeamController.php +++ b/app/Http/Controllers/Api/TeamController.php @@ -179,6 +179,8 @@ public function members_by_id(Request $request) $members = $team->members; $members->makeHidden([ 'pivot', + 'email_change_code', + 'email_change_code_expires_at', ]); return response()->json( @@ -264,6 +266,8 @@ public function current_team_members(Request $request) $team = auth()->user()->currentTeam(); $team->members->makeHidden([ 'pivot', + 'email_change_code', + 'email_change_code_expires_at', ]); return response()->json( diff --git a/app/Http/Controllers/Webhook/Github.php b/app/Http/Controllers/Webhook/Github.php index 8872754e5..5ba9c08e7 100644 --- a/app/Http/Controllers/Webhook/Github.php +++ b/app/Http/Controllers/Webhook/Github.php @@ -5,6 +5,7 @@ use App\Enums\ProcessStatus; use App\Http\Controllers\Controller; use App\Jobs\ApplicationPullRequestUpdateJob; +use App\Jobs\DeleteResourceJob; use App\Jobs\GithubAppPermissionJob; use App\Models\Application; use App\Models\ApplicationPreview; @@ -78,6 +79,7 @@ public function manual(Request $request) $pull_request_html_url = data_get($payload, 'pull_request.html_url'); $branch = data_get($payload, 'pull_request.head.ref'); $base_branch = data_get($payload, 'pull_request.base.ref'); + $author_association = data_get($payload, 'pull_request.author_association'); } if (! $branch) { return response('Nothing to do. No branch found in the request.'); @@ -95,151 +97,168 @@ public function manual(Request $request) return response("Nothing to do. No applications found with branch '$base_branch'."); } } - foreach ($applications as $application) { - $webhook_secret = data_get($application, 'manual_webhook_secret_github'); - $hmac = hash_hmac('sha256', $request->getContent(), $webhook_secret); - if (! hash_equals($x_hub_signature_256, $hmac) && ! isDev()) { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'failed', - 'message' => 'Invalid signature.', - ]); + $applicationsByServer = $applications->groupBy(function ($app) { + return $app->destination->server_id; + }); - continue; - } - $isFunctional = $application->destination->server->isFunctional(); - if (! $isFunctional) { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'failed', - 'message' => 'Server is not functional.', - ]); + foreach ($applicationsByServer as $serverId => $serverApplications) { + foreach ($serverApplications as $application) { + $webhook_secret = data_get($application, 'manual_webhook_secret_github'); + $hmac = hash_hmac('sha256', $request->getContent(), $webhook_secret); + if (! hash_equals($x_hub_signature_256, $hmac) && ! isDev()) { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'failed', + 'message' => 'Invalid signature.', + ]); - continue; - } - if ($x_github_event === 'push') { - if ($application->isDeployable()) { - $is_watch_path_triggered = $application->isWatchPathsTriggered($changed_files); - if ($is_watch_path_triggered || is_null($application->watch_paths)) { - $deployment_uuid = new Cuid2; - $result = queue_application_deployment( - application: $application, - deployment_uuid: $deployment_uuid, - force_rebuild: false, - commit: data_get($payload, 'after', 'HEAD'), - is_webhook: true, - ); - if ($result['status'] === 'skipped') { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'skipped', - 'message' => $result['message'], - ]); + continue; + } + $isFunctional = $application->destination->server->isFunctional(); + if (! $isFunctional) { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'failed', + 'message' => 'Server is not functional.', + ]); + + continue; + } + if ($x_github_event === 'push') { + if ($application->isDeployable()) { + $is_watch_path_triggered = $application->isWatchPathsTriggered($changed_files); + if ($is_watch_path_triggered || is_null($application->watch_paths)) { + $deployment_uuid = new Cuid2; + $result = queue_application_deployment( + application: $application, + deployment_uuid: $deployment_uuid, + force_rebuild: false, + commit: data_get($payload, 'after', 'HEAD'), + is_webhook: true, + ); + if ($result['status'] === 'skipped') { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'skipped', + 'message' => $result['message'], + ]); + } else { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'success', + 'message' => 'Deployment queued.', + 'application_uuid' => $application->uuid, + 'application_name' => $application->name, + 'deployment_uuid' => $result['deployment_uuid'], + ]); + } } else { + $paths = str($application->watch_paths)->explode("\n"); $return_payloads->push([ - 'application' => $application->name, - 'status' => 'success', - 'message' => 'Deployment queued.', + 'status' => 'failed', + 'message' => 'Changed files do not match watch paths. Ignoring deployment.', 'application_uuid' => $application->uuid, 'application_name' => $application->name, - 'deployment_uuid' => $result['deployment_uuid'], + 'details' => [ + 'changed_files' => $changed_files, + 'watch_paths' => $paths, + ], ]); } } else { - $paths = str($application->watch_paths)->explode("\n"); $return_payloads->push([ 'status' => 'failed', - 'message' => 'Changed files do not match watch paths. Ignoring deployment.', + 'message' => 'Deployments disabled.', 'application_uuid' => $application->uuid, 'application_name' => $application->name, - 'details' => [ - 'changed_files' => $changed_files, - 'watch_paths' => $paths, - ], ]); } - } else { - $return_payloads->push([ - 'status' => 'failed', - 'message' => 'Deployments disabled.', - 'application_uuid' => $application->uuid, - 'application_name' => $application->name, - ]); } - } - if ($x_github_event === 'pull_request') { - if ($action === 'opened' || $action === 'synchronize' || $action === 'reopened') { - if ($application->isPRDeployable()) { - $deployment_uuid = new Cuid2; - $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); - if (! $found) { - if ($application->build_pack === 'dockercompose') { - $pr_app = ApplicationPreview::create([ - 'git_type' => 'github', - 'application_id' => $application->id, - 'pull_request_id' => $pull_request_id, - 'pull_request_html_url' => $pull_request_html_url, - 'docker_compose_domains' => $application->docker_compose_domains, - ]); - $pr_app->generate_preview_fqdn_compose(); - } else { - $pr_app = ApplicationPreview::create([ - 'git_type' => 'github', - 'application_id' => $application->id, - 'pull_request_id' => $pull_request_id, - 'pull_request_html_url' => $pull_request_html_url, - ]); - $pr_app->generate_preview_fqdn(); - } - } + if ($x_github_event === 'pull_request') { + if ($action === 'opened' || $action === 'synchronize' || $action === 'reopened') { + if ($application->isPRDeployable()) { + // Check if PR deployments from public contributors are restricted + if (! $application->settings->is_pr_deployments_public_enabled) { + $trustedAssociations = ['OWNER', 'MEMBER', 'COLLABORATOR', 'CONTRIBUTOR']; + if (! in_array($author_association, $trustedAssociations)) { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'failed', + 'message' => 'PR deployments are restricted to repository members and contributors. Author association: '.$author_association, + ]); - $result = queue_application_deployment( - application: $application, - pull_request_id: $pull_request_id, - deployment_uuid: $deployment_uuid, - force_rebuild: false, - commit: data_get($payload, 'head.sha', 'HEAD'), - is_webhook: true, - git_type: 'github' - ); - if ($result['status'] === 'skipped') { + continue; + } + } + $deployment_uuid = new Cuid2; + $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); + if (! $found) { + if ($application->build_pack === 'dockercompose') { + $pr_app = ApplicationPreview::create([ + 'git_type' => 'github', + 'application_id' => $application->id, + 'pull_request_id' => $pull_request_id, + 'pull_request_html_url' => $pull_request_html_url, + 'docker_compose_domains' => $application->docker_compose_domains, + ]); + $pr_app->generate_preview_fqdn_compose(); + } else { + $pr_app = ApplicationPreview::create([ + 'git_type' => 'github', + 'application_id' => $application->id, + 'pull_request_id' => $pull_request_id, + 'pull_request_html_url' => $pull_request_html_url, + ]); + $pr_app->generate_preview_fqdn(); + } + } + + $result = queue_application_deployment( + application: $application, + pull_request_id: $pull_request_id, + deployment_uuid: $deployment_uuid, + force_rebuild: false, + commit: data_get($payload, 'head.sha', 'HEAD'), + is_webhook: true, + git_type: 'github' + ); + if ($result['status'] === 'skipped') { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'skipped', + 'message' => $result['message'], + ]); + } else { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'success', + 'message' => 'Preview deployment queued.', + ]); + } + } else { $return_payloads->push([ 'application' => $application->name, - 'status' => 'skipped', - 'message' => $result['message'], + 'status' => 'failed', + 'message' => 'Preview deployments disabled.', + ]); + } + } + if ($action === 'closed') { + $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); + if ($found) { + DeleteResourceJob::dispatch($found); + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'success', + 'message' => 'Preview deployment closed.', ]); } else { $return_payloads->push([ 'application' => $application->name, - 'status' => 'success', - 'message' => 'Preview deployment queued.', + 'status' => 'failed', + 'message' => 'No preview deployment found.', ]); } - } else { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'failed', - 'message' => 'Preview deployments disabled.', - ]); - } - } - if ($action === 'closed') { - $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); - if ($found) { - $found->delete(); - $container_name = generateApplicationContainerName($application, $pull_request_id); - instant_remote_process(["docker rm -f $container_name"], $application->destination->server); - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'success', - 'message' => 'Preview deployment closed.', - ]); - } else { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'failed', - 'message' => 'No preview deployment found.', - ]); } } } @@ -327,6 +346,7 @@ public function normal(Request $request) $pull_request_html_url = data_get($payload, 'pull_request.html_url'); $branch = data_get($payload, 'pull_request.head.ref'); $base_branch = data_get($payload, 'pull_request.base.ref'); + $author_association = data_get($payload, 'pull_request.author_association'); } if (! $id || ! $branch) { return response('Nothing to do. No id or branch found.'); @@ -344,127 +364,147 @@ public function normal(Request $request) return response("Nothing to do. No applications found with branch '$base_branch'."); } } - foreach ($applications as $application) { - $isFunctional = $application->destination->server->isFunctional(); - if (! $isFunctional) { - $return_payloads->push([ - 'status' => 'failed', - 'message' => 'Server is not functional.', - 'application_uuid' => $application->uuid, - 'application_name' => $application->name, - ]); + $applicationsByServer = $applications->groupBy(function ($app) { + return $app->destination->server_id; + }); - continue; - } - if ($x_github_event === 'push') { - if ($application->isDeployable()) { - $is_watch_path_triggered = $application->isWatchPathsTriggered($changed_files); - if ($is_watch_path_triggered || is_null($application->watch_paths)) { - $deployment_uuid = new Cuid2; - $result = queue_application_deployment( - application: $application, - deployment_uuid: $deployment_uuid, - commit: data_get($payload, 'after', 'HEAD'), - force_rebuild: false, - is_webhook: true, - ); - $return_payloads->push([ - 'status' => $result['status'], - 'message' => $result['message'], - 'application_uuid' => $application->uuid, - 'application_name' => $application->name, - 'deployment_uuid' => $result['deployment_uuid'], - ]); - } else { - $paths = str($application->watch_paths)->explode("\n"); - $return_payloads->push([ - 'status' => 'failed', - 'message' => 'Changed files do not match watch paths. Ignoring deployment.', - 'application_uuid' => $application->uuid, - 'application_name' => $application->name, - 'details' => [ - 'changed_files' => $changed_files, - 'watch_paths' => $paths, - ], - ]); - } - } else { + foreach ($applicationsByServer as $serverId => $serverApplications) { + foreach ($serverApplications as $application) { + $isFunctional = $application->destination->server->isFunctional(); + if (! $isFunctional) { $return_payloads->push([ 'status' => 'failed', - 'message' => 'Deployments disabled.', + 'message' => 'Server is not functional.', 'application_uuid' => $application->uuid, 'application_name' => $application->name, ]); + + continue; } - } - if ($x_github_event === 'pull_request') { - if ($action === 'opened' || $action === 'synchronize' || $action === 'reopened') { - if ($application->isPRDeployable()) { - $deployment_uuid = new Cuid2; - $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); - if (! $found) { - ApplicationPreview::create([ - 'git_type' => 'github', - 'application_id' => $application->id, - 'pull_request_id' => $pull_request_id, - 'pull_request_html_url' => $pull_request_html_url, + if ($x_github_event === 'push') { + if ($application->isDeployable()) { + $is_watch_path_triggered = $application->isWatchPathsTriggered($changed_files); + if ($is_watch_path_triggered || is_null($application->watch_paths)) { + $deployment_uuid = new Cuid2; + $result = queue_application_deployment( + application: $application, + deployment_uuid: $deployment_uuid, + commit: data_get($payload, 'after', 'HEAD'), + force_rebuild: false, + is_webhook: true, + ); + $return_payloads->push([ + 'status' => $result['status'], + 'message' => $result['message'], + 'application_uuid' => $application->uuid, + 'application_name' => $application->name, + 'deployment_uuid' => $result['deployment_uuid'], + ]); + } else { + $paths = str($application->watch_paths)->explode("\n"); + $return_payloads->push([ + 'status' => 'failed', + 'message' => 'Changed files do not match watch paths. Ignoring deployment.', + 'application_uuid' => $application->uuid, + 'application_name' => $application->name, + 'details' => [ + 'changed_files' => $changed_files, + 'watch_paths' => $paths, + ], ]); } - $result = queue_application_deployment( - application: $application, - pull_request_id: $pull_request_id, - deployment_uuid: $deployment_uuid, - force_rebuild: false, - commit: data_get($payload, 'head.sha', 'HEAD'), - is_webhook: true, - git_type: 'github' - ); - if ($result['status'] === 'skipped') { + } else { + $return_payloads->push([ + 'status' => 'failed', + 'message' => 'Deployments disabled.', + 'application_uuid' => $application->uuid, + 'application_name' => $application->name, + ]); + } + } + if ($x_github_event === 'pull_request') { + if ($action === 'opened' || $action === 'synchronize' || $action === 'reopened') { + if ($application->isPRDeployable()) { + // Check if PR deployments from public contributors are restricted + if (! $application->settings->is_pr_deployments_public_enabled) { + $trustedAssociations = ['OWNER', 'MEMBER', 'COLLABORATOR', 'CONTRIBUTOR']; + if (! in_array($author_association, $trustedAssociations)) { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'failed', + 'message' => 'PR deployments are restricted to repository members and contributors. Author association: '.$author_association, + ]); + + continue; + } + } + $deployment_uuid = new Cuid2; + $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); + if (! $found) { + ApplicationPreview::create([ + 'git_type' => 'github', + 'application_id' => $application->id, + 'pull_request_id' => $pull_request_id, + 'pull_request_html_url' => $pull_request_html_url, + ]); + } + $result = queue_application_deployment( + application: $application, + pull_request_id: $pull_request_id, + deployment_uuid: $deployment_uuid, + force_rebuild: false, + commit: data_get($payload, 'head.sha', 'HEAD'), + is_webhook: true, + git_type: 'github' + ); + if ($result['status'] === 'skipped') { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'skipped', + 'message' => $result['message'], + ]); + } else { + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'success', + 'message' => 'Preview deployment queued.', + ]); + } + } else { $return_payloads->push([ 'application' => $application->name, - 'status' => 'skipped', - 'message' => $result['message'], + 'status' => 'failed', + 'message' => 'Preview deployments disabled.', + ]); + } + } + if ($action === 'closed' || $action === 'close') { + $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); + if ($found) { + $containers = getCurrentApplicationContainerStatus($application->destination->server, $application->id, $pull_request_id); + if ($containers->isNotEmpty()) { + $containers->each(function ($container) use ($application) { + $container_name = data_get($container, 'Names'); + instant_remote_process(["docker rm -f $container_name"], $application->destination->server); + }); + } + + ApplicationPullRequestUpdateJob::dispatchSync(application: $application, preview: $found, status: ProcessStatus::CLOSED); + + DeleteResourceJob::dispatch($found); + + $return_payloads->push([ + 'application' => $application->name, + 'status' => 'success', + 'message' => 'Preview deployment closed.', ]); } else { $return_payloads->push([ 'application' => $application->name, - 'status' => 'success', - 'message' => 'Preview deployment queued.', + 'status' => 'failed', + 'message' => 'No preview deployment found.', ]); } - } else { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'failed', - 'message' => 'Preview deployments disabled.', - ]); - } - } - if ($action === 'closed' || $action === 'close') { - $found = ApplicationPreview::where('application_id', $application->id)->where('pull_request_id', $pull_request_id)->first(); - if ($found) { - $containers = getCurrentApplicationContainerStatus($application->destination->server, $application->id, $pull_request_id); - if ($containers->isNotEmpty()) { - $containers->each(function ($container) use ($application) { - $container_name = data_get($container, 'Names'); - instant_remote_process(["docker rm -f $container_name"], $application->destination->server); - }); - } - - ApplicationPullRequestUpdateJob::dispatchSync(application: $application, preview: $found, status: ProcessStatus::CLOSED); - $found->delete(); - - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'success', - 'message' => 'Preview deployment closed.', - ]); - } else { - $return_payloads->push([ - 'application' => $application->name, - 'status' => 'failed', - 'message' => 'No preview deployment found.', - ]); } } } diff --git a/app/Http/Controllers/Webhook/Stripe.php b/app/Http/Controllers/Webhook/Stripe.php index 83ba16699..ae50aac42 100644 --- a/app/Http/Controllers/Webhook/Stripe.php +++ b/app/Http/Controllers/Webhook/Stripe.php @@ -4,15 +4,12 @@ use App\Http\Controllers\Controller; use App\Jobs\StripeProcessJob; -use App\Models\Webhook; use Exception; use Illuminate\Http\Request; use Illuminate\Support\Facades\Storage; class Stripe extends Controller { - protected $webhook; - public function events(Request $request) { try { @@ -40,19 +37,10 @@ public function events(Request $request) return response('Webhook received. Cool cool cool cool cool.', 200); } - $this->webhook = Webhook::create([ - 'type' => 'stripe', - 'payload' => $request->getContent(), - ]); StripeProcessJob::dispatch($event); return response('Webhook received. Cool cool cool cool cool.', 200); } catch (Exception $e) { - $this->webhook->update([ - 'status' => 'failed', - 'failure_reason' => $e->getMessage(), - ]); - return response($e->getMessage(), 400); } } diff --git a/app/Http/Middleware/ApiAllowed.php b/app/Http/Middleware/ApiAllowed.php index dd85c3521..21441a117 100644 --- a/app/Http/Middleware/ApiAllowed.php +++ b/app/Http/Middleware/ApiAllowed.php @@ -28,7 +28,7 @@ public function handle(Request $request, Closure $next): Response $allowedIps = array_map('trim', $allowedIps); $allowedIps = array_filter($allowedIps); // Remove empty entries - if (! empty($allowedIps) && ! check_ip_against_allowlist($request->ip(), $allowedIps)) { + if (! empty($allowedIps) && ! checkIPAgainstAllowlist($request->ip(), $allowedIps)) { return response()->json(['success' => true, 'message' => 'You are not allowed to access the API.'], 403); } } diff --git a/app/Jobs/ApplicationDeploymentJob.php b/app/Jobs/ApplicationDeploymentJob.php index 9037fa3e5..04b11b9b4 100644 --- a/app/Jobs/ApplicationDeploymentJob.php +++ b/app/Jobs/ApplicationDeploymentJob.php @@ -5,6 +5,7 @@ use App\Actions\Docker\GetContainersStatus; use App\Enums\ApplicationDeploymentStatus; use App\Enums\ProcessStatus; +use App\Events\ApplicationConfigurationChanged; use App\Events\ServiceStatusChanged; use App\Models\Application; use App\Models\ApplicationDeploymentQueue; @@ -17,6 +18,7 @@ use App\Models\SwarmDocker; use App\Notifications\Application\DeploymentFailed; use App\Notifications\Application\DeploymentSuccess; +use App\Traits\EnvironmentVariableAnalyzer; use App\Traits\ExecuteRemoteCommand; use Carbon\Carbon; use Exception; @@ -34,11 +36,10 @@ use Symfony\Component\Yaml\Yaml; use Throwable; use Visus\Cuid2\Cuid2; -use Yosymfony\Toml\Toml; class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue { - use Dispatchable, ExecuteRemoteCommand, InteractsWithQueue, Queueable, SerializesModels; + use Dispatchable, EnvironmentVariableAnalyzer, ExecuteRemoteCommand, InteractsWithQueue, Queueable, SerializesModels; public $tries = 1; @@ -147,6 +148,8 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue private Collection $saved_outputs; + private ?string $secrets_hash_key = null; + private ?string $full_healthcheck_url = null; private string $serverUser = 'root'; @@ -167,6 +170,12 @@ class ApplicationDeploymentJob implements ShouldBeEncrypted, ShouldQueue private bool $preserveRepository = false; + private bool $dockerBuildkitSupported = false; + + private bool $skip_build = false; + + private Collection|string $build_secrets; + public function tags() { // Do not remove this one, it needs to properly identify which worker is running the job @@ -183,6 +192,7 @@ public function __construct(public int $application_deployment_queue_id) $this->application = Application::find($this->application_deployment_queue->application_id); $this->build_pack = data_get($this->application, 'build_pack'); $this->build_args = collect([]); + $this->build_secrets = ''; $this->deployment_uuid = $this->application_deployment_queue->deployment_uuid; $this->pull_request_id = $this->application_deployment_queue->pull_request_id; @@ -221,7 +231,7 @@ public function __construct(public int $application_deployment_queue_id) if ($this->pull_request_id === 0) { $this->container_name = $this->application->settings->custom_internal_name; } else { - $this->container_name = "{$this->application->settings->custom_internal_name}-pr-{$this->pull_request_id}"; + $this->container_name = addPreviewDeploymentSuffix($this->application->settings->custom_internal_name, $this->pull_request_id); } } @@ -250,6 +260,14 @@ public function __construct(public int $application_deployment_queue_id) public function handle(): void { + // Check if deployment was cancelled before we even started + $this->application_deployment_queue->refresh(); + if ($this->application_deployment_queue->status === ApplicationDeploymentStatus::CANCELLED_BY_USER->value) { + $this->application_deployment_queue->addLogEntry('Deployment was cancelled before starting.'); + + return; + } + $this->application_deployment_queue->update([ 'status' => ApplicationDeploymentStatus::IN_PROGRESS->value, 'horizon_job_worker' => gethostname(), @@ -263,7 +281,6 @@ public function handle(): void try { // Make sure the private key is stored in the filesystem $this->server->privateKey->storeInFileSystem(); - // Generate custom host<->ip mapping $allContainers = instant_remote_process(["docker network inspect {$this->destination->network} -f '{{json .Containers}}' "], $this->server); @@ -319,6 +336,7 @@ public function handle(): void $this->build_server = $this->server; $this->original_server = $this->server; } + $this->detectBuildKitCapabilities(); $this->decide_what_to_do(); } catch (Exception $e) { if ($this->pull_request_id !== 0 && $this->application->is_github_based()) { @@ -336,6 +354,7 @@ public function handle(): void } else { $this->write_deployment_configurations(); } + $this->application_deployment_queue->addLogEntry("Gracefully shutting down build container: {$this->deployment_uuid}"); $this->graceful_shutdown_container($this->deployment_uuid); @@ -343,6 +362,80 @@ public function handle(): void } } + private function detectBuildKitCapabilities(): void + { + // If build secrets are not enabled, skip detection and use traditional args + if (! $this->application->settings->use_build_secrets) { + $this->dockerBuildkitSupported = false; + + return; + } + + $serverToCheck = $this->use_build_server ? $this->build_server : $this->server; + $serverName = $this->use_build_server ? "build server ({$serverToCheck->name})" : "deployment server ({$serverToCheck->name})"; + + try { + $dockerVersion = instant_remote_process( + ["docker version --format '{{.Server.Version}}'"], + $serverToCheck + ); + + $versionParts = explode('.', $dockerVersion); + $majorVersion = (int) $versionParts[0]; + $minorVersion = (int) ($versionParts[1] ?? 0); + + if ($majorVersion < 18 || ($majorVersion == 18 && $minorVersion < 9)) { + $this->dockerBuildkitSupported = false; + $this->application_deployment_queue->addLogEntry("Docker {$dockerVersion} on {$serverName} does not support BuildKit (requires 18.09+). Build secrets feature disabled."); + + return; + } + + $buildkitEnabled = instant_remote_process( + ["docker buildx version >/dev/null 2>&1 && echo 'available' || echo 'not-available'"], + $serverToCheck + ); + + if (trim($buildkitEnabled) !== 'available') { + $buildkitTest = instant_remote_process( + ["DOCKER_BUILDKIT=1 docker build --help 2>&1 | grep -q 'secret' && echo 'supported' || echo 'not-supported'"], + $serverToCheck + ); + + if (trim($buildkitTest) === 'supported') { + $this->dockerBuildkitSupported = true; + $this->application_deployment_queue->addLogEntry("Docker {$dockerVersion} with BuildKit secrets support detected on {$serverName}."); + $this->application_deployment_queue->addLogEntry('Build secrets are enabled and will be used for enhanced security.'); + } else { + $this->dockerBuildkitSupported = false; + $this->application_deployment_queue->addLogEntry("Docker {$dockerVersion} on {$serverName} does not have BuildKit secrets support."); + $this->application_deployment_queue->addLogEntry('Build secrets feature is enabled but not supported. Using traditional build arguments.'); + } + } else { + // Buildx is available, which means BuildKit is available + // Now specifically test for secrets support + $secretsTest = instant_remote_process( + ["docker build --help 2>&1 | grep -q 'secret' && echo 'supported' || echo 'not-supported'"], + $serverToCheck + ); + + if (trim($secretsTest) === 'supported') { + $this->dockerBuildkitSupported = true; + $this->application_deployment_queue->addLogEntry("Docker {$dockerVersion} with BuildKit and Buildx detected on {$serverName}."); + $this->application_deployment_queue->addLogEntry('Build secrets are enabled and will be used for enhanced security.'); + } else { + $this->dockerBuildkitSupported = false; + $this->application_deployment_queue->addLogEntry("Docker {$dockerVersion} with Buildx on {$serverName}, but secrets not supported."); + $this->application_deployment_queue->addLogEntry('Build secrets feature is enabled but not supported. Using traditional build arguments.'); + } + } + } catch (\Exception $e) { + $this->dockerBuildkitSupported = false; + $this->application_deployment_queue->addLogEntry("Could not detect BuildKit capabilities on {$serverName}: {$e->getMessage()}"); + $this->application_deployment_queue->addLogEntry('Build secrets feature is enabled but detection failed. Using traditional build arguments.'); + } + } + private function decide_what_to_do() { if ($this->restart_only) { @@ -471,14 +564,23 @@ private function deploy_docker_compose_buildpack() } $this->generate_image_names(); $this->cleanup_git(); + + $this->generate_build_env_variables(); + $this->application->loadComposeFile(isInit: false); if ($this->application->settings->is_raw_compose_deployment_enabled) { $this->application->oldRawParser(); $yaml = $composeFile = $this->application->docker_compose_raw; - $this->save_environment_variables(); + $this->generate_runtime_environment_variables(); + + // For raw compose, we cannot automatically add secrets configuration + // User must define it manually in their docker-compose file + if ($this->application->settings->use_build_secrets && $this->dockerBuildkitSupported && ! empty($this->build_secrets)) { + $this->application_deployment_queue->addLogEntry('Build secrets are configured. Ensure your docker-compose file includes build.secrets configuration for services that need them.'); + } } else { $composeFile = $this->application->parse(pull_request_id: $this->pull_request_id, preview_id: data_get($this->preview, 'id')); - $this->save_environment_variables(); + $this->generate_runtime_environment_variables(); if (filled($this->env_filename)) { $services = collect(data_get($composeFile, 'services', [])); $services = $services->map(function ($service, $name) { @@ -494,6 +596,12 @@ private function deploy_docker_compose_buildpack() return; } + + // Add build secrets to compose file if enabled and BuildKit is supported + if ($this->application->settings->use_build_secrets && $this->dockerBuildkitSupported && ! empty($this->build_secrets)) { + $composeFile = $this->add_build_secrets_to_compose($composeFile); + } + $yaml = Yaml::dump(convertToArray($composeFile), 10); } $this->docker_compose_base64 = base64_encode($yaml); @@ -501,16 +609,28 @@ private function deploy_docker_compose_buildpack() executeInDocker($this->deployment_uuid, "echo '{$this->docker_compose_base64}' | base64 -d | tee {$this->workdir}{$this->docker_compose_location} > /dev/null"), 'hidden' => true, ]); + + // Modify Dockerfiles for ARGs and build secrets + $this->modify_dockerfiles_for_compose($composeFile); // Build new container to limit downtime. $this->application_deployment_queue->addLogEntry('Pulling & building required images.'); if ($this->docker_compose_custom_build_command) { + // Prepend DOCKER_BUILDKIT=1 if BuildKit is supported + $build_command = $this->docker_compose_custom_build_command; + if ($this->dockerBuildkitSupported) { + $build_command = "DOCKER_BUILDKIT=1 {$build_command}"; + } $this->execute_remote_command( - [executeInDocker($this->deployment_uuid, "cd {$this->basedir} && {$this->docker_compose_custom_build_command}"), 'hidden' => true], + [executeInDocker($this->deployment_uuid, "cd {$this->basedir} && {$build_command}"), 'hidden' => true], ); } else { $command = "{$this->coolify_variables} docker compose"; - if ($this->env_filename) { + // Prepend DOCKER_BUILDKIT=1 if BuildKit is supported + if ($this->dockerBuildkitSupported) { + $command = "DOCKER_BUILDKIT=1 {$command}"; + } + if (filled($this->env_filename)) { $command .= " --env-file {$this->workdir}/{$this->env_filename}"; } if ($this->force_rebuild) { @@ -518,6 +638,15 @@ private function deploy_docker_compose_buildpack() } else { $command .= " --project-name {$this->application->uuid} --project-directory {$this->workdir} -f {$this->workdir}{$this->docker_compose_location} build --pull"; } + + if (! $this->application->settings->use_build_secrets && $this->build_args instanceof \Illuminate\Support\Collection && $this->build_args->isNotEmpty()) { + $build_args_string = $this->build_args->implode(' '); + // Escape single quotes for bash -c context used by executeInDocker + $build_args_string = str_replace("'", "'\\''", $build_args_string); + $command .= " {$build_args_string}"; + $this->application_deployment_queue->addLogEntry('Adding build arguments to Docker Compose build command.'); + } + $this->execute_remote_command( [executeInDocker($this->deployment_uuid, $command), 'hidden' => true], ); @@ -556,7 +685,7 @@ private function deploy_docker_compose_buildpack() $this->docker_compose_location = '/docker-compose.yaml'; $command = "{$this->coolify_variables} docker compose"; - if ($this->env_filename) { + if (filled($this->env_filename)) { $command .= " --env-file {$server_workdir}/{$this->env_filename}"; } $command .= " --project-directory {$server_workdir} -f {$server_workdir}{$this->docker_compose_location} up -d"; @@ -573,7 +702,7 @@ private function deploy_docker_compose_buildpack() } else { $command = "{$this->coolify_variables} docker compose"; if ($this->preserveRepository) { - if ($this->env_filename) { + if (filled($this->env_filename)) { $command .= " --env-file {$server_workdir}/{$this->env_filename}"; } $command .= " --project-name {$this->application->uuid} --project-directory {$server_workdir} -f {$server_workdir}{$this->docker_compose_location} up -d"; @@ -583,7 +712,7 @@ private function deploy_docker_compose_buildpack() ['command' => $command, 'hidden' => true], ); } else { - if ($this->env_filename) { + if (filled($this->env_filename)) { $command .= " --env-file {$this->workdir}/{$this->env_filename}"; } $command .= " --project-name {$this->application->uuid} --project-directory {$this->workdir} -f {$this->workdir}{$this->docker_compose_location} up -d"; @@ -647,6 +776,10 @@ private function deploy_nixpacks_buildpack() $this->generate_compose_file(); $this->generate_build_env_variables(); $this->build_image(); + + // For Nixpacks, save runtime environment variables AFTER the build + // to prevent them from being accessible during the build process + $this->save_runtime_environment_variables(); $this->push_to_docker_registry(); $this->rolling_update(); } @@ -669,7 +802,7 @@ private function deploy_static_buildpack() $this->clone_repository(); $this->cleanup_git(); $this->generate_compose_file(); - $this->build_image(); + $this->build_static_image(); $this->push_to_docker_registry(); $this->rolling_update(); } @@ -712,8 +845,8 @@ private function write_deployment_configurations() if ($this->pull_request_id === 0) { $composeFileName = "$mainDir/docker-compose.yaml"; } else { - $composeFileName = "$mainDir/docker-compose-pr-{$this->pull_request_id}.yaml"; - $this->docker_compose_location = "/docker-compose-pr-{$this->pull_request_id}.yaml"; + $composeFileName = "$mainDir/".addPreviewDeploymentSuffix('docker-compose', $this->pull_request_id).'.yaml'; + $this->docker_compose_location = '/'.addPreviewDeploymentSuffix('docker-compose', $this->pull_request_id).'.yaml'; } $this->execute_remote_command( [ @@ -840,18 +973,17 @@ private function should_skip_build() { if (str($this->saved_outputs->get('local_image_found'))->isNotEmpty()) { if ($this->is_this_additional_server) { + $this->skip_build = true; $this->application_deployment_queue->addLogEntry("Image found ({$this->production_image_name}) with the same Git Commit SHA. Build step skipped."); $this->generate_compose_file(); $this->push_to_docker_registry(); $this->rolling_update(); - if ($this->restart_only) { - $this->post_deployment(); - } return true; } if (! $this->application->isConfigurationChanged()) { $this->application_deployment_queue->addLogEntry("No configuration changed & image found ({$this->production_image_name}) with the same Git Commit SHA. Build step skipped."); + $this->skip_build = true; $this->generate_compose_file(); $this->push_to_docker_registry(); $this->rolling_update(); @@ -892,7 +1024,7 @@ private function check_image_locally_or_remotely() } } - private function save_environment_variables() + private function generate_runtime_environment_variables() { $envs = collect([]); $sort = $this->application->settings->is_env_sorting_enabled; @@ -905,10 +1037,10 @@ private function save_environment_variables() } if ($this->build_pack === 'dockercompose') { $sorted_environment_variables = $sorted_environment_variables->filter(function ($env) { - return ! str($env->key)->startsWith('SERVICE_FQDN_') && ! str($env->key)->startsWith('SERVICE_URL_'); + return ! str($env->key)->startsWith('SERVICE_FQDN_') && ! str($env->key)->startsWith('SERVICE_URL_') && ! str($env->key)->startsWith('SERVICE_NAME_'); }); $sorted_environment_variables_preview = $sorted_environment_variables_preview->filter(function ($env) { - return ! str($env->key)->startsWith('SERVICE_FQDN_') && ! str($env->key)->startsWith('SERVICE_URL_'); + return ! str($env->key)->startsWith('SERVICE_FQDN_') && ! str($env->key)->startsWith('SERVICE_URL_') && ! str($env->key)->startsWith('SERVICE_NAME_'); }); } $ports = $this->application->main_port(); @@ -919,20 +1051,7 @@ private function save_environment_variables() if ($this->pull_request_id === 0) { $this->env_filename = '.env'; - foreach ($sorted_environment_variables as $env) { - $envs->push($env->key.'='.$env->real_value); - } - // Add PORT if not exists, use the first port as default - if ($this->build_pack !== 'dockercompose') { - if ($this->application->environment_variables->where('key', 'PORT')->isEmpty()) { - $envs->push("PORT={$ports[0]}"); - } - } - // Add HOST if not exists - if ($this->application->environment_variables->where('key', 'HOST')->isEmpty()) { - $envs->push('HOST=0.0.0.0'); - } - + // Generate SERVICE_ variables first for dockercompose if ($this->build_pack === 'dockercompose') { $domains = collect(json_decode($this->application->docker_compose_domains)) ?? collect([]); @@ -949,23 +1068,50 @@ private function save_environment_variables() $envs->push('SERVICE_FQDN_'.str($forServiceName)->upper().'='.$coolifyFqdn); } } + + // Generate SERVICE_NAME for dockercompose services from processed compose + if ($this->application->settings->is_raw_compose_deployment_enabled) { + $dockerCompose = Yaml::parse($this->application->docker_compose_raw); + } else { + $dockerCompose = Yaml::parse($this->application->docker_compose); + } + $services = data_get($dockerCompose, 'services', []); + foreach ($services as $serviceName => $_) { + $envs->push('SERVICE_NAME_'.str($serviceName)->upper().'='.$serviceName); + } } - } else { - $this->env_filename = ".env-pr-$this->pull_request_id"; - foreach ($sorted_environment_variables_preview as $env) { + + // Filter runtime variables (only include variables that are available at runtime) + $runtime_environment_variables = $sorted_environment_variables->filter(function ($env) { + return $env->is_runtime; + }); + + // Sort runtime environment variables: those referencing SERVICE_ variables come after others + $runtime_environment_variables = $runtime_environment_variables->sortBy(function ($env) { + if (str($env->value)->startsWith('$SERVICE_') || str($env->value)->contains('${SERVICE_')) { + return 2; + } + + return 1; + }); + + foreach ($runtime_environment_variables as $env) { $envs->push($env->key.'='.$env->real_value); } // Add PORT if not exists, use the first port as default if ($this->build_pack !== 'dockercompose') { - if ($this->application->environment_variables_preview->where('key', 'PORT')->isEmpty()) { + if ($this->application->environment_variables->where('key', 'PORT')->isEmpty()) { $envs->push("PORT={$ports[0]}"); } } // Add HOST if not exists - if ($this->application->environment_variables_preview->where('key', 'HOST')->isEmpty()) { + if ($this->application->environment_variables->where('key', 'HOST')->isEmpty()) { $envs->push('HOST=0.0.0.0'); } + } else { + $this->env_filename = '.env'; + // Generate SERVICE_ variables first for dockercompose preview if ($this->build_pack === 'dockercompose') { $domains = collect(json_decode(data_get($this->preview, 'docker_compose_domains'))) ?? collect([]); @@ -982,44 +1128,121 @@ private function save_environment_variables() $envs->push('SERVICE_FQDN_'.str($forServiceName)->upper().'='.$coolifyFqdn); } } + + // Generate SERVICE_NAME for dockercompose services + $rawDockerCompose = Yaml::parse($this->application->docker_compose_raw); + $rawServices = data_get($rawDockerCompose, 'services', []); + foreach ($rawServices as $rawServiceName => $_) { + $envs->push('SERVICE_NAME_'.str($rawServiceName)->upper().'='.addPreviewDeploymentSuffix($rawServiceName, $this->pull_request_id)); + } + } + + // Filter runtime variables for preview (only include variables that are available at runtime) + $runtime_environment_variables_preview = $sorted_environment_variables_preview->filter(function ($env) { + return $env->is_runtime; + }); + + // Sort runtime environment variables: those referencing SERVICE_ variables come after others + $runtime_environment_variables_preview = $runtime_environment_variables_preview->sortBy(function ($env) { + if (str($env->value)->startsWith('$SERVICE_') || str($env->value)->contains('${SERVICE_')) { + return 2; + } + + return 1; + }); + + foreach ($runtime_environment_variables_preview as $env) { + $envs->push($env->key.'='.$env->real_value); + } + // Add PORT if not exists, use the first port as default + if ($this->build_pack !== 'dockercompose') { + if ($this->application->environment_variables_preview->where('key', 'PORT')->isEmpty()) { + $envs->push("PORT={$ports[0]}"); + } + } + // Add HOST if not exists + if ($this->application->environment_variables_preview->where('key', 'HOST')->isEmpty()) { + $envs->push('HOST=0.0.0.0'); } } if ($envs->isEmpty()) { - $this->env_filename = null; - if ($this->use_build_server) { - $this->server = $this->original_server; - $this->execute_remote_command( - [ - 'command' => "rm -f $this->configuration_dir/{$this->env_filename}", - 'hidden' => true, - 'ignore_errors' => true, - ] - ); - $this->server = $this->build_server; - $this->execute_remote_command( - [ - 'command' => "rm -f $this->configuration_dir/{$this->env_filename}", - 'hidden' => true, - 'ignore_errors' => true, - ] - ); - } else { - $this->execute_remote_command( - [ - 'command' => "rm -f $this->configuration_dir/{$this->env_filename}", - 'hidden' => true, - 'ignore_errors' => true, - ] - ); + if ($this->env_filename) { + if ($this->use_build_server) { + $this->server = $this->original_server; + $this->execute_remote_command( + [ + 'command' => "rm -f $this->configuration_dir/{$this->env_filename}", + 'hidden' => true, + 'ignore_errors' => true, + ] + ); + $this->server = $this->build_server; + $this->execute_remote_command( + [ + 'command' => "rm -f $this->configuration_dir/{$this->env_filename}", + 'hidden' => true, + 'ignore_errors' => true, + ] + ); + } else { + $this->execute_remote_command( + [ + 'command' => "rm -f $this->configuration_dir/{$this->env_filename}", + 'hidden' => true, + 'ignore_errors' => true, + ] + ); + } } + $this->env_filename = null; } else { - $envs_base64 = base64_encode($envs->implode("\n")); + // For Nixpacks builds, we save the .env file AFTER the build to prevent + // runtime-only variables from being accessible during the build process + if ($this->application->build_pack !== 'nixpacks' || $this->skip_build) { + $envs_base64 = base64_encode($envs->implode("\n")); + $this->execute_remote_command( + [ + executeInDocker($this->deployment_uuid, "echo '$envs_base64' | base64 -d | tee $this->workdir/{$this->env_filename} > /dev/null"), + ], + + ); + if ($this->use_build_server) { + $this->server = $this->original_server; + $this->execute_remote_command( + [ + "echo '$envs_base64' | base64 -d | tee $this->configuration_dir/{$this->env_filename} > /dev/null", + ] + ); + $this->server = $this->build_server; + } else { + $this->execute_remote_command( + [ + "echo '$envs_base64' | base64 -d | tee $this->configuration_dir/{$this->env_filename} > /dev/null", + ] + ); + } + } + } + $this->environment_variables = $envs; + } + + private function save_runtime_environment_variables() + { + // This method saves the .env file with runtime variables + // It should be called AFTER the build for Nixpacks to prevent runtime-only variables + // from being accessible during the build process + + if ($this->environment_variables && $this->environment_variables->isNotEmpty() && $this->env_filename) { + $envs_base64 = base64_encode($this->environment_variables->implode("\n")); + + // Write .env file to workdir (for container runtime) $this->execute_remote_command( [ executeInDocker($this->deployment_uuid, "echo '$envs_base64' | base64 -d | tee $this->workdir/{$this->env_filename} > /dev/null"), ], - ); + + // Write .env file to configuration directory if ($this->use_build_server) { $this->server = $this->original_server; $this->execute_remote_command( @@ -1036,7 +1259,6 @@ private function save_environment_variables() ); } } - $this->environment_variables = $envs; } private function elixir_finetunes() @@ -1047,32 +1269,17 @@ private function elixir_finetunes() $envType = 'environment_variables_preview'; } $mix_env = $this->application->{$envType}->where('key', 'MIX_ENV')->first(); - if ($mix_env) { - if ($mix_env->is_build_time === false) { - $this->application_deployment_queue->addLogEntry('MIX_ENV environment variable is not set as build time.', type: 'error'); - $this->application_deployment_queue->addLogEntry('Please set MIX_ENV environment variable to be build time variable if you facing any issues with the deployment.', type: 'error'); - } - } else { + if (! $mix_env) { $this->application_deployment_queue->addLogEntry('MIX_ENV environment variable not found.', type: 'error'); $this->application_deployment_queue->addLogEntry('Please add MIX_ENV environment variable and set it to be build time variable if you facing any issues with the deployment.', type: 'error'); } $secret_key_base = $this->application->{$envType}->where('key', 'SECRET_KEY_BASE')->first(); - if ($secret_key_base) { - if ($secret_key_base->is_build_time === false) { - $this->application_deployment_queue->addLogEntry('SECRET_KEY_BASE environment variable is not set as build time.', type: 'error'); - $this->application_deployment_queue->addLogEntry('Please set SECRET_KEY_BASE environment variable to be build time variable if you facing any issues with the deployment.', type: 'error'); - } - } else { + if (! $secret_key_base) { $this->application_deployment_queue->addLogEntry('SECRET_KEY_BASE environment variable not found.', type: 'error'); $this->application_deployment_queue->addLogEntry('Please add SECRET_KEY_BASE environment variable and set it to be build time variable if you facing any issues with the deployment.', type: 'error'); } $database_url = $this->application->{$envType}->where('key', 'DATABASE_URL')->first(); - if ($database_url) { - if ($database_url->is_build_time === false) { - $this->application_deployment_queue->addLogEntry('DATABASE_URL environment variable is not set as build time.', type: 'error'); - $this->application_deployment_queue->addLogEntry('Please set DATABASE_URL environment variable to be build time variable if you facing any issues with the deployment.', type: 'error'); - } - } else { + if (! $database_url) { $this->application_deployment_queue->addLogEntry('DATABASE_URL environment variable not found.', type: 'error'); $this->application_deployment_queue->addLogEntry('Please add DATABASE_URL environment variable and set it to be build time variable if you facing any issues with the deployment.', type: 'error'); } @@ -1092,7 +1299,6 @@ private function laravel_finetunes() $nixpacks_php_fallback_path = new EnvironmentVariable; $nixpacks_php_fallback_path->key = 'NIXPACKS_PHP_FALLBACK_PATH'; $nixpacks_php_fallback_path->value = '/index.php'; - $nixpacks_php_fallback_path->is_build_time = false; $nixpacks_php_fallback_path->resourceable_id = $this->application->id; $nixpacks_php_fallback_path->resourceable_type = 'App\Models\Application'; $nixpacks_php_fallback_path->save(); @@ -1101,7 +1307,6 @@ private function laravel_finetunes() $nixpacks_php_root_dir = new EnvironmentVariable; $nixpacks_php_root_dir->key = 'NIXPACKS_PHP_ROOT_DIR'; $nixpacks_php_root_dir->value = '/app/public'; - $nixpacks_php_root_dir->is_build_time = false; $nixpacks_php_root_dir->resourceable_id = $this->application->id; $nixpacks_php_root_dir->resourceable_type = 'App\Models\Application'; $nixpacks_php_root_dir->save(); @@ -1112,6 +1317,7 @@ private function laravel_finetunes() private function rolling_update() { + $this->checkForCancellation(); if ($this->server->isSwarm()) { $this->application_deployment_queue->addLogEntry('Rolling update started.'); $this->execute_remote_command( @@ -1271,8 +1477,11 @@ private function deploy_pull_request() $this->add_build_env_variables_to_dockerfile(); } $this->build_image(); + // For Nixpacks, save runtime environment variables AFTER the build + if ($this->application->build_pack === 'nixpacks') { + $this->save_runtime_environment_variables(); + } $this->push_to_docker_registry(); - // $this->stop_running_container(); $this->rolling_update(); } @@ -1308,22 +1517,26 @@ private function create_workdir() private function prepare_builder_image() { + $this->checkForCancellation(); $settings = instanceSettings(); $helperImage = config('constants.coolify.helper_image'); $helperImage = "{$helperImage}:{$settings->helper_version}"; // Get user home directory $this->serverUserHomeDir = instant_remote_process(['echo $HOME'], $this->server); $this->dockerConfigFileExists = instant_remote_process(["test -f {$this->serverUserHomeDir}/.docker/config.json && echo 'OK' || echo 'NOK'"], $this->server); + + $env_flags = $this->generate_docker_env_flags_for_secrets(); + if ($this->use_build_server) { if ($this->dockerConfigFileExists === 'NOK') { throw new RuntimeException('Docker config file (~/.docker/config.json) not found on the build server. Please run "docker login" to login to the docker registry on the server.'); } - $runCommand = "docker run -d --name {$this->deployment_uuid} --rm -v {$this->serverUserHomeDir}/.docker/config.json:/root/.docker/config.json:ro -v /var/run/docker.sock:/var/run/docker.sock {$helperImage}"; + $runCommand = "docker run -d --name {$this->deployment_uuid} {$env_flags} --rm -v {$this->serverUserHomeDir}/.docker/config.json:/root/.docker/config.json:ro -v /var/run/docker.sock:/var/run/docker.sock {$helperImage}"; } else { if ($this->dockerConfigFileExists === 'OK') { - $runCommand = "docker run -d --network {$this->destination->network} --name {$this->deployment_uuid} --rm -v {$this->serverUserHomeDir}/.docker/config.json:/root/.docker/config.json:ro -v /var/run/docker.sock:/var/run/docker.sock {$helperImage}"; + $runCommand = "docker run -d --network {$this->destination->network} --name {$this->deployment_uuid} {$env_flags} --rm -v {$this->serverUserHomeDir}/.docker/config.json:/root/.docker/config.json:ro -v /var/run/docker.sock:/var/run/docker.sock {$helperImage}"; } else { - $runCommand = "docker run -d --network {$this->destination->network} --name {$this->deployment_uuid} --rm -v /var/run/docker.sock:/var/run/docker.sock {$helperImage}"; + $runCommand = "docker run -d --network {$this->destination->network} --name {$this->deployment_uuid} {$env_flags} --rm -v /var/run/docker.sock:/var/run/docker.sock {$helperImage}"; } } $this->application_deployment_queue->addLogEntry("Preparing container with helper image: $helperImage."); @@ -1531,6 +1744,7 @@ private function generate_nixpacks_confs() { $nixpacks_command = $this->nixpacks_build_cmd(); $this->application_deployment_queue->addLogEntry("Generating nixpacks configuration with: $nixpacks_command"); + $this->execute_remote_command( [executeInDocker($this->deployment_uuid, $nixpacks_command), 'save' => 'nixpacks_plan', 'hidden' => true], [executeInDocker($this->deployment_uuid, "nixpacks detect {$this->workdir}"), 'save' => 'nixpacks_type', 'hidden' => true], @@ -1547,9 +1761,10 @@ private function generate_nixpacks_confs() if ($this->nixpacks_plan) { $this->application_deployment_queue->addLogEntry("Found application type: {$this->nixpacks_type}."); $this->application_deployment_queue->addLogEntry("If you need further customization, please check the documentation of Nixpacks: https://nixpacks.com/docs/providers/{$this->nixpacks_type}"); - $parsed = Toml::Parse($this->nixpacks_plan); + $parsed = json_decode($this->nixpacks_plan, true); // Do any modifications here + // We need to generate envs here because nixpacks need to know to generate a proper Dockerfile $this->generate_env_variables(); $merged_envs = collect(data_get($parsed, 'variables', []))->merge($this->env_args); $aptPkgs = data_get($parsed, 'phases.setup.aptPkgs', []); @@ -1590,7 +1805,7 @@ private function generate_nixpacks_confs() private function nixpacks_build_cmd() { $this->generate_nixpacks_env_variables(); - $nixpacks_command = "nixpacks plan -f toml {$this->env_nixpacks_args}"; + $nixpacks_command = "nixpacks plan -f json {$this->env_nixpacks_args}"; if ($this->application->build_command) { $nixpacks_command .= " --build-cmd \"{$this->application->build_command}\""; } @@ -1622,6 +1837,12 @@ private function generate_nixpacks_env_variables() } } + // Add COOLIFY_* environment variables to Nixpacks build context + $coolify_envs = $this->generate_coolify_env_variables(); + $coolify_envs->each(function ($value, $key) { + $this->env_nixpacks_args->push("--env {$key}={$value}"); + }); + $this->env_nixpacks_args = $this->env_nixpacks_args->implode(' '); } @@ -1715,8 +1936,16 @@ private function generate_env_variables() $this->env_args = collect([]); $this->env_args->put('SOURCE_COMMIT', $this->commit); $coolify_envs = $this->generate_coolify_env_variables(); + + // For build process, include only environment variables where is_buildtime = true if ($this->pull_request_id === 0) { - foreach ($this->application->build_environment_variables as $env) { + // Get environment variables that are marked as available during build + $envs = $this->application->environment_variables() + ->where('key', 'not like', 'NIXPACKS_%') + ->where('is_buildtime', true) + ->get(); + + foreach ($envs as $env) { if (! is_null($env->real_value)) { $this->env_args->put($env->key, $env->real_value); if (str($env->real_value)->startsWith('$')) { @@ -1736,7 +1965,13 @@ private function generate_env_variables() } } } else { - foreach ($this->application->build_environment_variables_preview as $env) { + // Get preview environment variables that are marked as available during build + $envs = $this->application->environment_variables_preview() + ->where('key', 'not like', 'NIXPACKS_%') + ->where('is_buildtime', true) + ->get(); + + foreach ($envs as $env) { if (! is_null($env->real_value)) { $this->env_args->put($env->key, $env->real_value); if (str($env->real_value)->startsWith('$')) { @@ -1760,13 +1995,13 @@ private function generate_env_variables() private function generate_compose_file() { + $this->checkForCancellation(); $this->create_workdir(); $ports = $this->application->main_port(); $persistent_storages = $this->generate_local_persistent_volumes(); $persistent_file_volumes = $this->application->fileStorages()->get(); $volume_names = $this->generate_local_persistent_volumes_only_volume_names(); - // $environment_variables = $this->generate_environment_variables($ports); - $this->save_environment_variables(); + $this->generate_runtime_environment_variables(); if (data_get($this->application, 'custom_labels')) { $this->application->parseContainerLabels(); $labels = collect(preg_split("/\r\n|\n|\r/", base64_decode($this->application->custom_labels))); @@ -1835,7 +2070,7 @@ private function generate_compose_file() ], ], ]; - if (! is_null($this->env_filename)) { + if (filled($this->env_filename)) { $docker_compose['services'][$this->container_name]['env_file'] = [$this->env_filename]; } $docker_compose['services'][$this->container_name]['healthcheck'] = [ @@ -2006,7 +2241,7 @@ private function generate_local_persistent_volumes() $volume_name = $persistentStorage->name; } if ($this->pull_request_id !== 0) { - $volume_name = $volume_name.'-pr-'.$this->pull_request_id; + $volume_name = addPreviewDeploymentSuffix($volume_name, $this->pull_request_id); } $local_persistent_volumes[] = $volume_name.':'.$persistentStorage->mount_path; } @@ -2024,7 +2259,7 @@ private function generate_local_persistent_volumes_only_volume_names() $name = $persistentStorage->name; if ($this->pull_request_id !== 0) { - $name = $name.'-pr-'.$this->pull_request_id; + $name = addPreviewDeploymentSuffix($name, $this->pull_request_id); } $local_persistent_volumes_names[$name] = [ @@ -2072,16 +2307,74 @@ private function pull_latest_image($image) ); } + private function build_static_image() + { + $this->application_deployment_queue->addLogEntry('----------------------------------------'); + $this->application_deployment_queue->addLogEntry('Static deployment. Copying static assets to the image.'); + if ($this->application->static_image) { + $this->pull_latest_image($this->application->static_image); + } + $dockerfile = base64_encode("FROM {$this->application->static_image} + WORKDIR /usr/share/nginx/html/ + LABEL coolify.deploymentId={$this->deployment_uuid} + COPY . . + RUN rm -f /usr/share/nginx/html/nginx.conf + RUN rm -f /usr/share/nginx/html/Dockerfile + RUN rm -f /usr/share/nginx/html/docker-compose.yaml + RUN rm -f /usr/share/nginx/html/.env + COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); + if (str($this->application->custom_nginx_configuration)->isNotEmpty()) { + $nginx_config = base64_encode($this->application->custom_nginx_configuration); + } else { + if ($this->application->settings->is_spa) { + $nginx_config = base64_encode(defaultNginxConfiguration('spa')); + } else { + $nginx_config = base64_encode(defaultNginxConfiguration()); + } + } + $build_command = "docker build {$this->addHosts} --network host -f {$this->workdir}/Dockerfile --progress plain -t {$this->production_image_name} {$this->workdir}"; + $base64_build_command = base64_encode($build_command); + $this->execute_remote_command( + [ + executeInDocker($this->deployment_uuid, "echo '{$dockerfile}' | base64 -d | tee {$this->workdir}/Dockerfile > /dev/null"), + ], + [ + executeInDocker($this->deployment_uuid, "echo '{$nginx_config}' | base64 -d | tee {$this->workdir}/nginx.conf > /dev/null"), + ], + [ + executeInDocker($this->deployment_uuid, "echo '{$base64_build_command}' | base64 -d | tee /artifacts/build.sh > /dev/null"), + 'hidden' => true, + ], + [ + executeInDocker($this->deployment_uuid, 'cat /artifacts/build.sh'), + 'hidden' => true, + ], + [ + executeInDocker($this->deployment_uuid, 'bash /artifacts/build.sh'), + 'hidden' => true, + ] + ); + $this->application_deployment_queue->addLogEntry('Building docker image completed.'); + } + private function build_image() { - // Add Coolify related variables to the build args - $this->environment_variables->filter(function ($key, $value) { - return str($key)->startsWith('COOLIFY_'); - })->each(function ($key, $value) { - $this->build_args->push("--build-arg '{$key}'"); - }); + // Add Coolify related variables to the build args/secrets + if ($this->dockerBuildkitSupported) { + // Coolify variables are already included in the secrets from generate_build_env_variables + // build_secrets is already a string at this point + } else { + // Traditional build args approach + $this->environment_variables->filter(function ($key, $value) { + return str($key)->startsWith('COOLIFY_'); + })->each(function ($key, $value) { + $this->build_args->push("--build-arg '{$key}'"); + }); - $this->build_args = $this->build_args->implode(' '); + $this->build_args = $this->build_args instanceof \Illuminate\Support\Collection + ? $this->build_args->implode(' ') + : (string) $this->build_args; + } $this->application_deployment_queue->addLogEntry('----------------------------------------'); if ($this->disableBuildCache) { @@ -2094,100 +2387,114 @@ private function build_image() $this->application_deployment_queue->addLogEntry('To check the current progress, click on Show Debug Logs.'); } - if ($this->application->settings->is_static || $this->application->build_pack === 'static') { + if ($this->application->settings->is_static) { if ($this->application->static_image) { $this->pull_latest_image($this->application->static_image); $this->application_deployment_queue->addLogEntry('Continuing with the building process.'); } - if ($this->application->build_pack === 'static') { - $dockerfile = base64_encode("FROM {$this->application->static_image} -WORKDIR /usr/share/nginx/html/ -LABEL coolify.deploymentId={$this->deployment_uuid} -COPY . . -RUN rm -f /usr/share/nginx/html/nginx.conf -RUN rm -f /usr/share/nginx/html/Dockerfile -RUN rm -f /usr/share/nginx/html/docker-compose.yaml -RUN rm -f /usr/share/nginx/html/.env -COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); - if (str($this->application->custom_nginx_configuration)->isNotEmpty()) { - $nginx_config = base64_encode($this->application->custom_nginx_configuration); - } else { - if ($this->application->settings->is_spa) { - $nginx_config = base64_encode(defaultNginxConfiguration('spa')); + if ($this->application->build_pack === 'nixpacks') { + $this->nixpacks_plan = base64_encode($this->nixpacks_plan); + $this->execute_remote_command([executeInDocker($this->deployment_uuid, "echo '{$this->nixpacks_plan}' | base64 -d | tee /artifacts/thegameplan.json > /dev/null"), 'hidden' => true]); + if ($this->force_rebuild) { + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "nixpacks build -c /artifacts/thegameplan.json --no-cache --no-error-without-start -n {$this->build_image_name} {$this->workdir} -o {$this->workdir}"), + 'hidden' => true, + ], [ + executeInDocker($this->deployment_uuid, "cat {$this->workdir}/.nixpacks/Dockerfile"), + 'hidden' => true, + ]); + if ($this->dockerBuildkitSupported && $this->application->settings->use_build_secrets) { + // Modify the nixpacks Dockerfile to use build secrets + $this->modify_dockerfile_for_secrets("{$this->workdir}/.nixpacks/Dockerfile"); + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + $build_command = "DOCKER_BUILDKIT=1 docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile{$secrets_flags} --progress plain -t {$this->build_image_name} {$this->workdir}"; + } elseif ($this->dockerBuildkitSupported) { + // BuildKit without secrets + $build_command = "DOCKER_BUILDKIT=1 docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile --progress plain -t {$this->build_image_name} {$this->build_args} {$this->workdir}"; } else { - $nginx_config = base64_encode(defaultNginxConfiguration()); + $build_command = "docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile --progress plain -t {$this->build_image_name} {$this->build_args} {$this->workdir}"; + } + } else { + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "nixpacks build -c /artifacts/thegameplan.json --cache-key '{$this->application->uuid}' --no-error-without-start -n {$this->build_image_name} {$this->workdir} -o {$this->workdir}"), + 'hidden' => true, + ], [ + executeInDocker($this->deployment_uuid, "cat {$this->workdir}/.nixpacks/Dockerfile"), + 'hidden' => true, + ]); + if ($this->dockerBuildkitSupported) { + // Modify the nixpacks Dockerfile to use build secrets + $this->modify_dockerfile_for_secrets("{$this->workdir}/.nixpacks/Dockerfile"); + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + $build_command = "DOCKER_BUILDKIT=1 docker build {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile{$secrets_flags} --progress plain -t {$this->build_image_name} {$this->workdir}"; + } else { + $build_command = "docker build {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile --progress plain -t {$this->build_image_name} {$this->build_args} {$this->workdir}"; } } - } else { - if ($this->application->build_pack === 'nixpacks') { - $this->nixpacks_plan = base64_encode($this->nixpacks_plan); - $this->execute_remote_command([executeInDocker($this->deployment_uuid, "echo '{$this->nixpacks_plan}' | base64 -d | tee /artifacts/thegameplan.json > /dev/null"), 'hidden' => true]); - if ($this->force_rebuild) { - $this->execute_remote_command([ - executeInDocker($this->deployment_uuid, "nixpacks build -c /artifacts/thegameplan.json --no-cache --no-error-without-start -n {$this->build_image_name} {$this->workdir} -o {$this->workdir}"), - 'hidden' => true, - ]); - $build_command = "docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile {$this->build_args} --progress plain -t {$this->build_image_name} {$this->workdir}"; - } else { - $this->execute_remote_command([ - executeInDocker($this->deployment_uuid, "nixpacks build -c /artifacts/thegameplan.json --cache-key '{$this->application->uuid}' --no-error-without-start -n {$this->build_image_name} {$this->workdir} -o {$this->workdir}"), - 'hidden' => true, - ]); - $build_command = "docker build {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile {$this->build_args} --progress plain -t {$this->build_image_name} {$this->workdir}"; - } - $base64_build_command = base64_encode($build_command); - $this->execute_remote_command( - [ - executeInDocker($this->deployment_uuid, "echo '{$base64_build_command}' | base64 -d | tee /artifacts/build.sh > /dev/null"), - 'hidden' => true, - ], - [ - executeInDocker($this->deployment_uuid, 'cat /artifacts/build.sh'), - 'hidden' => true, - ], - [ - executeInDocker($this->deployment_uuid, 'bash /artifacts/build.sh'), - 'hidden' => true, - ] - ); - $this->execute_remote_command([executeInDocker($this->deployment_uuid, 'rm /artifacts/thegameplan.json'), 'hidden' => true]); + $base64_build_command = base64_encode($build_command); + $this->execute_remote_command( + [ + executeInDocker($this->deployment_uuid, "echo '{$base64_build_command}' | base64 -d | tee /artifacts/build.sh > /dev/null"), + 'hidden' => true, + ], + [ + executeInDocker($this->deployment_uuid, 'cat /artifacts/build.sh'), + 'hidden' => true, + ], + [ + executeInDocker($this->deployment_uuid, 'bash /artifacts/build.sh'), + 'hidden' => true, + ] + ); + $this->execute_remote_command([executeInDocker($this->deployment_uuid, 'rm /artifacts/thegameplan.json'), 'hidden' => true]); + } else { + // Dockerfile buildpack + if ($this->dockerBuildkitSupported && $this->application->settings->use_build_secrets) { + // Modify the Dockerfile to use build secrets + $this->modify_dockerfile_for_secrets("{$this->workdir}{$this->dockerfile_location}"); + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + if ($this->force_rebuild) { + $build_command = "DOCKER_BUILDKIT=1 docker build --no-cache {$this->buildTarget} --network {$this->destination->network} -f {$this->workdir}{$this->dockerfile_location}{$secrets_flags} --progress plain -t $this->build_image_name {$this->workdir}"; + } else { + $build_command = "DOCKER_BUILDKIT=1 docker build {$this->buildTarget} --network {$this->destination->network} -f {$this->workdir}{$this->dockerfile_location}{$secrets_flags} --progress plain -t $this->build_image_name {$this->workdir}"; + } } else { + // Traditional build with args if ($this->force_rebuild) { $build_command = "docker build --no-cache {$this->buildTarget} --network {$this->destination->network} -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t $this->build_image_name {$this->workdir}"; - $base64_build_command = base64_encode($build_command); } else { $build_command = "docker build {$this->buildTarget} --network {$this->destination->network} -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t $this->build_image_name {$this->workdir}"; - $base64_build_command = base64_encode($build_command); } - $this->execute_remote_command( - [ - executeInDocker($this->deployment_uuid, "echo '{$base64_build_command}' | base64 -d | tee /artifacts/build.sh > /dev/null"), - 'hidden' => true, - ], - [ - executeInDocker($this->deployment_uuid, 'cat /artifacts/build.sh'), - 'hidden' => true, - ], - [ - executeInDocker($this->deployment_uuid, 'bash /artifacts/build.sh'), - 'hidden' => true, - ] - ); } - $dockerfile = base64_encode("FROM {$this->application->static_image} + $base64_build_command = base64_encode($build_command); + $this->execute_remote_command( + [ + executeInDocker($this->deployment_uuid, "echo '{$base64_build_command}' | base64 -d | tee /artifacts/build.sh > /dev/null"), + 'hidden' => true, + ], + [ + executeInDocker($this->deployment_uuid, 'cat /artifacts/build.sh'), + 'hidden' => true, + ], + [ + executeInDocker($this->deployment_uuid, 'bash /artifacts/build.sh'), + 'hidden' => true, + ] + ); + } + $dockerfile = base64_encode("FROM {$this->application->static_image} WORKDIR /usr/share/nginx/html/ LABEL coolify.deploymentId={$this->deployment_uuid} COPY --from=$this->build_image_name /app/{$this->application->publish_directory} . COPY ./nginx.conf /etc/nginx/conf.d/default.conf"); - if (str($this->application->custom_nginx_configuration)->isNotEmpty()) { - $nginx_config = base64_encode($this->application->custom_nginx_configuration); + if (str($this->application->custom_nginx_configuration)->isNotEmpty()) { + $nginx_config = base64_encode($this->application->custom_nginx_configuration); + } else { + if ($this->application->settings->is_spa) { + $nginx_config = base64_encode(defaultNginxConfiguration('spa')); } else { - if ($this->application->settings->is_spa) { - $nginx_config = base64_encode(defaultNginxConfiguration('spa')); - } else { - $nginx_config = base64_encode(defaultNginxConfiguration()); - } + $nginx_config = base64_encode(defaultNginxConfiguration()); } } $build_command = "docker build {$this->addHosts} --network host -f {$this->workdir}/Dockerfile {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; @@ -2215,10 +2522,22 @@ private function build_image() } else { // Pure Dockerfile based deployment if ($this->application->dockerfile) { - if ($this->force_rebuild) { - $build_command = "docker build --no-cache --pull {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + if ($this->dockerBuildkitSupported && $this->application->settings->use_build_secrets) { + // Modify the Dockerfile to use build secrets + $this->modify_dockerfile_for_secrets("{$this->workdir}{$this->dockerfile_location}"); + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + if ($this->force_rebuild) { + $build_command = "DOCKER_BUILDKIT=1 docker build --no-cache --pull {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location}{$secrets_flags} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } else { + $build_command = "DOCKER_BUILDKIT=1 docker build --pull {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location}{$secrets_flags} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } } else { - $build_command = "docker build --pull {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + // Traditional build with args + if ($this->force_rebuild) { + $build_command = "docker build --no-cache --pull {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } else { + $build_command = "docker build --pull {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } } $base64_build_command = base64_encode($build_command); $this->execute_remote_command( @@ -2243,14 +2562,34 @@ private function build_image() $this->execute_remote_command([ executeInDocker($this->deployment_uuid, "nixpacks build -c /artifacts/thegameplan.json --no-cache --no-error-without-start -n {$this->production_image_name} {$this->workdir} -o {$this->workdir}"), 'hidden' => true, + ], [ + executeInDocker($this->deployment_uuid, "cat {$this->workdir}/.nixpacks/Dockerfile"), + 'hidden' => true, ]); - $build_command = "docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + if ($this->dockerBuildkitSupported) { + // Modify the nixpacks Dockerfile to use build secrets + $this->modify_dockerfile_for_secrets("{$this->workdir}/.nixpacks/Dockerfile"); + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + $build_command = "DOCKER_BUILDKIT=1 docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile{$secrets_flags} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } else { + $build_command = "docker build --no-cache {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile --progress plain -t {$this->production_image_name} {$this->build_args} {$this->workdir}"; + } } else { $this->execute_remote_command([ executeInDocker($this->deployment_uuid, "nixpacks build -c /artifacts/thegameplan.json --cache-key '{$this->application->uuid}' --no-error-without-start -n {$this->production_image_name} {$this->workdir} -o {$this->workdir}"), 'hidden' => true, + ], [ + executeInDocker($this->deployment_uuid, "cat {$this->workdir}/.nixpacks/Dockerfile"), + 'hidden' => true, ]); - $build_command = "docker build {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + if ($this->dockerBuildkitSupported) { + // Modify the nixpacks Dockerfile to use build secrets + $this->modify_dockerfile_for_secrets("{$this->workdir}/.nixpacks/Dockerfile"); + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + $build_command = "DOCKER_BUILDKIT=1 docker build {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile{$secrets_flags} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } else { + $build_command = "docker build {$this->addHosts} --network host -f {$this->workdir}/.nixpacks/Dockerfile --progress plain -t {$this->production_image_name} {$this->build_args} {$this->workdir}"; + } } $base64_build_command = base64_encode($build_command); $this->execute_remote_command( @@ -2269,13 +2608,24 @@ private function build_image() ); $this->execute_remote_command([executeInDocker($this->deployment_uuid, 'rm /artifacts/thegameplan.json'), 'hidden' => true]); } else { - if ($this->force_rebuild) { - $build_command = "docker build --no-cache {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; - $base64_build_command = base64_encode($build_command); + // Dockerfile buildpack + if ($this->dockerBuildkitSupported) { + // Use BuildKit with secrets + $secrets_flags = $this->build_secrets ? " {$this->build_secrets}" : ''; + if ($this->force_rebuild) { + $build_command = "DOCKER_BUILDKIT=1 docker build --no-cache {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location}{$secrets_flags} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } else { + $build_command = "DOCKER_BUILDKIT=1 docker build {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location}{$secrets_flags} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } } else { - $build_command = "docker build {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; - $base64_build_command = base64_encode($build_command); + // Traditional build with args + if ($this->force_rebuild) { + $build_command = "docker build --no-cache {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } else { + $build_command = "docker build {$this->buildTarget} {$this->addHosts} --network host -f {$this->workdir}{$this->dockerfile_location} {$this->build_args} --progress plain -t {$this->production_image_name} {$this->workdir}"; + } } + $base64_build_command = base64_encode($build_command); $this->execute_remote_command( [ executeInDocker($this->deployment_uuid, "echo '{$base64_build_command}' | base64 -d | tee /artifacts/build.sh > /dev/null"), @@ -2319,7 +2669,7 @@ private function stop_running_container(bool $force = false) $containers = getCurrentApplicationContainerStatus($this->server, $this->application->id, $this->pull_request_id); if ($this->pull_request_id === 0) { $containers = $containers->filter(function ($container) { - return data_get($container, 'Names') !== $this->container_name && data_get($container, 'Names') !== $this->container_name.'-pr-'.$this->pull_request_id; + return data_get($container, 'Names') !== $this->container_name && data_get($container, 'Names') !== addPreviewDeploymentSuffix($this->container_name, $this->pull_request_id); }); } $containers->each(function ($container) { @@ -2362,6 +2712,50 @@ private function start_by_compose_file() $this->application_deployment_queue->addLogEntry('New container started.'); } + private function analyzeBuildTimeVariables($variables) + { + $userDefinedVariables = collect([]); + + $dbVariables = $this->pull_request_id === 0 + ? $this->application->environment_variables() + ->where('is_buildtime', true) + ->pluck('key') + : $this->application->environment_variables_preview() + ->where('is_buildtime', true) + ->pluck('key'); + + foreach ($variables as $key => $value) { + if ($dbVariables->contains($key)) { + $userDefinedVariables->put($key, $value); + } + } + + if ($userDefinedVariables->isEmpty()) { + return; + } + + $variablesArray = $userDefinedVariables->toArray(); + $warnings = self::analyzeBuildVariables($variablesArray); + + if (empty($warnings)) { + return; + } + $this->application_deployment_queue->addLogEntry('----------------------------------------'); + foreach ($warnings as $warning) { + $messages = self::formatBuildWarning($warning); + foreach ($messages as $message) { + $this->application_deployment_queue->addLogEntry($message, type: 'warning'); + } + $this->application_deployment_queue->addLogEntry(''); + } + + // Add general advice + $this->application_deployment_queue->addLogEntry('💡 Tips to resolve build issues:', type: 'info'); + $this->application_deployment_queue->addLogEntry(' 1. Set these variables as "Runtime only" in the environment variables settings', type: 'info'); + $this->application_deployment_queue->addLogEntry(' 2. Use different values for build-time (e.g., NODE_ENV=development for build)', type: 'info'); + $this->application_deployment_queue->addLogEntry(' 3. Consider using multi-stage Docker builds to separate build and runtime environments', type: 'info'); + } + private function generate_build_env_variables() { if ($this->application->build_pack === 'nixpacks') { @@ -2371,43 +2765,463 @@ private function generate_build_env_variables() $variables = collect([])->merge($this->env_args); } - $this->build_args = $variables->map(function ($value, $key) { - $value = escapeshellarg($value); + // Analyze build variables for potential issues + if ($variables->isNotEmpty()) { + $this->analyzeBuildTimeVariables($variables); + } - return "--build-arg {$key}={$value}"; + if ($this->dockerBuildkitSupported && $this->application->settings->use_build_secrets) { + $this->generate_build_secrets($variables); + $this->build_args = ''; + } else { + $secrets_hash = ''; + if ($variables->isNotEmpty()) { + $secrets_hash = $this->generate_secrets_hash($variables); + } + + $env_vars = $this->pull_request_id === 0 + ? $this->application->environment_variables()->where('is_buildtime', true)->get() + : $this->application->environment_variables_preview()->where('is_buildtime', true)->get(); + + // Map variables to include is_multiline flag + $vars_with_metadata = $variables->map(function ($value, $key) use ($env_vars) { + $env = $env_vars->firstWhere('key', $key); + + return [ + 'key' => $key, + 'value' => $value, + 'is_multiline' => $env ? $env->is_multiline : false, + ]; + }); + + $this->build_args = generateDockerBuildArgs($vars_with_metadata); + + if ($secrets_hash) { + $this->build_args->push("--build-arg COOLIFY_BUILD_SECRETS_HASH={$secrets_hash}"); + } + } + } + + private function generate_docker_env_flags_for_secrets() + { + // Only generate env flags if build secrets are enabled + if (! $this->application->settings->use_build_secrets) { + return ''; + } + + $variables = $this->pull_request_id === 0 + ? $this->application->environment_variables()->where('key', 'not like', 'NIXPACKS_%')->where('is_buildtime', true)->get() + : $this->application->environment_variables_preview()->where('key', 'not like', 'NIXPACKS_%')->where('is_buildtime', true)->get(); + + if ($variables->isEmpty()) { + return ''; + } + + $secrets_hash = $this->generate_secrets_hash($variables); + + // Map to simple array format for the helper function + $vars_array = $variables->map(function ($env) { + return [ + 'key' => $env->key, + 'value' => $env->real_value, + 'is_multiline' => $env->is_multiline, + ]; }); + + $env_flags = generateDockerEnvFlags($vars_array); + $env_flags .= " -e COOLIFY_BUILD_SECRETS_HASH={$secrets_hash}"; + + return $env_flags; + } + + private function generate_build_secrets(Collection $variables) + { + if ($variables->isEmpty()) { + $this->build_secrets = ''; + + return; + } + + $this->build_secrets = $variables + ->map(function ($value, $key) { + return "--secret id={$key},env={$key}"; + }) + ->implode(' '); + + $this->build_secrets .= ' --secret id=COOLIFY_BUILD_SECRETS_HASH,env=COOLIFY_BUILD_SECRETS_HASH'; + } + + private function generate_secrets_hash($variables) + { + if (! $this->secrets_hash_key) { + $this->secrets_hash_key = bin2hex(random_bytes(32)); + } + + if ($variables instanceof Collection) { + $secrets_string = $variables + ->mapWithKeys(function ($value, $key) { + return [$key => $value]; + }) + ->sortKeys() + ->map(function ($value, $key) { + return "{$key}={$value}"; + }) + ->implode('|'); + } else { + $secrets_string = $variables + ->map(function ($env) { + return "{$env->key}={$env->real_value}"; + }) + ->sort() + ->implode('|'); + } + + return hash_hmac('sha256', $secrets_string, $this->secrets_hash_key); } private function add_build_env_variables_to_dockerfile() { - $this->execute_remote_command([ - executeInDocker($this->deployment_uuid, "cat {$this->workdir}{$this->dockerfile_location}"), - 'hidden' => true, - 'save' => 'dockerfile', - ]); - $dockerfile = collect(str($this->saved_outputs->get('dockerfile'))->trim()->explode("\n")); - if ($this->pull_request_id === 0) { - foreach ($this->application->build_environment_variables as $env) { - if (data_get($env, 'is_multiline') === true) { - $dockerfile->splice(1, 0, "ARG {$env->key}"); - } else { - $dockerfile->splice(1, 0, "ARG {$env->key}={$env->real_value}"); + if ($this->dockerBuildkitSupported) { + // We dont need to add build secrets to dockerfile for buildkit, as we already added them with --secret flag in function generate_docker_env_flags_for_secrets + } else { + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "cat {$this->workdir}{$this->dockerfile_location}"), + 'hidden' => true, + 'save' => 'dockerfile', + ]); + $dockerfile = collect(str($this->saved_outputs->get('dockerfile'))->trim()->explode("\n")); + + if ($this->pull_request_id === 0) { + // Only add environment variables that are available during build + $envs = $this->application->environment_variables() + ->where('key', 'not like', 'NIXPACKS_%') + ->where('is_buildtime', true) + ->get(); + foreach ($envs as $env) { + if (data_get($env, 'is_multiline') === true) { + $dockerfile->splice(1, 0, ["ARG {$env->key}"]); + } else { + $dockerfile->splice(1, 0, ["ARG {$env->key}={$env->real_value}"]); + } + } + // Add Coolify variables as ARGs + if ($this->coolify_variables) { + $coolify_vars = collect(explode(' ', trim($this->coolify_variables))) + ->filter() + ->map(function ($var) { + return "ARG {$var}"; + }); + foreach ($coolify_vars as $arg) { + $dockerfile->splice(1, 0, [$arg]); + } + } + } else { + // Only add preview environment variables that are available during build + $envs = $this->application->environment_variables_preview() + ->where('key', 'not like', 'NIXPACKS_%') + ->where('is_buildtime', true) + ->get(); + foreach ($envs as $env) { + if (data_get($env, 'is_multiline') === true) { + $dockerfile->splice(1, 0, ["ARG {$env->key}"]); + } else { + $dockerfile->splice(1, 0, ["ARG {$env->key}={$env->real_value}"]); + } + } + // Add Coolify variables as ARGs + if ($this->coolify_variables) { + $coolify_vars = collect(explode(' ', trim($this->coolify_variables))) + ->filter() + ->map(function ($var) { + return "ARG {$var}"; + }); + foreach ($coolify_vars as $arg) { + $dockerfile->splice(1, 0, [$arg]); + } } } - } else { - foreach ($this->application->build_environment_variables_preview as $env) { - if (data_get($env, 'is_multiline') === true) { - $dockerfile->splice(1, 0, "ARG {$env->key}"); - } else { - $dockerfile->splice(1, 0, "ARG {$env->key}={$env->real_value}"); + + if ($envs->isNotEmpty()) { + $secrets_hash = $this->generate_secrets_hash($envs); + $dockerfile->splice(1, 0, ["ARG COOLIFY_BUILD_SECRETS_HASH={$secrets_hash}"]); + } + + $dockerfile_base64 = base64_encode($dockerfile->implode("\n")); + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "echo '{$dockerfile_base64}' | base64 -d | tee {$this->workdir}{$this->dockerfile_location} > /dev/null"), + 'hidden' => true, + ]); + } + } + + private function modify_dockerfile_for_secrets($dockerfile_path) + { + // Only process if build secrets are enabled and we have secrets to mount + if (! $this->application->settings->use_build_secrets || empty($this->build_secrets)) { + return; + } + + // Read the Dockerfile + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "cat {$dockerfile_path}"), + 'hidden' => true, + 'save' => 'dockerfile_content', + ]); + + $dockerfile = str($this->saved_outputs->get('dockerfile_content'))->trim()->explode("\n"); + + // Add BuildKit syntax directive if not present + if (! str_starts_with($dockerfile->first(), '# syntax=')) { + $dockerfile->prepend('# syntax=docker/dockerfile:1'); + } + + // Get environment variables for secrets + $variables = $this->pull_request_id === 0 + ? $this->application->environment_variables()->where('key', 'not like', 'NIXPACKS_%')->where('is_buildtime', true)->get() + : $this->application->environment_variables_preview()->where('key', 'not like', 'NIXPACKS_%')->where('is_buildtime', true)->get(); + if ($variables->isEmpty()) { + return; + } + + // Generate mount strings for all secrets + $mountStrings = $variables->map(fn ($env) => "--mount=type=secret,id={$env->key},env={$env->key}")->implode(' '); + + // Add mount for the secrets hash to ensure cache invalidation + $mountStrings .= ' --mount=type=secret,id=COOLIFY_BUILD_SECRETS_HASH,env=COOLIFY_BUILD_SECRETS_HASH'; + + $modified = false; + $dockerfile = $dockerfile->map(function ($line) use ($mountStrings, &$modified) { + $trimmed = ltrim($line); + + // Skip lines that already have secret mounts or are not RUN commands + if (str_contains($line, '--mount=type=secret') || ! str_starts_with($trimmed, 'RUN')) { + return $line; + } + + // Add mount strings to RUN command + $originalCommand = trim(substr($trimmed, 3)); + $modified = true; + + return "RUN {$mountStrings} {$originalCommand}"; + }); + + if ($modified) { + // Write the modified Dockerfile back + $dockerfile_base64 = base64_encode($dockerfile->implode("\n")); + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "echo '{$dockerfile_base64}' | base64 -d | tee {$dockerfile_path} > /dev/null"), + 'hidden' => true, + ]); + + $this->application_deployment_queue->addLogEntry('Modified Dockerfile to use build secrets.'); + } + } + + private function modify_dockerfiles_for_compose($composeFile) + { + if ($this->application->build_pack !== 'dockercompose') { + return; + } + + $variables = $this->pull_request_id === 0 + ? $this->application->environment_variables() + ->where('key', 'not like', 'NIXPACKS_%') + ->where('is_buildtime', true) + ->get() + : $this->application->environment_variables_preview() + ->where('key', 'not like', 'NIXPACKS_%') + ->where('is_buildtime', true) + ->get(); + + if ($variables->isEmpty()) { + $this->application_deployment_queue->addLogEntry('No build-time variables to add to Dockerfiles.'); + + return; + } + + $services = data_get($composeFile, 'services', []); + + foreach ($services as $serviceName => $service) { + if (! isset($service['build'])) { + continue; + } + + $context = '.'; + $dockerfile = 'Dockerfile'; + + if (is_string($service['build'])) { + $context = $service['build']; + } elseif (is_array($service['build'])) { + $context = data_get($service['build'], 'context', '.'); + $dockerfile = data_get($service['build'], 'dockerfile', 'Dockerfile'); + } + + $dockerfilePath = rtrim($context, '/').'/'.ltrim($dockerfile, '/'); + if (str_starts_with($dockerfilePath, './')) { + $dockerfilePath = substr($dockerfilePath, 2); + } + if (str_starts_with($dockerfilePath, '/')) { + $dockerfilePath = substr($dockerfilePath, 1); + } + + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "test -f {$this->workdir}/{$dockerfilePath} && echo 'exists' || echo 'not found'"), + 'hidden' => true, + 'save' => 'dockerfile_check_'.$serviceName, + ]); + + if (str($this->saved_outputs->get('dockerfile_check_'.$serviceName))->trim()->toString() !== 'exists') { + $this->application_deployment_queue->addLogEntry("Dockerfile not found for service {$serviceName} at {$dockerfilePath}, skipping ARG injection."); + + continue; + } + + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "cat {$this->workdir}/{$dockerfilePath}"), + 'hidden' => true, + 'save' => 'dockerfile_content_'.$serviceName, + ]); + + $dockerfileContent = $this->saved_outputs->get('dockerfile_content_'.$serviceName); + if (! $dockerfileContent) { + continue; + } + + $dockerfile_lines = collect(str($dockerfileContent)->trim()->explode("\n")); + + $fromIndices = []; + $dockerfile_lines->each(function ($line, $index) use (&$fromIndices) { + if (str($line)->trim()->startsWith('FROM')) { + $fromIndices[] = $index; + } + }); + + if (empty($fromIndices)) { + $this->application_deployment_queue->addLogEntry("No FROM instruction found in Dockerfile for service {$serviceName}, skipping."); + + continue; + } + + $isMultiStage = count($fromIndices) > 1; + + $argsToAdd = collect([]); + foreach ($variables as $env) { + $argsToAdd->push("ARG {$env->key}"); + } + + ray($argsToAdd); + if ($argsToAdd->isEmpty()) { + $this->application_deployment_queue->addLogEntry("Service {$serviceName}: No build-time variables to add."); + + continue; + } + + $totalAdded = 0; + $offset = 0; + + foreach ($fromIndices as $stageIndex => $fromIndex) { + $adjustedIndex = $fromIndex + $offset; + + $stageStart = $adjustedIndex + 1; + $stageEnd = isset($fromIndices[$stageIndex + 1]) + ? $fromIndices[$stageIndex + 1] + $offset + : $dockerfile_lines->count(); + + $existingStageArgs = collect([]); + for ($i = $stageStart; $i < $stageEnd; $i++) { + $line = $dockerfile_lines->get($i); + if (! $line || ! str($line)->trim()->startsWith('ARG')) { + break; + } + $parts = explode(' ', trim($line), 2); + if (count($parts) >= 2) { + $argPart = $parts[1]; + $keyValue = explode('=', $argPart, 2); + $existingStageArgs->push($keyValue[0]); + } + } + + $stageArgsToAdd = $argsToAdd->filter(function ($arg) use ($existingStageArgs) { + $key = str($arg)->after('ARG ')->trim()->toString(); + + return ! $existingStageArgs->contains($key); + }); + + if ($stageArgsToAdd->isNotEmpty()) { + $dockerfile_lines->splice($adjustedIndex + 1, 0, $stageArgsToAdd->toArray()); + $totalAdded += $stageArgsToAdd->count(); + $offset += $stageArgsToAdd->count(); + } + } + + if ($totalAdded > 0) { + $dockerfile_base64 = base64_encode($dockerfile_lines->implode("\n")); + $this->execute_remote_command([ + executeInDocker($this->deployment_uuid, "echo '{$dockerfile_base64}' | base64 -d | tee {$this->workdir}/{$dockerfilePath} > /dev/null"), + 'hidden' => true, + ]); + + $stageInfo = $isMultiStage ? ' (multi-stage build, added to '.count($fromIndices).' stages)' : ''; + $this->application_deployment_queue->addLogEntry("Added {$totalAdded} ARG declarations to Dockerfile for service {$serviceName}{$stageInfo}."); + } else { + $this->application_deployment_queue->addLogEntry("Service {$serviceName}: All required ARG declarations already exist."); + } + + if ($this->application->settings->use_build_secrets && $this->dockerBuildkitSupported && ! empty($this->build_secrets)) { + $fullDockerfilePath = "{$this->workdir}/{$dockerfilePath}"; + $this->modify_dockerfile_for_secrets($fullDockerfilePath); + $this->application_deployment_queue->addLogEntry("Modified Dockerfile for service {$serviceName} to use build secrets."); + } + } + } + + private function add_build_secrets_to_compose($composeFile) + { + // Get environment variables for secrets + $variables = $this->pull_request_id === 0 + ? $this->application->environment_variables()->where('key', 'not like', 'NIXPACKS_%')->get() + : $this->application->environment_variables_preview()->where('key', 'not like', 'NIXPACKS_%')->get(); + + if ($variables->isEmpty()) { + return $composeFile; + } + + $secrets = []; + foreach ($variables as $env) { + $secrets[$env->key] = [ + 'environment' => $env->key, + ]; + } + + $services = data_get($composeFile, 'services', []); + foreach ($services as $serviceName => &$service) { + if (isset($service['build'])) { + if (is_string($service['build'])) { + $service['build'] = [ + 'context' => $service['build'], + ]; + } + if (! isset($service['build']['secrets'])) { + $service['build']['secrets'] = []; + } + foreach ($variables as $env) { + if (! in_array($env->key, $service['build']['secrets'])) { + $service['build']['secrets'][] = $env->key; + } } } } - $dockerfile_base64 = base64_encode($dockerfile->implode("\n")); - $this->execute_remote_command([ - executeInDocker($this->deployment_uuid, "echo '{$dockerfile_base64}' | base64 -d | tee {$this->workdir}{$this->dockerfile_location} > /dev/null"), - 'hidden' => true, - ]); + + $composeFile['services'] = $services; + $existingSecrets = data_get($composeFile, 'secrets', []); + if ($existingSecrets instanceof \Illuminate\Support\Collection) { + $existingSecrets = $existingSecrets->toArray(); + } + $composeFile['secrets'] = array_replace($existingSecrets, $secrets); + + $this->application_deployment_queue->addLogEntry('Added build secrets configuration to docker-compose file (using environment variables).'); + + return $composeFile; } private function run_pre_deployment_command() @@ -2475,9 +3289,22 @@ private function run_post_deployment_command() throw new RuntimeException('Post-deployment command: Could not find a valid container. Is the container name correct?'); } + /** + * Check if the deployment was cancelled and abort if it was + */ + private function checkForCancellation(): void + { + $this->application_deployment_queue->refresh(); + if ($this->application_deployment_queue->status === ApplicationDeploymentStatus::CANCELLED_BY_USER->value) { + $this->application_deployment_queue->addLogEntry('Deployment cancelled by user, stopping execution.'); + throw new \RuntimeException('Deployment cancelled by user', 69420); + } + } + private function next(string $status) { - queue_next_deployment($this->application); + // Refresh to get latest status + $this->application_deployment_queue->refresh(); // Never allow changing status from FAILED or CANCELLED_BY_USER to anything else if ($this->application_deployment_queue->status === ApplicationDeploymentStatus::FAILED->value) { @@ -2486,14 +3313,21 @@ private function next(string $status) return; } if ($this->application_deployment_queue->status === ApplicationDeploymentStatus::CANCELLED_BY_USER->value) { - return; + // Job was cancelled, stop execution + $this->application_deployment_queue->addLogEntry('Deployment cancelled by user, stopping execution.'); + throw new \RuntimeException('Deployment cancelled by user', 69420); } $this->application_deployment_queue->update([ 'status' => $status, ]); + queue_next_deployment($this->application); + if ($status === ApplicationDeploymentStatus::FINISHED->value) { + ray($this->application->team()->id); + event(new ApplicationConfigurationChanged($this->application->team()->id)); + if (! $this->only_this_server) { $this->deploy_to_additional_destinations(); } @@ -2513,8 +3347,8 @@ public function failed(Throwable $exception): void $code = $exception->getCode(); if ($code !== 69420) { // 69420 means failed to push the image to the registry, so we don't need to remove the new version as it is the currently running one - if ($this->application->settings->is_consistent_container_name_enabled || str($this->application->settings->custom_internal_name)->isNotEmpty()) { - // do not remove already running container + if ($this->application->settings->is_consistent_container_name_enabled || str($this->application->settings->custom_internal_name)->isNotEmpty() || $this->pull_request_id !== 0) { + // do not remove already running container for PR deployments } else { $this->application_deployment_queue->addLogEntry('Deployment failed. Removing the new version of your application.', 'stderr'); $this->execute_remote_command( diff --git a/app/Jobs/DEPRECATEDContainerStatusJob.php b/app/Jobs/DEPRECATEDContainerStatusJob.php deleted file mode 100644 index df6dec7fe..000000000 --- a/app/Jobs/DEPRECATEDContainerStatusJob.php +++ /dev/null @@ -1,31 +0,0 @@ -<?php - -namespace App\Jobs; - -use App\Actions\Docker\GetContainersStatus; -use App\Models\Server; -use Illuminate\Bus\Queueable; -use Illuminate\Contracts\Queue\ShouldBeEncrypted; -use Illuminate\Contracts\Queue\ShouldQueue; -use Illuminate\Foundation\Bus\Dispatchable; -use Illuminate\Queue\InteractsWithQueue; -use Illuminate\Queue\SerializesModels; - -class DEPRECATEDContainerStatusJob implements ShouldBeEncrypted, ShouldQueue -{ - use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; - - public $tries = 4; - - public function backoff(): int - { - return isDev() ? 1 : 3; - } - - public function __construct(public Server $server) {} - - public function handle() - { - GetContainersStatus::run($this->server); - } -} diff --git a/app/Jobs/DEPRECATEDServerCheckNewJob.php b/app/Jobs/DEPRECATEDServerCheckNewJob.php deleted file mode 100644 index 1118366fe..000000000 --- a/app/Jobs/DEPRECATEDServerCheckNewJob.php +++ /dev/null @@ -1,34 +0,0 @@ -<?php - -namespace App\Jobs; - -use App\Actions\Server\ResourcesCheck; -use App\Actions\Server\ServerCheck; -use App\Models\Server; -use Illuminate\Bus\Queueable; -use Illuminate\Contracts\Queue\ShouldBeEncrypted; -use Illuminate\Contracts\Queue\ShouldQueue; -use Illuminate\Foundation\Bus\Dispatchable; -use Illuminate\Queue\InteractsWithQueue; -use Illuminate\Queue\SerializesModels; - -class DEPRECATEDServerCheckNewJob implements ShouldBeEncrypted, ShouldQueue -{ - use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; - - public $tries = 1; - - public $timeout = 60; - - public function __construct(public Server $server) {} - - public function handle() - { - try { - ServerCheck::run($this->server); - ResourcesCheck::dispatch($this->server); - } catch (\Throwable $e) { - return handleError($e); - } - } -} diff --git a/app/Jobs/DEPRECATEDServerResourceManager.php b/app/Jobs/DEPRECATEDServerResourceManager.php deleted file mode 100644 index c50567a01..000000000 --- a/app/Jobs/DEPRECATEDServerResourceManager.php +++ /dev/null @@ -1,162 +0,0 @@ -<?php - -namespace App\Jobs; - -use App\Models\InstanceSettings; -use App\Models\Server; -use App\Models\Team; -use Cron\CronExpression; -use Illuminate\Bus\Queueable; -use Illuminate\Contracts\Queue\ShouldQueue; -use Illuminate\Foundation\Bus\Dispatchable; -use Illuminate\Queue\InteractsWithQueue; -use Illuminate\Queue\Middleware\WithoutOverlapping; -use Illuminate\Queue\SerializesModels; -use Illuminate\Support\Carbon; -use Illuminate\Support\Facades\Log; - -class DEPRECATEDServerResourceManager implements ShouldQueue -{ - use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; - - /** - * The time when this job execution started. - */ - private ?Carbon $executionTime = null; - - private InstanceSettings $settings; - - private string $instanceTimezone; - - /** - * Create a new job instance. - */ - public function __construct() - { - $this->onQueue('high'); - } - - /** - * Get the middleware the job should pass through. - */ - public function middleware(): array - { - return [ - (new WithoutOverlapping('server-resource-manager')) - ->releaseAfter(60), - ]; - } - - public function handle(): void - { - // Freeze the execution time at the start of the job - $this->executionTime = Carbon::now(); - - $this->settings = instanceSettings(); - $this->instanceTimezone = $this->settings->instance_timezone ?: config('app.timezone'); - - if (validate_timezone($this->instanceTimezone) === false) { - $this->instanceTimezone = config('app.timezone'); - } - - // Process server checks - don't let failures stop the job - try { - $this->processServerChecks(); - } catch (\Exception $e) { - Log::channel('scheduled-errors')->error('Failed to process server checks', [ - 'error' => $e->getMessage(), - 'trace' => $e->getTraceAsString(), - ]); - } - } - - private function processServerChecks(): void - { - $servers = $this->getServers(); - - foreach ($servers as $server) { - try { - $this->processServer($server); - } catch (\Exception $e) { - Log::channel('scheduled-errors')->error('Error processing server', [ - 'server_id' => $server->id, - 'server_name' => $server->name, - 'error' => $e->getMessage(), - ]); - } - } - } - - private function getServers() - { - $allServers = Server::where('ip', '!=', '1.2.3.4'); - - if (isCloud()) { - $servers = $allServers->whereRelation('team.subscription', 'stripe_invoice_paid', true)->get(); - $own = Team::find(0)->servers; - - return $servers->merge($own); - } else { - return $allServers->get(); - } - } - - private function processServer(Server $server): void - { - $serverTimezone = data_get($server->settings, 'server_timezone', $this->instanceTimezone); - if (validate_timezone($serverTimezone) === false) { - $serverTimezone = config('app.timezone'); - } - - // Sentinel check - $lastSentinelUpdate = $server->sentinel_updated_at; - if (Carbon::parse($lastSentinelUpdate)->isBefore($this->executionTime->subSeconds($server->waitBeforeDoingSshCheck()))) { - // Dispatch ServerCheckJob if due - $checkFrequency = isCloud() ? '*/5 * * * *' : '* * * * *'; // Every 5 min for cloud, every minute for self-hosted - if ($this->shouldRunNow($checkFrequency, $serverTimezone)) { - ServerCheckJob::dispatch($server); - } - - // Dispatch ServerStorageCheckJob if due - $serverDiskUsageCheckFrequency = data_get($server->settings, 'server_disk_usage_check_frequency', '0 * * * *'); - if (isset(VALID_CRON_STRINGS[$serverDiskUsageCheckFrequency])) { - $serverDiskUsageCheckFrequency = VALID_CRON_STRINGS[$serverDiskUsageCheckFrequency]; - } - if ($this->shouldRunNow($serverDiskUsageCheckFrequency, $serverTimezone)) { - ServerStorageCheckJob::dispatch($server); - } - } - - // Dispatch DockerCleanupJob if due - $dockerCleanupFrequency = data_get($server->settings, 'docker_cleanup_frequency', '0 * * * *'); - if (isset(VALID_CRON_STRINGS[$dockerCleanupFrequency])) { - $dockerCleanupFrequency = VALID_CRON_STRINGS[$dockerCleanupFrequency]; - } - if ($this->shouldRunNow($dockerCleanupFrequency, $serverTimezone)) { - DockerCleanupJob::dispatch($server, false, $server->settings->delete_unused_volumes, $server->settings->delete_unused_networks); - } - - // Dispatch ServerPatchCheckJob if due (weekly) - if ($this->shouldRunNow('0 0 * * 0', $serverTimezone)) { // Weekly on Sunday at midnight - ServerPatchCheckJob::dispatch($server); - } - - // Dispatch Sentinel restart if due (daily for Sentinel-enabled servers) - if ($server->isSentinelEnabled() && $this->shouldRunNow('0 0 * * *', $serverTimezone)) { - dispatch(function () use ($server) { - $server->restartContainer('coolify-sentinel'); - }); - } - } - - private function shouldRunNow(string $frequency, string $timezone): bool - { - $cron = new CronExpression($frequency); - - // Use the frozen execution time, not the current time - $baseTime = $this->executionTime ?? Carbon::now(); - $executionTime = $baseTime->copy()->setTimezone($timezone); - - return $cron->isDue($executionTime); - } -} diff --git a/app/Jobs/DatabaseBackupJob.php b/app/Jobs/DatabaseBackupJob.php index 752d1f1ca..92db14a61 100644 --- a/app/Jobs/DatabaseBackupJob.php +++ b/app/Jobs/DatabaseBackupJob.php @@ -54,6 +54,10 @@ class DatabaseBackupJob implements ShouldBeEncrypted, ShouldQueue public ?string $backup_output = null; + public ?string $error_output = null; + + public bool $s3_uploaded = false; + public ?string $postgres_password = null; public ?string $mongo_root_username = null; @@ -70,8 +74,6 @@ public function __construct(public ScheduledDatabaseBackup $backup) { $this->onQueue('high'); $this->timeout = $backup->timeout; - - $this->backup_log_uuid = (string) new Cuid2; } public function handle(): void @@ -284,6 +286,17 @@ public function handle(): void $this->backup_dir = backup_dir().'/coolify'."/coolify-db-$ip"; } foreach ($databasesToBackup as $database) { + // Generate unique UUID for each database backup execution + $attempts = 0; + do { + $this->backup_log_uuid = (string) new Cuid2; + $exists = ScheduledDatabaseBackupExecution::where('uuid', $this->backup_log_uuid)->exists(); + $attempts++; + if ($attempts >= 3 && $exists) { + throw new \Exception('Unable to generate unique UUID for backup execution after 3 attempts'); + } + } while ($exists); + $size = 0; try { if (str($databaseType)->contains('postgres')) { @@ -355,7 +368,6 @@ public function handle(): void // If local backup is disabled, delete the local file immediately after S3 upload if ($this->backup->disable_local_backup) { deleteBackupsLocally($this->backup_location, $this->server); - $this->add_to_backup_output('Local backup file deleted after S3 upload (disable_local_backup enabled).'); } } @@ -367,15 +379,34 @@ public function handle(): void 'size' => $size, ]); } catch (\Throwable $e) { - if ($this->backup_log) { - $this->backup_log->update([ - 'status' => 'failed', - 'message' => $this->backup_output, - 'size' => $size, - 'filename' => null, - ]); + // Check if backup actually failed or if it's just a post-backup issue + $actualBackupFailed = ! $this->s3_uploaded && $this->backup->save_s3; + + if ($actualBackupFailed || $size === 0) { + // Real backup failure + if ($this->backup_log) { + $this->backup_log->update([ + 'status' => 'failed', + 'message' => $this->error_output ?? $this->backup_output ?? $e->getMessage(), + 'size' => $size, + 'filename' => null, + ]); + } + $this->team?->notify(new BackupFailed($this->backup, $this->database, $this->error_output ?? $this->backup_output ?? $e->getMessage(), $database)); + } else { + // Backup succeeded but post-processing failed (cleanup, notification, etc.) + if ($this->backup_log) { + $this->backup_log->update([ + 'status' => 'success', + 'message' => $this->backup_output ? $this->backup_output."\nWarning: Post-backup cleanup encountered an issue: ".$e->getMessage() : 'Warning: '.$e->getMessage(), + 'size' => $size, + ]); + } + // Send success notification since the backup itself succeeded + $this->team->notify(new BackupSuccess($this->backup, $this->database, $database)); + // Log the post-backup issue + ray('Post-backup operation failed but backup was successful: '.$e->getMessage()); } - $this->team?->notify(new BackupFailed($this->backup, $this->database, $this->backup_output, $database)); } } if ($this->backup_log && $this->backup_log->status === 'success') { @@ -446,7 +477,7 @@ private function backup_standalone_mongodb(string $databaseWithCollections): voi $this->backup_output = null; } } catch (\Throwable $e) { - $this->add_to_backup_output($e->getMessage()); + $this->add_to_error_output($e->getMessage()); throw $e; } } @@ -472,7 +503,7 @@ private function backup_standalone_postgresql(string $database): void $this->backup_output = null; } } catch (\Throwable $e) { - $this->add_to_backup_output($e->getMessage()); + $this->add_to_error_output($e->getMessage()); throw $e; } } @@ -492,7 +523,7 @@ private function backup_standalone_mysql(string $database): void $this->backup_output = null; } } catch (\Throwable $e) { - $this->add_to_backup_output($e->getMessage()); + $this->add_to_error_output($e->getMessage()); throw $e; } } @@ -512,7 +543,7 @@ private function backup_standalone_mariadb(string $database): void $this->backup_output = null; } } catch (\Throwable $e) { - $this->add_to_backup_output($e->getMessage()); + $this->add_to_error_output($e->getMessage()); throw $e; } } @@ -526,6 +557,15 @@ private function add_to_backup_output($output): void } } + private function add_to_error_output($output): void + { + if ($this->error_output) { + $this->error_output = $this->error_output."\n".$output; + } else { + $this->error_output = $output; + } + } + private function calculate_size() { return instant_remote_process(["du -b $this->backup_location | cut -f1"], $this->server, false); @@ -571,9 +611,10 @@ private function upload_to_s3(): void $commands[] = "docker exec backup-of-{$this->backup->uuid} mc cp $this->backup_location temporary/$bucket{$this->backup_dir}/"; instant_remote_process($commands, $this->server); - $this->add_to_backup_output('Uploaded to S3.'); + $this->s3_uploaded = true; } catch (\Throwable $e) { - $this->add_to_backup_output($e->getMessage()); + $this->s3_uploaded = false; + $this->add_to_error_output($e->getMessage()); throw $e; } finally { $command = "docker rm -f backup-of-{$this->backup->uuid}"; diff --git a/app/Jobs/PullChangelogFromGitHub.php b/app/Jobs/PullChangelog.php similarity index 77% rename from app/Jobs/PullChangelogFromGitHub.php rename to app/Jobs/PullChangelog.php index e84766f7f..052e6d557 100644 --- a/app/Jobs/PullChangelogFromGitHub.php +++ b/app/Jobs/PullChangelog.php @@ -11,8 +11,9 @@ use Illuminate\Queue\SerializesModels; use Illuminate\Support\Facades\File; use Illuminate\Support\Facades\Http; +use Illuminate\Support\Facades\Log; -class PullChangelogFromGitHub implements ShouldBeEncrypted, ShouldQueue +class PullChangelog implements ShouldBeEncrypted, ShouldQueue { use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; @@ -26,21 +27,36 @@ public function __construct() public function handle(): void { try { + // Fetch from CDN instead of GitHub API to avoid rate limits + $cdnUrl = config('constants.coolify.releases_url'); + $response = Http::retry(3, 1000) ->timeout(30) - ->get('https://api.github.com/repos/coollabsio/coolify/releases?per_page=10'); + ->get($cdnUrl); if ($response->successful()) { $releases = $response->json(); + + // Limit to 10 releases for processing (same as before) + $releases = array_slice($releases, 0, 10); + $changelog = $this->transformReleasesToChangelog($releases); // Group entries by month and save them $this->saveChangelogEntries($changelog); } else { - send_internal_notification('PullChangelogFromGitHub failed with: '.$response->status().' '.$response->body()); + // Log error instead of sending notification + Log::error('PullChangelogFromGitHub: Failed to fetch from CDN', [ + 'status' => $response->status(), + 'url' => $cdnUrl, + ]); } } catch (\Throwable $e) { - send_internal_notification('PullChangelogFromGitHub failed with: '.$e->getMessage()); + // Log error instead of sending notification + Log::error('PullChangelogFromGitHub: Exception occurred', [ + 'message' => $e->getMessage(), + 'trace' => $e->getTraceAsString(), + ]); } } diff --git a/app/Jobs/PushServerUpdateJob.php b/app/Jobs/PushServerUpdateJob.php index 3e3aa1eb7..7726c2c73 100644 --- a/app/Jobs/PushServerUpdateJob.php +++ b/app/Jobs/PushServerUpdateJob.php @@ -65,6 +65,8 @@ class PushServerUpdateJob implements ShouldBeEncrypted, ShouldQueue, Silenced public Collection $foundApplicationPreviewsIds; + public Collection $applicationContainerStatuses; + public bool $foundProxy = false; public bool $foundLogDrainContainer = false; @@ -87,6 +89,7 @@ public function __construct(public Server $server, public $data) $this->foundServiceApplicationIds = collect(); $this->foundApplicationPreviewsIds = collect(); $this->foundServiceDatabaseIds = collect(); + $this->applicationContainerStatuses = collect(); $this->allApplicationIds = collect(); $this->allDatabaseUuids = collect(); $this->allTcpProxyUuids = collect(); @@ -155,7 +158,14 @@ public function handle() if ($this->allApplicationIds->contains($applicationId) && $this->isRunning($containerStatus)) { $this->foundApplicationIds->push($applicationId); } - $this->updateApplicationStatus($applicationId, $containerStatus); + // Store container status for aggregation + if (! $this->applicationContainerStatuses->has($applicationId)) { + $this->applicationContainerStatuses->put($applicationId, collect()); + } + $containerName = $labels->get('com.docker.compose.service'); + if ($containerName) { + $this->applicationContainerStatuses->get($applicationId)->put($containerName, $containerStatus); + } } else { $previewKey = $applicationId.':'.$pullRequestId; if ($this->allApplicationPreviewsIds->contains($previewKey) && $this->isRunning($containerStatus)) { @@ -205,9 +215,86 @@ public function handle() $this->updateAdditionalServersStatus(); + // Aggregate multi-container application statuses + $this->aggregateMultiContainerStatuses(); + $this->checkLogDrainContainer(); } + private function aggregateMultiContainerStatuses() + { + if ($this->applicationContainerStatuses->isEmpty()) { + return; + } + + foreach ($this->applicationContainerStatuses as $applicationId => $containerStatuses) { + $application = $this->applications->where('id', $applicationId)->first(); + if (! $application) { + continue; + } + + // Parse docker compose to check for excluded containers + $dockerComposeRaw = data_get($application, 'docker_compose_raw'); + $excludedContainers = collect(); + + if ($dockerComposeRaw) { + try { + $dockerCompose = \Symfony\Component\Yaml\Yaml::parse($dockerComposeRaw); + $services = data_get($dockerCompose, 'services', []); + + foreach ($services as $serviceName => $serviceConfig) { + // Check if container should be excluded + $excludeFromHc = data_get($serviceConfig, 'exclude_from_hc', false); + $restartPolicy = data_get($serviceConfig, 'restart', 'always'); + + if ($excludeFromHc || $restartPolicy === 'no') { + $excludedContainers->push($serviceName); + } + } + } catch (\Exception $e) { + // If we can't parse, treat all containers as included + } + } + + // Filter out excluded containers + $relevantStatuses = $containerStatuses->filter(function ($status, $containerName) use ($excludedContainers) { + return ! $excludedContainers->contains($containerName); + }); + + // If all containers are excluded, don't update status + if ($relevantStatuses->isEmpty()) { + continue; + } + + // Aggregate status: if any container is running, app is running + $hasRunning = false; + $hasUnhealthy = false; + + foreach ($relevantStatuses as $status) { + if (str($status)->contains('running')) { + $hasRunning = true; + if (str($status)->contains('unhealthy')) { + $hasUnhealthy = true; + } + } + } + + $aggregatedStatus = null; + if ($hasRunning) { + $aggregatedStatus = $hasUnhealthy ? 'running (unhealthy)' : 'running (healthy)'; + } else { + // All containers are exited + $aggregatedStatus = 'exited (unhealthy)'; + } + + // Update application status with aggregated result + if ($aggregatedStatus && $application->status !== $aggregatedStatus) { + $application->status = $aggregatedStatus; + $application->save(); + } + } + } + private function updateApplicationStatus(string $applicationId, string $containerStatus) { $application = $this->applications->where('id', $applicationId)->first(); diff --git a/app/Jobs/ScheduledTaskJob.php b/app/Jobs/ScheduledTaskJob.php index 6c0c017e7..609595356 100644 --- a/app/Jobs/ScheduledTaskJob.php +++ b/app/Jobs/ScheduledTaskJob.php @@ -3,6 +3,7 @@ namespace App\Jobs; use App\Events\ScheduledTaskDone; +use App\Exceptions\NonReportableException; use App\Models\Application; use App\Models\ScheduledTask; use App\Models\ScheduledTaskExecution; @@ -120,7 +121,7 @@ public function handle(): void } // No valid container was found. - throw new \Exception('ScheduledTaskJob failed: No valid container was found. Is the container name correct?'); + throw new NonReportableException('ScheduledTaskJob failed: No valid container was found. Is the container name correct?'); } catch (\Throwable $e) { if ($this->task_log) { $this->task_log->update([ diff --git a/app/Jobs/ServerConnectionCheckJob.php b/app/Jobs/ServerConnectionCheckJob.php index 167bcea38..8b55434f6 100644 --- a/app/Jobs/ServerConnectionCheckJob.php +++ b/app/Jobs/ServerConnectionCheckJob.php @@ -78,11 +78,11 @@ public function handle() } // Server is reachable, check if Docker is available - // $isUsable = $this->checkDockerAvailability(); + $isUsable = $this->checkDockerAvailability(); $this->server->settings->update([ 'is_reachable' => true, - 'is_usable' => true, + 'is_usable' => $isUsable, ]); } catch (\Throwable $e) { diff --git a/app/Jobs/StripeProcessJob.php b/app/Jobs/StripeProcessJob.php index f1c5bc1a8..aebceaa6d 100644 --- a/app/Jobs/StripeProcessJob.php +++ b/app/Jobs/StripeProcessJob.php @@ -58,7 +58,7 @@ public function handle(): void case 'checkout.session.completed': $clientReferenceId = data_get($data, 'client_reference_id'); if (is_null($clientReferenceId)) { - send_internal_notification('Checkout session completed without client reference id.'); + // send_internal_notification('Checkout session completed without client reference id.'); break; } $userId = Str::before($clientReferenceId, ':'); @@ -68,7 +68,7 @@ public function handle(): void $team = Team::find($teamId); $found = $team->members->where('id', $userId)->first(); if (! $found->isAdmin()) { - send_internal_notification("User {$userId} is not an admin or owner of team {$team->id}, customerid: {$customerId}, subscriptionid: {$subscriptionId}."); + // send_internal_notification("User {$userId} is not an admin or owner of team {$team->id}, customerid: {$customerId}, subscriptionid: {$subscriptionId}."); throw new \RuntimeException("User {$userId} is not an admin or owner of team {$team->id}, customerid: {$customerId}, subscriptionid: {$subscriptionId}."); } $subscription = Subscription::where('team_id', $teamId)->first(); @@ -93,33 +93,101 @@ public function handle(): void break; case 'invoice.paid': $customerId = data_get($data, 'customer'); + $invoiceAmount = data_get($data, 'amount_paid', 0); + $subscriptionId = data_get($data, 'subscription'); $planId = data_get($data, 'lines.data.0.plan.id'); if (Str::contains($excludedPlans, $planId)) { - send_internal_notification('Subscription excluded.'); + // send_internal_notification('Subscription excluded.'); break; } $subscription = Subscription::where('stripe_customer_id', $customerId)->first(); - if ($subscription) { - $subscription->update([ - 'stripe_invoice_paid' => true, - 'stripe_past_due' => false, - ]); - } else { + if (! $subscription) { throw new \RuntimeException("No subscription found for customer: {$customerId}"); } + + if ($subscription->stripe_subscription_id) { + try { + $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); + $stripeSubscription = $stripe->subscriptions->retrieve( + $subscription->stripe_subscription_id + ); + + switch ($stripeSubscription->status) { + case 'active': + $subscription->update([ + 'stripe_invoice_paid' => true, + 'stripe_past_due' => false, + ]); + break; + + case 'past_due': + $subscription->update([ + 'stripe_invoice_paid' => true, + 'stripe_past_due' => true, + ]); + break; + + case 'canceled': + case 'incomplete_expired': + case 'unpaid': + send_internal_notification( + "Invoice paid for {$stripeSubscription->status} subscription. ". + "Customer: {$customerId}, Amount: \${$invoiceAmount}" + ); + break; + + default: + VerifyStripeSubscriptionStatusJob::dispatch($subscription) + ->delay(now()->addSeconds(20)); + break; + } + } catch (\Exception $e) { + VerifyStripeSubscriptionStatusJob::dispatch($subscription) + ->delay(now()->addSeconds(20)); + + send_internal_notification( + 'Failed to verify subscription status in invoice.paid: '.$e->getMessage() + ); + } + } else { + VerifyStripeSubscriptionStatusJob::dispatch($subscription) + ->delay(now()->addSeconds(20)); + } break; case 'invoice.payment_failed': $customerId = data_get($data, 'customer'); + $invoiceId = data_get($data, 'id'); + $paymentIntentId = data_get($data, 'payment_intent'); + $subscription = Subscription::where('stripe_customer_id', $customerId)->first(); if (! $subscription) { - send_internal_notification('invoice.payment_failed failed but no subscription found in Coolify for customer: '.$customerId); + // send_internal_notification('invoice.payment_failed failed but no subscription found in Coolify for customer: '.$customerId); throw new \RuntimeException("No subscription found for customer: {$customerId}"); } $team = data_get($subscription, 'team'); if (! $team) { - send_internal_notification('invoice.payment_failed failed but no team found in Coolify for customer: '.$customerId); + // send_internal_notification('invoice.payment_failed failed but no team found in Coolify for customer: '.$customerId); throw new \RuntimeException("No team found in Coolify for customer: {$customerId}"); } + + // Verify payment status with Stripe API before sending failure notification + if ($paymentIntentId) { + try { + $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); + $paymentIntent = $stripe->paymentIntents->retrieve($paymentIntentId); + + if (in_array($paymentIntent->status, ['processing', 'succeeded', 'requires_action', 'requires_confirmation'])) { + break; + } + + if (! $subscription->stripe_invoice_paid && $subscription->created_at->diffInMinutes(now()) < 5) { + SubscriptionInvoiceFailedJob::dispatch($team)->delay(now()->addSeconds(60)); + break; + } + } catch (\Exception $e) { + } + } + if (! $subscription->stripe_invoice_paid) { SubscriptionInvoiceFailedJob::dispatch($team); // send_internal_notification('Invoice payment failed: '.$customerId); @@ -129,11 +197,11 @@ public function handle(): void $customerId = data_get($data, 'customer'); $subscription = Subscription::where('stripe_customer_id', $customerId)->first(); if (! $subscription) { - send_internal_notification('payment_intent.payment_failed, no subscription found in Coolify for customer: '.$customerId); + // send_internal_notification('payment_intent.payment_failed, no subscription found in Coolify for customer: '.$customerId); throw new \RuntimeException("No subscription found in Coolify for customer: {$customerId}"); } if ($subscription->stripe_invoice_paid) { - send_internal_notification('payment_intent.payment_failed but invoice is active for customer: '.$customerId); + // send_internal_notification('payment_intent.payment_failed but invoice is active for customer: '.$customerId); return; } @@ -154,7 +222,7 @@ public function handle(): void $team = Team::find($teamId); $found = $team->members->where('id', $userId)->first(); if (! $found->isAdmin()) { - send_internal_notification("User {$userId} is not an admin or owner of team {$team->id}, customerid: {$customerId}."); + // send_internal_notification("User {$userId} is not an admin or owner of team {$team->id}, customerid: {$customerId}."); throw new \RuntimeException("User {$userId} is not an admin or owner of team {$team->id}, customerid: {$customerId}."); } $subscription = Subscription::where('team_id', $teamId)->first(); @@ -177,7 +245,7 @@ public function handle(): void $subscriptionId = data_get($data, 'items.data.0.subscription') ?? data_get($data, 'id'); $planId = data_get($data, 'items.data.0.plan.id') ?? data_get($data, 'plan.id'); if (Str::contains($excludedPlans, $planId)) { - send_internal_notification('Subscription excluded.'); + // send_internal_notification('Subscription excluded.'); break; } $subscription = Subscription::where('stripe_customer_id', $customerId)->first(); @@ -194,7 +262,7 @@ public function handle(): void 'stripe_invoice_paid' => false, ]); } else { - send_internal_notification('No subscription and team id found'); + // send_internal_notification('No subscription and team id found'); throw new \RuntimeException('No subscription and team id found'); } } @@ -230,7 +298,7 @@ public function handle(): void $subscription->update([ 'stripe_past_due' => true, ]); - send_internal_notification('Past Due: '.$customerId.'Subscription ID: '.$subscriptionId); + // send_internal_notification('Past Due: '.$customerId.'Subscription ID: '.$subscriptionId); } } if ($status === 'unpaid') { @@ -238,13 +306,13 @@ public function handle(): void $subscription->update([ 'stripe_invoice_paid' => false, ]); - send_internal_notification('Unpaid: '.$customerId.'Subscription ID: '.$subscriptionId); + // send_internal_notification('Unpaid: '.$customerId.'Subscription ID: '.$subscriptionId); } $team = data_get($subscription, 'team'); if ($team) { $team->subscriptionEnded(); } else { - send_internal_notification('Subscription unpaid but no team found in Coolify for customer: '.$customerId); + // send_internal_notification('Subscription unpaid but no team found in Coolify for customer: '.$customerId); throw new \RuntimeException("No team found in Coolify for customer: {$customerId}"); } } @@ -273,11 +341,11 @@ public function handle(): void if ($team) { $team->subscriptionEnded(); } else { - send_internal_notification('Subscription deleted but no team found in Coolify for customer: '.$customerId); + // send_internal_notification('Subscription deleted but no team found in Coolify for customer: '.$customerId); throw new \RuntimeException("No team found in Coolify for customer: {$customerId}"); } } else { - send_internal_notification('Subscription deleted but no subscription found in Coolify for customer: '.$customerId); + // send_internal_notification('Subscription deleted but no subscription found in Coolify for customer: '.$customerId); throw new \RuntimeException("No subscription found in Coolify for customer: {$customerId}"); } break; diff --git a/app/Jobs/SubscriptionInvoiceFailedJob.php b/app/Jobs/SubscriptionInvoiceFailedJob.php index dc511f445..927d50467 100755 --- a/app/Jobs/SubscriptionInvoiceFailedJob.php +++ b/app/Jobs/SubscriptionInvoiceFailedJob.php @@ -23,6 +23,47 @@ public function __construct(protected Team $team) public function handle() { try { + // Double-check subscription status before sending failure notification + $subscription = $this->team->subscription; + if ($subscription && $subscription->stripe_customer_id) { + try { + $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); + + if ($subscription->stripe_subscription_id) { + $stripeSubscription = $stripe->subscriptions->retrieve($subscription->stripe_subscription_id); + + if (in_array($stripeSubscription->status, ['active', 'trialing'])) { + if (! $subscription->stripe_invoice_paid) { + $subscription->update([ + 'stripe_invoice_paid' => true, + 'stripe_past_due' => false, + ]); + } + + return; + } + } + + $invoices = $stripe->invoices->all([ + 'customer' => $subscription->stripe_customer_id, + 'limit' => 3, + ]); + + foreach ($invoices->data as $invoice) { + if ($invoice->paid && $invoice->created > (time() - 3600)) { + $subscription->update([ + 'stripe_invoice_paid' => true, + 'stripe_past_due' => false, + ]); + + return; + } + } + } catch (\Exception $e) { + } + } + + // If we reach here, payment genuinely failed $session = getStripeCustomerPortalSession($this->team); $mail = new MailMessage; $mail->view('emails.subscription-invoice-failed', [ diff --git a/app/Jobs/VerifyStripeSubscriptionStatusJob.php b/app/Jobs/VerifyStripeSubscriptionStatusJob.php new file mode 100644 index 000000000..58b6944a2 --- /dev/null +++ b/app/Jobs/VerifyStripeSubscriptionStatusJob.php @@ -0,0 +1,106 @@ +<?php + +namespace App\Jobs; + +use App\Models\Subscription; +use Illuminate\Bus\Queueable; +use Illuminate\Contracts\Queue\ShouldQueue; +use Illuminate\Foundation\Bus\Dispatchable; +use Illuminate\Queue\InteractsWithQueue; +use Illuminate\Queue\SerializesModels; + +class VerifyStripeSubscriptionStatusJob implements ShouldQueue +{ + use Dispatchable, InteractsWithQueue, Queueable, SerializesModels; + + public int $tries = 3; + + public array $backoff = [10, 30, 60]; + + public function __construct(public Subscription $subscription) + { + $this->onQueue('high'); + } + + public function handle(): void + { + // If no subscription ID yet, try to find it via customer + if (! $this->subscription->stripe_subscription_id && + $this->subscription->stripe_customer_id) { + try { + $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); + $subscriptions = $stripe->subscriptions->all([ + 'customer' => $this->subscription->stripe_customer_id, + 'limit' => 1, + ]); + + if ($subscriptions->data) { + $this->subscription->update([ + 'stripe_subscription_id' => $subscriptions->data[0]->id, + ]); + } + } catch (\Exception $e) { + // Continue without subscription ID + } + } + + if (! $this->subscription->stripe_subscription_id) { + return; + } + + try { + $stripe = new \Stripe\StripeClient(config('subscription.stripe_api_key')); + $stripeSubscription = $stripe->subscriptions->retrieve( + $this->subscription->stripe_subscription_id + ); + + switch ($stripeSubscription->status) { + case 'active': + $this->subscription->update([ + 'stripe_invoice_paid' => true, + 'stripe_past_due' => false, + 'stripe_cancel_at_period_end' => $stripeSubscription->cancel_at_period_end, + ]); + break; + + case 'past_due': + // Keep subscription active but mark as past_due + $this->subscription->update([ + 'stripe_invoice_paid' => true, + 'stripe_past_due' => true, + 'stripe_cancel_at_period_end' => $stripeSubscription->cancel_at_period_end, + ]); + break; + + case 'canceled': + case 'incomplete_expired': + case 'unpaid': + // Ensure subscription is marked as inactive + $this->subscription->update([ + 'stripe_invoice_paid' => false, + 'stripe_past_due' => false, + ]); + + // Trigger subscription ended logic if canceled + if ($stripeSubscription->status === 'canceled') { + $team = $this->subscription->team; + if ($team) { + $team->subscriptionEnded(); + } + } + break; + + default: + send_internal_notification( + 'Unknown subscription status in VerifyStripeSubscriptionStatusJob: '.$stripeSubscription->status. + ' for customer: '.$this->subscription->stripe_customer_id + ); + break; + } + } catch (\Exception $e) { + send_internal_notification( + 'VerifyStripeSubscriptionStatusJob failed for subscription ID '.$this->subscription->id.': '.$e->getMessage() + ); + } + } +} diff --git a/app/Livewire/Dashboard.php b/app/Livewire/Dashboard.php index 18dbde0d3..57ecaa8a2 100644 --- a/app/Livewire/Dashboard.php +++ b/app/Livewire/Dashboard.php @@ -2,63 +2,25 @@ namespace App\Livewire; -use App\Models\Application; -use App\Models\ApplicationDeploymentQueue; use App\Models\PrivateKey; use App\Models\Project; use App\Models\Server; use Illuminate\Support\Collection; -use Illuminate\Support\Facades\Artisan; use Livewire\Component; class Dashboard extends Component { - public $projects = []; + public Collection $projects; public Collection $servers; public Collection $privateKeys; - public array $deploymentsPerServer = []; - public function mount() { $this->privateKeys = PrivateKey::ownedByCurrentTeam()->get(); $this->servers = Server::ownedByCurrentTeam()->get(); $this->projects = Project::ownedByCurrentTeam()->get(); - $this->loadDeployments(); - } - - public function cleanupQueue() - { - try { - $this->authorize('cleanupDeploymentQueue', Application::class); - } catch (\Illuminate\Auth\Access\AuthorizationException $e) { - return handleError($e, $this); - } - - Artisan::queue('cleanup:deployment-queue', [ - '--team-id' => currentTeam()->id, - ]); - } - - public function loadDeployments() - { - $this->deploymentsPerServer = ApplicationDeploymentQueue::whereIn('status', ['in_progress', 'queued'])->whereIn('server_id', $this->servers->pluck('id'))->get([ - 'id', - 'application_id', - 'application_name', - 'deployment_url', - 'pull_request_id', - 'server_name', - 'server_id', - 'status', - ])->sortBy('id')->groupBy('server_name')->toArray(); - } - - public function navigateToProject($projectUuid) - { - return $this->redirect(collect($this->projects)->firstWhere('uuid', $projectUuid)->navigateTo(), navigate: false); } public function render() diff --git a/app/Livewire/DeploymentsIndicator.php b/app/Livewire/DeploymentsIndicator.php new file mode 100644 index 000000000..0293ad6c6 --- /dev/null +++ b/app/Livewire/DeploymentsIndicator.php @@ -0,0 +1,49 @@ +<?php + +namespace App\Livewire; + +use App\Models\ApplicationDeploymentQueue; +use App\Models\Server; +use Livewire\Attributes\Computed; +use Livewire\Component; + +class DeploymentsIndicator extends Component +{ + public bool $expanded = false; + + #[Computed] + public function deployments() + { + $servers = Server::ownedByCurrentTeam()->get(); + + return ApplicationDeploymentQueue::whereIn('status', ['in_progress', 'queued']) + ->whereIn('server_id', $servers->pluck('id')) + ->orderBy('id') + ->get([ + 'id', + 'application_id', + 'application_name', + 'deployment_url', + 'pull_request_id', + 'server_name', + 'server_id', + 'status', + ]); + } + + #[Computed] + public function deploymentCount() + { + return $this->deployments->count(); + } + + public function toggleExpanded() + { + $this->expanded = ! $this->expanded; + } + + public function render() + { + return view('livewire.deployments-indicator'); + } +} diff --git a/app/Livewire/GlobalSearch.php b/app/Livewire/GlobalSearch.php new file mode 100644 index 000000000..15de5d838 --- /dev/null +++ b/app/Livewire/GlobalSearch.php @@ -0,0 +1,444 @@ +<?php + +namespace App\Livewire; + +use App\Models\Application; +use App\Models\Environment; +use App\Models\Project; +use App\Models\Server; +use App\Models\Service; +use App\Models\StandaloneClickhouse; +use App\Models\StandaloneDragonfly; +use App\Models\StandaloneKeydb; +use App\Models\StandaloneMariadb; +use App\Models\StandaloneMongodb; +use App\Models\StandaloneMysql; +use App\Models\StandalonePostgresql; +use App\Models\StandaloneRedis; +use Illuminate\Support\Facades\Cache; +use Livewire\Component; + +class GlobalSearch extends Component +{ + public $searchQuery = ''; + + public $isModalOpen = false; + + public $searchResults = []; + + public $allSearchableItems = []; + + public function mount() + { + $this->searchQuery = ''; + $this->isModalOpen = false; + $this->searchResults = []; + $this->allSearchableItems = []; + } + + public function openSearchModal() + { + $this->isModalOpen = true; + $this->loadSearchableItems(); + $this->dispatch('search-modal-opened'); + } + + public function closeSearchModal() + { + $this->isModalOpen = false; + $this->searchQuery = ''; + $this->searchResults = []; + } + + public static function getCacheKey($teamId) + { + return 'global_search_items_'.$teamId; + } + + public static function clearTeamCache($teamId) + { + Cache::forget(self::getCacheKey($teamId)); + } + + public function updatedSearchQuery() + { + $this->search(); + } + + private function loadSearchableItems() + { + // Try to get from Redis cache first + $cacheKey = self::getCacheKey(auth()->user()->currentTeam()->id); + + $this->allSearchableItems = Cache::remember($cacheKey, 300, function () { + ray()->showQueries(); + $items = collect(); + $team = auth()->user()->currentTeam(); + + // Get all applications + $applications = Application::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($app) { + // Collect all FQDNs from the application + $fqdns = collect([]); + + // For regular applications + if ($app->fqdn) { + $fqdns = collect(explode(',', $app->fqdn))->map(fn ($fqdn) => trim($fqdn)); + } + + // For docker compose based applications + if ($app->build_pack === 'dockercompose' && $app->docker_compose_domains) { + try { + $composeDomains = json_decode($app->docker_compose_domains, true); + if (is_array($composeDomains)) { + foreach ($composeDomains as $serviceName => $domains) { + if (is_array($domains)) { + $fqdns = $fqdns->merge($domains); + } + } + } + } catch (\Exception $e) { + // Ignore JSON parsing errors + } + } + + $fqdnsString = $fqdns->implode(' '); + + return [ + 'id' => $app->id, + 'name' => $app->name, + 'type' => 'application', + 'uuid' => $app->uuid, + 'description' => $app->description, + 'link' => $app->link(), + 'project' => $app->environment->project->name ?? null, + 'environment' => $app->environment->name ?? null, + 'fqdns' => $fqdns->take(2)->implode(', '), // Show first 2 FQDNs in UI + 'search_text' => strtolower($app->name.' '.$app->description.' '.$fqdnsString), + ]; + }); + + // Get all services + $services = Service::ownedByCurrentTeam() + ->with(['environment.project', 'applications']) + ->get() + ->map(function ($service) { + // Collect all FQDNs from service applications + $fqdns = collect([]); + foreach ($service->applications as $app) { + if ($app->fqdn) { + $appFqdns = collect(explode(',', $app->fqdn))->map(fn ($fqdn) => trim($fqdn)); + $fqdns = $fqdns->merge($appFqdns); + } + } + $fqdnsString = $fqdns->implode(' '); + + return [ + 'id' => $service->id, + 'name' => $service->name, + 'type' => 'service', + 'uuid' => $service->uuid, + 'description' => $service->description, + 'link' => $service->link(), + 'project' => $service->environment->project->name ?? null, + 'environment' => $service->environment->name ?? null, + 'fqdns' => $fqdns->take(2)->implode(', '), // Show first 2 FQDNs in UI + 'search_text' => strtolower($service->name.' '.$service->description.' '.$fqdnsString), + ]; + }); + + // Get all standalone databases + $databases = collect(); + + // PostgreSQL + $databases = $databases->merge( + StandalonePostgresql::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'postgresql', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' postgresql '.$db->description), + ]; + }) + ); + + // MySQL + $databases = $databases->merge( + StandaloneMysql::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'mysql', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' mysql '.$db->description), + ]; + }) + ); + + // MariaDB + $databases = $databases->merge( + StandaloneMariadb::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'mariadb', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' mariadb '.$db->description), + ]; + }) + ); + + // MongoDB + $databases = $databases->merge( + StandaloneMongodb::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'mongodb', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' mongodb '.$db->description), + ]; + }) + ); + + // Redis + $databases = $databases->merge( + StandaloneRedis::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'redis', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' redis '.$db->description), + ]; + }) + ); + + // KeyDB + $databases = $databases->merge( + StandaloneKeydb::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'keydb', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' keydb '.$db->description), + ]; + }) + ); + + // Dragonfly + $databases = $databases->merge( + StandaloneDragonfly::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'dragonfly', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' dragonfly '.$db->description), + ]; + }) + ); + + // Clickhouse + $databases = $databases->merge( + StandaloneClickhouse::ownedByCurrentTeam() + ->with(['environment.project']) + ->get() + ->map(function ($db) { + return [ + 'id' => $db->id, + 'name' => $db->name, + 'type' => 'database', + 'subtype' => 'clickhouse', + 'uuid' => $db->uuid, + 'description' => $db->description, + 'link' => $db->link(), + 'project' => $db->environment->project->name ?? null, + 'environment' => $db->environment->name ?? null, + 'search_text' => strtolower($db->name.' clickhouse '.$db->description), + ]; + }) + ); + + // Get all servers + $servers = Server::ownedByCurrentTeam() + ->get() + ->map(function ($server) { + return [ + 'id' => $server->id, + 'name' => $server->name, + 'type' => 'server', + 'uuid' => $server->uuid, + 'description' => $server->description, + 'link' => $server->url(), + 'project' => null, + 'environment' => null, + 'search_text' => strtolower($server->name.' '.$server->ip.' '.$server->description), + ]; + }); + + // Get all projects + $projects = Project::ownedByCurrentTeam() + ->withCount(['environments', 'applications', 'services']) + ->get() + ->map(function ($project) { + $resourceCount = $project->applications_count + $project->services_count; + $resourceSummary = $resourceCount > 0 + ? "{$resourceCount} resource".($resourceCount !== 1 ? 's' : '') + : 'No resources'; + + return [ + 'id' => $project->id, + 'name' => $project->name, + 'type' => 'project', + 'uuid' => $project->uuid, + 'description' => $project->description, + 'link' => $project->navigateTo(), + 'project' => null, + 'environment' => null, + 'resource_count' => $resourceSummary, + 'environment_count' => $project->environments_count, + 'search_text' => strtolower($project->name.' '.$project->description.' project'), + ]; + }); + + // Get all environments + $environments = Environment::query() + ->whereHas('project', function ($query) { + $query->where('team_id', auth()->user()->currentTeam()->id); + }) + ->with('project') + ->withCount(['applications', 'services']) + ->get() + ->map(function ($environment) { + $resourceCount = $environment->applications_count + $environment->services_count; + $resourceSummary = $resourceCount > 0 + ? "{$resourceCount} resource".($resourceCount !== 1 ? 's' : '') + : 'No resources'; + + // Build description with project context + $descriptionParts = []; + if ($environment->project) { + $descriptionParts[] = "Project: {$environment->project->name}"; + } + if ($environment->description) { + $descriptionParts[] = $environment->description; + } + if (empty($descriptionParts)) { + $descriptionParts[] = $resourceSummary; + } + + return [ + 'id' => $environment->id, + 'name' => $environment->name, + 'type' => 'environment', + 'uuid' => $environment->uuid, + 'description' => implode(' • ', $descriptionParts), + 'link' => route('project.resource.index', [ + 'project_uuid' => $environment->project->uuid, + 'environment_uuid' => $environment->uuid, + ]), + 'project' => $environment->project->name ?? null, + 'environment' => null, + 'resource_count' => $resourceSummary, + 'search_text' => strtolower($environment->name.' '.$environment->description.' '.$environment->project->name.' environment'), + ]; + }); + + // Merge all collections + $items = $items->merge($applications) + ->merge($services) + ->merge($databases) + ->merge($servers) + ->merge($projects) + ->merge($environments); + + return $items->toArray(); + }); + } + + private function search() + { + if (strlen($this->searchQuery) < 2) { + $this->searchResults = []; + + return; + } + + $query = strtolower($this->searchQuery); + + // Case-insensitive search in the items + $this->searchResults = collect($this->allSearchableItems) + ->filter(function ($item) use ($query) { + return str_contains($item['search_text'], $query); + }) + ->take(20) + ->values() + ->toArray(); + } + + public function render() + { + return view('livewire.global-search'); + } +} diff --git a/app/Livewire/Help.php b/app/Livewire/Help.php index 913710588..490515875 100644 --- a/app/Livewire/Help.php +++ b/app/Livewire/Help.php @@ -42,7 +42,7 @@ public function submit() 'content' => 'User: `'.auth()->user()?->email.'` with subject: `'.$this->subject.'` has the following problem: `'.$this->description.'`', ]); } else { - send_user_an_email($mail, auth()->user()?->email, 'hi@coollabs.io'); + send_user_an_email($mail, auth()->user()?->email, 'feedback@coollabs.io'); } $this->dispatch('success', 'Feedback sent.', 'We will get in touch with you as soon as possible.'); $this->reset('description', 'subject'); diff --git a/app/Livewire/Profile/Index.php b/app/Livewire/Profile/Index.php index a6b4dbe9e..4a419a12f 100644 --- a/app/Livewire/Profile/Index.php +++ b/app/Livewire/Profile/Index.php @@ -78,6 +78,8 @@ public function requestEmailChange() 'new_email' => ['required', 'email', 'unique:users,email'], ]); + $this->new_email = strtolower($this->new_email); + // Skip rate limiting in development mode if (! isDev()) { // Rate limit by current user's email (1 request per 2 minutes) @@ -90,7 +92,7 @@ public function requestEmailChange() } // Rate limit by new email address (3 requests per hour per email) - $newEmailKey = 'email-change:email:'.md5(strtolower($this->new_email)); + $newEmailKey = 'email-change:email:'.md5($this->new_email); if (! RateLimiter::attempt($newEmailKey, 3, function () {}, 3600)) { $this->dispatch('error', 'This email address has received too many verification requests. Please try again later.'); diff --git a/app/Livewire/Project/Application/Advanced.php b/app/Livewire/Project/Application/Advanced.php index 862dc20d8..ed15ab258 100644 --- a/app/Livewire/Project/Application/Advanced.php +++ b/app/Livewire/Project/Application/Advanced.php @@ -28,6 +28,9 @@ class Advanced extends Component #[Validate(['boolean'])] public bool $isPreviewDeploymentsEnabled = false; + #[Validate(['boolean'])] + public bool $isPrDeploymentsPublicEnabled = false; + #[Validate(['boolean'])] public bool $isAutoDeployEnabled = true; @@ -91,6 +94,7 @@ public function syncData(bool $toModel = false) $this->application->settings->is_git_lfs_enabled = $this->isGitLfsEnabled; $this->application->settings->is_git_shallow_clone_enabled = $this->isGitShallowCloneEnabled; $this->application->settings->is_preview_deployments_enabled = $this->isPreviewDeploymentsEnabled; + $this->application->settings->is_pr_deployments_public_enabled = $this->isPrDeploymentsPublicEnabled; $this->application->settings->is_auto_deploy_enabled = $this->isAutoDeployEnabled; $this->application->settings->is_log_drain_enabled = $this->isLogDrainEnabled; $this->application->settings->is_gpu_enabled = $this->isGpuEnabled; @@ -117,6 +121,7 @@ public function syncData(bool $toModel = false) $this->isGitLfsEnabled = $this->application->settings->is_git_lfs_enabled; $this->isGitShallowCloneEnabled = $this->application->settings->is_git_shallow_clone_enabled ?? false; $this->isPreviewDeploymentsEnabled = $this->application->settings->is_preview_deployments_enabled; + $this->isPrDeploymentsPublicEnabled = $this->application->settings->is_pr_deployments_public_enabled ?? false; $this->isAutoDeployEnabled = $this->application->settings->is_auto_deploy_enabled; $this->isGpuEnabled = $this->application->settings->is_gpu_enabled; $this->gpuDriver = $this->application->settings->gpu_driver; diff --git a/app/Livewire/Project/Application/DeploymentNavbar.php b/app/Livewire/Project/Application/DeploymentNavbar.php index 66f387fcf..dccd1e499 100644 --- a/app/Livewire/Project/Application/DeploymentNavbar.php +++ b/app/Livewire/Project/Application/DeploymentNavbar.php @@ -52,15 +52,24 @@ public function force_start() public function cancel() { - $kill_command = "docker rm -f {$this->application_deployment_queue->deployment_uuid}"; + $deployment_uuid = $this->application_deployment_queue->deployment_uuid; + $kill_command = "docker rm -f {$deployment_uuid}"; $build_server_id = $this->application_deployment_queue->build_server_id ?? $this->application->destination->server_id; $server_id = $this->application_deployment_queue->server_id ?? $this->application->destination->server_id; + + // First, mark the deployment as cancelled to prevent further processing + $this->application_deployment_queue->update([ + 'status' => ApplicationDeploymentStatus::CANCELLED_BY_USER->value, + ]); + try { if ($this->application->settings->is_build_server_enabled) { $server = Server::ownedByCurrentTeam()->find($build_server_id); } else { $server = Server::ownedByCurrentTeam()->find($server_id); } + + // Add cancellation log entry if ($this->application_deployment_queue->logs) { $previous_logs = json_decode($this->application_deployment_queue->logs, associative: true, flags: JSON_THROW_ON_ERROR); @@ -77,13 +86,35 @@ public function cancel() 'logs' => json_encode($previous_logs, flags: JSON_THROW_ON_ERROR), ]); } - instant_remote_process([$kill_command], $server); + + // Try to stop the helper container if it exists + // Check if container exists first + $checkCommand = "docker ps -a --filter name={$deployment_uuid} --format '{{.Names}}'"; + $containerExists = instant_remote_process([$checkCommand], $server); + + if ($containerExists && str($containerExists)->trim()->isNotEmpty()) { + // Container exists, kill it + instant_remote_process([$kill_command], $server); + } else { + // Container hasn't started yet + $this->application_deployment_queue->addLogEntry('Helper container not yet started. Deployment will be cancelled when job checks status.'); + } + + // Also try to kill any running process if we have a process ID + if ($this->application_deployment_queue->current_process_id) { + try { + $processKillCommand = "kill -9 {$this->application_deployment_queue->current_process_id}"; + instant_remote_process([$processKillCommand], $server); + } catch (\Throwable $e) { + // Process might already be gone, that's ok + } + } } catch (\Throwable $e) { + // Still mark as cancelled even if cleanup fails return handleError($e, $this); } finally { $this->application_deployment_queue->update([ 'current_process_id' => null, - 'status' => ApplicationDeploymentStatus::CANCELLED_BY_USER->value, ]); next_after_cancel($server); } diff --git a/app/Livewire/Project/Application/General.php b/app/Livewire/Project/Application/General.php index aa72b7c5f..ae9bd314b 100644 --- a/app/Livewire/Project/Application/General.php +++ b/app/Livewire/Project/Application/General.php @@ -210,10 +210,10 @@ public function mount() } } $this->parsedServiceDomains = $this->application->docker_compose_domains ? json_decode($this->application->docker_compose_domains, true) : []; - // Convert service names with dots to use underscores for HTML form binding + // Convert service names with dots and dashes to use underscores for HTML form binding $sanitizedDomains = []; foreach ($this->parsedServiceDomains as $serviceName => $domain) { - $sanitizedKey = str($serviceName)->slug('_')->toString(); + $sanitizedKey = str($serviceName)->replace('-', '_')->replace('.', '_')->toString(); $sanitizedDomains[$sanitizedKey] = $domain; } $this->parsedServiceDomains = $sanitizedDomains; @@ -305,10 +305,10 @@ public function loadComposeFile($isInit = false, $showToast = true) // Refresh parsedServiceDomains to reflect any changes in docker_compose_domains $this->application->refresh(); $this->parsedServiceDomains = $this->application->docker_compose_domains ? json_decode($this->application->docker_compose_domains, true) : []; - // Convert service names with dots to use underscores for HTML form binding + // Convert service names with dots and dashes to use underscores for HTML form binding $sanitizedDomains = []; foreach ($this->parsedServiceDomains as $serviceName => $domain) { - $sanitizedKey = str($serviceName)->slug('_')->toString(); + $sanitizedKey = str($serviceName)->replace('-', '_')->replace('.', '_')->toString(); $sanitizedDomains[$sanitizedKey] = $domain; } $this->parsedServiceDomains = $sanitizedDomains; @@ -334,7 +334,7 @@ public function generateDomain(string $serviceName) $uuid = new Cuid2; $domain = generateUrl(server: $this->application->destination->server, random: $uuid); - $sanitizedKey = str($serviceName)->slug('_')->toString(); + $sanitizedKey = str($serviceName)->replace('-', '_')->replace('.', '_')->toString(); $this->parsedServiceDomains[$sanitizedKey]['domain'] = $domain; // Convert back to original service names for storage @@ -344,7 +344,7 @@ public function generateDomain(string $serviceName) $originalServiceName = $key; if (isset($this->parsedServices['services'])) { foreach ($this->parsedServices['services'] as $originalName => $service) { - if (str($originalName)->slug('_')->toString() === $key) { + if (str($originalName)->replace('-', '_')->replace('.', '_')->toString() === $key) { $originalServiceName = $originalName; break; } @@ -487,7 +487,7 @@ public function checkFqdns($showToaster = true) $domains = str($this->application->fqdn)->trim()->explode(','); if ($this->application->additional_servers->count() === 0) { foreach ($domains as $domain) { - if (! validate_dns_entry($domain, $this->application->destination->server)) { + if (! validateDNSEntry($domain, $this->application->destination->server)) { $showToaster && $this->dispatch('error', 'Validating DNS failed.', "Make sure you have added the DNS records correctly.<br><br>$domain->{$this->application->destination->server->ip}<br><br>Check this <a target='_blank' class='underline dark:text-white' href='https://coolify.io/docs/knowledge-base/dns-configuration'>documentation</a> for further help."); } } @@ -547,9 +547,10 @@ public function submit($showToaster = true) $this->application->fqdn = str($this->application->fqdn)->replaceEnd(',', '')->trim(); $this->application->fqdn = str($this->application->fqdn)->replaceStart(',', '')->trim(); $this->application->fqdn = str($this->application->fqdn)->trim()->explode(',')->map(function ($domain) { + $domain = trim($domain); Url::fromString($domain, ['http', 'https']); - return str($domain)->trim()->lower(); + return str($domain)->lower(); }); $this->application->fqdn = $this->application->fqdn->unique()->implode(','); @@ -615,7 +616,7 @@ public function submit($showToaster = true) foreach ($this->parsedServiceDomains as $service) { $domain = data_get($service, 'domain'); if ($domain) { - if (! validate_dns_entry($domain, $this->application->destination->server)) { + if (! validateDNSEntry($domain, $this->application->destination->server)) { $showToaster && $this->dispatch('error', 'Validating DNS failed.', "Make sure you have added the DNS records correctly.<br><br>$domain->{$this->application->destination->server->ip}<br><br>Check this <a target='_blank' class='underline dark:text-white' href='https://coolify.io/docs/knowledge-base/dns-configuration'>documentation</a> for further help."); } } @@ -671,7 +672,7 @@ private function updateServiceEnvironmentVariables() $domains = collect(json_decode($this->application->docker_compose_domains, true)) ?? collect([]); foreach ($domains as $serviceName => $service) { - $serviceNameFormatted = str($serviceName)->upper()->replace('-', '_'); + $serviceNameFormatted = str($serviceName)->upper()->replace('-', '_')->replace('.', '_'); $domain = data_get($service, 'domain'); // Delete SERVICE_FQDN_ and SERVICE_URL_ variables if domain is removed $this->application->environment_variables()->where('resourceable_type', Application::class) @@ -703,7 +704,6 @@ private function updateServiceEnvironmentVariables() 'key' => "SERVICE_FQDN_{$serviceNameFormatted}", ], [ 'value' => $fqdnValue, - 'is_build_time' => false, 'is_preview' => false, ]); @@ -712,7 +712,6 @@ private function updateServiceEnvironmentVariables() 'key' => "SERVICE_URL_{$serviceNameFormatted}", ], [ 'value' => $urlValue, - 'is_build_time' => false, 'is_preview' => false, ]); // Create/update port-specific variables if port exists @@ -721,7 +720,6 @@ private function updateServiceEnvironmentVariables() 'key' => "SERVICE_FQDN_{$serviceNameFormatted}_{$port}", ], [ 'value' => $fqdnValue, - 'is_build_time' => false, 'is_preview' => false, ]); @@ -729,7 +727,6 @@ private function updateServiceEnvironmentVariables() 'key' => "SERVICE_URL_{$serviceNameFormatted}_{$port}", ], [ 'value' => $urlValue, - 'is_build_time' => false, 'is_preview' => false, ]); } diff --git a/app/Livewire/Project/Application/Previews.php b/app/Livewire/Project/Application/Previews.php index ebfd84489..1cb2ef2c5 100644 --- a/app/Livewire/Project/Application/Previews.php +++ b/app/Livewire/Project/Application/Previews.php @@ -77,7 +77,7 @@ public function save_preview($preview_id) $preview->fqdn = str($preview->fqdn)->replaceEnd(',', '')->trim(); $preview->fqdn = str($preview->fqdn)->replaceStart(',', '')->trim(); $preview->fqdn = str($preview->fqdn)->trim()->lower(); - if (! validate_dns_entry($preview->fqdn, $this->application->destination->server)) { + if (! validateDNSEntry($preview->fqdn, $this->application->destination->server)) { $this->dispatch('error', 'Validating DNS failed.', "Make sure you have added the DNS records correctly.<br><br>$preview->fqdn->{$this->application->destination->server->ip}<br><br>Check this <a target='_blank' class='underline dark:text-white' href='https://coolify.io/docs/knowledge-base/dns-configuration'>documentation</a> for further help."); $success = false; } @@ -231,6 +231,18 @@ protected function setDeploymentUuid() $this->parameters['deployment_uuid'] = $this->deployment_uuid; } + private function stopContainers(array $containers, $server) + { + $containersToStop = collect($containers)->pluck('Names')->toArray(); + + foreach ($containersToStop as $containerName) { + instant_remote_process(command: [ + "docker stop --time=30 $containerName", + "docker rm -f $containerName", + ], server: $server, throwError: false); + } + } + public function stop(int $pull_request_id) { $this->authorize('deploy', $this->application); diff --git a/app/Livewire/Project/Application/PreviewsCompose.php b/app/Livewire/Project/Application/PreviewsCompose.php index 2632509ea..cfb364b6d 100644 --- a/app/Livewire/Project/Application/PreviewsCompose.php +++ b/app/Livewire/Project/Application/PreviewsCompose.php @@ -72,10 +72,13 @@ public function generate() $template = $this->preview->application->preview_url_template; $host = $url->getHost(); $schema = $url->getScheme(); + $portInt = $url->getPort(); + $port = $portInt !== null ? ':'.$portInt : ''; $random = new Cuid2; $preview_fqdn = str_replace('{{random}}', $random, $template); $preview_fqdn = str_replace('{{domain}}', $host, $preview_fqdn); $preview_fqdn = str_replace('{{pr_id}}', $this->preview->pull_request_id, $preview_fqdn); + $preview_fqdn = str_replace('{{port}}', $port, $preview_fqdn); $preview_fqdn = "$schema://$preview_fqdn"; } diff --git a/app/Livewire/Project/Application/Source.php b/app/Livewire/Project/Application/Source.php index 29be68b6c..ab2517f2b 100644 --- a/app/Livewire/Project/Application/Source.php +++ b/app/Livewire/Project/Application/Source.php @@ -47,6 +47,21 @@ public function mount() } } + public function updatedGitRepository() + { + $this->gitRepository = trim($this->gitRepository); + } + + public function updatedGitBranch() + { + $this->gitBranch = trim($this->gitBranch); + } + + public function updatedGitCommitSha() + { + $this->gitCommitSha = trim($this->gitCommitSha); + } + public function syncData(bool $toModel = false) { if ($toModel) { @@ -57,6 +72,9 @@ public function syncData(bool $toModel = false) 'git_commit_sha' => $this->gitCommitSha, 'private_key_id' => $this->privateKeyId, ]); + // Refresh to get the trimmed values from the model + $this->application->refresh(); + $this->syncData(false); } else { $this->gitRepository = $this->application->git_repository; $this->gitBranch = $this->application->git_branch; diff --git a/app/Livewire/Project/CloneMe.php b/app/Livewire/Project/CloneMe.php index be9de139f..3b3e42619 100644 --- a/app/Livewire/Project/CloneMe.php +++ b/app/Livewire/Project/CloneMe.php @@ -2,7 +2,6 @@ namespace App\Livewire\Project; -use App\Actions\Application\StopApplication; use App\Actions\Database\StartDatabase; use App\Actions\Database\StopDatabase; use App\Actions\Service\StartService; @@ -128,144 +127,10 @@ public function clone(string $type) $databases = $this->environment->databases(); $services = $this->environment->services; foreach ($applications as $application) { - $applicationSettings = $application->settings; - - $uuid = (string) new Cuid2; - $url = $application->fqdn; - if ($this->server->proxyType() !== 'NONE' && $applicationSettings->is_container_label_readonly_enabled === true) { - $url = generateUrl(server: $this->server, random: $uuid); - } - - $newApplication = $application->replicate([ - 'id', - 'created_at', - 'updated_at', - 'additional_servers_count', - 'additional_networks_count', - ])->fill([ - 'uuid' => $uuid, - 'fqdn' => $url, - 'status' => 'exited', + $selectedDestination = $this->servers->flatMap(fn ($server) => $server->destinations())->where('id', $this->selectedDestination)->first(); + clone_application($application, $selectedDestination, [ 'environment_id' => $environment->id, - 'destination_id' => $this->selectedDestination, - ]); - $newApplication->save(); - - if ($newApplication->destination->server->proxyType() !== 'NONE' && $applicationSettings->is_container_label_readonly_enabled === true) { - $customLabels = str(implode('|coolify|', generateLabelsApplication($newApplication)))->replace('|coolify|', "\n"); - $newApplication->custom_labels = base64_encode($customLabels); - $newApplication->save(); - } - - $newApplication->settings()->delete(); - if ($applicationSettings) { - $newApplicationSettings = $applicationSettings->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'application_id' => $newApplication->id, - ]); - $newApplicationSettings->save(); - } - - $tags = $application->tags; - foreach ($tags as $tag) { - $newApplication->tags()->attach($tag->id); - } - - $scheduledTasks = $application->scheduled_tasks()->get(); - foreach ($scheduledTasks as $task) { - $newTask = $task->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => (string) new Cuid2, - 'application_id' => $newApplication->id, - 'team_id' => currentTeam()->id, - ]); - $newTask->save(); - } - - $applicationPreviews = $application->previews()->get(); - foreach ($applicationPreviews as $preview) { - $newPreview = $preview->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'application_id' => $newApplication->id, - 'status' => 'exited', - ]); - $newPreview->save(); - } - - $persistentVolumes = $application->persistentStorages()->get(); - foreach ($persistentVolumes as $volume) { - $newName = ''; - if (str_starts_with($volume->name, $application->uuid)) { - $newName = str($volume->name)->replace($application->uuid, $newApplication->uuid); - } else { - $newName = $newApplication->uuid.'-'.$volume->name; - } - - $newPersistentVolume = $volume->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'name' => $newName, - 'resource_id' => $newApplication->id, - ]); - $newPersistentVolume->save(); - - if ($this->cloneVolumeData) { - try { - StopApplication::dispatch($application, false, false); - $sourceVolume = $volume->name; - $targetVolume = $newPersistentVolume->name; - $sourceServer = $application->destination->server; - $targetServer = $newApplication->destination->server; - - VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); - - queue_application_deployment( - deployment_uuid: (string) new Cuid2, - application: $application, - server: $sourceServer, - destination: $application->destination, - no_questions_asked: true - ); - } catch (\Exception $e) { - \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); - } - } - } - - $fileStorages = $application->fileStorages()->get(); - foreach ($fileStorages as $storage) { - $newStorage = $storage->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resource_id' => $newApplication->id, - ]); - $newStorage->save(); - } - - $environmentVaribles = $application->environment_variables()->get(); - foreach ($environmentVaribles as $environmentVarible) { - $newEnvironmentVariable = $environmentVarible->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resourceable_id' => $newApplication->id, - ]); - $newEnvironmentVariable->save(); - } + ], $this->cloneVolumeData); } foreach ($databases as $database) { diff --git a/app/Livewire/Project/New/DockerCompose.php b/app/Livewire/Project/New/DockerCompose.php index 7c81e810c..5cda1dedd 100644 --- a/app/Livewire/Project/New/DockerCompose.php +++ b/app/Livewire/Project/New/DockerCompose.php @@ -63,7 +63,6 @@ public function submit() EnvironmentVariable::create([ 'key' => $key, 'value' => $variable, - 'is_build_time' => false, 'is_preview' => false, 'resourceable_id' => $service->id, 'resourceable_type' => $service->getMorphClass(), diff --git a/app/Livewire/Project/New/GithubPrivateRepository.php b/app/Livewire/Project/New/GithubPrivateRepository.php index 0f496e6db..a2071931e 100644 --- a/app/Livewire/Project/New/GithubPrivateRepository.php +++ b/app/Livewire/Project/New/GithubPrivateRepository.php @@ -143,7 +143,13 @@ public function loadBranches() protected function loadBranchByPage() { - $response = Http::withToken($this->token)->get("{$this->github_app->api_url}/repos/{$this->selected_repository_owner}/{$this->selected_repository_repo}/branches?per_page=100&page={$this->page}"); + $response = Http::GitHub($this->github_app->api_url, $this->token) + ->timeout(20) + ->retry(3, 200, throw: false) + ->get("/repos/{$this->selected_repository_owner}/{$this->selected_repository_repo}/branches", [ + 'per_page' => 100, + 'page' => $this->page, + ]); $json = $response->json(); if ($response->status() !== 200) { return $this->dispatch('error', $json['message']); diff --git a/app/Livewire/Project/New/GithubPrivateRepositoryDeployKey.php b/app/Livewire/Project/New/GithubPrivateRepositoryDeployKey.php index 5ff8f9137..77b106200 100644 --- a/app/Livewire/Project/New/GithubPrivateRepositoryDeployKey.php +++ b/app/Livewire/Project/New/GithubPrivateRepositoryDeployKey.php @@ -90,7 +90,7 @@ protected function rules() public function mount() { if (isDev()) { - $this->repository_url = 'https://github.com/coollabsio/coolify-examples'; + $this->repository_url = 'https://github.com/coollabsio/coolify-examples/tree/v4.x'; } $this->parameters = get_route_parameters(); $this->query = request()->query(); diff --git a/app/Livewire/Project/New/PublicGitRepository.php b/app/Livewire/Project/New/PublicGitRepository.php index f5978aea1..89814ee7f 100644 --- a/app/Livewire/Project/New/PublicGitRepository.php +++ b/app/Livewire/Project/New/PublicGitRepository.php @@ -100,7 +100,7 @@ protected function rules() public function mount() { if (isDev()) { - $this->repository_url = 'https://github.com/coollabsio/coolify-examples'; + $this->repository_url = 'https://github.com/coollabsio/coolify-examples/tree/v4.x'; $this->port = 3000; } $this->parameters = get_route_parameters(); @@ -176,13 +176,16 @@ public function loadBranch() str($this->repository_url)->startsWith('http://')) && ! str($this->repository_url)->endsWith('.git') && (! str($this->repository_url)->contains('github.com') || - ! str($this->repository_url)->contains('git.sr.ht')) + ! str($this->repository_url)->contains('git.sr.ht')) && + ! str($this->repository_url)->contains('tangled') ) { + $this->repository_url = $this->repository_url.'.git'; } if (str($this->repository_url)->contains('github.com') && str($this->repository_url)->endsWith('.git')) { $this->repository_url = str($this->repository_url)->beforeLast('.git')->value(); } + } catch (\Throwable $e) { return handleError($e, $this); } @@ -190,6 +193,9 @@ public function loadBranch() $this->branchFound = false; $this->getGitSource(); $this->getBranch(); + if (str($this->repository_url)->contains('tangled')) { + $this->git_branch = 'master'; + } $this->selectedBranch = $this->git_branch; } catch (\Throwable $e) { if ($this->rate_limit_remaining == 0) { diff --git a/app/Livewire/Project/Resource/Create.php b/app/Livewire/Project/Resource/Create.php index 3dbe4230c..73960d288 100644 --- a/app/Livewire/Project/Resource/Create.php +++ b/app/Livewire/Project/Resource/Create.php @@ -97,7 +97,6 @@ public function mount() 'value' => $value, 'resourceable_id' => $service->id, 'resourceable_type' => $service->getMorphClass(), - 'is_build_time' => false, 'is_preview' => false, ]); } diff --git a/app/Livewire/Project/Service/EditDomain.php b/app/Livewire/Project/Service/EditDomain.php index 5ce170b99..7c718393d 100644 --- a/app/Livewire/Project/Service/EditDomain.php +++ b/app/Livewire/Project/Service/EditDomain.php @@ -41,9 +41,10 @@ public function submit() $this->application->fqdn = str($this->application->fqdn)->replaceEnd(',', '')->trim(); $this->application->fqdn = str($this->application->fqdn)->replaceStart(',', '')->trim(); $this->application->fqdn = str($this->application->fqdn)->trim()->explode(',')->map(function ($domain) { + $domain = trim($domain); Url::fromString($domain, ['http', 'https']); - return str($domain)->trim()->lower(); + return str($domain)->lower(); }); $this->application->fqdn = $this->application->fqdn->unique()->implode(','); $warning = sslipDomainWarning($this->application->fqdn); diff --git a/app/Livewire/Project/Service/ServiceApplicationView.php b/app/Livewire/Project/Service/ServiceApplicationView.php index 3ac12cfe9..e37b6ad86 100644 --- a/app/Livewire/Project/Service/ServiceApplicationView.php +++ b/app/Livewire/Project/Service/ServiceApplicationView.php @@ -149,9 +149,10 @@ public function submit() $this->application->fqdn = str($this->application->fqdn)->replaceEnd(',', '')->trim(); $this->application->fqdn = str($this->application->fqdn)->replaceStart(',', '')->trim(); $this->application->fqdn = str($this->application->fqdn)->trim()->explode(',')->map(function ($domain) { + $domain = trim($domain); Url::fromString($domain, ['http', 'https']); - return str($domain)->trim()->lower(); + return str($domain)->lower(); }); $this->application->fqdn = $this->application->fqdn->unique()->implode(','); $warning = sslipDomainWarning($this->application->fqdn); diff --git a/app/Livewire/Project/Service/Storage.php b/app/Livewire/Project/Service/Storage.php index 26cd54425..db171db24 100644 --- a/app/Livewire/Project/Service/Storage.php +++ b/app/Livewire/Project/Service/Storage.php @@ -14,6 +14,22 @@ class Storage extends Component public $fileStorage; + public $isSwarm = false; + + public string $name = ''; + + public string $mount_path = ''; + + public ?string $host_path = null; + + public string $file_storage_path = ''; + + public ?string $file_storage_content = null; + + public string $file_storage_directory_source = ''; + + public string $file_storage_directory_destination = ''; + public function getListeners() { $teamId = auth()->user()->currentTeam()->id; @@ -27,6 +43,18 @@ public function getListeners() public function mount() { + if (str($this->resource->getMorphClass())->contains('Standalone')) { + $this->file_storage_directory_source = database_configuration_dir()."/{$this->resource->uuid}"; + } else { + $this->file_storage_directory_source = application_configuration_dir()."/{$this->resource->uuid}"; + } + + if ($this->resource->getMorphClass() === \App\Models\Application::class) { + if ($this->resource->destination->server->isSwarm()) { + $this->isSwarm = true; + } + } + $this->refreshStorages(); } @@ -39,30 +67,151 @@ public function refreshStoragesFromEvent() public function refreshStorages() { $this->fileStorage = $this->resource->fileStorages()->get(); - $this->dispatch('$refresh'); + $this->resource->refresh(); } - public function addNewVolume($data) + public function getFilesProperty() + { + return $this->fileStorage->where('is_directory', false); + } + + public function getDirectoriesProperty() + { + return $this->fileStorage->where('is_directory', true); + } + + public function getVolumeCountProperty() + { + return $this->resource->persistentStorages()->count(); + } + + public function getFileCountProperty() + { + return $this->files->count(); + } + + public function getDirectoryCountProperty() + { + return $this->directories->count(); + } + + public function submitPersistentVolume() { try { $this->authorize('update', $this->resource); + $this->validate([ + 'name' => 'required|string', + 'mount_path' => 'required|string', + 'host_path' => $this->isSwarm ? 'required|string' : 'string|nullable', + ]); + + $name = $this->resource->uuid.'-'.$this->name; + LocalPersistentVolume::create([ - 'name' => $data['name'], - 'mount_path' => $data['mount_path'], - 'host_path' => $data['host_path'], + 'name' => $name, + 'mount_path' => $this->mount_path, + 'host_path' => $this->host_path, 'resource_id' => $this->resource->id, 'resource_type' => $this->resource->getMorphClass(), ]); $this->resource->refresh(); - $this->dispatch('success', 'Storage added successfully'); - $this->dispatch('clearAddStorage'); - $this->dispatch('refreshStorages'); + $this->dispatch('success', 'Volume added successfully'); + $this->dispatch('closeStorageModal', 'volume'); + $this->clearForm(); + $this->refreshStorages(); } catch (\Throwable $e) { return handleError($e, $this); } } + public function submitFileStorage() + { + try { + $this->authorize('update', $this->resource); + + $this->validate([ + 'file_storage_path' => 'required|string', + 'file_storage_content' => 'nullable|string', + ]); + + $this->file_storage_path = trim($this->file_storage_path); + $this->file_storage_path = str($this->file_storage_path)->start('/')->value(); + + if ($this->resource->getMorphClass() === \App\Models\Application::class) { + $fs_path = application_configuration_dir().'/'.$this->resource->uuid.$this->file_storage_path; + } elseif (str($this->resource->getMorphClass())->contains('Standalone')) { + $fs_path = database_configuration_dir().'/'.$this->resource->uuid.$this->file_storage_path; + } else { + throw new \Exception('No valid resource type for file mount storage type!'); + } + + \App\Models\LocalFileVolume::create([ + 'fs_path' => $fs_path, + 'mount_path' => $this->file_storage_path, + 'content' => $this->file_storage_content, + 'is_directory' => false, + 'resource_id' => $this->resource->id, + 'resource_type' => get_class($this->resource), + ]); + + $this->dispatch('success', 'File mount added successfully'); + $this->dispatch('closeStorageModal', 'file'); + $this->clearForm(); + $this->refreshStorages(); + } catch (\Throwable $e) { + return handleError($e, $this); + } + } + + public function submitFileStorageDirectory() + { + try { + $this->authorize('update', $this->resource); + + $this->validate([ + 'file_storage_directory_source' => 'required|string', + 'file_storage_directory_destination' => 'required|string', + ]); + + $this->file_storage_directory_source = trim($this->file_storage_directory_source); + $this->file_storage_directory_source = str($this->file_storage_directory_source)->start('/')->value(); + $this->file_storage_directory_destination = trim($this->file_storage_directory_destination); + $this->file_storage_directory_destination = str($this->file_storage_directory_destination)->start('/')->value(); + + \App\Models\LocalFileVolume::create([ + 'fs_path' => $this->file_storage_directory_source, + 'mount_path' => $this->file_storage_directory_destination, + 'is_directory' => true, + 'resource_id' => $this->resource->id, + 'resource_type' => get_class($this->resource), + ]); + + $this->dispatch('success', 'Directory mount added successfully'); + $this->dispatch('closeStorageModal', 'directory'); + $this->clearForm(); + $this->refreshStorages(); + } catch (\Throwable $e) { + return handleError($e, $this); + } + } + + public function clearForm() + { + $this->name = ''; + $this->mount_path = ''; + $this->host_path = null; + $this->file_storage_path = ''; + $this->file_storage_content = null; + $this->file_storage_directory_destination = ''; + + if (str($this->resource->getMorphClass())->contains('Standalone')) { + $this->file_storage_directory_source = database_configuration_dir()."/{$this->resource->uuid}"; + } else { + $this->file_storage_directory_source = application_configuration_dir()."/{$this->resource->uuid}"; + } + } + public function render() { return view('livewire.project.service.storage'); diff --git a/app/Livewire/Project/Shared/ConfigurationChecker.php b/app/Livewire/Project/Shared/ConfigurationChecker.php index ab9f3785d..ce9ce7780 100644 --- a/app/Livewire/Project/Shared/ConfigurationChecker.php +++ b/app/Livewire/Project/Shared/ConfigurationChecker.php @@ -20,7 +20,15 @@ class ConfigurationChecker extends Component public Application|Service|StandaloneRedis|StandalonePostgresql|StandaloneMongodb|StandaloneMysql|StandaloneMariadb|StandaloneKeydb|StandaloneDragonfly|StandaloneClickhouse $resource; - protected $listeners = ['configurationChanged']; + public function getListeners() + { + $teamId = auth()->user()->currentTeam()->id; + + return [ + "echo-private:team.{$teamId},ApplicationConfigurationChanged" => 'configurationChanged', + 'configurationChanged' => 'configurationChanged', + ]; + } public function mount() { diff --git a/app/Livewire/Project/Shared/EnvironmentVariable/Add.php b/app/Livewire/Project/Shared/EnvironmentVariable/Add.php index cf7843f84..5f5e12e0a 100644 --- a/app/Livewire/Project/Shared/EnvironmentVariable/Add.php +++ b/app/Livewire/Project/Shared/EnvironmentVariable/Add.php @@ -2,12 +2,13 @@ namespace App\Livewire\Project\Shared\EnvironmentVariable; +use App\Traits\EnvironmentVariableAnalyzer; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; use Livewire\Component; class Add extends Component { - use AuthorizesRequests; + use AuthorizesRequests, EnvironmentVariableAnalyzer; public $parameters; @@ -19,33 +20,40 @@ class Add extends Component public ?string $value = null; - public bool $is_build_time = false; - public bool $is_multiline = false; public bool $is_literal = false; + public bool $is_runtime = true; + + public bool $is_buildtime = true; + + public array $problematicVariables = []; + protected $listeners = ['clearAddEnv' => 'clear']; protected $rules = [ 'key' => 'required|string', 'value' => 'nullable', - 'is_build_time' => 'required|boolean', 'is_multiline' => 'required|boolean', 'is_literal' => 'required|boolean', + 'is_runtime' => 'required|boolean', + 'is_buildtime' => 'required|boolean', ]; protected $validationAttributes = [ 'key' => 'key', 'value' => 'value', - 'is_build_time' => 'build', 'is_multiline' => 'multiline', 'is_literal' => 'literal', + 'is_runtime' => 'runtime', + 'is_buildtime' => 'buildtime', ]; public function mount() { $this->parameters = get_route_parameters(); + $this->problematicVariables = self::getProblematicVariablesForFrontend(); } public function submit() @@ -54,9 +62,10 @@ public function submit() $this->dispatch('saveKey', [ 'key' => $this->key, 'value' => $this->value, - 'is_build_time' => $this->is_build_time, 'is_multiline' => $this->is_multiline, 'is_literal' => $this->is_literal, + 'is_runtime' => $this->is_runtime, + 'is_buildtime' => $this->is_buildtime, 'is_preview' => $this->is_preview, ]); $this->clear(); @@ -66,8 +75,9 @@ public function clear() { $this->key = ''; $this->value = ''; - $this->is_build_time = false; $this->is_multiline = false; $this->is_literal = false; + $this->is_runtime = true; + $this->is_buildtime = true; } } diff --git a/app/Livewire/Project/Shared/EnvironmentVariable/All.php b/app/Livewire/Project/Shared/EnvironmentVariable/All.php index 3631a43c8..639c025c7 100644 --- a/app/Livewire/Project/Shared/EnvironmentVariable/All.php +++ b/app/Livewire/Project/Shared/EnvironmentVariable/All.php @@ -25,6 +25,8 @@ class All extends Component public bool $is_env_sorting_enabled = false; + public bool $use_build_secrets = false; + protected $listeners = [ 'saveKey' => 'submit', 'refreshEnvs', @@ -34,13 +36,14 @@ class All extends Component public function mount() { $this->is_env_sorting_enabled = data_get($this->resource, 'settings.is_env_sorting_enabled', false); + $this->use_build_secrets = data_get($this->resource, 'settings.use_build_secrets', false); $this->resourceClass = get_class($this->resource); $resourceWithPreviews = [\App\Models\Application::class]; $simpleDockerfile = filled(data_get($this->resource, 'dockerfile')); if (str($this->resourceClass)->contains($resourceWithPreviews) && ! $simpleDockerfile) { $this->showPreview = true; } - $this->sortEnvironmentVariables(); + $this->getDevView(); } public function instantSave() @@ -49,34 +52,38 @@ public function instantSave() $this->authorize('manageEnvironment', $this->resource); $this->resource->settings->is_env_sorting_enabled = $this->is_env_sorting_enabled; + $this->resource->settings->use_build_secrets = $this->use_build_secrets; $this->resource->settings->save(); - $this->sortEnvironmentVariables(); + $this->getDevView(); $this->dispatch('success', 'Environment variable settings updated.'); } catch (\Throwable $e) { return handleError($e, $this); } } - public function sortEnvironmentVariables() + public function getEnvironmentVariablesProperty() { if ($this->is_env_sorting_enabled === false) { - if ($this->resource->environment_variables) { - $this->resource->environment_variables = $this->resource->environment_variables->sortBy('order')->values(); - } - - if ($this->resource->environment_variables_preview) { - $this->resource->environment_variables_preview = $this->resource->environment_variables_preview->sortBy('order')->values(); - } + return $this->resource->environment_variables()->orderBy('order')->get(); } - $this->getDevView(); + return $this->resource->environment_variables; + } + + public function getEnvironmentVariablesPreviewProperty() + { + if ($this->is_env_sorting_enabled === false) { + return $this->resource->environment_variables_preview()->orderBy('order')->get(); + } + + return $this->resource->environment_variables_preview; } public function getDevView() { - $this->variables = $this->formatEnvironmentVariables($this->resource->environment_variables); + $this->variables = $this->formatEnvironmentVariables($this->environmentVariables); if ($this->showPreview) { - $this->variablesPreview = $this->formatEnvironmentVariables($this->resource->environment_variables_preview); + $this->variablesPreview = $this->formatEnvironmentVariables($this->environmentVariablesPreview); } } @@ -97,7 +104,7 @@ private function formatEnvironmentVariables($variables) public function switch() { $this->view = $this->view === 'normal' ? 'dev' : 'normal'; - $this->sortEnvironmentVariables(); + $this->getDevView(); } public function submit($data = null) @@ -111,7 +118,7 @@ public function submit($data = null) } $this->updateOrder(); - $this->sortEnvironmentVariables(); + $this->getDevView(); } catch (\Throwable $e) { return handleError($e, $this); } finally { @@ -212,9 +219,10 @@ private function createEnvironmentVariable($data) $environment = new EnvironmentVariable; $environment->key = $data['key']; $environment->value = $data['value']; - $environment->is_build_time = $data['is_build_time'] ?? false; $environment->is_multiline = $data['is_multiline'] ?? false; $environment->is_literal = $data['is_literal'] ?? false; + $environment->is_runtime = $data['is_runtime'] ?? true; + $environment->is_buildtime = $data['is_buildtime'] ?? true; $environment->is_preview = $data['is_preview'] ?? false; $environment->resourceable_id = $this->resource->id; $environment->resourceable_type = $this->resource->getMorphClass(); @@ -257,7 +265,7 @@ private function updateOrCreateVariables($isPreview, $variables) { $count = 0; foreach ($variables as $key => $value) { - if (str($key)->startsWith('SERVICE_FQDN') || str($key)->startsWith('SERVICE_URL')) { + if (str($key)->startsWith('SERVICE_FQDN') || str($key)->startsWith('SERVICE_URL') || str($key)->startsWith('SERVICE_NAME')) { continue; } $method = $isPreview ? 'environment_variables_preview' : 'environment_variables'; @@ -276,7 +284,6 @@ private function updateOrCreateVariables($isPreview, $variables) $environment = new EnvironmentVariable; $environment->key = $key; $environment->value = $value; - $environment->is_build_time = false; $environment->is_multiline = false; $environment->is_preview = $isPreview; $environment->resourceable_id = $this->resource->id; @@ -293,7 +300,6 @@ private function updateOrCreateVariables($isPreview, $variables) public function refreshEnvs() { $this->resource->refresh(); - $this->sortEnvironmentVariables(); $this->getDevView(); } } diff --git a/app/Livewire/Project/Shared/EnvironmentVariable/Show.php b/app/Livewire/Project/Shared/EnvironmentVariable/Show.php index 1a9daf77b..3b8d244cc 100644 --- a/app/Livewire/Project/Shared/EnvironmentVariable/Show.php +++ b/app/Livewire/Project/Shared/EnvironmentVariable/Show.php @@ -4,13 +4,14 @@ use App\Models\EnvironmentVariable as ModelsEnvironmentVariable; use App\Models\SharedEnvironmentVariable; +use App\Traits\EnvironmentVariableAnalyzer; use App\Traits\EnvironmentVariableProtection; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; use Livewire\Component; class Show extends Component { - use AuthorizesRequests, EnvironmentVariableProtection; + use AuthorizesRequests, EnvironmentVariableAnalyzer, EnvironmentVariableProtection; public $parameters; @@ -32,20 +33,24 @@ class Show extends Component public bool $is_shared = false; - public bool $is_build_time = false; - public bool $is_multiline = false; public bool $is_literal = false; public bool $is_shown_once = false; + public bool $is_runtime = true; + + public bool $is_buildtime = true; + public bool $is_required = false; public bool $is_really_required = false; public bool $is_redis_credential = false; + public array $problematicVariables = []; + protected $listeners = [ 'refreshEnvs' => 'refresh', 'refresh', @@ -55,10 +60,11 @@ class Show extends Component protected $rules = [ 'key' => 'required|string', 'value' => 'nullable', - 'is_build_time' => 'required|boolean', 'is_multiline' => 'required|boolean', 'is_literal' => 'required|boolean', 'is_shown_once' => 'required|boolean', + 'is_runtime' => 'required|boolean', + 'is_buildtime' => 'required|boolean', 'real_value' => 'nullable', 'is_required' => 'required|boolean', ]; @@ -74,6 +80,7 @@ public function mount() if ($this->type === 'standalone-redis' && ($this->env->key === 'REDIS_PASSWORD' || $this->env->key === 'REDIS_USERNAME')) { $this->is_redis_credential = true; } + $this->problematicVariables = self::getProblematicVariablesForFrontend(); } public function getResourceProperty() @@ -101,8 +108,9 @@ public function syncData(bool $toModel = false) ]); } else { $this->validate(); - $this->env->is_build_time = $this->is_build_time; $this->env->is_required = $this->is_required; + $this->env->is_runtime = $this->is_runtime; + $this->env->is_buildtime = $this->is_buildtime; $this->env->is_shared = $this->is_shared; } $this->env->key = $this->key; @@ -114,10 +122,11 @@ public function syncData(bool $toModel = false) } else { $this->key = $this->env->key; $this->value = $this->env->value; - $this->is_build_time = $this->env->is_build_time ?? false; $this->is_multiline = $this->env->is_multiline; $this->is_literal = $this->env->is_literal; $this->is_shown_once = $this->env->is_shown_once; + $this->is_runtime = $this->env->is_runtime ?? true; + $this->is_buildtime = $this->env->is_buildtime ?? true; $this->is_required = $this->env->is_required ?? false; $this->is_really_required = $this->env->is_really_required ?? false; $this->is_shared = $this->env->is_shared ?? false; @@ -128,7 +137,7 @@ public function syncData(bool $toModel = false) public function checkEnvs() { $this->isDisabled = false; - if (str($this->env->key)->startsWith('SERVICE_FQDN') || str($this->env->key)->startsWith('SERVICE_URL')) { + if (str($this->env->key)->startsWith('SERVICE_FQDN') || str($this->env->key)->startsWith('SERVICE_URL') || str($this->env->key)->startsWith('SERVICE_NAME')) { $this->isDisabled = true; } if ($this->env->is_shown_once) { @@ -139,9 +148,6 @@ public function checkEnvs() public function serialize() { data_forget($this->env, 'real_value'); - if ($this->env->getMorphClass() === \App\Models\SharedEnvironmentVariable::class) { - data_forget($this->env, 'is_build_time'); - } } public function lock() diff --git a/app/Livewire/Project/Shared/HealthChecks.php b/app/Livewire/Project/Shared/HealthChecks.php index ae94f7cf2..c0714fe03 100644 --- a/app/Livewire/Project/Shared/HealthChecks.php +++ b/app/Livewire/Project/Shared/HealthChecks.php @@ -47,6 +47,24 @@ public function submit() } } + public function toggleHealthcheck() + { + try { + $this->authorize('update', $this->resource); + $wasEnabled = $this->resource->health_check_enabled; + $this->resource->health_check_enabled = ! $this->resource->health_check_enabled; + $this->resource->save(); + + if ($this->resource->health_check_enabled && ! $wasEnabled && $this->resource->isRunning()) { + $this->dispatch('info', 'Health check has been enabled. A restart is required to apply the new settings.'); + } else { + $this->dispatch('success', 'Health check '.($this->resource->health_check_enabled ? 'enabled' : 'disabled').'.'); + } + } catch (\Throwable $e) { + return handleError($e, $this); + } + } + public function render() { return view('livewire.project.shared.health-checks'); diff --git a/app/Livewire/Project/Shared/Metrics.php b/app/Livewire/Project/Shared/Metrics.php index fdc35fc0f..e5b87b48c 100644 --- a/app/Livewire/Project/Shared/Metrics.php +++ b/app/Livewire/Project/Shared/Metrics.php @@ -8,7 +8,7 @@ class Metrics extends Component { public $resource; - public $chartId = 'container-cpu'; + public $chartId = 'metrics'; public $data; diff --git a/app/Livewire/Project/Shared/ResourceOperations.php b/app/Livewire/Project/Shared/ResourceOperations.php index 28a6380d5..47b3534a2 100644 --- a/app/Livewire/Project/Shared/ResourceOperations.php +++ b/app/Livewire/Project/Shared/ResourceOperations.php @@ -2,7 +2,6 @@ namespace App\Livewire\Project\Shared; -use App\Actions\Application\StopApplication; use App\Actions\Database\StartDatabase; use App\Actions\Database\StopDatabase; use App\Actions\Service\StartService; @@ -61,145 +60,7 @@ public function cloneTo($destination_id) $server = $new_destination->server; if ($this->resource->getMorphClass() === \App\Models\Application::class) { - $name = 'clone-of-'.str($this->resource->name)->limit(20).'-'.$uuid; - $applicationSettings = $this->resource->settings; - $url = $this->resource->fqdn; - - if ($server->proxyType() !== 'NONE' && $applicationSettings->is_container_label_readonly_enabled === true) { - $url = generateUrl(server: $server, random: $uuid); - } - - $new_resource = $this->resource->replicate([ - 'id', - 'created_at', - 'updated_at', - 'additional_servers_count', - 'additional_networks_count', - ])->fill([ - 'uuid' => $uuid, - 'name' => $name, - 'fqdn' => $url, - 'status' => 'exited', - 'destination_id' => $new_destination->id, - ]); - $new_resource->save(); - - if ($new_resource->destination->server->proxyType() !== 'NONE' && $applicationSettings->is_container_label_readonly_enabled === true) { - $customLabels = str(implode('|coolify|', generateLabelsApplication($new_resource)))->replace('|coolify|', "\n"); - $new_resource->custom_labels = base64_encode($customLabels); - $new_resource->save(); - } - - $new_resource->settings()->delete(); - if ($applicationSettings) { - $newApplicationSettings = $applicationSettings->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'application_id' => $new_resource->id, - ]); - $newApplicationSettings->save(); - } - - $tags = $this->resource->tags; - foreach ($tags as $tag) { - $new_resource->tags()->attach($tag->id); - } - - $scheduledTasks = $this->resource->scheduled_tasks()->get(); - foreach ($scheduledTasks as $task) { - $newTask = $task->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'uuid' => (string) new Cuid2, - 'application_id' => $new_resource->id, - 'team_id' => currentTeam()->id, - ]); - $newTask->save(); - } - - $applicationPreviews = $this->resource->previews()->get(); - foreach ($applicationPreviews as $preview) { - $newPreview = $preview->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'application_id' => $new_resource->id, - 'status' => 'exited', - ]); - $newPreview->save(); - } - - $persistentVolumes = $this->resource->persistentStorages()->get(); - foreach ($persistentVolumes as $volume) { - $newName = ''; - if (str_starts_with($volume->name, $this->resource->uuid)) { - $newName = str($volume->name)->replace($this->resource->uuid, $new_resource->uuid); - } else { - $newName = $new_resource->uuid.'-'.str($volume->name)->afterLast('-'); - } - - $newPersistentVolume = $volume->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'name' => $newName, - 'resource_id' => $new_resource->id, - ]); - $newPersistentVolume->save(); - - if ($this->cloneVolumeData) { - try { - StopApplication::dispatch($this->resource, false, false); - $sourceVolume = $volume->name; - $targetVolume = $newPersistentVolume->name; - $sourceServer = $this->resource->destination->server; - $targetServer = $new_resource->destination->server; - - VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); - - queue_application_deployment( - deployment_uuid: (string) new Cuid2, - application: $this->resource, - server: $sourceServer, - destination: $this->resource->destination, - no_questions_asked: true - ); - } catch (\Exception $e) { - \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); - } - } - } - - $fileStorages = $this->resource->fileStorages()->get(); - foreach ($fileStorages as $storage) { - $newStorage = $storage->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resource_id' => $new_resource->id, - ]); - $newStorage->save(); - } - - $environmentVaribles = $this->resource->environment_variables()->get(); - foreach ($environmentVaribles as $environmentVarible) { - $newEnvironmentVariable = $environmentVarible->replicate([ - 'id', - 'created_at', - 'updated_at', - ])->fill([ - 'resourceable_id' => $new_resource->id, - 'resourceable_type' => $new_resource->getMorphClass(), - ]); - $newEnvironmentVariable->save(); - } + $new_resource = clone_application($this->resource, $new_destination, ['uuid' => $uuid], $this->cloneVolumeData); $route = route('project.application.configuration', [ 'project_uuid' => $this->projectUuid, diff --git a/app/Livewire/Project/Shared/ScheduledTask/Executions.php b/app/Livewire/Project/Shared/ScheduledTask/Executions.php index 6f62a5b5b..ca2bbd9b4 100644 --- a/app/Livewire/Project/Shared/ScheduledTask/Executions.php +++ b/app/Livewire/Project/Shared/ScheduledTask/Executions.php @@ -105,6 +105,19 @@ public function loadMoreLogs() $this->currentPage++; } + public function loadAllLogs() + { + if (! $this->selectedExecution || ! $this->selectedExecution->message) { + return; + } + + $lines = collect(explode("\n", $this->selectedExecution->message)); + $totalLines = $lines->count(); + $totalPages = ceil($totalLines / $this->logsPerPage); + + $this->currentPage = $totalPages; + } + public function getLogLinesProperty() { if (! $this->selectedExecution) { diff --git a/app/Livewire/Project/Shared/Storages/Add.php b/app/Livewire/Project/Shared/Storages/Add.php deleted file mode 100644 index 006d41c14..000000000 --- a/app/Livewire/Project/Shared/Storages/Add.php +++ /dev/null @@ -1,174 +0,0 @@ -<?php - -namespace App\Livewire\Project\Shared\Storages; - -use App\Models\Application; -use App\Models\LocalFileVolume; -use Illuminate\Foundation\Auth\Access\AuthorizesRequests; -use Livewire\Component; - -class Add extends Component -{ - use AuthorizesRequests; - - public $resource; - - public $uuid; - - public $parameters; - - public $isSwarm = false; - - public string $name; - - public string $mount_path; - - public ?string $host_path = null; - - public string $file_storage_path; - - public ?string $file_storage_content = null; - - public string $file_storage_directory_source; - - public string $file_storage_directory_destination; - - public $rules = [ - 'name' => 'required|string', - 'mount_path' => 'required|string', - 'host_path' => 'string|nullable', - 'file_storage_path' => 'string', - 'file_storage_content' => 'nullable|string', - 'file_storage_directory_source' => 'string', - 'file_storage_directory_destination' => 'string', - ]; - - protected $listeners = ['clearAddStorage' => 'clear']; - - protected $validationAttributes = [ - 'name' => 'name', - 'mount_path' => 'mount', - 'host_path' => 'host', - 'file_storage_path' => 'file storage path', - 'file_storage_content' => 'file storage content', - 'file_storage_directory_source' => 'file storage directory source', - 'file_storage_directory_destination' => 'file storage directory destination', - ]; - - public function mount() - { - if (str($this->resource->getMorphClass())->contains('Standalone')) { - $this->file_storage_directory_source = database_configuration_dir()."/{$this->resource->uuid}"; - } else { - $this->file_storage_directory_source = application_configuration_dir()."/{$this->resource->uuid}"; - } - $this->uuid = $this->resource->uuid; - $this->parameters = get_route_parameters(); - if (data_get($this->parameters, 'application_uuid')) { - $applicationUuid = $this->parameters['application_uuid']; - $application = Application::where('uuid', $applicationUuid)->first(); - if (! $application) { - abort(404); - } - if ($application->destination->server->isSwarm()) { - $this->isSwarm = true; - $this->rules['host_path'] = 'required|string'; - } - } - } - - public function submitFileStorage() - { - try { - $this->authorize('update', $this->resource); - - $this->validate([ - 'file_storage_path' => 'string', - 'file_storage_content' => 'nullable|string', - ]); - - $this->file_storage_path = trim($this->file_storage_path); - $this->file_storage_path = str($this->file_storage_path)->start('/')->value(); - - if ($this->resource->getMorphClass() === \App\Models\Application::class) { - $fs_path = application_configuration_dir().'/'.$this->resource->uuid.$this->file_storage_path; - } elseif (str($this->resource->getMorphClass())->contains('Standalone')) { - $fs_path = database_configuration_dir().'/'.$this->resource->uuid.$this->file_storage_path; - } else { - throw new \Exception('No valid resource type for file mount storage type!'); - } - - LocalFileVolume::create( - [ - 'fs_path' => $fs_path, - 'mount_path' => $this->file_storage_path, - 'content' => $this->file_storage_content, - 'is_directory' => false, - 'resource_id' => $this->resource->id, - 'resource_type' => get_class($this->resource), - ], - ); - $this->dispatch('refreshStorages'); - } catch (\Throwable $e) { - return handleError($e, $this); - } - } - - public function submitFileStorageDirectory() - { - try { - $this->authorize('update', $this->resource); - - $this->validate([ - 'file_storage_directory_source' => 'string', - 'file_storage_directory_destination' => 'string', - ]); - - $this->file_storage_directory_source = trim($this->file_storage_directory_source); - $this->file_storage_directory_source = str($this->file_storage_directory_source)->start('/')->value(); - $this->file_storage_directory_destination = trim($this->file_storage_directory_destination); - $this->file_storage_directory_destination = str($this->file_storage_directory_destination)->start('/')->value(); - - LocalFileVolume::create( - [ - 'fs_path' => $this->file_storage_directory_source, - 'mount_path' => $this->file_storage_directory_destination, - 'is_directory' => true, - 'resource_id' => $this->resource->id, - 'resource_type' => get_class($this->resource), - ], - ); - $this->dispatch('refreshStorages'); - } catch (\Throwable $e) { - return handleError($e, $this); - } - } - - public function submitPersistentVolume() - { - try { - $this->authorize('update', $this->resource); - - $this->validate([ - 'name' => 'required|string', - 'mount_path' => 'required|string', - 'host_path' => 'string|nullable', - ]); - $name = $this->uuid.'-'.$this->name; - $this->dispatch('addNewVolume', [ - 'name' => $name, - 'mount_path' => $this->mount_path, - 'host_path' => $this->host_path, - ]); - } catch (\Throwable $e) { - return handleError($e, $this); - } - } - - public function clear() - { - $this->name = ''; - $this->mount_path = ''; - $this->host_path = null; - } -} diff --git a/app/Livewire/Project/Shared/Storages/All.php b/app/Livewire/Project/Shared/Storages/All.php index c26315d3b..63fc06a36 100644 --- a/app/Livewire/Project/Shared/Storages/All.php +++ b/app/Livewire/Project/Shared/Storages/All.php @@ -9,4 +9,15 @@ class All extends Component public $resource; protected $listeners = ['refreshStorages' => '$refresh']; + + public function getFirstStorageIdProperty() + { + if ($this->resource->persistentStorages->isEmpty()) { + return null; + } + + // Use the storage with the smallest ID as the "first" one + // This ensures stability even when storages are deleted + return $this->resource->persistentStorages->sortBy('id')->first()->id; + } } diff --git a/app/Livewire/Server/Advanced.php b/app/Livewire/Server/Advanced.php index 760c4df0d..8d17bb557 100644 --- a/app/Livewire/Server/Advanced.php +++ b/app/Livewire/Server/Advanced.php @@ -2,10 +2,7 @@ namespace App\Livewire\Server; -use App\Models\InstanceSettings; use App\Models\Server; -use Illuminate\Support\Facades\Auth; -use Illuminate\Support\Facades\Hash; use Livewire\Attributes\Validate; use Livewire\Component; @@ -27,9 +24,6 @@ class Advanced extends Component #[Validate(['integer', 'min:1'])] public int $dynamicTimeout = 1; - #[Validate(['boolean'])] - public bool $isTerminalEnabled = false; - public function mount(string $server_uuid) { try { @@ -42,37 +36,6 @@ public function mount(string $server_uuid) } } - public function toggleTerminal($password) - { - try { - // Check if user is admin or owner - if (! auth()->user()->isAdmin()) { - throw new \Exception('Only team administrators and owners can modify terminal access.'); - } - - // Verify password unless two-step confirmation is disabled - if (! data_get(InstanceSettings::get(), 'disable_two_step_confirmation')) { - if (! Hash::check($password, Auth::user()->password)) { - $this->addError('password', 'The provided password is incorrect.'); - - return; - } - } - - // Toggle the terminal setting - $this->server->settings->is_terminal_enabled = ! $this->server->settings->is_terminal_enabled; - $this->server->settings->save(); - - // Update the local property - $this->isTerminalEnabled = $this->server->settings->is_terminal_enabled; - - $status = $this->isTerminalEnabled ? 'enabled' : 'disabled'; - $this->dispatch('success', "Terminal access has been {$status}."); - } catch (\Throwable $e) { - return handleError($e, $this); - } - } - public function syncData(bool $toModel = false) { if ($toModel) { @@ -88,7 +51,6 @@ public function syncData(bool $toModel = false) $this->dynamicTimeout = $this->server->settings->dynamic_timeout; $this->serverDiskUsageNotificationThreshold = $this->server->settings->server_disk_usage_notification_threshold; $this->serverDiskUsageCheckFrequency = $this->server->settings->server_disk_usage_check_frequency; - $this->isTerminalEnabled = $this->server->settings->is_terminal_enabled; } } diff --git a/app/Livewire/Server/Navbar.php b/app/Livewire/Server/Navbar.php index 055290580..beefed12a 100644 --- a/app/Livewire/Server/Navbar.php +++ b/app/Livewire/Server/Navbar.php @@ -32,7 +32,7 @@ public function getListeners() $teamId = auth()->user()->currentTeam()->id; return [ - 'refreshServerShow' => '$refresh', + 'refreshServerShow' => 'refreshServer', "echo-private:team.{$teamId},ProxyStatusChangedUI" => 'showNotification', ]; } @@ -134,6 +134,12 @@ public function showNotification() } + public function refreshServer() + { + $this->server->refresh(); + $this->server->load('settings'); + } + public function render() { return view('livewire.server.navbar'); diff --git a/app/Livewire/Server/PrivateKey/Show.php b/app/Livewire/Server/PrivateKey/Show.php index 845d568ce..fd55717fa 100644 --- a/app/Livewire/Server/PrivateKey/Show.php +++ b/app/Livewire/Server/PrivateKey/Show.php @@ -5,6 +5,7 @@ use App\Models\PrivateKey; use App\Models\Server; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Illuminate\Support\Facades\DB; use Livewire\Component; class Show extends Component @@ -35,19 +36,20 @@ public function setPrivateKey($privateKeyId) return; } - - $originalPrivateKeyId = $this->server->getOriginal('private_key_id'); try { $this->authorize('update', $this->server); - $this->server->update(['private_key_id' => $privateKeyId]); - ['uptime' => $uptime, 'error' => $error] = $this->server->validateConnection(justCheckingNewKey: true); - if ($uptime) { - $this->dispatch('success', 'Private key updated successfully.'); - } else { - throw new \Exception($error); - } + DB::transaction(function () use ($ownedPrivateKey) { + $this->server->privateKey()->associate($ownedPrivateKey); + $this->server->save(); + ['uptime' => $uptime, 'error' => $error] = $this->server->validateConnection(justCheckingNewKey: true); + if (! $uptime) { + throw new \Exception($error); + } + }); + $this->dispatch('success', 'Private key updated successfully.'); + $this->dispatch('refreshServerShow'); } catch (\Exception $e) { - $this->server->update(['private_key_id' => $originalPrivateKeyId]); + $this->server->refresh(); $this->server->validateConnection(); $this->dispatch('error', $e->getMessage()); } @@ -59,6 +61,7 @@ public function checkConnection() ['uptime' => $uptime, 'error' => $error] = $this->server->validateConnection(); if ($uptime) { $this->dispatch('success', 'Server is reachable.'); + $this->dispatch('refreshServerShow'); } else { $this->dispatch('error', 'Server is not reachable.<br><br>Check this <a target="_blank" class="underline" href="https://coolify.io/docs/knowledge-base/server/openssh">documentation</a> for further help.<br><br>Error: '.$error); diff --git a/app/Livewire/Server/Proxy.php b/app/Livewire/Server/Proxy.php index 49adf7fe6..5ef559862 100644 --- a/app/Livewire/Server/Proxy.php +++ b/app/Livewire/Server/Proxy.php @@ -2,8 +2,8 @@ namespace App\Livewire\Server; -use App\Actions\Proxy\CheckConfiguration; -use App\Actions\Proxy\SaveConfiguration; +use App\Actions\Proxy\GetProxyConfiguration; +use App\Actions\Proxy\SaveProxyConfiguration; use App\Models\Server; use Illuminate\Foundation\Auth\Access\AuthorizesRequests; use Livewire\Component; @@ -16,11 +16,11 @@ class Proxy extends Component public ?string $selectedProxy = null; - public $proxy_settings = null; + public $proxySettings = null; - public bool $redirect_enabled = true; + public bool $redirectEnabled = true; - public ?string $redirect_url = null; + public ?string $redirectUrl = null; public function getListeners() { @@ -39,14 +39,14 @@ public function getListeners() public function mount() { $this->selectedProxy = $this->server->proxyType(); - $this->redirect_enabled = data_get($this->server, 'proxy.redirect_enabled', true); - $this->redirect_url = data_get($this->server, 'proxy.redirect_url'); + $this->redirectEnabled = data_get($this->server, 'proxy.redirect_enabled', true); + $this->redirectUrl = data_get($this->server, 'proxy.redirect_url'); } - // public function proxyStatusUpdated() - // { - // $this->dispatch('refresh')->self(); - // } + public function getConfigurationFilePathProperty() + { + return $this->server->proxyPath().'docker-compose.yml'; + } public function changeProxy() { @@ -86,7 +86,7 @@ public function instantSaveRedirect() { try { $this->authorize('update', $this->server); - $this->server->proxy->redirect_enabled = $this->redirect_enabled; + $this->server->proxy->redirect_enabled = $this->redirectEnabled; $this->server->save(); $this->server->setupDefaultRedirect(); $this->dispatch('success', 'Proxy configuration saved.'); @@ -99,8 +99,8 @@ public function submit() { try { $this->authorize('update', $this->server); - SaveConfiguration::run($this->server, $this->proxy_settings); - $this->server->proxy->redirect_url = $this->redirect_url; + SaveProxyConfiguration::run($this->server, $this->proxySettings); + $this->server->proxy->redirect_url = $this->redirectUrl; $this->server->save(); $this->server->setupDefaultRedirect(); $this->dispatch('success', 'Proxy configuration saved.'); @@ -109,14 +109,15 @@ public function submit() } } - public function reset_proxy_configuration() + public function resetProxyConfiguration() { try { $this->authorize('update', $this->server); - $this->proxy_settings = CheckConfiguration::run($this->server, true); - SaveConfiguration::run($this->server, $this->proxy_settings); + // Explicitly regenerate default configuration + $this->proxySettings = GetProxyConfiguration::run($this->server, forceRegenerate: true); + SaveProxyConfiguration::run($this->server, $this->proxySettings); $this->server->save(); - $this->dispatch('success', 'Proxy configuration saved.'); + $this->dispatch('success', 'Proxy configuration reset to default.'); } catch (\Throwable $e) { return handleError($e, $this); } @@ -125,7 +126,7 @@ public function reset_proxy_configuration() public function loadProxyConfiguration() { try { - $this->proxy_settings = CheckConfiguration::run($this->server); + $this->proxySettings = GetProxyConfiguration::run($this->server); } catch (\Throwable $e) { return handleError($e, $this); } diff --git a/app/Livewire/Server/Security/TerminalAccess.php b/app/Livewire/Server/Security/TerminalAccess.php new file mode 100644 index 000000000..284eea7dd --- /dev/null +++ b/app/Livewire/Server/Security/TerminalAccess.php @@ -0,0 +1,85 @@ +<?php + +namespace App\Livewire\Server\Security; + +use App\Models\InstanceSettings; +use App\Models\Server; +use Illuminate\Foundation\Auth\Access\AuthorizesRequests; +use Illuminate\Support\Facades\Auth; +use Illuminate\Support\Facades\Hash; +use Livewire\Attributes\Validate; +use Livewire\Component; + +class TerminalAccess extends Component +{ + use AuthorizesRequests; + + public Server $server; + + public array $parameters = []; + + #[Validate(['boolean'])] + public bool $isTerminalEnabled = false; + + public function mount(string $server_uuid) + { + try { + $this->server = Server::ownedByCurrentTeam()->whereUuid($server_uuid)->firstOrFail(); + $this->authorize('update', $this->server); + $this->parameters = get_route_parameters(); + $this->syncData(); + + } catch (\Throwable) { + return redirect()->route('server.index'); + } + } + + public function toggleTerminal($password) + { + try { + $this->authorize('update', $this->server); + + // Check if user is admin or owner + if (! auth()->user()->isAdmin()) { + throw new \Exception('Only team administrators and owners can modify terminal access.'); + } + + // Verify password unless two-step confirmation is disabled + if (! data_get(InstanceSettings::get(), 'disable_two_step_confirmation')) { + if (! Hash::check($password, Auth::user()->password)) { + $this->addError('password', 'The provided password is incorrect.'); + + return; + } + } + + // Toggle the terminal setting + $this->server->settings->is_terminal_enabled = ! $this->server->settings->is_terminal_enabled; + $this->server->settings->save(); + + // Update the local property + $this->isTerminalEnabled = $this->server->settings->is_terminal_enabled; + + $status = $this->isTerminalEnabled ? 'enabled' : 'disabled'; + $this->dispatch('success', "Terminal access has been {$status}."); + } catch (\Throwable $e) { + return handleError($e, $this); + } + } + + public function syncData(bool $toModel = false) + { + if ($toModel) { + $this->authorize('update', $this->server); + $this->validate(); + // No other fields to sync for terminal access + } else { + $this->isTerminalEnabled = $this->server->settings->is_terminal_enabled; + } + } + + public function render() + { + return view('livewire.server.security.terminal-access'); + } +} diff --git a/app/Livewire/Server/Show.php b/app/Livewire/Server/Show.php index f4ae6dd7e..db4dc9b88 100644 --- a/app/Livewire/Server/Show.php +++ b/app/Livewire/Server/Show.php @@ -63,6 +63,8 @@ class Show extends Component public bool $isSentinelDebugEnabled; + public ?string $sentinelCustomDockerImage = null; + public string $serverTimezone; public function getListeners() @@ -267,8 +269,9 @@ public function restartSentinel() { try { $this->authorize('manageSentinel', $this->server); - $this->server->restartSentinel(); - $this->dispatch('success', 'Restarting Sentinel.'); + $customImage = isDev() ? $this->sentinelCustomDockerImage : null; + $this->server->restartSentinel($customImage); + $this->dispatch('info', 'Restarting Sentinel.'); } catch (\Throwable $e) { return handleError($e, $this); } @@ -295,12 +298,38 @@ public function updatedIsMetricsEnabled($value) } } + public function updatedIsBuildServer($value) + { + try { + $this->authorize('update', $this->server); + if ($value === true && $this->isSentinelEnabled) { + $this->isSentinelEnabled = false; + $this->isMetricsEnabled = false; + $this->isSentinelDebugEnabled = false; + StopSentinel::dispatch($this->server); + $this->dispatch('info', 'Sentinel has been disabled as build servers cannot run Sentinel.'); + } + $this->submit(); + // Dispatch event to refresh the navbar + $this->dispatch('refreshServerShow'); + } catch (\Throwable $e) { + return handleError($e, $this); + } + } + public function updatedIsSentinelEnabled($value) { try { $this->authorize('manageSentinel', $this->server); if ($value === true) { - StartSentinel::run($this->server, true); + if ($this->isBuildServer) { + $this->isSentinelEnabled = false; + $this->dispatch('error', 'Sentinel cannot be enabled on build servers.'); + + return; + } + $customImage = isDev() ? $this->sentinelCustomDockerImage : null; + StartSentinel::run($this->server, true, null, $customImage); } else { $this->isMetricsEnabled = false; $this->isSentinelDebugEnabled = false; @@ -326,7 +355,7 @@ public function regenerateSentinelToken() public function instantSave() { try { - $this->submit(); + $this->syncData(true); } catch (\Throwable $e) { return handleError($e, $this); } @@ -336,7 +365,7 @@ public function submit() { try { $this->syncData(true); - $this->dispatch('success', 'Server updated.'); + $this->dispatch('success', 'Server settings updated.'); } catch (\Throwable $e) { return handleError($e, $this); } diff --git a/app/Livewire/Server/ValidateAndInstall.php b/app/Livewire/Server/ValidateAndInstall.php index c75474e44..bf0b7b6a5 100644 --- a/app/Livewire/Server/ValidateAndInstall.php +++ b/app/Livewire/Server/ValidateAndInstall.php @@ -146,7 +146,7 @@ public function validateDockerVersion() StartProxy::dispatch($this->server); } else { $requiredDockerVersion = str(config('constants.docker.minimum_required_version'))->before('.'); - $this->error = 'Minimum Docker Engine version '.$requiredDockerVersion.' is not instaled. Please install Docker manually before continuing: <a target="_blank" class="underline" href="https://docs.docker.com/engine/install/#server">documentation</a>.'; + $this->error = 'Minimum Docker Engine version '.$requiredDockerVersion.' is not installed. Please install Docker manually before continuing: <a target="_blank" class="underline" href="https://docs.docker.com/engine/install/#server">documentation</a>.'; $this->server->update([ 'validation_logs' => $this->error, ]); diff --git a/app/Livewire/Settings/Index.php b/app/Livewire/Settings/Index.php index d05433082..13d690352 100644 --- a/app/Livewire/Settings/Index.php +++ b/app/Livewire/Settings/Index.php @@ -115,7 +115,7 @@ public function submit() $this->validate(); if ($this->settings->is_dns_validation_enabled && $this->fqdn) { - if (! validate_dns_entry($this->fqdn, $this->server)) { + if (! validateDNSEntry($this->fqdn, $this->server)) { $this->dispatch('error', "Validating DNS failed.<br><br>Make sure you have added the DNS records correctly.<br><br>{$this->fqdn}->{$this->server->ip}<br><br>Check this <a target='_blank' class='underline dark:text-white' href='https://coolify.io/docs/knowledge-base/dns-configuration'>documentation</a> for further help."); $error_show = true; } diff --git a/app/Livewire/SettingsDropdown.php b/app/Livewire/SettingsDropdown.php index 314957462..7afa763df 100644 --- a/app/Livewire/SettingsDropdown.php +++ b/app/Livewire/SettingsDropdown.php @@ -2,7 +2,7 @@ namespace App\Livewire; -use App\Jobs\PullChangelogFromGitHub; +use App\Jobs\PullChangelog; use App\Services\ChangelogService; use Illuminate\Support\Facades\Auth; use Livewire\Component; @@ -23,6 +23,11 @@ public function getEntriesProperty() return app(ChangelogService::class)->getEntriesForUser($user); } + public function getCurrentVersionProperty() + { + return 'v'.config('constants.coolify.version'); + } + public function openWhatsNewModal() { $this->showWhatsNewModal = true; @@ -50,7 +55,7 @@ public function manualFetchChangelog() } try { - PullChangelogFromGitHub::dispatch(); + PullChangelog::dispatch(); $this->dispatch('success', 'Changelog fetch initiated! Check back in a few moments.'); } catch (\Throwable $e) { $this->dispatch('error', 'Failed to fetch changelog: '.$e->getMessage()); @@ -62,6 +67,7 @@ public function render() return view('livewire.settings-dropdown', [ 'entries' => $this->entries, 'unreadCount' => $this->unreadCount, + 'currentVersion' => $this->currentVersion, ]); } } diff --git a/app/Livewire/Team/InviteLink.php b/app/Livewire/Team/InviteLink.php index 0bac39db8..45f7e467f 100644 --- a/app/Livewire/Team/InviteLink.php +++ b/app/Livewire/Team/InviteLink.php @@ -48,6 +48,8 @@ private function generateInviteLink(bool $sendEmail = false) if (auth()->user()->role() === 'admin' && $this->role === 'owner') { throw new \Exception('Admins cannot invite owners.'); } + $this->email = strtolower($this->email); + $member_emails = currentTeam()->members()->get()->pluck('email'); if ($member_emails->contains($this->email)) { return handleError(livewire: $this, customErrorMessage: "$this->email is already a member of ".currentTeam()->name.'.'); diff --git a/app/Models/Application.php b/app/Models/Application.php index 378161602..4f1796790 100644 --- a/app/Models/Application.php +++ b/app/Models/Application.php @@ -4,6 +4,7 @@ use App\Enums\ApplicationDeploymentStatus; use App\Services\ConfigurationGenerator; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasConfiguration; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; @@ -110,7 +111,7 @@ class Application extends BaseModel { - use HasConfiguration, HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasConfiguration, HasFactory, HasSafeStringAttribute, SoftDeletes; private static $parserVersion = '5'; @@ -123,6 +124,82 @@ class Application extends BaseModel 'http_basic_auth_password' => 'encrypted', ]; + protected static function booted() + { + static::addGlobalScope('withRelations', function ($builder) { + $builder->withCount([ + 'additional_servers', + 'additional_networks', + ]); + }); + static::saving(function ($application) { + $payload = []; + if ($application->isDirty('fqdn')) { + if ($application->fqdn === '') { + $application->fqdn = null; + } + $payload['fqdn'] = $application->fqdn; + } + if ($application->isDirty('install_command')) { + $payload['install_command'] = str($application->install_command)->trim(); + } + if ($application->isDirty('build_command')) { + $payload['build_command'] = str($application->build_command)->trim(); + } + if ($application->isDirty('start_command')) { + $payload['start_command'] = str($application->start_command)->trim(); + } + if ($application->isDirty('base_directory')) { + $payload['base_directory'] = str($application->base_directory)->trim(); + } + if ($application->isDirty('publish_directory')) { + $payload['publish_directory'] = str($application->publish_directory)->trim(); + } + if ($application->isDirty('git_repository')) { + $payload['git_repository'] = str($application->git_repository)->trim(); + } + if ($application->isDirty('git_branch')) { + $payload['git_branch'] = str($application->git_branch)->trim(); + } + if ($application->isDirty('git_commit_sha')) { + $payload['git_commit_sha'] = str($application->git_commit_sha)->trim(); + } + if ($application->isDirty('status')) { + $payload['last_online_at'] = now(); + } + if ($application->isDirty('custom_nginx_configuration')) { + if ($application->custom_nginx_configuration === '') { + $payload['custom_nginx_configuration'] = null; + } + } + if (count($payload) > 0) { + $application->forceFill($payload); + } + }); + static::created(function ($application) { + ApplicationSetting::create([ + 'application_id' => $application->id, + ]); + $application->compose_parsing_version = self::$parserVersion; + $application->save(); + }); + static::forceDeleting(function ($application) { + $application->update(['fqdn' => null]); + $application->settings()->delete(); + $application->persistentStorages()->delete(); + $application->environment_variables()->delete(); + $application->environment_variables_preview()->delete(); + foreach ($application->scheduled_tasks as $task) { + $task->delete(); + } + $application->tags()->detach(); + $application->previews()->delete(); + foreach ($application->deployment_queue as $deployment) { + $deployment->delete(); + } + }); + } + public function customNetworkAliases(): Attribute { return Attribute::make( @@ -183,73 +260,6 @@ private function isJson($string) return json_last_error() === JSON_ERROR_NONE; } - protected static function booted() - { - static::addGlobalScope('withRelations', function ($builder) { - $builder->withCount([ - 'additional_servers', - 'additional_networks', - ]); - }); - static::saving(function ($application) { - $payload = []; - if ($application->isDirty('fqdn')) { - if ($application->fqdn === '') { - $application->fqdn = null; - } - $payload['fqdn'] = $application->fqdn; - } - if ($application->isDirty('install_command')) { - $payload['install_command'] = str($application->install_command)->trim(); - } - if ($application->isDirty('build_command')) { - $payload['build_command'] = str($application->build_command)->trim(); - } - if ($application->isDirty('start_command')) { - $payload['start_command'] = str($application->start_command)->trim(); - } - if ($application->isDirty('base_directory')) { - $payload['base_directory'] = str($application->base_directory)->trim(); - } - if ($application->isDirty('publish_directory')) { - $payload['publish_directory'] = str($application->publish_directory)->trim(); - } - if ($application->isDirty('status')) { - $payload['last_online_at'] = now(); - } - if ($application->isDirty('custom_nginx_configuration')) { - if ($application->custom_nginx_configuration === '') { - $payload['custom_nginx_configuration'] = null; - } - } - if (count($payload) > 0) { - $application->forceFill($payload); - } - }); - static::created(function ($application) { - ApplicationSetting::create([ - 'application_id' => $application->id, - ]); - $application->compose_parsing_version = self::$parserVersion; - $application->save(); - }); - static::forceDeleting(function ($application) { - $application->update(['fqdn' => null]); - $application->settings()->delete(); - $application->persistentStorages()->delete(); - $application->environment_variables()->delete(); - $application->environment_variables_preview()->delete(); - foreach ($application->scheduled_tasks as $task) { - $task->delete(); - } - $application->tags()->detach(); - $application->previews()->delete(); - foreach ($application->deployment_queue as $deployment) { - $deployment->delete(); - } - }); - } - public static function ownedByCurrentTeamAPI(int $teamId) { return Application::whereRelation('environment.project.team', 'id', $teamId)->orderBy('name'); @@ -728,7 +738,14 @@ public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') ->where('is_preview', false) - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } public function runtime_environment_variables() @@ -738,14 +755,6 @@ public function runtime_environment_variables() ->where('key', 'not like', 'NIXPACKS_%'); } - public function build_environment_variables() - { - return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->where('is_preview', false) - ->where('is_build_time', true) - ->where('key', 'not like', 'NIXPACKS_%'); - } - public function nixpacks_environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') @@ -757,7 +766,14 @@ public function environment_variables_preview() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') ->where('is_preview', true) - ->orderByRaw("LOWER(key) LIKE LOWER('SERVICE%') DESC, LOWER(key) ASC"); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } public function runtime_environment_variables_preview() @@ -767,14 +783,6 @@ public function runtime_environment_variables_preview() ->where('key', 'not like', 'NIXPACKS_%'); } - public function build_environment_variables_preview() - { - return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->where('is_preview', true) - ->where('is_build_time', true) - ->where('key', 'not like', 'NIXPACKS_%'); - } - public function nixpacks_environment_variables_preview() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') @@ -934,11 +942,11 @@ public function isLogDrainEnabled() public function isConfigurationChanged(bool $save = false) { - $newConfigHash = base64_encode($this->fqdn.$this->git_repository.$this->git_branch.$this->git_commit_sha.$this->build_pack.$this->static_image.$this->install_command.$this->build_command.$this->start_command.$this->ports_exposes.$this->ports_mappings.$this->base_directory.$this->publish_directory.$this->dockerfile.$this->dockerfile_location.$this->custom_labels.$this->custom_docker_run_options.$this->dockerfile_target_build.$this->redirect.$this->custom_nginx_configuration.$this->custom_labels); + $newConfigHash = base64_encode($this->fqdn.$this->git_repository.$this->git_branch.$this->git_commit_sha.$this->build_pack.$this->static_image.$this->install_command.$this->build_command.$this->start_command.$this->ports_exposes.$this->ports_mappings.$this->base_directory.$this->publish_directory.$this->dockerfile.$this->dockerfile_location.$this->custom_labels.$this->custom_docker_run_options.$this->dockerfile_target_build.$this->redirect.$this->custom_nginx_configuration.$this->custom_labels.$this->settings->use_build_secrets); if ($this->pull_request_id === 0 || $this->pull_request_id === null) { - $newConfigHash .= json_encode($this->environment_variables()->get('value')->sort()); + $newConfigHash .= json_encode($this->environment_variables()->get(['value', 'is_multiline', 'is_literal', 'is_buildtime', 'is_runtime'])->sort()); } else { - $newConfigHash .= json_encode($this->environment_variables_preview->get('value')->sort()); + $newConfigHash .= json_encode($this->environment_variables_preview->get(['value', 'is_multiline', 'is_literal', 'is_buildtime', 'is_runtime'])->sort()); } $newConfigHash = md5($newConfigHash); $oldConfigHash = data_get($this, 'config_hash'); @@ -1478,17 +1486,18 @@ public function loadComposeFile($isInit = false) $this->save(); $parsedServices = $this->parse(); if ($this->docker_compose_domains) { - $json = collect(json_decode($this->docker_compose_domains)); + $decoded = json_decode($this->docker_compose_domains, true); + $json = collect(is_array($decoded) ? $decoded : []); + $normalized = collect(); foreach ($json as $key => $value) { - if (str($key)->contains('-')) { - $key = str($key)->replace('-', '_'); - } - $json->put((string) $key, $value); + $normalizedKey = (string) str($key)->replace('-', '_')->replace('.', '_'); + $normalized->put($normalizedKey, $value); } + $json = $normalized; $services = collect(data_get($parsedServices, 'services', [])); foreach ($services as $name => $service) { - if (str($name)->contains('-')) { - $replacedName = str($name)->replace('-', '_'); + if (str($name)->contains('-') || str($name)->contains('.')) { + $replacedName = str($name)->replace('-', '_')->replace('.', '_'); $services->put((string) $replacedName, $service); $services->forget((string) $name); } @@ -1556,28 +1565,206 @@ protected function buildGitCheckoutCommand($target): string return $command; } + private function parseWatchPaths($value) + { + if ($value) { + $watch_paths = collect(explode("\n", $value)) + ->map(function (string $path): string { + // Trim whitespace + $path = trim($path); + + if (str_starts_with($path, '!')) { + $negation = '!'; + $pathWithoutNegation = substr($path, 1); + $pathWithoutNegation = ltrim(trim($pathWithoutNegation), '/'); + + return $negation.$pathWithoutNegation; + } + + return ltrim($path, '/'); + }) + ->filter(function (string $path): bool { + return strlen($path) > 0; + }); + + return trim($watch_paths->implode("\n")); + } + } + public function watchPaths(): Attribute { return Attribute::make( set: function ($value) { if ($value) { - return trim($value); + return $this->parseWatchPaths($value); } } ); } + public function matchWatchPaths(Collection $modified_files, ?Collection $watch_paths): Collection + { + return self::matchPaths($modified_files, $watch_paths); + } + + /** + * Static method to match paths against watch patterns with negation support + * Uses order-based matching: last matching pattern wins + */ + public static function matchPaths(Collection $modified_files, ?Collection $watch_paths): Collection + { + if (is_null($watch_paths) || $watch_paths->isEmpty()) { + return collect([]); + } + + return $modified_files->filter(function ($file) use ($watch_paths) { + $shouldInclude = null; // null means no patterns matched + + // Process patterns in order - last match wins + foreach ($watch_paths as $pattern) { + $pattern = trim($pattern); + if (empty($pattern)) { + continue; + } + + $isExclusion = str_starts_with($pattern, '!'); + $matchPattern = $isExclusion ? substr($pattern, 1) : $pattern; + + if (self::globMatch($matchPattern, $file)) { + // This pattern matches - it determines the current state + $shouldInclude = ! $isExclusion; + } + } + + // If no patterns matched and we only have exclusion patterns, include by default + if ($shouldInclude === null) { + // Check if we only have exclusion patterns + $hasInclusionPatterns = $watch_paths->contains(fn ($p) => ! str_starts_with(trim($p), '!')); + + return ! $hasInclusionPatterns; + } + + return $shouldInclude; + })->values(); + } + + /** + * Check if a path matches a glob pattern + * Supports: *, **, ?, [abc], [!abc] + */ + public static function globMatch(string $pattern, string $path): bool + { + $regex = self::globToRegex($pattern); + + return preg_match($regex, $path) === 1; + } + + /** + * Convert a glob pattern to a regular expression + */ + public static function globToRegex(string $pattern): string + { + $regex = ''; + $inGroup = false; + $chars = str_split($pattern); + $len = count($chars); + + for ($i = 0; $i < $len; $i++) { + $c = $chars[$i]; + + switch ($c) { + case '*': + // Check for ** + if ($i + 1 < $len && $chars[$i + 1] === '*') { + // ** matches any number of directories + $regex .= '.*'; + $i++; // Skip next * + // Skip optional / + if ($i + 1 < $len && $chars[$i + 1] === '/') { + $i++; + } + } else { + // * matches anything except / + $regex .= '[^/]*'; + } + break; + + case '?': + // ? matches any single character except / + $regex .= '[^/]'; + break; + + case '[': + // Character class + $inGroup = true; + $regex .= '['; + // Check for negation + if ($i + 1 < $len && ($chars[$i + 1] === '!' || $chars[$i + 1] === '^')) { + $regex .= '^'; + $i++; + } + break; + + case ']': + if ($inGroup) { + $inGroup = false; + $regex .= ']'; + } else { + $regex .= preg_quote($c, '#'); + } + break; + + case '.': + case '(': + case ')': + case '+': + case '{': + case '}': + case '$': + case '^': + case '|': + case '\\': + // Escape regex special characters + $regex .= '\\'.$c; + break; + + default: + $regex .= $c; + break; + } + } + + // Wrap in delimiters and anchors + return '#^'.$regex.'$#'; + } + + public function normalizeWatchPaths(): void + { + if (is_null($this->watch_paths)) { + return; + } + + $normalized = $this->parseWatchPaths($this->watch_paths); + if ($normalized !== $this->watch_paths) { + $this->watch_paths = $normalized; + $this->save(); + } + } + public function isWatchPathsTriggered(Collection $modified_files): bool { if (is_null($this->watch_paths)) { return false; } + + $this->normalizeWatchPaths(); + $watch_paths = collect(explode("\n", $this->watch_paths)); - $matches = $modified_files->filter(function ($file) use ($watch_paths) { - return $watch_paths->contains(function ($glob) use ($file) { - return fnmatch($glob, $file); - }); - }); + + if ($watch_paths->isEmpty()) { + return false; + } + $matches = $this->matchWatchPaths($modified_files, $watch_paths); return $matches->count() > 0; } diff --git a/app/Models/ApplicationDeploymentQueue.php b/app/Models/ApplicationDeploymentQueue.php index 2a9bea67a..8df6877ab 100644 --- a/app/Models/ApplicationDeploymentQueue.php +++ b/app/Models/ApplicationDeploymentQueue.php @@ -85,6 +85,47 @@ public function commitMessage() return str($this->commit_message)->value(); } + private function redactSensitiveInfo($text) + { + $text = remove_iip($text); + + $app = $this->application; + if (! $app) { + return $text; + } + + $lockedVars = collect([]); + + if ($app->environment_variables) { + $lockedVars = $lockedVars->merge( + $app->environment_variables + ->where('is_shown_once', true) + ->pluck('real_value', 'key') + ->filter() + ); + } + + if ($this->pull_request_id !== 0 && $app->environment_variables_preview) { + $lockedVars = $lockedVars->merge( + $app->environment_variables_preview + ->where('is_shown_once', true) + ->pluck('real_value', 'key') + ->filter() + ); + } + + foreach ($lockedVars as $key => $value) { + $escapedValue = preg_quote($value, '/'); + $text = preg_replace( + '/'.$escapedValue.'/', + REDACTED, + $text + ); + } + + return $text; + } + public function addLogEntry(string $message, string $type = 'stdout', bool $hidden = false) { if ($type === 'error') { @@ -96,7 +137,7 @@ public function addLogEntry(string $message, string $type = 'stdout', bool $hidd } $newLogEntry = [ 'command' => null, - 'output' => remove_iip($message), + 'output' => $this->redactSensitiveInfo($message), 'type' => $type, 'timestamp' => Carbon::now('UTC'), 'hidden' => $hidden, diff --git a/app/Models/ApplicationSetting.php b/app/Models/ApplicationSetting.php index d05081d21..4b03c69e1 100644 --- a/app/Models/ApplicationSetting.php +++ b/app/Models/ApplicationSetting.php @@ -13,6 +13,7 @@ class ApplicationSetting extends Model 'is_force_https_enabled' => 'boolean', 'is_debug_enabled' => 'boolean', 'is_preview_deployments_enabled' => 'boolean', + 'is_pr_deployments_public_enabled' => 'boolean', 'is_git_submodules_enabled' => 'boolean', 'is_git_lfs_enabled' => 'boolean', 'is_git_shallow_clone_enabled' => 'boolean', diff --git a/app/Models/Environment.php b/app/Models/Environment.php index 437be7d87..bfeee01c9 100644 --- a/app/Models/Environment.php +++ b/app/Models/Environment.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use OpenApi\Attributes as OA; @@ -19,6 +20,7 @@ )] class Environment extends BaseModel { + use ClearsGlobalSearchCache; use HasSafeStringAttribute; protected $guarded = []; diff --git a/app/Models/EnvironmentVariable.php b/app/Models/EnvironmentVariable.php index b8bde5c84..80399a16b 100644 --- a/app/Models/EnvironmentVariable.php +++ b/app/Models/EnvironmentVariable.php @@ -14,10 +14,11 @@ 'uuid' => ['type' => 'string'], 'resourceable_type' => ['type' => 'string'], 'resourceable_id' => ['type' => 'integer'], - 'is_build_time' => ['type' => 'boolean'], 'is_literal' => ['type' => 'boolean'], 'is_multiline' => ['type' => 'boolean'], 'is_preview' => ['type' => 'boolean'], + 'is_runtime' => ['type' => 'boolean'], + 'is_buildtime' => ['type' => 'boolean'], 'is_shared' => ['type' => 'boolean'], 'is_shown_once' => ['type' => 'boolean'], 'key' => ['type' => 'string'], @@ -35,15 +36,16 @@ class EnvironmentVariable extends BaseModel protected $casts = [ 'key' => 'string', 'value' => 'encrypted', - 'is_build_time' => 'boolean', 'is_multiline' => 'boolean', 'is_preview' => 'boolean', + 'is_runtime' => 'boolean', + 'is_buildtime' => 'boolean', 'version' => 'string', 'resourceable_type' => 'string', 'resourceable_id' => 'integer', ]; - protected $appends = ['real_value', 'is_shared', 'is_really_required']; + protected $appends = ['real_value', 'is_shared', 'is_really_required', 'is_nixpacks', 'is_coolify']; protected static function booted() { @@ -61,8 +63,8 @@ protected static function booted() ModelsEnvironmentVariable::create([ 'key' => $environment_variable->key, 'value' => $environment_variable->value, - 'is_build_time' => $environment_variable->is_build_time, 'is_multiline' => $environment_variable->is_multiline ?? false, + 'is_literal' => $environment_variable->is_literal ?? false, 'resourceable_type' => Application::class, 'resourceable_id' => $environment_variable->resourceable_id, 'is_preview' => true, @@ -137,6 +139,32 @@ protected function isReallyRequired(): Attribute ); } + protected function isNixpacks(): Attribute + { + return Attribute::make( + get: function () { + if (str($this->key)->startsWith('NIXPACKS_')) { + return true; + } + + return false; + } + ); + } + + protected function isCoolify(): Attribute + { + return Attribute::make( + get: function () { + if (str($this->key)->startsWith('SERVICE_')) { + return true; + } + + return false; + } + ); + } + protected function isShared(): Attribute { return Attribute::make( diff --git a/app/Models/Kubernetes.php b/app/Models/Kubernetes.php deleted file mode 100644 index 174cb5bc8..000000000 --- a/app/Models/Kubernetes.php +++ /dev/null @@ -1,5 +0,0 @@ -<?php - -namespace App\Models; - -class Kubernetes extends BaseModel {} diff --git a/app/Models/LocalFileVolume.php b/app/Models/LocalFileVolume.php index c56cd7694..b3e71d75d 100644 --- a/app/Models/LocalFileVolume.php +++ b/app/Models/LocalFileVolume.php @@ -119,6 +119,7 @@ public function saveStorageOnServer() $commands = collect([]); if ($this->is_directory) { $commands->push("mkdir -p $this->fs_path > /dev/null 2>&1 || true"); + $commands->push("mkdir -p $workdir > /dev/null 2>&1 || true"); $commands->push("cd $workdir"); } if (str($this->fs_path)->startsWith('.') || str($this->fs_path)->startsWith('/') || str($this->fs_path)->startsWith('~')) { diff --git a/app/Models/PrivateKey.php b/app/Models/PrivateKey.php index f70f32bc4..c210f3c5b 100644 --- a/app/Models/PrivateKey.php +++ b/app/Models/PrivateKey.php @@ -4,6 +4,7 @@ use App\Traits\HasSafeStringAttribute; use DanHarrin\LivewireRateLimiting\WithRateLimiting; +use Illuminate\Support\Facades\DB; use Illuminate\Support\Facades\Storage; use Illuminate\Validation\ValidationException; use OpenApi\Attributes as OA; @@ -99,11 +100,18 @@ public static function validatePrivateKey($privateKey) public static function createAndStore(array $data) { - $privateKey = new self($data); - $privateKey->save(); - $privateKey->storeInFileSystem(); + return DB::transaction(function () use ($data) { + $privateKey = new self($data); + $privateKey->save(); - return $privateKey; + try { + $privateKey->storeInFileSystem(); + } catch (\Exception $e) { + throw new \Exception('Failed to store SSH key: '.$e->getMessage()); + } + + return $privateKey; + }); } public static function generateNewKeyPair($type = 'rsa') @@ -151,15 +159,64 @@ public static function validateAndExtractPublicKey($privateKey) public function storeInFileSystem() { $filename = "ssh_key@{$this->uuid}"; - Storage::disk('ssh-keys')->put($filename, $this->private_key); + $disk = Storage::disk('ssh-keys'); - return "/var/www/html/storage/app/ssh/keys/{$filename}"; + // Ensure the storage directory exists and is writable + $this->ensureStorageDirectoryExists(); + + // Attempt to store the private key + $success = $disk->put($filename, $this->private_key); + + if (! $success) { + throw new \Exception("Failed to write SSH key to filesystem. Check disk space and permissions for: {$this->getKeyLocation()}"); + } + + // Verify the file was actually created and has content + if (! $disk->exists($filename)) { + throw new \Exception("SSH key file was not created: {$this->getKeyLocation()}"); + } + + $storedContent = $disk->get($filename); + if (empty($storedContent) || $storedContent !== $this->private_key) { + $disk->delete($filename); // Clean up the bad file + throw new \Exception("SSH key file content verification failed: {$this->getKeyLocation()}"); + } + + return $this->getKeyLocation(); } public static function deleteFromStorage(self $privateKey) { $filename = "ssh_key@{$privateKey->uuid}"; - Storage::disk('ssh-keys')->delete($filename); + $disk = Storage::disk('ssh-keys'); + + if ($disk->exists($filename)) { + $disk->delete($filename); + } + } + + protected function ensureStorageDirectoryExists() + { + $disk = Storage::disk('ssh-keys'); + $directoryPath = ''; + + if (! $disk->exists($directoryPath)) { + $success = $disk->makeDirectory($directoryPath); + if (! $success) { + throw new \Exception('Failed to create SSH keys storage directory'); + } + } + + // Check if directory is writable by attempting a test file + $testFilename = '.test_write_'.uniqid(); + $testSuccess = $disk->put($testFilename, 'test'); + + if (! $testSuccess) { + throw new \Exception('SSH keys storage directory is not writable'); + } + + // Clean up test file + $disk->delete($testFilename); } public function getKeyLocation() @@ -169,10 +226,17 @@ public function getKeyLocation() public function updatePrivateKey(array $data) { - $this->update($data); - $this->storeInFileSystem(); + return DB::transaction(function () use ($data) { + $this->update($data); - return $this; + try { + $this->storeInFileSystem(); + } catch (\Exception $e) { + throw new \Exception('Failed to update SSH key: '.$e->getMessage()); + } + + return $this; + }); } public function servers() diff --git a/app/Models/Project.php b/app/Models/Project.php index 1c46042e3..a9bf76803 100644 --- a/app/Models/Project.php +++ b/app/Models/Project.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use OpenApi\Attributes as OA; use Visus\Cuid2\Cuid2; @@ -24,6 +25,7 @@ )] class Project extends BaseModel { + use ClearsGlobalSearchCache; use HasSafeStringAttribute; protected $guarded = []; diff --git a/app/Models/ScheduledDatabaseBackup.php b/app/Models/ScheduledDatabaseBackup.php index 90204d8df..3ade21df8 100644 --- a/app/Models/ScheduledDatabaseBackup.php +++ b/app/Models/ScheduledDatabaseBackup.php @@ -10,6 +10,21 @@ class ScheduledDatabaseBackup extends BaseModel { protected $guarded = []; + public static function ownedByCurrentTeam() + { + return ScheduledDatabaseBackup::whereRelation('team', 'id', currentTeam()->id)->orderBy('created_at', 'desc'); + } + + public static function ownedByCurrentTeamAPI(int $teamId) + { + return ScheduledDatabaseBackup::whereRelation('team', 'id', $teamId)->orderBy('created_at', 'desc'); + } + + public function team() + { + return $this->belongsTo(Team::class); + } + public function database(): MorphTo { return $this->morphTo(); diff --git a/app/Models/Server.php b/app/Models/Server.php index 0f92bd390..829a4b5aa 100644 --- a/app/Models/Server.php +++ b/app/Models/Server.php @@ -13,6 +13,7 @@ use App\Notifications\Server\Reachable; use App\Notifications\Server\Unreachable; use App\Services\ConfigurationRepository; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Builder; use Illuminate\Database\Eloquent\Casts\Attribute; @@ -55,7 +56,7 @@ class Server extends BaseModel { - use HasFactory, SchemalessAttributesTrait, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, SchemalessAttributesTrait, SoftDeletes; public static $batch_counter = 0; @@ -1259,13 +1260,13 @@ public function isIpv6(): bool return str($this->ip)->contains(':'); } - public function restartSentinel(bool $async = true) + public function restartSentinel(?string $customImage = null, bool $async = true) { try { if ($async) { - StartSentinel::dispatch($this, true); + StartSentinel::dispatch($this, true, null, $customImage); } else { - StartSentinel::run($this, true); + StartSentinel::run($this, true, null, $customImage); } } catch (\Throwable $e) { return handleError($e); diff --git a/app/Models/Service.php b/app/Models/Service.php index 43cb32d85..d42d471c6 100644 --- a/app/Models/Service.php +++ b/app/Models/Service.php @@ -3,6 +3,7 @@ namespace App\Models; use App\Enums\ProcessStatus; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -41,7 +42,7 @@ )] class Service extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; private static $parserVersion = '5'; @@ -1113,7 +1114,6 @@ public function saveExtraFields($fields) $this->environment_variables()->create([ 'key' => $key, 'value' => $value, - 'is_build_time' => false, 'resourceable_id' => $this->id, 'resourceable_type' => $this->getMorphClass(), 'is_preview' => false, @@ -1230,14 +1230,14 @@ public function scheduled_tasks(): HasMany public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); - } - - public function environment_variables_preview() - { - return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->where('is_preview', true) - ->orderByRaw("LOWER(key) LIKE LOWER('SERVICE%') DESC, LOWER(key) ASC"); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } public function workdir() diff --git a/app/Models/StandaloneClickhouse.php b/app/Models/StandaloneClickhouse.php index 60a750a99..146ee0a2d 100644 --- a/app/Models/StandaloneClickhouse.php +++ b/app/Models/StandaloneClickhouse.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandaloneClickhouse extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -28,7 +29,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -44,6 +44,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneClickhouse::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -267,7 +272,14 @@ public function destination() public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } public function runtime_environment_variables() diff --git a/app/Models/StandaloneDragonfly.php b/app/Models/StandaloneDragonfly.php index 673851713..90e7304f1 100644 --- a/app/Models/StandaloneDragonfly.php +++ b/app/Models/StandaloneDragonfly.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandaloneDragonfly extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -28,7 +29,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -44,6 +44,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneDragonfly::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -342,6 +347,13 @@ public function isBackupSolutionAvailable() public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } } diff --git a/app/Models/StandaloneKeydb.php b/app/Models/StandaloneKeydb.php index e6562193b..ad0cabf7e 100644 --- a/app/Models/StandaloneKeydb.php +++ b/app/Models/StandaloneKeydb.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandaloneKeydb extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -28,7 +29,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -44,6 +44,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneKeydb::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -342,6 +347,13 @@ public function isBackupSolutionAvailable() public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } } diff --git a/app/Models/StandaloneMariadb.php b/app/Models/StandaloneMariadb.php index 1aa9d63c1..3d9e38147 100644 --- a/app/Models/StandaloneMariadb.php +++ b/app/Models/StandaloneMariadb.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -10,7 +11,7 @@ class StandaloneMariadb extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -29,7 +30,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -45,6 +45,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneMariadb::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -263,7 +268,14 @@ public function destination(): MorphTo public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } public function runtime_environment_variables() diff --git a/app/Models/StandaloneMongodb.php b/app/Models/StandaloneMongodb.php index 299ea75b2..7cccd332a 100644 --- a/app/Models/StandaloneMongodb.php +++ b/app/Models/StandaloneMongodb.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandaloneMongodb extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -24,7 +25,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); LocalPersistentVolume::create([ 'name' => 'mongodb-db-'.$database->uuid, @@ -32,7 +32,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -48,6 +47,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneMongodb::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -365,6 +369,13 @@ public function isBackupSolutionAvailable() public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } } diff --git a/app/Models/StandaloneMysql.php b/app/Models/StandaloneMysql.php index f376c7644..80269972f 100644 --- a/app/Models/StandaloneMysql.php +++ b/app/Models/StandaloneMysql.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandaloneMysql extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -29,7 +30,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -45,6 +45,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneMysql::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -346,6 +351,13 @@ public function isBackupSolutionAvailable() public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } } diff --git a/app/Models/StandalonePostgresql.php b/app/Models/StandalonePostgresql.php index 0bca2f4a7..acde7a20c 100644 --- a/app/Models/StandalonePostgresql.php +++ b/app/Models/StandalonePostgresql.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandalonePostgresql extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -29,7 +30,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -45,6 +45,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandalonePostgresql::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + public function workdir() { return database_configuration_dir()."/{$this->uuid}"; @@ -297,7 +302,14 @@ public function scheduledBackups() public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } public function isBackupSolutionAvailable() diff --git a/app/Models/StandaloneRedis.php b/app/Models/StandaloneRedis.php index 6a44ee714..001ebe36a 100644 --- a/app/Models/StandaloneRedis.php +++ b/app/Models/StandaloneRedis.php @@ -2,6 +2,7 @@ namespace App\Models; +use App\Traits\ClearsGlobalSearchCache; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; use Illuminate\Database\Eloquent\Factories\HasFactory; @@ -9,7 +10,7 @@ class StandaloneRedis extends BaseModel { - use HasFactory, HasSafeStringAttribute, SoftDeletes; + use ClearsGlobalSearchCache, HasFactory, HasSafeStringAttribute, SoftDeletes; protected $guarded = []; @@ -24,7 +25,6 @@ protected static function booted() 'host_path' => null, 'resource_id' => $database->id, 'resource_type' => $database->getMorphClass(), - 'is_readonly' => true, ]); }); static::forceDeleting(function ($database) { @@ -46,6 +46,11 @@ protected static function booted() }); } + public static function ownedByCurrentTeam() + { + return StandaloneRedis::whereRelation('environment.project.team', 'id', currentTeam()->id)->orderBy('name'); + } + protected function serverStatus(): Attribute { return Attribute::make( @@ -389,6 +394,13 @@ public function redisUsername(): Attribute public function environment_variables() { return $this->morphMany(EnvironmentVariable::class, 'resourceable') - ->orderBy('key', 'asc'); + ->orderByRaw(" + CASE + WHEN LOWER(key) LIKE 'service_%' THEN 1 + WHEN is_required = true AND (value IS NULL OR value = '') THEN 2 + ELSE 3 + END, + LOWER(key) ASC + "); } } diff --git a/app/Models/Team.php b/app/Models/Team.php index 81638e31c..51fdeffa4 100644 --- a/app/Models/Team.php +++ b/app/Models/Team.php @@ -10,6 +10,7 @@ use App\Traits\HasNotificationSettings; use App\Traits\HasSafeStringAttribute; use Illuminate\Database\Eloquent\Casts\Attribute; +use Illuminate\Database\Eloquent\Factories\HasFactory; use Illuminate\Database\Eloquent\Model; use Illuminate\Notifications\Notifiable; use OpenApi\Attributes as OA; @@ -37,7 +38,7 @@ class Team extends Model implements SendsDiscord, SendsEmail, SendsPushover, SendsSlack { - use HasNotificationSettings, HasSafeStringAttribute, Notifiable; + use HasFactory, HasNotificationSettings, HasSafeStringAttribute, Notifiable; protected $guarded = []; @@ -193,6 +194,7 @@ public function isAnyNotificationEnabled() public function subscriptionEnded() { $this->subscription->update([ + 'stripe_subscription_id' => null, 'stripe_cancel_at_period_end' => false, 'stripe_invoice_paid' => false, 'stripe_trial_already_ended' => false, diff --git a/app/Models/TeamInvitation.php b/app/Models/TeamInvitation.php index 0fea1806b..c322982ed 100644 --- a/app/Models/TeamInvitation.php +++ b/app/Models/TeamInvitation.php @@ -15,6 +15,14 @@ class TeamInvitation extends Model 'via', ]; + /** + * Set the email attribute to lowercase. + */ + public function setEmailAttribute(string $value): void + { + $this->attributes['email'] = strtolower($value); + } + public function team() { return $this->belongsTo(Team::class); diff --git a/app/Models/User.php b/app/Models/User.php index 48651d292..9ab9fefe9 100644 --- a/app/Models/User.php +++ b/app/Models/User.php @@ -56,6 +56,22 @@ class User extends Authenticatable implements SendsEmail 'email_change_code_expires_at' => 'datetime', ]; + /** + * Set the email attribute to lowercase. + */ + public function setEmailAttribute($value) + { + $this->attributes['email'] = strtolower($value); + } + + /** + * Set the pending_email attribute to lowercase. + */ + public function setPendingEmailAttribute($value) + { + $this->attributes['pending_email'] = $value ? strtolower($value) : null; + } + protected static function boot() { parent::boot(); diff --git a/app/Models/Webhook.php b/app/Models/Webhook.php deleted file mode 100644 index 8e2b62955..000000000 --- a/app/Models/Webhook.php +++ /dev/null @@ -1,15 +0,0 @@ -<?php - -namespace App\Models; - -use Illuminate\Database\Eloquent\Model; - -class Webhook extends Model -{ - protected $guarded = []; - - protected $casts = [ - 'type' => 'string', - 'payload' => 'encrypted', - ]; -} diff --git a/app/Notifications/Channels/EmailChannel.php b/app/Notifications/Channels/EmailChannel.php index 47994c690..245bd85f0 100644 --- a/app/Notifications/Channels/EmailChannel.php +++ b/app/Notifications/Channels/EmailChannel.php @@ -2,6 +2,7 @@ namespace App\Notifications\Channels; +use App\Exceptions\NonReportableException; use App\Models\Team; use Exception; use Illuminate\Notifications\Notification; @@ -101,13 +102,11 @@ public function send(SendsEmail $notifiable, Notification $notification): void $mailer->send($email); } } catch (\Throwable $e) { - \Illuminate\Support\Facades\Log::error('EmailChannel failed: '.$e->getMessage(), [ - 'notification' => get_class($notification), - 'notifiable' => get_class($notifiable), - 'team_id' => data_get($notifiable, 'id'), - 'error' => $e->getMessage(), - 'trace' => $e->getTraceAsString(), - ]); + // Check if this is a Resend domain verification error on cloud instances + if (isCloud() && str_contains($e->getMessage(), 'domain is not verified')) { + // Throw as NonReportableException so it won't go to Sentry + throw NonReportableException::fromException($e); + } throw $e; } } diff --git a/app/Policies/ServiceDatabasePolicy.php b/app/Policies/ServiceDatabasePolicy.php index 023434a24..f72f1f327 100644 --- a/app/Policies/ServiceDatabasePolicy.php +++ b/app/Policies/ServiceDatabasePolicy.php @@ -13,7 +13,7 @@ class ServiceDatabasePolicy */ public function view(User $user, ServiceDatabase $serviceDatabase): bool { - return Gate::allows('view', $serviceDatabase->service); + return true; } /** @@ -30,6 +30,7 @@ public function create(User $user): bool */ public function update(User $user, ServiceDatabase $serviceDatabase): bool { + // return Gate::allows('update', $serviceDatabase->service); return true; } @@ -60,4 +61,9 @@ public function forceDelete(User $user, ServiceDatabase $serviceDatabase): bool // return Gate::allows('delete', $serviceDatabase->service); return true; } + + public function manageBackups(User $user, ServiceDatabase $serviceDatabase): bool + { + return true; + } } diff --git a/app/Providers/FortifyServiceProvider.php b/app/Providers/FortifyServiceProvider.php index ed27a158a..30d909388 100644 --- a/app/Providers/FortifyServiceProvider.php +++ b/app/Providers/FortifyServiceProvider.php @@ -80,9 +80,23 @@ public function boot(): void ) { $user->updated_at = now(); $user->save(); - $user->currentTeam = $user->teams->firstWhere('personal_team', true); - if (! $user->currentTeam) { - $user->currentTeam = $user->recreate_personal_team(); + + // Check if user has a pending invitation they haven't accepted yet + $invitation = \App\Models\TeamInvitation::whereEmail($email)->first(); + if ($invitation && $invitation->isValid()) { + // User is logging in for the first time after being invited + // Attach them to the invited team if not already attached + if (! $user->teams()->where('team_id', $invitation->team->id)->exists()) { + $user->teams()->attach($invitation->team->id, ['role' => $invitation->role]); + } + $user->currentTeam = $invitation->team; + $invitation->delete(); + } else { + // Normal login - use personal team + $user->currentTeam = $user->teams->firstWhere('personal_team', true); + if (! $user->currentTeam) { + $user->currentTeam = $user->recreate_personal_team(); + } } session(['currentTeam' => $user->currentTeam]); diff --git a/app/Rules/ValidGitRepositoryUrl.php b/app/Rules/ValidGitRepositoryUrl.php index 3cbe9246e..ba1aed11b 100644 --- a/app/Rules/ValidGitRepositoryUrl.php +++ b/app/Rules/ValidGitRepositoryUrl.php @@ -31,7 +31,7 @@ public function validate(string $attribute, mixed $value, Closure $fail): void $dangerousChars = [ ';', '|', '&', '$', '`', '(', ')', '{', '}', '[', ']', '<', '>', '\n', '\r', '\0', '"', "'", - '\\', '!', '?', '*', '~', '^', '%', '=', '+', + '\\', '!', '?', '*', '^', '%', '=', '+', '#', // Comment character that could hide commands ]; @@ -85,7 +85,7 @@ public function validate(string $attribute, mixed $value, Closure $fail): void } // Validate SSH URL format (git@host:user/repo.git) - if (! preg_match('/^git@[a-zA-Z0-9\.\-]+:[a-zA-Z0-9\-_\/\.]+$/', $value)) { + if (! preg_match('/^git@[a-zA-Z0-9\.\-]+:[a-zA-Z0-9\-_\/\.~]+$/', $value)) { $fail('The :attribute is not a valid SSH repository URL.'); return; @@ -136,14 +136,14 @@ public function validate(string $attribute, mixed $value, Closure $fail): void // Validate path contains only safe characters $path = $parsed['path'] ?? ''; - if (! empty($path) && ! preg_match('/^[a-zA-Z0-9\-_\/\.]+$/', $path)) { + if (! empty($path) && ! preg_match('/^[a-zA-Z0-9\-_\/\.@~]+$/', $path)) { $fail('The :attribute path contains invalid characters.'); return; } } elseif (str_starts_with($value, 'git://')) { - // Validate git:// protocol URL - if (! preg_match('/^git:\/\/[a-zA-Z0-9\.\-]+\/[a-zA-Z0-9\-_\/\.]+$/', $value)) { + // Validate git:// protocol URL (supports both git://host/path and git://host:port/path with tilde) + if (! preg_match('/^git:\/\/[a-zA-Z0-9\.\-]+(:[0-9]+)?[:\/][a-zA-Z0-9\-_\/\.~]+$/', $value)) { $fail('The :attribute is not a valid git:// URL.'); return; diff --git a/app/Services/ConfigurationGenerator.php b/app/Services/ConfigurationGenerator.php index a7e4b31be..320e3f32a 100644 --- a/app/Services/ConfigurationGenerator.php +++ b/app/Services/ConfigurationGenerator.php @@ -129,7 +129,6 @@ protected function getEnvironmentVariables(): array $variables->push([ 'key' => $env->key, 'value' => $env->value, - 'is_build_time' => $env->is_build_time, 'is_preview' => $env->is_preview, 'is_multiline' => $env->is_multiline, ]); @@ -145,7 +144,6 @@ protected function getPreviewEnvironmentVariables(): array $variables->push([ 'key' => $env->key, 'value' => $env->value, - 'is_build_time' => $env->is_build_time, 'is_preview' => $env->is_preview, 'is_multiline' => $env->is_multiline, ]); diff --git a/app/Traits/ClearsGlobalSearchCache.php b/app/Traits/ClearsGlobalSearchCache.php new file mode 100644 index 000000000..b9af70aba --- /dev/null +++ b/app/Traits/ClearsGlobalSearchCache.php @@ -0,0 +1,128 @@ +<?php + +namespace App\Traits; + +use App\Livewire\GlobalSearch; +use Illuminate\Database\Eloquent\Model; + +trait ClearsGlobalSearchCache +{ + protected static function bootClearsGlobalSearchCache() + { + static::saving(function ($model) { + try { + // Only clear cache if searchable fields are being changed + if ($model->hasSearchableChanges()) { + $teamId = $model->getTeamIdForCache(); + if (filled($teamId)) { + GlobalSearch::clearTeamCache($teamId); + } + } + } catch (\Throwable $e) { + // Silently fail cache clearing - don't break the save operation + ray('Failed to clear global search cache on saving: '.$e->getMessage()); + } + }); + + static::created(function ($model) { + try { + // Always clear cache when model is created + $teamId = $model->getTeamIdForCache(); + if (filled($teamId)) { + GlobalSearch::clearTeamCache($teamId); + } + } catch (\Throwable $e) { + // Silently fail cache clearing - don't break the create operation + ray('Failed to clear global search cache on creation: '.$e->getMessage()); + } + }); + + static::deleted(function ($model) { + try { + // Always clear cache when model is deleted + $teamId = $model->getTeamIdForCache(); + if (filled($teamId)) { + GlobalSearch::clearTeamCache($teamId); + } + } catch (\Throwable $e) { + // Silently fail cache clearing - don't break the delete operation + ray('Failed to clear global search cache on deletion: '.$e->getMessage()); + } + }); + } + + private function hasSearchableChanges(): bool + { + try { + // Define searchable fields based on model type + $searchableFields = ['name', 'description']; + + // Add model-specific searchable fields + if ($this instanceof \App\Models\Application) { + $searchableFields[] = 'fqdn'; + $searchableFields[] = 'docker_compose_domains'; + } elseif ($this instanceof \App\Models\Server) { + $searchableFields[] = 'ip'; + } elseif ($this instanceof \App\Models\Service) { + // Services don't have direct fqdn, but name and description are covered + } elseif ($this instanceof \App\Models\Project || $this instanceof \App\Models\Environment) { + // Projects and environments only have name and description as searchable + } + // Database models only have name and description as searchable + + // Check if any searchable field is dirty + foreach ($searchableFields as $field) { + // Check if attribute exists before checking if dirty + if (array_key_exists($field, $this->getAttributes()) && $this->isDirty($field)) { + return true; + } + } + + return false; + } catch (\Throwable $e) { + // If checking changes fails, assume changes exist to be safe + ray('Failed to check searchable changes: '.$e->getMessage()); + + return true; + } + } + + private function getTeamIdForCache() + { + try { + // For Project models (has direct team_id) + if ($this instanceof \App\Models\Project) { + return $this->team_id ?? null; + } + + // For Environment models (get team_id through project) + if ($this instanceof \App\Models\Environment) { + return $this->project?->team_id; + } + + // For database models, team is accessed through environment.project.team + if (method_exists($this, 'team')) { + if ($this instanceof \App\Models\Server) { + $team = $this->team; + } else { + $team = $this->team(); + } + if (filled($team)) { + return is_object($team) ? $team->id : null; + } + } + + // For models with direct team_id property + if (property_exists($this, 'team_id') || isset($this->team_id)) { + return $this->team_id ?? null; + } + + return null; + } catch (\Throwable $e) { + // If we can't determine team ID, return null + ray('Failed to get team ID for cache: '.$e->getMessage()); + + return null; + } + } +} diff --git a/app/Traits/EnvironmentVariableAnalyzer.php b/app/Traits/EnvironmentVariableAnalyzer.php new file mode 100644 index 000000000..0b452a940 --- /dev/null +++ b/app/Traits/EnvironmentVariableAnalyzer.php @@ -0,0 +1,221 @@ +<?php + +namespace App\Traits; + +trait EnvironmentVariableAnalyzer +{ + /** + * List of environment variables that commonly cause build issues when set to production values. + * Each entry contains the variable pattern and associated metadata. + */ + protected static function getProblematicBuildVariables(): array + { + return [ + 'NODE_ENV' => [ + 'problematic_values' => ['production', 'prod'], + 'affects' => 'Node.js/npm/yarn/bun/pnpm', + 'issue' => 'Skips devDependencies installation which are often required for building (webpack, typescript, etc.)', + 'recommendation' => 'Uncheck "Available at Buildtime" or use "development" during build', + ], + 'NPM_CONFIG_PRODUCTION' => [ + 'problematic_values' => ['true', '1', 'yes'], + 'affects' => 'npm/pnpm', + 'issue' => 'Forces npm to skip devDependencies', + 'recommendation' => 'Remove from build-time variables or set to false', + ], + 'YARN_PRODUCTION' => [ + 'problematic_values' => ['true', '1', 'yes'], + 'affects' => 'Yarn/pnpm', + 'issue' => 'Forces yarn to skip devDependencies', + 'recommendation' => 'Remove from build-time variables or set to false', + ], + 'COMPOSER_NO_DEV' => [ + 'problematic_values' => ['1', 'true', 'yes'], + 'affects' => 'PHP/Composer', + 'issue' => 'Skips require-dev packages which may include build tools', + 'recommendation' => 'Set as "Runtime only" or remove from build-time variables', + ], + 'MIX_ENV' => [ + 'problematic_values' => ['prod', 'production'], + 'affects' => 'Elixir/Phoenix', + 'issue' => 'Production mode may skip development dependencies needed for compilation', + 'recommendation' => 'Use "dev" for build or set as "Runtime only"', + ], + 'RAILS_ENV' => [ + 'problematic_values' => ['production'], + 'affects' => 'Ruby on Rails', + 'issue' => 'May affect asset precompilation and dependency handling', + 'recommendation' => 'Consider using "development" for build phase', + ], + 'RACK_ENV' => [ + 'problematic_values' => ['production'], + 'affects' => 'Ruby/Rack', + 'issue' => 'May affect dependency handling and build behavior', + 'recommendation' => 'Consider using "development" for build phase', + ], + 'BUNDLE_WITHOUT' => [ + 'problematic_values' => ['development', 'test', 'development:test'], + 'affects' => 'Ruby/Bundler', + 'issue' => 'Excludes gem groups that may contain build dependencies', + 'recommendation' => 'Remove from build-time variables or adjust groups', + ], + 'FLASK_ENV' => [ + 'problematic_values' => ['production'], + 'affects' => 'Python/Flask', + 'issue' => 'May affect debug mode and development tools availability', + 'recommendation' => 'Usually safe, but consider "development" for complex builds', + ], + 'DJANGO_SETTINGS_MODULE' => [ + 'problematic_values' => [], // Check if contains 'production' or 'prod' + 'affects' => 'Python/Django', + 'issue' => 'Production settings may disable debug tools needed during build', + 'recommendation' => 'Use development settings for build phase', + 'check_function' => 'checkDjangoSettings', + ], + 'APP_ENV' => [ + 'problematic_values' => ['production', 'prod'], + 'affects' => 'Laravel/Symfony', + 'issue' => 'May affect dependency installation and build optimizations', + 'recommendation' => 'Consider using "local" or "development" for build', + ], + 'ASPNETCORE_ENVIRONMENT' => [ + 'problematic_values' => ['Production'], + 'affects' => '.NET/ASP.NET Core', + 'issue' => 'May affect build-time configurations and optimizations', + 'recommendation' => 'Usually safe, but verify build requirements', + ], + 'CI' => [ + 'problematic_values' => ['true', '1', 'yes'], + 'affects' => 'Various tools', + 'issue' => 'Changes behavior in many tools (disables interactivity, changes caching)', + 'recommendation' => 'Usually beneficial for builds, but be aware of behavior changes', + ], + ]; + } + + /** + * Analyze an environment variable for potential build issues. + * Always returns a warning if the key is in our list, regardless of value. + */ + public static function analyzeBuildVariable(string $key, string $value): ?array + { + $problematicVars = self::getProblematicBuildVariables(); + + // Direct key match + if (isset($problematicVars[$key])) { + $config = $problematicVars[$key]; + + // Check if it has a custom check function + if (isset($config['check_function'])) { + $method = $config['check_function']; + if (method_exists(self::class, $method)) { + return self::{$method}($key, $value, $config); + } + } + + // Always return warning for known problematic variables + return [ + 'variable' => $key, + 'value' => $value, + 'affects' => $config['affects'], + 'issue' => $config['issue'], + 'recommendation' => $config['recommendation'], + ]; + } + + return null; + } + + /** + * Analyze multiple environment variables for potential build issues. + */ + public static function analyzeBuildVariables(array $variables): array + { + $warnings = []; + + foreach ($variables as $key => $value) { + $warning = self::analyzeBuildVariable($key, $value); + if ($warning) { + $warnings[] = $warning; + } + } + + return $warnings; + } + + /** + * Custom check for Django settings module. + */ + protected static function checkDjangoSettings(string $key, string $value, array $config): ?array + { + // Always return warning for DJANGO_SETTINGS_MODULE when it's set as build-time + return [ + 'variable' => $key, + 'value' => $value, + 'affects' => $config['affects'], + 'issue' => $config['issue'], + 'recommendation' => $config['recommendation'], + ]; + } + + /** + * Generate a formatted warning message for deployment logs. + */ + public static function formatBuildWarning(array $warning): array + { + $messages = [ + "⚠️ Build-time environment variable warning: {$warning['variable']}={$warning['value']}", + " Affects: {$warning['affects']}", + " Issue: {$warning['issue']}", + " Recommendation: {$warning['recommendation']}", + ]; + + return $messages; + } + + /** + * Check if a variable should show a warning in the UI. + */ + public static function shouldShowBuildWarning(string $key): bool + { + return isset(self::getProblematicBuildVariables()[$key]); + } + + /** + * Get UI warning message for a specific variable. + */ + public static function getUIWarningMessage(string $key): ?string + { + $problematicVars = self::getProblematicBuildVariables(); + + if (! isset($problematicVars[$key])) { + return null; + } + + $config = $problematicVars[$key]; + $problematicValuesStr = implode(', ', $config['problematic_values']); + + return "Setting {$key} to {$problematicValuesStr} as a build-time variable may cause issues. {$config['issue']} Consider: {$config['recommendation']}"; + } + + /** + * Get problematic variables configuration for frontend use. + */ + public static function getProblematicVariablesForFrontend(): array + { + $vars = self::getProblematicBuildVariables(); + $result = []; + + foreach ($vars as $key => $config) { + // Skip the check_function as it's PHP-specific + $result[$key] = [ + 'problematic_values' => $config['problematic_values'], + 'affects' => $config['affects'], + 'issue' => $config['issue'], + 'recommendation' => $config['recommendation'], + ]; + } + + return $result; + } +} diff --git a/app/Traits/EnvironmentVariableProtection.php b/app/Traits/EnvironmentVariableProtection.php index b6b8d2687..ecc484966 100644 --- a/app/Traits/EnvironmentVariableProtection.php +++ b/app/Traits/EnvironmentVariableProtection.php @@ -14,7 +14,7 @@ trait EnvironmentVariableProtection */ protected function isProtectedEnvironmentVariable(string $key): bool { - return str($key)->startsWith('SERVICE_FQDN') || str($key)->startsWith('SERVICE_URL'); + return str($key)->startsWith('SERVICE_FQDN_') || str($key)->startsWith('SERVICE_URL_') || str($key)->startsWith('SERVICE_NAME_'); } /** diff --git a/app/Traits/ExecuteRemoteCommand.php b/app/Traits/ExecuteRemoteCommand.php index a228a5d10..4aa5aae8b 100644 --- a/app/Traits/ExecuteRemoteCommand.php +++ b/app/Traits/ExecuteRemoteCommand.php @@ -11,10 +11,52 @@ trait ExecuteRemoteCommand { + use SshRetryable; + public ?string $save = null; public static int $batch_counter = 0; + private function redact_sensitive_info($text) + { + $text = remove_iip($text); + + if (! isset($this->application)) { + return $text; + } + + $lockedVars = collect([]); + + if (isset($this->application->environment_variables)) { + $lockedVars = $lockedVars->merge( + $this->application->environment_variables + ->where('is_shown_once', true) + ->pluck('real_value', 'key') + ->filter() + ); + } + + if (isset($this->pull_request_id) && $this->pull_request_id !== 0 && isset($this->application->environment_variables_preview)) { + $lockedVars = $lockedVars->merge( + $this->application->environment_variables_preview + ->where('is_shown_once', true) + ->pluck('real_value', 'key') + ->filter() + ); + } + + foreach ($lockedVars as $key => $value) { + $escapedValue = preg_quote($value, '/'); + $text = preg_replace( + '/'.$escapedValue.'/', + REDACTED, + $text + ); + } + + return $text; + } + public function execute_remote_command(...$commands) { static::$batch_counter++; @@ -43,76 +85,188 @@ public function execute_remote_command(...$commands) $command = parseLineForSudo($command, $this->server); } } - $remote_command = SshMultiplexingHelper::generateSshCommand($this->server, $command); - $process = Process::timeout(3600)->idleTimeout(3600)->start($remote_command, function (string $type, string $output) use ($command, $hidden, $customType, $append) { - $output = str($output)->trim(); - if ($output->startsWith('╔')) { - $output = "\n".$output; + + // Check for cancellation before executing commands + if (isset($this->application_deployment_queue)) { + $this->application_deployment_queue->refresh(); + if ($this->application_deployment_queue->status === \App\Enums\ApplicationDeploymentStatus::CANCELLED_BY_USER->value) { + throw new \RuntimeException('Deployment cancelled by user', 69420); } + } - // Sanitize output to ensure valid UTF-8 encoding before JSON encoding - $sanitized_output = sanitize_utf8_text($output); - - $new_log_entry = [ - 'command' => remove_iip($command), - 'output' => remove_iip($sanitized_output), - 'type' => $customType ?? $type === 'err' ? 'stderr' : 'stdout', - 'timestamp' => Carbon::now('UTC'), - 'hidden' => $hidden, - 'batch' => static::$batch_counter, - ]; - if (! $this->application_deployment_queue->logs) { - $new_log_entry['order'] = 1; - } else { - try { - $previous_logs = json_decode($this->application_deployment_queue->logs, associative: true, flags: JSON_THROW_ON_ERROR); - } catch (\JsonException $e) { - // If existing logs are corrupted, start fresh - $previous_logs = []; - $new_log_entry['order'] = 1; - } - if (is_array($previous_logs)) { - $new_log_entry['order'] = count($previous_logs) + 1; - } else { - $previous_logs = []; - $new_log_entry['order'] = 1; - } - } - $previous_logs[] = $new_log_entry; + $maxRetries = config('constants.ssh.max_retries'); + $attempt = 0; + $lastError = null; + $commandExecuted = false; + while ($attempt < $maxRetries && ! $commandExecuted) { try { - $this->application_deployment_queue->logs = json_encode($previous_logs, flags: JSON_THROW_ON_ERROR); - } catch (\JsonException $e) { - // If JSON encoding still fails, use fallback with invalid sequences replacement - $this->application_deployment_queue->logs = json_encode($previous_logs, flags: JSON_INVALID_UTF8_SUBSTITUTE); - } + $this->executeCommandWithProcess($command, $hidden, $customType, $append, $ignore_errors); + $commandExecuted = true; + } catch (\RuntimeException $e) { + $lastError = $e; + $errorMessage = $e->getMessage(); + // Only retry if it's an SSH connection error and we haven't exhausted retries + if ($this->isRetryableSshError($errorMessage) && $attempt < $maxRetries - 1) { + $attempt++; + $delay = $this->calculateRetryDelay($attempt - 1); - $this->application_deployment_queue->save(); + // Track SSH retry event in Sentry + $this->trackSshRetryEvent($attempt, $maxRetries, $delay, $errorMessage, [ + 'server' => $this->server->name ?? $this->server->ip ?? 'unknown', + 'command' => $this->redact_sensitive_info($command), + 'trait' => 'ExecuteRemoteCommand', + ]); - if ($this->save) { - if (data_get($this->saved_outputs, $this->save, null) === null) { - data_set($this->saved_outputs, $this->save, str()); - } - if ($append) { - $this->saved_outputs[$this->save] .= str($sanitized_output)->trim(); - $this->saved_outputs[$this->save] = str($this->saved_outputs[$this->save]); + // Add log entry for the retry + if (isset($this->application_deployment_queue)) { + $this->addRetryLogEntry($attempt, $maxRetries, $delay, $errorMessage); + + // Check for cancellation during retry wait + $this->application_deployment_queue->refresh(); + if ($this->application_deployment_queue->status === \App\Enums\ApplicationDeploymentStatus::CANCELLED_BY_USER->value) { + throw new \RuntimeException('Deployment cancelled by user during retry', 69420); + } + } + + sleep($delay); } else { - $this->saved_outputs[$this->save] = str($sanitized_output)->trim(); + // Not retryable or max retries reached + throw $e; } } - }); - $this->application_deployment_queue->update([ - 'current_process_id' => $process->id(), - ]); + } - $process_result = $process->wait(); - if ($process_result->exitCode() !== 0) { - if (! $ignore_errors) { + // If we exhausted all retries and still failed + if (! $commandExecuted && $lastError) { + // Now we can set the status to FAILED since all retries have been exhausted + if (isset($this->application_deployment_queue)) { $this->application_deployment_queue->status = ApplicationDeploymentStatus::FAILED->value; $this->application_deployment_queue->save(); - throw new \RuntimeException($process_result->errorOutput()); } + throw $lastError; } }); } + + /** + * Execute the actual command with process handling + */ + private function executeCommandWithProcess($command, $hidden, $customType, $append, $ignore_errors) + { + $remote_command = SshMultiplexingHelper::generateSshCommand($this->server, $command); + $process = Process::timeout(config('constants.ssh.command_timeout'))->idleTimeout(3600)->start($remote_command, function (string $type, string $output) use ($command, $hidden, $customType, $append) { + $output = str($output)->trim(); + if ($output->startsWith('╔')) { + $output = "\n".$output; + } + + // Sanitize output to ensure valid UTF-8 encoding before JSON encoding + $sanitized_output = sanitize_utf8_text($output); + + $new_log_entry = [ + 'command' => $this->redact_sensitive_info($command), + 'output' => $this->redact_sensitive_info($sanitized_output), + 'type' => $customType ?? $type === 'err' ? 'stderr' : 'stdout', + 'timestamp' => Carbon::now('UTC'), + 'hidden' => $hidden, + 'batch' => static::$batch_counter, + ]; + if (! $this->application_deployment_queue->logs) { + $new_log_entry['order'] = 1; + } else { + try { + $previous_logs = json_decode($this->application_deployment_queue->logs, associative: true, flags: JSON_THROW_ON_ERROR); + } catch (\JsonException $e) { + // If existing logs are corrupted, start fresh + $previous_logs = []; + $new_log_entry['order'] = 1; + } + if (is_array($previous_logs)) { + $new_log_entry['order'] = count($previous_logs) + 1; + } else { + $previous_logs = []; + $new_log_entry['order'] = 1; + } + } + $previous_logs[] = $new_log_entry; + + try { + $this->application_deployment_queue->logs = json_encode($previous_logs, flags: JSON_THROW_ON_ERROR); + } catch (\JsonException $e) { + // If JSON encoding still fails, use fallback with invalid sequences replacement + $this->application_deployment_queue->logs = json_encode($previous_logs, flags: JSON_INVALID_UTF8_SUBSTITUTE); + } + + $this->application_deployment_queue->save(); + + if ($this->save) { + if (data_get($this->saved_outputs, $this->save, null) === null) { + $this->saved_outputs->put($this->save, str()); + } + if ($append) { + $current_value = $this->saved_outputs->get($this->save); + $this->saved_outputs->put($this->save, str($current_value.str($sanitized_output)->trim())); + } else { + $this->saved_outputs->put($this->save, str($sanitized_output)->trim()); + } + } + }); + $this->application_deployment_queue->update([ + 'current_process_id' => $process->id(), + ]); + + $process_result = $process->wait(); + if ($process_result->exitCode() !== 0) { + if (! $ignore_errors) { + // Don't immediately set to FAILED - let the retry logic handle it + // This prevents premature status changes during retryable SSH errors + throw new \RuntimeException($process_result->errorOutput()); + } + } + } + + /** + * Add a log entry for SSH retry attempts + */ + private function addRetryLogEntry(int $attempt, int $maxRetries, int $delay, string $errorMessage) + { + $retryMessage = "SSH connection failed. Retrying... (Attempt {$attempt}/{$maxRetries}, waiting {$delay}s)\nError: {$errorMessage}"; + + $new_log_entry = [ + 'output' => $this->redact_sensitive_info($retryMessage), + 'type' => 'stdout', + 'timestamp' => Carbon::now('UTC'), + 'hidden' => false, + 'batch' => static::$batch_counter, + ]; + + if (! $this->application_deployment_queue->logs) { + $new_log_entry['order'] = 1; + $previous_logs = []; + } else { + try { + $previous_logs = json_decode($this->application_deployment_queue->logs, associative: true, flags: JSON_THROW_ON_ERROR); + } catch (\JsonException $e) { + $previous_logs = []; + $new_log_entry['order'] = 1; + } + if (is_array($previous_logs)) { + $new_log_entry['order'] = count($previous_logs) + 1; + } else { + $previous_logs = []; + $new_log_entry['order'] = 1; + } + } + + $previous_logs[] = $new_log_entry; + + try { + $this->application_deployment_queue->logs = json_encode($previous_logs, flags: JSON_THROW_ON_ERROR); + } catch (\JsonException $e) { + $this->application_deployment_queue->logs = json_encode($previous_logs, flags: JSON_INVALID_UTF8_SUBSTITUTE); + } + + $this->application_deployment_queue->save(); + } } diff --git a/app/Traits/SshRetryable.php b/app/Traits/SshRetryable.php new file mode 100644 index 000000000..a26481056 --- /dev/null +++ b/app/Traits/SshRetryable.php @@ -0,0 +1,174 @@ +<?php + +namespace App\Traits; + +use Illuminate\Support\Facades\Log; + +trait SshRetryable +{ + /** + * Check if an error message indicates a retryable SSH connection error + */ + protected function isRetryableSshError(string $errorOutput): bool + { + $retryablePatterns = [ + 'kex_exchange_identification', + 'Connection reset by peer', + 'Connection refused', + 'Connection timed out', + 'Connection closed by remote host', + 'ssh_exchange_identification', + 'Bad file descriptor', + 'Broken pipe', + 'No route to host', + 'Network is unreachable', + 'Host is down', + 'No buffer space available', + 'Connection reset by', + 'Permission denied, please try again', + 'Received disconnect from', + 'Disconnected from', + 'Connection to .* closed', + 'ssh: connect to host .* port .*: Connection', + 'Lost connection', + 'Timeout, server not responding', + 'Cannot assign requested address', + 'Network is down', + 'Host key verification failed', + 'Operation timed out', + 'Connection closed unexpectedly', + 'Remote host closed connection', + 'Authentication failed', + 'Too many authentication failures', + ]; + + $lowerErrorOutput = strtolower($errorOutput); + foreach ($retryablePatterns as $pattern) { + if (str_contains($lowerErrorOutput, strtolower($pattern))) { + return true; + } + } + + return false; + } + + /** + * Calculate delay for exponential backoff + */ + protected function calculateRetryDelay(int $attempt): int + { + $baseDelay = config('constants.ssh.retry_base_delay'); + $maxDelay = config('constants.ssh.retry_max_delay'); + $multiplier = config('constants.ssh.retry_multiplier'); + + $delay = min($baseDelay * pow($multiplier, $attempt), $maxDelay); + + return (int) $delay; + } + + /** + * Execute a callback with SSH retry logic + * + * @param callable $callback The operation to execute + * @param array $context Context for logging (server, command, etc.) + * @param bool $throwError Whether to throw error on final failure + * @return mixed The result from the callback + */ + protected function executeWithSshRetry(callable $callback, array $context = [], bool $throwError = true) + { + $maxRetries = config('constants.ssh.max_retries'); + $lastError = null; + $lastErrorMessage = ''; + // Randomly fail the command with a key exchange error for testing + // if (random_int(1, 10) === 1) { // 10% chance to fail + // ray('SSH key exchange failed: kex_exchange_identification: read: Connection reset by peer'); + // throw new \RuntimeException('SSH key exchange failed: kex_exchange_identification: read: Connection reset by peer'); + // } + for ($attempt = 0; $attempt < $maxRetries; $attempt++) { + try { + return $callback(); + } catch (\Throwable $e) { + $lastError = $e; + $lastErrorMessage = $e->getMessage(); + + // Check if it's retryable and not the last attempt + if ($this->isRetryableSshError($lastErrorMessage) && $attempt < $maxRetries - 1) { + $delay = $this->calculateRetryDelay($attempt); + + // Track SSH retry event in Sentry + $this->trackSshRetryEvent($attempt + 1, $maxRetries, $delay, $lastErrorMessage, $context); + + // Add deployment log if available (for ExecuteRemoteCommand trait) + if (isset($this->application_deployment_queue) && method_exists($this, 'addRetryLogEntry')) { + $this->addRetryLogEntry($attempt + 1, $maxRetries, $delay, $lastErrorMessage); + } + + sleep($delay); + + continue; + } + + // Not retryable or max retries reached + break; + } + } + + // All retries exhausted + if ($attempt >= $maxRetries) { + Log::error('SSH operation failed after all retries', array_merge($context, [ + 'attempts' => $attempt, + 'error' => $lastErrorMessage, + ])); + } + + if ($throwError && $lastError) { + // If the error message is empty, provide a more meaningful one + if (empty($lastErrorMessage) || trim($lastErrorMessage) === '') { + $contextInfo = isset($context['server']) ? " to server {$context['server']}" : ''; + $attemptInfo = $attempt > 1 ? " after {$attempt} attempts" : ''; + throw new \RuntimeException("SSH connection failed{$contextInfo}{$attemptInfo}", $lastError->getCode()); + } + throw $lastError; + } + + return null; + } + + /** + * Track SSH retry event in Sentry + */ + protected function trackSshRetryEvent(int $attempt, int $maxRetries, int $delay, string $errorMessage, array $context = []): void + { + // Only track in production/cloud instances + if (isDev() || ! config('constants.sentry.sentry_dsn')) { + return; + } + + try { + app('sentry')->captureMessage( + 'SSH connection retry triggered', + \Sentry\Severity::warning(), + [ + 'extra' => [ + 'attempt' => $attempt, + 'max_retries' => $maxRetries, + 'delay_seconds' => $delay, + 'error_message' => $errorMessage, + 'context' => $context, + 'retryable_error' => true, + ], + 'tags' => [ + 'component' => 'ssh_retry', + 'error_type' => 'connection_retry', + ], + ] + ); + } catch (\Throwable $e) { + // Don't let Sentry tracking errors break the SSH retry flow + Log::warning('Failed to track SSH retry event in Sentry', [ + 'error' => $e->getMessage(), + 'original_attempt' => $attempt, + ]); + } + } +} diff --git a/bootstrap/helpers/applications.php b/bootstrap/helpers/applications.php index 919b2bde5..db7767c1e 100644 --- a/bootstrap/helpers/applications.php +++ b/bootstrap/helpers/applications.php @@ -1,12 +1,15 @@ <?php +use App\Actions\Application\StopApplication; use App\Enums\ApplicationDeploymentStatus; use App\Jobs\ApplicationDeploymentJob; +use App\Jobs\VolumeCloneJob; use App\Models\Application; use App\Models\ApplicationDeploymentQueue; use App\Models\Server; use App\Models\StandaloneDocker; use Spatie\Url\Url; +use Visus\Cuid2\Cuid2; function queue_application_deployment(Application $application, string $deployment_uuid, ?int $pull_request_id = 0, string $commit = 'HEAD', bool $force_rebuild = false, bool $is_webhook = false, bool $is_api = false, bool $restart_only = false, ?string $git_type = null, bool $no_questions_asked = false, ?Server $server = null, ?StandaloneDocker $destination = null, bool $only_this_server = false, bool $rollback = false) { @@ -68,7 +71,7 @@ function queue_application_deployment(Application $application, string $deployme ApplicationDeploymentJob::dispatch( application_deployment_queue_id: $deployment->id, ); - } elseif (next_queuable($server_id, $application_id, $commit)) { + } elseif (next_queuable($server_id, $application_id, $commit, $pull_request_id)) { ApplicationDeploymentJob::dispatch( application_deployment_queue_id: $deployment->id, ); @@ -93,32 +96,31 @@ function force_start_deployment(ApplicationDeploymentQueue $deployment) function queue_next_deployment(Application $application) { $server_id = $application->destination->server_id; - $next_found = ApplicationDeploymentQueue::where('server_id', $server_id)->where('status', ApplicationDeploymentStatus::QUEUED)->get()->sortBy('created_at')->first(); - if ($next_found) { - $next_found->update([ - 'status' => ApplicationDeploymentStatus::IN_PROGRESS->value, - ]); + $queued_deployments = ApplicationDeploymentQueue::where('server_id', $server_id) + ->where('status', ApplicationDeploymentStatus::QUEUED) + ->get() + ->sortBy('created_at'); - ApplicationDeploymentJob::dispatch( - application_deployment_queue_id: $next_found->id, - ); + foreach ($queued_deployments as $next_deployment) { + // Check if this queued deployment can actually run + if (next_queuable($next_deployment->server_id, $next_deployment->application_id, $next_deployment->commit, $next_deployment->pull_request_id)) { + $next_deployment->update([ + 'status' => ApplicationDeploymentStatus::IN_PROGRESS->value, + ]); + + ApplicationDeploymentJob::dispatch( + application_deployment_queue_id: $next_deployment->id, + ); + } } } -function next_queuable(string $server_id, string $application_id, string $commit = 'HEAD'): bool +function next_queuable(string $server_id, string $application_id, string $commit = 'HEAD', int $pull_request_id = 0): bool { - // Check if there's already a deployment in progress for this application and commit - $existing_deployment = ApplicationDeploymentQueue::where('application_id', $application_id) - ->where('commit', $commit) - ->where('status', ApplicationDeploymentStatus::IN_PROGRESS->value) - ->first(); - - if ($existing_deployment) { - return false; - } - - // Check if there's any deployment in progress for this application + // Check if there's already a deployment in progress for this application with the same pull_request_id + // This allows normal deployments and PR deployments to run concurrently $in_progress = ApplicationDeploymentQueue::where('application_id', $application_id) + ->where('pull_request_id', $pull_request_id) ->where('status', ApplicationDeploymentStatus::IN_PROGRESS->value) ->exists(); @@ -142,13 +144,15 @@ function next_queuable(string $server_id, string $application_id, string $commit function next_after_cancel(?Server $server = null) { if ($server) { - $next_found = ApplicationDeploymentQueue::where('server_id', data_get($server, 'id'))->where('status', ApplicationDeploymentStatus::QUEUED)->get()->sortBy('created_at'); + $next_found = ApplicationDeploymentQueue::where('server_id', data_get($server, 'id')) + ->where('status', ApplicationDeploymentStatus::QUEUED) + ->get() + ->sortBy('created_at'); + if ($next_found->count() > 0) { foreach ($next_found as $next) { - $server = Server::find($next->server_id); - $concurrent_builds = $server->settings->concurrent_builds; - $inprogress_deployments = ApplicationDeploymentQueue::where('server_id', $next->server_id)->whereIn('status', [ApplicationDeploymentStatus::QUEUED])->get()->sortByDesc('created_at'); - if ($inprogress_deployments->count() < $concurrent_builds) { + // Use next_queuable to properly check if this deployment can run + if (next_queuable($next->server_id, $next->application_id, $next->commit, $next->pull_request_id)) { $next->update([ 'status' => ApplicationDeploymentStatus::IN_PROGRESS->value, ]); @@ -157,8 +161,195 @@ function next_after_cancel(?Server $server = null) application_deployment_queue_id: $next->id, ); } - break; } } } } + +function clone_application(Application $source, $destination, array $overrides = [], bool $cloneVolumeData = false): Application +{ + $uuid = $overrides['uuid'] ?? (string) new Cuid2; + $server = $destination->server; + + // Prepare name and URL + $name = $overrides['name'] ?? 'clone-of-'.str($source->name)->limit(20).'-'.$uuid; + $applicationSettings = $source->settings; + $url = $overrides['fqdn'] ?? $source->fqdn; + + if ($server->proxyType() !== 'NONE' && $applicationSettings->is_container_label_readonly_enabled === true) { + $url = generateUrl(server: $server, random: $uuid); + } + + // Clone the application + $newApplication = $source->replicate([ + 'id', + 'created_at', + 'updated_at', + 'additional_servers_count', + 'additional_networks_count', + ])->fill(array_merge([ + 'uuid' => $uuid, + 'name' => $name, + 'fqdn' => $url, + 'status' => 'exited', + 'destination_id' => $destination->id, + ], $overrides)); + $newApplication->save(); + + // Update custom labels if needed + if ($newApplication->destination->server->proxyType() !== 'NONE' && $applicationSettings->is_container_label_readonly_enabled === true) { + $customLabels = str(implode('|coolify|', generateLabelsApplication($newApplication)))->replace('|coolify|', "\n"); + $newApplication->custom_labels = base64_encode($customLabels); + $newApplication->save(); + } + + // Clone settings + $newApplication->settings()->delete(); + if ($applicationSettings) { + $newApplicationSettings = $applicationSettings->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'application_id' => $newApplication->id, + ]); + $newApplicationSettings->save(); + } + + // Clone tags + $tags = $source->tags; + foreach ($tags as $tag) { + $newApplication->tags()->attach($tag->id); + } + + // Clone scheduled tasks + $scheduledTasks = $source->scheduled_tasks()->get(); + foreach ($scheduledTasks as $task) { + $newTask = $task->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'uuid' => (string) new Cuid2, + 'application_id' => $newApplication->id, + 'team_id' => currentTeam()->id, + ]); + $newTask->save(); + } + + // Clone previews with FQDN regeneration + $applicationPreviews = $source->previews()->get(); + foreach ($applicationPreviews as $preview) { + $newPreview = $preview->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'uuid' => (string) new Cuid2, + 'application_id' => $newApplication->id, + 'status' => 'exited', + 'fqdn' => null, + 'docker_compose_domains' => null, + ]); + $newPreview->save(); + + // Regenerate FQDN for the cloned preview + if ($newApplication->build_pack === 'dockercompose') { + $newPreview->generate_preview_fqdn_compose(); + } else { + $newPreview->generate_preview_fqdn(); + } + } + + // Clone persistent volumes + $persistentVolumes = $source->persistentStorages()->get(); + foreach ($persistentVolumes as $volume) { + $newName = ''; + if (str_starts_with($volume->name, $source->uuid)) { + $newName = str($volume->name)->replace($source->uuid, $newApplication->uuid); + } else { + $newName = $newApplication->uuid.'-'.str($volume->name)->afterLast('-'); + } + + $newPersistentVolume = $volume->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'name' => $newName, + 'resource_id' => $newApplication->id, + ]); + $newPersistentVolume->save(); + + if ($cloneVolumeData) { + try { + StopApplication::dispatch($source, false, false); + $sourceVolume = $volume->name; + $targetVolume = $newPersistentVolume->name; + $sourceServer = $source->destination->server; + $targetServer = $newApplication->destination->server; + + VolumeCloneJob::dispatch($sourceVolume, $targetVolume, $sourceServer, $targetServer, $newPersistentVolume); + + queue_application_deployment( + deployment_uuid: (string) new Cuid2, + application: $source, + server: $sourceServer, + destination: $source->destination, + no_questions_asked: true + ); + } catch (\Exception $e) { + \Log::error('Failed to copy volume data for '.$volume->name.': '.$e->getMessage()); + } + } + } + + // Clone file storages + $fileStorages = $source->fileStorages()->get(); + foreach ($fileStorages as $storage) { + $newStorage = $storage->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'resource_id' => $newApplication->id, + ]); + $newStorage->save(); + } + + // Clone production environment variables without triggering the created hook + $environmentVariables = $source->environment_variables()->get(); + foreach ($environmentVariables as $environmentVariable) { + \App\Models\EnvironmentVariable::withoutEvents(function () use ($environmentVariable, $newApplication) { + $newEnvironmentVariable = $environmentVariable->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'resourceable_id' => $newApplication->id, + 'resourceable_type' => $newApplication->getMorphClass(), + 'is_preview' => false, + ]); + $newEnvironmentVariable->save(); + }); + } + + // Clone preview environment variables + $previewEnvironmentVariables = $source->environment_variables_preview()->get(); + foreach ($previewEnvironmentVariables as $previewEnvironmentVariable) { + \App\Models\EnvironmentVariable::withoutEvents(function () use ($previewEnvironmentVariable, $newApplication) { + $newPreviewEnvironmentVariable = $previewEnvironmentVariable->replicate([ + 'id', + 'created_at', + 'updated_at', + ])->fill([ + 'resourceable_id' => $newApplication->id, + 'resourceable_type' => $newApplication->getMorphClass(), + 'is_preview' => true, + ]); + $newPreviewEnvironmentVariable->save(); + }); + } + + return $newApplication; +} diff --git a/bootstrap/helpers/docker.php b/bootstrap/helpers/docker.php index f61abc806..af26c97bd 100644 --- a/bootstrap/helpers/docker.php +++ b/bootstrap/helpers/docker.php @@ -1093,11 +1093,11 @@ function getContainerLogs(Server $server, string $container_id, int $lines = 100 { if ($server->isSwarm()) { $output = instant_remote_process([ - "docker service logs -n {$lines} {$container_id}", + "docker service logs -n {$lines} {$container_id} 2>&1", ], $server); } else { $output = instant_remote_process([ - "docker logs -n {$lines} {$container_id}", + "docker logs -n {$lines} {$container_id} 2>&1", ], $server); } @@ -1105,7 +1105,6 @@ function getContainerLogs(Server $server, string $container_id, int $lines = 100 return $output; } - function escapeEnvVariables($value) { $search = ['\\', "\r", "\t", "\x0", '"', "'"]; @@ -1120,3 +1119,64 @@ function escapeDollarSign($value) return str_replace($search, $replace, $value); } + +/** + * Generate Docker build arguments from environment variables collection + * + * @param \Illuminate\Support\Collection|array $variables Collection of variables with 'key', 'value', and optionally 'is_multiline' + * @return \Illuminate\Support\Collection Collection of formatted --build-arg strings + */ +function generateDockerBuildArgs($variables): \Illuminate\Support\Collection +{ + $variables = collect($variables); + + return $variables->map(function ($var) { + $key = is_array($var) ? data_get($var, 'key') : $var->key; + $value = is_array($var) ? data_get($var, 'value') : $var->value; + $isMultiline = is_array($var) ? data_get($var, 'is_multiline', false) : ($var->is_multiline ?? false); + + if ($isMultiline) { + // For multiline variables, strip surrounding quotes and escape for bash + $raw_value = trim($value, "'"); + $escaped_value = str_replace(['\\', '"', '$', '`'], ['\\\\', '\\"', '\\$', '\\`'], $raw_value); + + return "--build-arg {$key}=\"{$escaped_value}\""; + } + + // For regular variables, use escapeshellarg for security + $value = escapeshellarg($value); + + return "--build-arg {$key}={$value}"; + }); +} + +/** + * Generate Docker environment flags from environment variables collection + * + * @param \Illuminate\Support\Collection|array $variables Collection of variables with 'key', 'value', and optionally 'is_multiline' + * @return string Space-separated environment flags + */ +function generateDockerEnvFlags($variables): string +{ + $variables = collect($variables); + + return $variables + ->map(function ($var) { + $key = is_array($var) ? data_get($var, 'key') : $var->key; + $value = is_array($var) ? data_get($var, 'value') : $var->value; + $isMultiline = is_array($var) ? data_get($var, 'is_multiline', false) : ($var->is_multiline ?? false); + + if ($isMultiline) { + // For multiline variables, strip surrounding quotes and escape for bash + $raw_value = trim($value, "'"); + $escaped_value = str_replace(['\\', '"', '$', '`'], ['\\\\', '\\"', '\\$', '\\`'], $raw_value); + + return "-e {$key}=\"{$escaped_value}\""; + } + + $escaped_value = escapeshellarg($value); + + return "-e {$key}={$escaped_value}"; + }) + ->implode(' '); +} diff --git a/bootstrap/helpers/github.php b/bootstrap/helpers/github.php index 0de2f2fd9..3b5f183fb 100644 --- a/bootstrap/helpers/github.php +++ b/bootstrap/helpers/github.php @@ -135,7 +135,13 @@ function getPermissionsPath(GithubApp $source) function loadRepositoryByPage(GithubApp $source, string $token, int $page) { - $response = Http::withToken($token)->get("{$source->api_url}/installation/repositories?per_page=100&page={$page}"); + $response = Http::GitHub($source->api_url, $token) + ->timeout(20) + ->retry(3, 200, throw: false) + ->get('/installation/repositories', [ + 'per_page' => 100, + 'page' => $page, + ]); $json = $response->json(); if ($response->status() !== 200) { return [ diff --git a/bootstrap/helpers/parsers.php b/bootstrap/helpers/parsers.php index f7041c3da..25cc5d0a6 100644 --- a/bootstrap/helpers/parsers.php +++ b/bootstrap/helpers/parsers.php @@ -342,7 +342,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $fqdn, - 'is_build_time' => false, 'is_preview' => false, ]); } @@ -355,7 +354,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $fqdn, - 'is_build_time' => false, 'is_preview' => false, ]); } @@ -373,7 +371,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $fqdnFor = $key->after('SERVICE_FQDN_')->lower()->value(); $originalFqdnFor = str($fqdnFor)->replace('_', '-'); if (str($fqdnFor)->contains('-')) { - $fqdnFor = str($fqdnFor)->replace('-', '_'); + $fqdnFor = str($fqdnFor)->replace('-', '_')->replace('.', '_'); } // Generated FQDN & URL $fqdn = generateFqdn(server: $server, random: "$originalFqdnFor-$uuid", parserVersion: $resource->compose_parsing_version); @@ -384,32 +382,44 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $fqdn, - 'is_build_time' => false, 'is_preview' => false, ]); if ($resource->build_pack === 'dockercompose') { - $domains = collect(json_decode(data_get($resource, 'docker_compose_domains'))) ?? collect([]); - $domainExists = data_get($domains->get($fqdnFor), 'domain'); - $envExists = $resource->environment_variables()->where('key', $key->value())->first(); - if (str($domainExists)->replace('http://', '')->replace('https://', '')->value() !== $envExists->value) { - $envExists->update([ - 'value' => $url, - ]); + // Check if a service with this name actually exists + $serviceExists = false; + foreach ($services as $serviceName => $service) { + $transformedServiceName = str($serviceName)->replace('-', '_')->replace('.', '_')->value(); + if ($transformedServiceName === $fqdnFor) { + $serviceExists = true; + break; + } } - if (is_null($domainExists)) { - // Put URL in the domains array instead of FQDN - $domains->put((string) $fqdnFor, [ - 'domain' => $url, - ]); - $resource->docker_compose_domains = $domains->toJson(); - $resource->save(); + + // Only add domain if the service exists + if ($serviceExists) { + $domains = collect(json_decode(data_get($resource, 'docker_compose_domains'))) ?? collect([]); + $domainExists = data_get($domains->get($fqdnFor), 'domain'); + $envExists = $resource->environment_variables()->where('key', $key->value())->first(); + if (str($domainExists)->replace('http://', '')->replace('https://', '')->value() !== $envExists->value) { + $envExists->update([ + 'value' => $url, + ]); + } + if (is_null($domainExists)) { + // Put URL in the domains array instead of FQDN + $domains->put((string) $fqdnFor, [ + 'domain' => $url, + ]); + $resource->docker_compose_domains = $domains->toJson(); + $resource->save(); + } } } } elseif ($command->value() === 'URL') { $urlFor = $key->after('SERVICE_URL_')->lower()->value(); $originalUrlFor = str($urlFor)->replace('_', '-'); if (str($urlFor)->contains('-')) { - $urlFor = str($urlFor)->replace('-', '_'); + $urlFor = str($urlFor)->replace('-', '_')->replace('.', '_'); } $url = generateUrl(server: $server, random: "$originalUrlFor-$uuid"); $resource->environment_variables()->firstOrCreate([ @@ -418,24 +428,36 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $url, - 'is_build_time' => false, 'is_preview' => false, ]); if ($resource->build_pack === 'dockercompose') { - $domains = collect(json_decode(data_get($resource, 'docker_compose_domains'))) ?? collect([]); - $domainExists = data_get($domains->get($urlFor), 'domain'); - $envExists = $resource->environment_variables()->where('key', $key->value())->first(); - if ($domainExists !== $envExists->value) { - $envExists->update([ - 'value' => $url, - ]); + // Check if a service with this name actually exists + $serviceExists = false; + foreach ($services as $serviceName => $service) { + $transformedServiceName = str($serviceName)->replace('-', '_')->replace('.', '_')->value(); + if ($transformedServiceName === $urlFor) { + $serviceExists = true; + break; + } } - if (is_null($domainExists)) { - $domains->put((string) $urlFor, [ - 'domain' => $url, - ]); - $resource->docker_compose_domains = $domains->toJson(); - $resource->save(); + + // Only add domain if the service exists + if ($serviceExists) { + $domains = collect(json_decode(data_get($resource, 'docker_compose_domains'))) ?? collect([]); + $domainExists = data_get($domains->get($urlFor), 'domain'); + $envExists = $resource->environment_variables()->where('key', $key->value())->first(); + if ($domainExists !== $envExists->value) { + $envExists->update([ + 'value' => $url, + ]); + } + if (is_null($domainExists)) { + $domains->put((string) $urlFor, [ + 'domain' => $url, + ]); + $resource->docker_compose_domains = $domains->toJson(); + $resource->save(); + } } } } else { @@ -446,7 +468,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, ]); } @@ -454,6 +475,12 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int } } + // generate SERVICE_NAME variables for docker compose services + $serviceNameEnvironments = collect([]); + if ($resource->build_pack === 'dockercompose') { + $serviceNameEnvironments = generateDockerComposeServiceName($services, $pullRequestId); + } + // Parse the rest of the services foreach ($services as $serviceName => $service) { $image = data_get_str($service, 'image'); @@ -567,7 +594,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int } $source = replaceLocalSource($source, $mainDirectory); if ($isPullRequest) { - $source = $source."-pr-$pullRequestId"; + $source = addPreviewDeploymentSuffix($source, $pull_request_id); } LocalFileVolume::updateOrCreate( [ @@ -610,7 +637,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $name = "{$uuid}_{$slugWithoutUuid}"; if ($isPullRequest) { - $name = "{$name}-pr-$pullRequestId"; + $name = addPreviewDeploymentSuffix($name, $pull_request_id); } if (is_string($volume)) { $parsed = parseDockerVolumeString($volume); @@ -651,11 +678,11 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $newDependsOn = collect([]); $depends_on->each(function ($dependency, $condition) use ($pullRequestId, $newDependsOn) { if (is_numeric($condition)) { - $dependency = "$dependency-pr-$pullRequestId"; + $dependency = addPreviewDeploymentSuffix($dependency, $pullRequestId); $newDependsOn->put($condition, $dependency); } else { - $condition = "$condition-pr-$pullRequestId"; + $condition = addPreviewDeploymentSuffix($condition, $pullRequestId); $newDependsOn->put($condition, $dependency); } }); @@ -754,7 +781,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, ]); @@ -771,7 +797,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, ]); } else { @@ -807,7 +832,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, ], [ - 'is_build_time' => false, 'is_preview' => false, 'is_required' => $isRequired, ]); @@ -822,7 +846,6 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, 'is_required' => $isRequired, ]); @@ -858,13 +881,13 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int if ($resource->build_pack !== 'dockercompose') { $domains = collect([]); } - $changedServiceName = str($serviceName)->replace('-', '_')->value(); + $changedServiceName = str($serviceName)->replace('-', '_')->replace('.', '_')->value(); $fqdns = data_get($domains, "$changedServiceName.domain"); // Generate SERVICE_FQDN & SERVICE_URL for dockercompose if ($resource->build_pack === 'dockercompose') { foreach ($domains as $forServiceName => $domain) { $parsedDomain = data_get($domain, 'domain'); - $serviceNameFormatted = str($serviceName)->upper()->replace('-', '_'); + $serviceNameFormatted = str($serviceName)->upper()->replace('-', '_')->replace('.', '_'); if (filled($parsedDomain)) { $parsedDomain = str($parsedDomain)->explode(',')->first(); @@ -872,24 +895,22 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $coolifyScheme = $coolifyUrl->getScheme(); $coolifyFqdn = $coolifyUrl->getHost(); $coolifyUrl = $coolifyUrl->withScheme($coolifyScheme)->withHost($coolifyFqdn)->withPort(null); - $coolifyEnvironments->put('SERVICE_URL_'.str($forServiceName)->upper()->replace('-', '_'), $coolifyUrl->__toString()); - $coolifyEnvironments->put('SERVICE_FQDN_'.str($forServiceName)->upper()->replace('-', '_'), $coolifyFqdn); + $coolifyEnvironments->put('SERVICE_URL_'.str($forServiceName)->upper()->replace('-', '_')->replace('.', '_'), $coolifyUrl->__toString()); + $coolifyEnvironments->put('SERVICE_FQDN_'.str($forServiceName)->upper()->replace('-', '_')->replace('.', '_'), $coolifyFqdn); $resource->environment_variables()->updateOrCreate([ 'resourceable_type' => Application::class, 'resourceable_id' => $resource->id, - 'key' => 'SERVICE_URL_'.str($forServiceName)->upper()->replace('-', '_'), + 'key' => 'SERVICE_URL_'.str($forServiceName)->upper()->replace('-', '_')->replace('.', '_'), ], [ 'value' => $coolifyUrl->__toString(), - 'is_build_time' => false, 'is_preview' => false, ]); $resource->environment_variables()->updateOrCreate([ 'resourceable_type' => Application::class, 'resourceable_id' => $resource->id, - 'key' => 'SERVICE_FQDN_'.str($forServiceName)->upper()->replace('-', '_'), + 'key' => 'SERVICE_FQDN_'.str($forServiceName)->upper()->replace('-', '_')->replace('.', '_'), ], [ 'value' => $coolifyFqdn, - 'is_build_time' => false, 'is_preview' => false, ]); } else { @@ -915,7 +936,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $preview = $resource->previews()->find($preview_id); $docker_compose_domains = collect(json_decode(data_get($preview, 'docker_compose_domains'))); if ($docker_compose_domains->count() > 0) { - $found_fqdn = data_get($docker_compose_domains, "$serviceName.domain"); + $found_fqdn = data_get($docker_compose_domains, "$changedServiceName.domain"); if ($found_fqdn) { $fqdns = collect($found_fqdn); } else { @@ -1082,7 +1103,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $payload['volumes'] = $volumesParsed; } if ($environment->count() > 0 || $coolifyEnvironments->count() > 0) { - $payload['environment'] = $environment->merge($coolifyEnvironments); + $payload['environment'] = $environment->merge($coolifyEnvironments)->merge($serviceNameEnvironments); } if ($logging) { $payload['logging'] = $logging; @@ -1091,7 +1112,7 @@ function applicationParser(Application $resource, int $pull_request_id = 0, ?int $payload['depends_on'] = $depends_on; } if ($isPullRequest) { - $serviceName = "{$serviceName}-pr-{$pullRequestId}"; + $serviceName = addPreviewDeploymentSuffix($serviceName, $pullRequestId); } $parsedServices->put($serviceName, $payload); @@ -1337,7 +1358,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $fqdn, - 'is_build_time' => false, 'is_preview' => false, ]); $resource->environment_variables()->updateOrCreate([ @@ -1346,7 +1366,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $url, - 'is_build_time' => false, 'is_preview' => false, ]); } @@ -1358,7 +1377,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $fqdn, - 'is_build_time' => false, 'is_preview' => false, ]); $resource->environment_variables()->updateOrCreate([ @@ -1367,7 +1385,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $url, - 'is_build_time' => false, 'is_preview' => false, ]); } @@ -1397,7 +1414,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $fqdn, - 'is_build_time' => false, 'is_preview' => false, ]); @@ -1417,7 +1433,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $url, - 'is_build_time' => false, 'is_preview' => false, ]); @@ -1429,7 +1444,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, ]); } @@ -1748,7 +1762,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, ]); @@ -1765,7 +1778,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, ]); } else { @@ -1801,7 +1813,6 @@ function serviceParser(Service $resource): Collection 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, ], [ - 'is_build_time' => false, 'is_preview' => false, 'is_required' => $isRequired, ]); @@ -1816,7 +1827,6 @@ function serviceParser(Service $resource): Collection 'resourceable_id' => $resource->id, ], [ 'value' => $value, - 'is_build_time' => false, 'is_preview' => false, 'is_required' => $isRequired, ]); diff --git a/bootstrap/helpers/proxy.php b/bootstrap/helpers/proxy.php index 2d479a193..5bc1d005e 100644 --- a/bootstrap/helpers/proxy.php +++ b/bootstrap/helpers/proxy.php @@ -1,6 +1,6 @@ <?php -use App\Actions\Proxy\SaveConfiguration; +use App\Actions\Proxy\SaveProxyConfiguration; use App\Enums\ProxyTypes; use App\Models\Application; use App\Models\Server; @@ -267,7 +267,7 @@ function generate_default_proxy_configuration(Server $server) } $config = Yaml::dump($config, 12, 2); - SaveConfiguration::run($server, $config); + SaveProxyConfiguration::run($server, $config); return $config; } diff --git a/bootstrap/helpers/remoteProcess.php b/bootstrap/helpers/remoteProcess.php index 6c1e2beab..3218bf878 100644 --- a/bootstrap/helpers/remoteProcess.php +++ b/bootstrap/helpers/remoteProcess.php @@ -60,15 +60,28 @@ function remote_process( function instant_scp(string $source, string $dest, Server $server, $throwError = true) { - $scp_command = SshMultiplexingHelper::generateScpCommand($server, $source, $dest); - $process = Process::timeout(config('constants.ssh.command_timeout'))->run($scp_command); - $output = trim($process->output()); - $exitCode = $process->exitCode(); - if ($exitCode !== 0) { - return $throwError ? excludeCertainErrors($process->errorOutput(), $exitCode) : null; - } + return \App\Helpers\SshRetryHandler::retry( + function () use ($source, $dest, $server) { + $scp_command = SshMultiplexingHelper::generateScpCommand($server, $source, $dest); + $process = Process::timeout(config('constants.ssh.command_timeout'))->run($scp_command); - return $output === 'null' ? null : $output; + $output = trim($process->output()); + $exitCode = $process->exitCode(); + + if ($exitCode !== 0) { + excludeCertainErrors($process->errorOutput(), $exitCode); + } + + return $output === 'null' ? null : $output; + }, + [ + 'server' => $server->ip, + 'source' => $source, + 'dest' => $dest, + 'function' => 'instant_scp', + ], + $throwError + ); } function instant_remote_process_with_timeout(Collection|array $command, Server $server, bool $throwError = true, bool $no_sudo = false): ?string @@ -79,54 +92,65 @@ function instant_remote_process_with_timeout(Collection|array $command, Server $ } $command_string = implode("\n", $command); - // $start_time = microtime(true); - $sshCommand = SshMultiplexingHelper::generateSshCommand($server, $command_string); - $process = Process::timeout(30)->run($sshCommand); - // $end_time = microtime(true); + return \App\Helpers\SshRetryHandler::retry( + function () use ($server, $command_string) { + $sshCommand = SshMultiplexingHelper::generateSshCommand($server, $command_string); + $process = Process::timeout(30)->run($sshCommand); - // $execution_time = ($end_time - $start_time) * 1000; // Convert to milliseconds - // ray('SSH command execution time:', $execution_time.' ms')->orange(); + $output = trim($process->output()); + $exitCode = $process->exitCode(); - $output = trim($process->output()); - $exitCode = $process->exitCode(); + if ($exitCode !== 0) { + excludeCertainErrors($process->errorOutput(), $exitCode); + } - if ($exitCode !== 0) { - return $throwError ? excludeCertainErrors($process->errorOutput(), $exitCode) : null; - } + // Sanitize output to ensure valid UTF-8 encoding + $output = $output === 'null' ? null : sanitize_utf8_text($output); - // Sanitize output to ensure valid UTF-8 encoding - $output = $output === 'null' ? null : sanitize_utf8_text($output); - - return $output; + return $output; + }, + [ + 'server' => $server->ip, + 'command_preview' => substr($command_string, 0, 100), + 'function' => 'instant_remote_process_with_timeout', + ], + $throwError + ); } function instant_remote_process(Collection|array $command, Server $server, bool $throwError = true, bool $no_sudo = false): ?string { $command = $command instanceof Collection ? $command->toArray() : $command; + if ($server->isNonRoot() && ! $no_sudo) { $command = parseCommandsByLineForSudo(collect($command), $server); } $command_string = implode("\n", $command); - // $start_time = microtime(true); - $sshCommand = SshMultiplexingHelper::generateSshCommand($server, $command_string); - $process = Process::timeout(config('constants.ssh.command_timeout'))->run($sshCommand); - // $end_time = microtime(true); + return \App\Helpers\SshRetryHandler::retry( + function () use ($server, $command_string) { + $sshCommand = SshMultiplexingHelper::generateSshCommand($server, $command_string); + $process = Process::timeout(config('constants.ssh.command_timeout'))->run($sshCommand); - // $execution_time = ($end_time - $start_time) * 1000; // Convert to milliseconds - // ray('SSH command execution time:', $execution_time.' ms')->orange(); + $output = trim($process->output()); + $exitCode = $process->exitCode(); - $output = trim($process->output()); - $exitCode = $process->exitCode(); + if ($exitCode !== 0) { + excludeCertainErrors($process->errorOutput(), $exitCode); + } - if ($exitCode !== 0) { - return $throwError ? excludeCertainErrors($process->errorOutput(), $exitCode) : null; - } + // Sanitize output to ensure valid UTF-8 encoding + $output = $output === 'null' ? null : sanitize_utf8_text($output); - // Sanitize output to ensure valid UTF-8 encoding - $output = $output === 'null' ? null : sanitize_utf8_text($output); - - return $output; + return $output; + }, + [ + 'server' => $server->ip, + 'command_preview' => substr($command_string, 0, 100), + 'function' => 'instant_remote_process', + ], + $throwError + ); } function excludeCertainErrors(string $errorOutput, ?int $exitCode = null) @@ -136,11 +160,18 @@ function excludeCertainErrors(string $errorOutput, ?int $exitCode = null) 'Could not resolve hostname', ]); $ignored = $ignoredErrors->contains(fn ($error) => Str::contains($errorOutput, $error)); + + // Ensure we always have a meaningful error message + $errorMessage = trim($errorOutput); + if (empty($errorMessage)) { + $errorMessage = "SSH command failed with exit code: $exitCode"; + } + if ($ignored) { // TODO: Create new exception and disable in sentry - throw new \RuntimeException($errorOutput, $exitCode); + throw new \RuntimeException($errorMessage, $exitCode); } - throw new \RuntimeException($errorOutput, $exitCode); + throw new \RuntimeException($errorMessage, $exitCode); } function decode_remote_command_output(?ApplicationDeploymentQueue $application_deployment_queue = null): Collection diff --git a/bootstrap/helpers/services.php b/bootstrap/helpers/services.php index cf12a28a5..a124272a2 100644 --- a/bootstrap/helpers/services.php +++ b/bootstrap/helpers/services.php @@ -115,14 +115,14 @@ function updateCompose(ServiceApplication|ServiceDatabase $resource) $resource->save(); } - $serviceName = str($resource->name)->upper()->replace('-', '_'); + $serviceName = str($resource->name)->upper()->replace('-', '_')->replace('.', '_'); $resource->service->environment_variables()->where('key', 'LIKE', "SERVICE_FQDN_{$serviceName}%")->delete(); $resource->service->environment_variables()->where('key', 'LIKE', "SERVICE_URL_{$serviceName}%")->delete(); if ($resource->fqdn) { $resourceFqdns = str($resource->fqdn)->explode(','); $resourceFqdns = $resourceFqdns->first(); - $variableName = 'SERVICE_URL_'.str($resource->name)->upper()->replace('-', '_'); + $variableName = 'SERVICE_URL_'.str($resource->name)->upper()->replace('-', '_')->replace('.', '_'); $url = Url::fromString($resourceFqdns); $port = $url->getPort(); $path = $url->getPath(); @@ -134,7 +134,6 @@ function updateCompose(ServiceApplication|ServiceDatabase $resource) 'key' => $variableName, ], [ 'value' => $urlValue, - 'is_build_time' => false, 'is_preview' => false, ]); if ($port) { @@ -145,11 +144,10 @@ function updateCompose(ServiceApplication|ServiceDatabase $resource) 'key' => $variableName, ], [ 'value' => $urlValue, - 'is_build_time' => false, 'is_preview' => false, ]); } - $variableName = 'SERVICE_FQDN_'.str($resource->name)->upper()->replace('-', '_'); + $variableName = 'SERVICE_FQDN_'.str($resource->name)->upper()->replace('-', '_')->replace('.', '_'); $fqdn = Url::fromString($resourceFqdns); $port = $fqdn->getPort(); $path = $fqdn->getPath(); @@ -164,7 +162,6 @@ function updateCompose(ServiceApplication|ServiceDatabase $resource) 'key' => $variableName, ], [ 'value' => $fqdnValue, - 'is_build_time' => false, 'is_preview' => false, ]); if ($port) { @@ -175,7 +172,6 @@ function updateCompose(ServiceApplication|ServiceDatabase $resource) 'key' => $variableName, ], [ 'value' => $fqdnValue, - 'is_build_time' => false, 'is_preview' => false, ]); } diff --git a/bootstrap/helpers/shared.php b/bootstrap/helpers/shared.php index e01f4d58b..656c607bf 100644 --- a/bootstrap/helpers/shared.php +++ b/bootstrap/helpers/shared.php @@ -204,7 +204,6 @@ function get_latest_version_of_coolify(): string return data_get($versions, 'coolify.v4.version'); } catch (\Throwable $e) { - ray($e->getMessage()); return '0.0.0'; } @@ -635,10 +634,14 @@ function getTopLevelNetworks(Service|Application $resource) $definedNetwork = collect([$resource->uuid]); $services = collect($services)->map(function ($service, $_) use ($topLevelNetworks, $definedNetwork) { $serviceNetworks = collect(data_get($service, 'networks', [])); - $hasHostNetworkMode = data_get($service, 'network_mode') === 'host' ? true : false; + $networkMode = data_get($service, 'network_mode'); - // Only add 'networks' key if 'network_mode' is not 'host' - if (! $hasHostNetworkMode) { + $hasValidNetworkMode = + $networkMode === 'host' || + (is_string($networkMode) && (str_starts_with($networkMode, 'service:') || str_starts_with($networkMode, 'container:'))); + + // Only add 'networks' key if 'network_mode' is not 'host' or does not start with 'service:' or 'container:' + if (! $hasValidNetworkMode) { // Collect/create/update networks if ($serviceNetworks->count() > 0) { foreach ($serviceNetworks as $networkName => $networkDetails) { @@ -962,7 +965,7 @@ function getRealtime() } } -function validate_dns_entry(string $fqdn, Server $server) +function validateDNSEntry(string $fqdn, Server $server) { // https://www.cloudflare.com/ips-v4/# $cloudflare_ips = collect(['173.245.48.0/20', '103.21.244.0/22', '103.22.200.0/22', '103.31.4.0/22', '141.101.64.0/18', '108.162.192.0/18', '190.93.240.0/20', '188.114.96.0/20', '197.234.240.0/22', '198.41.128.0/17', '162.158.0.0/15', '104.16.0.0/13', '172.64.0.0/13', '131.0.72.0/22']); @@ -995,7 +998,7 @@ function validate_dns_entry(string $fqdn, Server $server) } else { foreach ($results as $result) { if ($result->getType() == $type) { - if (ip_match($result->getData(), $cloudflare_ips->toArray(), $match)) { + if (ipMatch($result->getData(), $cloudflare_ips->toArray(), $match)) { $found_matching_ip = true; break; } @@ -1013,7 +1016,7 @@ function validate_dns_entry(string $fqdn, Server $server) return $found_matching_ip; } -function ip_match($ip, $cidrs, &$match = null) +function ipMatch($ip, $cidrs, &$match = null) { foreach ((array) $cidrs as $cidr) { [$subnet, $mask] = explode('/', $cidr); @@ -1027,7 +1030,7 @@ function ip_match($ip, $cidrs, &$match = null) return false; } -function check_ip_against_allowlist($ip, $allowlist) +function checkIPAgainstAllowlist($ip, $allowlist) { if (empty($allowlist)) { return false; @@ -1085,78 +1088,6 @@ function check_ip_against_allowlist($ip, $allowlist) return false; } -function parseCommandsByLineForSudo(Collection $commands, Server $server): array -{ - $commands = $commands->map(function ($line) { - if ( - ! str(trim($line))->startsWith([ - 'cd', - 'command', - 'echo', - 'true', - 'if', - 'fi', - ]) - ) { - return "sudo $line"; - } - - if (str(trim($line))->startsWith('if')) { - return str_replace('if', 'if sudo', $line); - } - - return $line; - }); - - $commands = $commands->map(function ($line) use ($server) { - if (Str::startsWith($line, 'sudo mkdir -p')) { - return "$line && sudo chown -R $server->user:$server->user ".Str::after($line, 'sudo mkdir -p').' && sudo chmod -R o-rwx '.Str::after($line, 'sudo mkdir -p'); - } - - return $line; - }); - - $commands = $commands->map(function ($line) { - $line = str($line); - if (str($line)->contains('$(')) { - $line = $line->replace('$(', '$(sudo '); - } - if (str($line)->contains('||')) { - $line = $line->replace('||', '|| sudo'); - } - if (str($line)->contains('&&')) { - $line = $line->replace('&&', '&& sudo'); - } - if (str($line)->contains(' | ')) { - $line = $line->replace(' | ', ' | sudo '); - } - - return $line->value(); - }); - - return $commands->toArray(); -} -function parseLineForSudo(string $command, Server $server): string -{ - if (! str($command)->startSwith('cd') && ! str($command)->startSwith('command')) { - $command = "sudo $command"; - } - if (Str::startsWith($command, 'sudo mkdir -p')) { - $command = "$command && sudo chown -R $server->user:$server->user ".Str::after($command, 'sudo mkdir -p').' && sudo chmod -R o-rwx '.Str::after($command, 'sudo mkdir -p'); - } - if (str($command)->contains('$(') || str($command)->contains('`')) { - $command = str($command)->replace('$(', '$(sudo ')->replace('`', '`sudo ')->value(); - } - if (str($command)->contains('||')) { - $command = str($command)->replace('||', '|| sudo ')->value(); - } - if (str($command)->contains('&&')) { - $command = str($command)->replace('&&', '&& sudo ')->value(); - } - - return $command; -} - function get_public_ips() { try { @@ -1198,30 +1129,77 @@ function get_public_ips() function isAnyDeploymentInprogress() { $runningJobs = ApplicationDeploymentQueue::where('horizon_job_worker', gethostname())->where('status', ApplicationDeploymentStatus::IN_PROGRESS->value)->get(); - $basicDetails = $runningJobs->map(function ($job) { - return [ - 'id' => $job->id, - 'created_at' => $job->created_at, - 'application_id' => $job->application_id, - 'server_id' => $job->server_id, - 'horizon_job_id' => $job->horizon_job_id, - 'status' => $job->status, - ]; - }); - echo 'Running jobs: '.json_encode($basicDetails)."\n"; + + if ($runningJobs->isEmpty()) { + echo "No deployments in progress.\n"; + exit(0); + } + $horizonJobIds = []; + $deploymentDetails = []; + foreach ($runningJobs as $runningJob) { $horizonJobStatus = getJobStatus($runningJob->horizon_job_id); if ($horizonJobStatus === 'unknown' || $horizonJobStatus === 'reserved') { $horizonJobIds[] = $runningJob->horizon_job_id; + + // Get application and team information + $application = Application::find($runningJob->application_id); + $teamMembers = []; + $deploymentUrl = ''; + + if ($application) { + // Get team members through the application's project + $team = $application->team(); + if ($team) { + $teamMembers = $team->members()->pluck('email')->toArray(); + } + + // Construct the full deployment URL + if ($runningJob->deployment_url) { + $baseUrl = base_url(); + $deploymentUrl = $baseUrl.$runningJob->deployment_url; + } + } + + $deploymentDetails[] = [ + 'id' => $runningJob->id, + 'application_name' => $runningJob->application_name ?? 'Unknown', + 'server_name' => $runningJob->server_name ?? 'Unknown', + 'deployment_url' => $deploymentUrl, + 'team_members' => $teamMembers, + 'created_at' => $runningJob->created_at->format('Y-m-d H:i:s'), + 'horizon_job_id' => $runningJob->horizon_job_id, + ]; } } + if (count($horizonJobIds) === 0) { - echo "No deployments in progress.\n"; + echo "No active deployments in progress (all jobs completed or failed).\n"; exit(0); } - $horizonJobIds = collect($horizonJobIds)->unique()->toArray(); - echo 'There are '.count($horizonJobIds)." deployments in progress.\n"; + + // Display enhanced deployment information + echo "\n=== Running Deployments ===\n"; + echo 'Total active deployments: '.count($horizonJobIds)."\n\n"; + + foreach ($deploymentDetails as $index => $deployment) { + echo 'Deployment #'.($index + 1).":\n"; + echo ' Application: '.$deployment['application_name']."\n"; + echo ' Server: '.$deployment['server_name']."\n"; + echo ' Started: '.$deployment['created_at']."\n"; + if ($deployment['deployment_url']) { + echo ' URL: '.$deployment['deployment_url']."\n"; + } + if (! empty($deployment['team_members'])) { + echo ' Team members: '.implode(', ', $deployment['team_members'])."\n"; + } else { + echo " Team members: No team members found\n"; + } + echo ' Horizon Job ID: '.$deployment['horizon_job_id']."\n"; + echo "\n"; + } + exit(1); } @@ -1298,7 +1276,12 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $serviceNetworks = collect(data_get($service, 'networks', [])); $serviceVariables = collect(data_get($service, 'environment', [])); $serviceLabels = collect(data_get($service, 'labels', [])); - $hasHostNetworkMode = data_get($service, 'network_mode') === 'host' ? true : false; + $networkMode = data_get($service, 'network_mode'); + + $hasValidNetworkMode = + $networkMode === 'host' || + (is_string($networkMode) && (str_starts_with($networkMode, 'service:') || str_starts_with($networkMode, 'container:'))); + if ($serviceLabels->count() > 0) { $removedLabels = collect([]); $serviceLabels = $serviceLabels->filter(function ($serviceLabel, $serviceLabelName) use ($removedLabels) { @@ -1409,7 +1392,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $savedService->ports = $collectedPorts->implode(','); $savedService->save(); - if (! $hasHostNetworkMode) { + if (! $hasValidNetworkMode) { // Add Coolify specific networks $definedNetworkExists = $topLevelNetworks->contains(function ($value, $_) use ($definedNetwork) { return $value == $definedNetwork; @@ -1637,7 +1620,6 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal EnvironmentVariable::create([ 'key' => $key, 'value' => $fqdn, - 'is_build_time' => false, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -1717,7 +1699,6 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal EnvironmentVariable::create([ 'key' => $key, 'value' => $fqdn, - 'is_build_time' => false, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -1756,7 +1737,6 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal EnvironmentVariable::create([ 'key' => $key, 'value' => $generatedValue, - 'is_build_time' => false, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -1795,7 +1775,6 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal 'resourceable_id' => $resource->id, ], [ 'value' => $defaultValue, - 'is_build_time' => false, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -2059,12 +2038,12 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $name = $name->replaceFirst('~', $dir); } if ($pull_request_id !== 0) { - $name = $name."-pr-$pull_request_id"; + $name = addPreviewDeploymentSuffix($name, $pull_request_id); } $volume = str("$name:$mount"); } else { if ($pull_request_id !== 0) { - $name = $name."-pr-$pull_request_id"; + $name = addPreviewDeploymentSuffix($name, $pull_request_id); $volume = str("$name:$mount"); if ($topLevelVolumes->has($name)) { $v = $topLevelVolumes->get($name); @@ -2103,7 +2082,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $name = $volume->before(':'); $mount = $volume->after(':'); if ($pull_request_id !== 0) { - $name = $name."-pr-$pull_request_id"; + $name = addPreviewDeploymentSuffix($name, $pull_request_id); } $volume = str("$name:$mount"); } @@ -2122,7 +2101,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $source = str($source)->replaceFirst('~', $dir); } if ($pull_request_id !== 0) { - $source = $source."-pr-$pull_request_id"; + $source = addPreviewDeploymentSuffix($source, $pull_request_id); } if ($read_only) { data_set($volume, 'source', $source.':'.$target.':ro'); @@ -2131,7 +2110,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal } } else { if ($pull_request_id !== 0) { - $source = $source."-pr-$pull_request_id"; + $source = addPreviewDeploymentSuffix($source, $pull_request_id); } if ($read_only) { data_set($volume, 'source', $source.':'.$target.':ro'); @@ -2183,13 +2162,13 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $name = $name->replaceFirst('~', $dir); } if ($pull_request_id !== 0) { - $name = $name."-pr-$pull_request_id"; + $name = addPreviewDeploymentSuffix($name, $pull_request_id); } $volume = str("$name:$mount"); } else { if ($pull_request_id !== 0) { $uuid = $resource->uuid; - $name = $uuid."-$name-pr-$pull_request_id"; + $name = $uuid.'-'.addPreviewDeploymentSuffix($name, $pull_request_id); $volume = str("$name:$mount"); if ($topLevelVolumes->has($name)) { $v = $topLevelVolumes->get($name); @@ -2231,7 +2210,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal $name = $volume->before(':'); $mount = $volume->after(':'); if ($pull_request_id !== 0) { - $name = $name."-pr-$pull_request_id"; + $name = addPreviewDeploymentSuffix($name, $pull_request_id); } $volume = str("$name:$mount"); } @@ -2259,7 +2238,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal if ($pull_request_id === 0) { $source = $uuid."-$source"; } else { - $source = $uuid."-$source-pr-$pull_request_id"; + $source = $uuid.'-'.addPreviewDeploymentSuffix($source, $pull_request_id); } if ($read_only) { data_set($volume, 'source', $source.':'.$target.':ro'); @@ -2299,7 +2278,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal if ($pull_request_id !== 0 && count($serviceDependencies) > 0) { $serviceDependencies = $serviceDependencies->map(function ($dependency) use ($pull_request_id) { - return $dependency."-pr-$pull_request_id"; + return addPreviewDeploymentSuffix($dependency, $pull_request_id); }); data_set($service, 'depends_on', $serviceDependencies->toArray()); } @@ -2486,7 +2465,6 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal EnvironmentVariable::create([ 'key' => $key, 'value' => $fqdn, - 'is_build_time' => false, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -2498,7 +2476,6 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal EnvironmentVariable::create([ 'key' => $key, 'value' => $generatedValue, - 'is_build_time' => false, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -2532,20 +2509,17 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal if ($foundEnv) { $defaultValue = data_get($foundEnv, 'value'); } - $isBuildTime = data_get($foundEnv, 'is_build_time', false); if ($foundEnv) { $foundEnv->update([ 'key' => $key, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, - 'is_build_time' => $isBuildTime, 'value' => $defaultValue, ]); } else { EnvironmentVariable::create([ 'key' => $key, 'value' => $defaultValue, - 'is_build_time' => $isBuildTime, 'resourceable_type' => get_class($resource), 'resourceable_id' => $resource->id, 'is_preview' => false, @@ -2693,7 +2667,7 @@ function parseDockerComposeFile(Service|Application $resource, bool $isNew = fal }); if ($pull_request_id !== 0) { $services->each(function ($service, $serviceName) use ($pull_request_id, $services) { - $services[$serviceName."-pr-$pull_request_id"] = $service; + $services[addPreviewDeploymentSuffix($serviceName, $pull_request_id)] = $service; data_forget($services, $serviceName); }); } @@ -3073,3 +3047,18 @@ function parseDockerfileInterval(string $something) return $seconds; } + +function addPreviewDeploymentSuffix(string $name, int $pull_request_id = 0): string +{ + return ($pull_request_id === 0) ? $name : $name.'-pr-'.$pull_request_id; +} + +function generateDockerComposeServiceName(mixed $services, int $pullRequestId = 0): Collection +{ + $collection = collect([]); + foreach ($services as $serviceName => $_) { + $collection->put('SERVICE_NAME_'.str($serviceName)->replace('-', '_')->replace('.', '_')->upper(), addPreviewDeploymentSuffix($serviceName, $pullRequestId)); + } + + return $collection; +} diff --git a/bootstrap/helpers/socialite.php b/bootstrap/helpers/socialite.php index 961f6809b..fd3fbe74b 100644 --- a/bootstrap/helpers/socialite.php +++ b/bootstrap/helpers/socialite.php @@ -70,8 +70,14 @@ function get_socialite_provider(string $provider) 'infomaniak' => \SocialiteProviders\Infomaniak\Provider::class, ]; - return Socialite::buildProvider( + $socialite = Socialite::buildProvider( $provider_class_map[$provider], $config ); + + if ($provider == 'gitlab' && ! empty($oauth_setting->base_url)) { + $socialite->setHost($oauth_setting->base_url); + } + + return $socialite; } diff --git a/bootstrap/helpers/sudo.php b/bootstrap/helpers/sudo.php new file mode 100644 index 000000000..ba252c64f --- /dev/null +++ b/bootstrap/helpers/sudo.php @@ -0,0 +1,101 @@ +<?php + +use App\Models\Server; +use Illuminate\Support\Collection; +use Illuminate\Support\Str; + +function shouldChangeOwnership(string $path): bool +{ + $path = trim($path); + + $systemPaths = ['/var', '/etc', '/usr', '/opt', '/sys', '/proc', '/dev', '/bin', '/sbin', '/lib', '/lib64', '/boot', '/root', '/home', '/media', '/mnt', '/srv', '/run']; + + foreach ($systemPaths as $systemPath) { + if ($path === $systemPath || Str::startsWith($path, $systemPath.'/')) { + return false; + } + } + + $isCoolifyPath = Str::startsWith($path, '/data/coolify') || Str::startsWith($path, '/tmp/coolify'); + + return $isCoolifyPath; +} +function parseCommandsByLineForSudo(Collection $commands, Server $server): array +{ + $commands = $commands->map(function ($line) { + if ( + ! str(trim($line))->startsWith([ + 'cd', + 'command', + 'echo', + 'true', + 'if', + 'fi', + ]) + ) { + return "sudo $line"; + } + + if (str(trim($line))->startsWith('if')) { + return str_replace('if', 'if sudo', $line); + } + + return $line; + }); + + $commands = $commands->map(function ($line) use ($server) { + if (Str::startsWith($line, 'sudo mkdir -p')) { + $path = trim(Str::after($line, 'sudo mkdir -p')); + if (shouldChangeOwnership($path)) { + return "$line && sudo chown -R $server->user:$server->user $path && sudo chmod -R o-rwx $path"; + } + + return $line; + } + + return $line; + }); + + $commands = $commands->map(function ($line) { + $line = str($line); + if (str($line)->contains('$(')) { + $line = $line->replace('$(', '$(sudo '); + } + if (str($line)->contains('||')) { + $line = $line->replace('||', '|| sudo'); + } + if (str($line)->contains('&&')) { + $line = $line->replace('&&', '&& sudo'); + } + if (str($line)->contains(' | ')) { + $line = $line->replace(' | ', ' | sudo '); + } + + return $line->value(); + }); + + return $commands->toArray(); +} +function parseLineForSudo(string $command, Server $server): string +{ + if (! str($command)->startSwith('cd') && ! str($command)->startSwith('command')) { + $command = "sudo $command"; + } + if (Str::startsWith($command, 'sudo mkdir -p')) { + $path = trim(Str::after($command, 'sudo mkdir -p')); + if (shouldChangeOwnership($path)) { + $command = "$command && sudo chown -R $server->user:$server->user $path && sudo chmod -R o-rwx $path"; + } + } + if (str($command)->contains('$(') || str($command)->contains('`')) { + $command = str($command)->replace('$(', '$(sudo ')->replace('`', '`sudo ')->value(); + } + if (str($command)->contains('||')) { + $command = str($command)->replace('||', '|| sudo ')->value(); + } + if (str($command)->contains('&&')) { + $command = str($command)->replace('&&', '&& sudo ')->value(); + } + + return $command; +} diff --git a/composer.json b/composer.json index 38756edf9..ea466049d 100644 --- a/composer.json +++ b/composer.json @@ -62,6 +62,7 @@ "barryvdh/laravel-debugbar": "^3.15.4", "driftingly/rector-laravel": "^2.0.5", "fakerphp/faker": "^1.24.1", + "laravel/boost": "^1.1", "laravel/dusk": "^8.3.3", "laravel/pint": "^1.24", "laravel/telescope": "^5.10", diff --git a/composer.lock b/composer.lock index c7de9ad34..6320db071 100644 --- a/composer.lock +++ b/composer.lock @@ -4,7 +4,7 @@ "Read more about it at https://getcomposer.org/doc/01-basic-usage.md#installing-dependencies", "This file is @generated automatically" ], - "content-hash": "a78cf8fdfec25eac43de77c05640dc91", + "content-hash": "a993799242581bd06b5939005ee458d9", "packages": [ { "name": "amphp/amp", @@ -12747,6 +12747,71 @@ }, "time": "2025-04-30T06:54:44+00:00" }, + { + "name": "laravel/boost", + "version": "v1.1.4", + "source": { + "type": "git", + "url": "https://github.com/laravel/boost.git", + "reference": "70f909465bf73dad7e791fad8b7716b3b2712076" + }, + "dist": { + "type": "zip", + "url": "https://api.github.com/repos/laravel/boost/zipball/70f909465bf73dad7e791fad8b7716b3b2712076", + "reference": "70f909465bf73dad7e791fad8b7716b3b2712076", + "shasum": "" + }, + "require": { + "guzzlehttp/guzzle": "^7.9", + "illuminate/console": "^10.0|^11.0|^12.0", + "illuminate/contracts": "^10.0|^11.0|^12.0", + "illuminate/routing": "^10.0|^11.0|^12.0", + "illuminate/support": "^10.0|^11.0|^12.0", + "laravel/mcp": "^0.1.1", + "laravel/prompts": "^0.1.9|^0.3", + "laravel/roster": "^0.2.5", + "php": "^8.1" + }, + "require-dev": { + "laravel/pint": "^1.14", + "mockery/mockery": "^1.6", + "orchestra/testbench": "^8.22.0|^9.0|^10.0", + "pestphp/pest": "^2.0|^3.0", + "phpstan/phpstan": "^2.0" + }, + "type": "library", + "extra": { + "laravel": { + "providers": [ + "Laravel\\Boost\\BoostServiceProvider" + ] + }, + "branch-alias": { + "dev-master": "1.x-dev" + } + }, + "autoload": { + "psr-4": { + "Laravel\\Boost\\": "src/" + } + }, + "notification-url": "https://packagist.org/downloads/", + "license": [ + "MIT" + ], + "description": "Laravel Boost accelerates AI-assisted development to generate high-quality, Laravel-specific code.", + "homepage": "https://github.com/laravel/boost", + "keywords": [ + "ai", + "dev", + "laravel" + ], + "support": { + "issues": "https://github.com/laravel/boost/issues", + "source": "https://github.com/laravel/boost" + }, + "time": "2025-09-04T12:16:09+00:00" + }, { "name": "laravel/dusk", "version": "v8.3.3", @@ -12821,6 +12886,70 @@ }, "time": "2025-06-10T13:59:27+00:00" }, + { + "name": "laravel/mcp", + "version": "v0.1.1", + "source": { + "type": "git", + "url": "https://github.com/laravel/mcp.git", + "reference": "6d6284a491f07c74d34f48dfd999ed52c567c713" + }, + "dist": { + "type": "zip", + "url": "https://api.github.com/repos/laravel/mcp/zipball/6d6284a491f07c74d34f48dfd999ed52c567c713", + "reference": "6d6284a491f07c74d34f48dfd999ed52c567c713", + "shasum": "" + }, + "require": { + "illuminate/console": "^10.0|^11.0|^12.0", + "illuminate/contracts": "^10.0|^11.0|^12.0", + "illuminate/http": "^10.0|^11.0|^12.0", + "illuminate/routing": "^10.0|^11.0|^12.0", + "illuminate/support": "^10.0|^11.0|^12.0", + "illuminate/validation": "^10.0|^11.0|^12.0", + "php": "^8.1|^8.2" + }, + "require-dev": { + "laravel/pint": "^1.14", + "orchestra/testbench": "^8.22.0|^9.0|^10.0", + "phpstan/phpstan": "^2.0" + }, + "type": "library", + "extra": { + "laravel": { + "aliases": { + "Mcp": "Laravel\\Mcp\\Server\\Facades\\Mcp" + }, + "providers": [ + "Laravel\\Mcp\\Server\\McpServiceProvider" + ] + } + }, + "autoload": { + "psr-4": { + "Laravel\\Mcp\\": "src/", + "Workbench\\App\\": "workbench/app/", + "Laravel\\Mcp\\Tests\\": "tests/", + "Laravel\\Mcp\\Server\\": "src/Server/" + } + }, + "notification-url": "https://packagist.org/downloads/", + "license": [ + "MIT" + ], + "description": "The easiest way to add MCP servers to your Laravel app.", + "homepage": "https://github.com/laravel/mcp", + "keywords": [ + "dev", + "laravel", + "mcp" + ], + "support": { + "issues": "https://github.com/laravel/mcp/issues", + "source": "https://github.com/laravel/mcp" + }, + "time": "2025-08-16T09:50:43+00:00" + }, { "name": "laravel/pint", "version": "v1.24.0", @@ -12890,6 +13019,67 @@ }, "time": "2025-07-10T18:09:32+00:00" }, + { + "name": "laravel/roster", + "version": "v0.2.6", + "source": { + "type": "git", + "url": "https://github.com/laravel/roster.git", + "reference": "5615acdf860c5a5c61d04aba44f2d3312550c514" + }, + "dist": { + "type": "zip", + "url": "https://api.github.com/repos/laravel/roster/zipball/5615acdf860c5a5c61d04aba44f2d3312550c514", + "reference": "5615acdf860c5a5c61d04aba44f2d3312550c514", + "shasum": "" + }, + "require": { + "illuminate/console": "^10.0|^11.0|^12.0", + "illuminate/contracts": "^10.0|^11.0|^12.0", + "illuminate/routing": "^10.0|^11.0|^12.0", + "illuminate/support": "^10.0|^11.0|^12.0", + "php": "^8.1|^8.2", + "symfony/yaml": "^6.4|^7.2" + }, + "require-dev": { + "laravel/pint": "^1.14", + "mockery/mockery": "^1.6", + "orchestra/testbench": "^8.22.0|^9.0|^10.0", + "pestphp/pest": "^2.0|^3.0", + "phpstan/phpstan": "^2.0" + }, + "type": "library", + "extra": { + "laravel": { + "providers": [ + "Laravel\\Roster\\RosterServiceProvider" + ] + }, + "branch-alias": { + "dev-master": "1.x-dev" + } + }, + "autoload": { + "psr-4": { + "Laravel\\Roster\\": "src/" + } + }, + "notification-url": "https://packagist.org/downloads/", + "license": [ + "MIT" + ], + "description": "Detect packages & approaches in use within a Laravel project", + "homepage": "https://github.com/laravel/roster", + "keywords": [ + "dev", + "laravel" + ], + "support": { + "issues": "https://github.com/laravel/roster/issues", + "source": "https://github.com/laravel/roster" + }, + "time": "2025-09-04T07:31:39+00:00" + }, { "name": "laravel/telescope", "version": "v5.10.2", diff --git a/config/constants.php b/config/constants.php index 44b51b978..ddda70d19 100644 --- a/config/constants.php +++ b/config/constants.php @@ -2,8 +2,8 @@ return [ 'coolify' => [ - 'version' => '4.0.0-beta.425', - 'helper_version' => '1.0.10', + 'version' => '4.0.0-beta.434', + 'helper_version' => '1.0.11', 'realtime_version' => '1.0.10', 'self_hosted' => env('SELF_HOSTED', true), 'autoupdate' => env('AUTOUPDATE'), @@ -12,6 +12,7 @@ 'helper_image' => env('HELPER_IMAGE', env('REGISTRY_URL', 'ghcr.io').'/coollabsio/coolify-helper'), 'realtime_image' => env('REALTIME_IMAGE', env('REGISTRY_URL', 'ghcr.io').'/coollabsio/coolify-realtime'), 'is_windows_docker_desktop' => env('IS_WINDOWS_DOCKER_DESKTOP', false), + 'releases_url' => 'https://cdn.coollabs.io/coolify/releases.json', ], 'urls' => [ @@ -58,9 +59,16 @@ 'ssh' => [ 'mux_enabled' => env('MUX_ENABLED', env('SSH_MUX_ENABLED', true)), 'mux_persist_time' => env('SSH_MUX_PERSIST_TIME', 3600), + 'mux_health_check_enabled' => env('SSH_MUX_HEALTH_CHECK_ENABLED', true), + 'mux_health_check_timeout' => env('SSH_MUX_HEALTH_CHECK_TIMEOUT', 5), + 'mux_max_age' => env('SSH_MUX_MAX_AGE', 1800), // 30 minutes 'connection_timeout' => 10, 'server_interval' => 20, - 'command_timeout' => 7200, + 'command_timeout' => 3600, + 'max_retries' => env('SSH_MAX_RETRIES', 3), + 'retry_base_delay' => env('SSH_RETRY_BASE_DELAY', 2), // seconds + 'retry_max_delay' => env('SSH_RETRY_MAX_DELAY', 30), // seconds + 'retry_multiplier' => env('SSH_RETRY_MULTIPLIER', 2), ], 'invitation' => [ diff --git a/database/factories/TeamFactory.php b/database/factories/TeamFactory.php new file mode 100644 index 000000000..26748c54e --- /dev/null +++ b/database/factories/TeamFactory.php @@ -0,0 +1,40 @@ +<?php + +namespace Database\Factories; + +use App\Models\Team; +use Illuminate\Database\Eloquent\Factories\Factory; + +/** + * @extends \Illuminate\Database\Eloquent\Factories\Factory<\App\Models\Team> + */ +class TeamFactory extends Factory +{ + protected $model = Team::class; + + /** + * Define the model's default state. + * + * @return array<string, mixed> + */ + public function definition(): array + { + return [ + 'name' => $this->faker->company().' Team', + 'description' => $this->faker->sentence(), + 'personal_team' => false, + 'show_boarding' => false, + ]; + } + + /** + * Indicate that the team is a personal team. + */ + public function personal(): static + { + return $this->state(fn (array $attributes) => [ + 'personal_team' => true, + 'name' => $this->faker->firstName()."'s Team", + ]); + } +} diff --git a/database/migrations/2025_09_05_142446_add_pr_deployments_public_enabled_to_application_settings.php b/database/migrations/2025_09_05_142446_add_pr_deployments_public_enabled_to_application_settings.php new file mode 100644 index 000000000..5d84ce42d --- /dev/null +++ b/database/migrations/2025_09_05_142446_add_pr_deployments_public_enabled_to_application_settings.php @@ -0,0 +1,28 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::table('application_settings', function (Blueprint $table) { + $table->boolean('is_pr_deployments_public_enabled')->default(false)->after('is_preview_deployments_enabled'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('application_settings', function (Blueprint $table) { + $table->dropColumn('is_pr_deployments_public_enabled'); + }); + } +}; diff --git a/database/migrations/2025_09_10_172952_remove_is_readonly_from_local_persistent_volumes_table.php b/database/migrations/2025_09_10_172952_remove_is_readonly_from_local_persistent_volumes_table.php new file mode 100644 index 000000000..31398bd35 --- /dev/null +++ b/database/migrations/2025_09_10_172952_remove_is_readonly_from_local_persistent_volumes_table.php @@ -0,0 +1,28 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::table('local_persistent_volumes', function (Blueprint $table) { + $table->dropColumn('is_readonly'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('local_persistent_volumes', function (Blueprint $table) { + $table->boolean('is_readonly')->default(false); + }); + } +}; diff --git a/database/migrations/2025_09_10_173300_drop_webhooks_table.php b/database/migrations/2025_09_10_173300_drop_webhooks_table.php new file mode 100644 index 000000000..4cb1b4e70 --- /dev/null +++ b/database/migrations/2025_09_10_173300_drop_webhooks_table.php @@ -0,0 +1,31 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::dropIfExists('webhooks'); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::create('webhooks', function (Blueprint $table) { + $table->id(); + $table->enum('status', ['pending', 'success', 'failed'])->default('pending'); + $table->string('type'); + $table->longText('payload'); + $table->longText('failure_reason')->nullable(); + $table->timestamps(); + }); + } +}; diff --git a/database/migrations/2025_09_10_173402_drop_kubernetes_table.php b/database/migrations/2025_09_10_173402_drop_kubernetes_table.php new file mode 100644 index 000000000..329ed0e7e --- /dev/null +++ b/database/migrations/2025_09_10_173402_drop_kubernetes_table.php @@ -0,0 +1,28 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::dropIfExists('kubernetes'); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::create('kubernetes', function (Blueprint $table) { + $table->id(); + $table->string('uuid')->unique(); + $table->timestamps(); + }); + } +}; diff --git a/database/migrations/2025_09_11_143432_remove_is_build_time_from_environment_variables_table.php b/database/migrations/2025_09_11_143432_remove_is_build_time_from_environment_variables_table.php new file mode 100644 index 000000000..076ee8e09 --- /dev/null +++ b/database/migrations/2025_09_11_143432_remove_is_build_time_from_environment_variables_table.php @@ -0,0 +1,38 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::table('environment_variables', function (Blueprint $table) { + // Check if the column exists before trying to drop it + if (Schema::hasColumn('environment_variables', 'is_build_time')) { + // Drop the is_build_time column + // Note: The unique constraints that included is_build_time were tied to old foreign key columns + // (application_id, service_id, database_id) which were removed in migration 2024_12_16_134437. + // Those constraints should no longer exist in the database. + $table->dropColumn('is_build_time'); + } + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('environment_variables', function (Blueprint $table) { + // Re-add the is_build_time column + if (! Schema::hasColumn('environment_variables', 'is_build_time')) { + $table->boolean('is_build_time')->default(false)->after('value'); + } + }); + } +}; diff --git a/database/migrations/2025_09_11_150344_add_is_buildtime_only_to_environment_variables_table.php b/database/migrations/2025_09_11_150344_add_is_buildtime_only_to_environment_variables_table.php new file mode 100644 index 000000000..d95f351d5 --- /dev/null +++ b/database/migrations/2025_09_11_150344_add_is_buildtime_only_to_environment_variables_table.php @@ -0,0 +1,28 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::table('environment_variables', function (Blueprint $table) { + $table->boolean('is_buildtime_only')->default(false)->after('is_preview'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('environment_variables', function (Blueprint $table) { + $table->dropColumn('is_buildtime_only'); + }); + } +}; diff --git a/database/migrations/2025_09_17_081112_add_use_build_secrets_to_application_settings.php b/database/migrations/2025_09_17_081112_add_use_build_secrets_to_application_settings.php new file mode 100644 index 000000000..b78f391fc --- /dev/null +++ b/database/migrations/2025_09_17_081112_add_use_build_secrets_to_application_settings.php @@ -0,0 +1,28 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::table('application_settings', function (Blueprint $table) { + $table->boolean('use_build_secrets')->default(false)->after('is_build_server_enabled'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('application_settings', function (Blueprint $table) { + $table->dropColumn('use_build_secrets'); + }); + } +}; diff --git a/database/migrations/2025_09_18_080152_add_runtime_and_buildtime_to_environment_variables_table.php b/database/migrations/2025_09_18_080152_add_runtime_and_buildtime_to_environment_variables_table.php new file mode 100644 index 000000000..6fd4bfed6 --- /dev/null +++ b/database/migrations/2025_09_18_080152_add_runtime_and_buildtime_to_environment_variables_table.php @@ -0,0 +1,67 @@ +<?php + +use Illuminate\Database\Migrations\Migration; +use Illuminate\Database\Schema\Blueprint; +use Illuminate\Support\Facades\DB; +use Illuminate\Support\Facades\Schema; + +return new class extends Migration +{ + /** + * Run the migrations. + */ + public function up(): void + { + Schema::table('environment_variables', function (Blueprint $table) { + // Add new boolean fields with defaults + $table->boolean('is_runtime')->default(true)->after('is_buildtime_only'); + $table->boolean('is_buildtime')->default(true)->after('is_runtime'); + }); + + // Migrate existing data from is_buildtime_only to new fields + DB::table('environment_variables') + ->where('is_buildtime_only', true) + ->update([ + 'is_runtime' => false, + 'is_buildtime' => true, + ]); + + DB::table('environment_variables') + ->where('is_buildtime_only', false) + ->update([ + 'is_runtime' => true, + 'is_buildtime' => true, + ]); + + // Remove the old is_buildtime_only column + Schema::table('environment_variables', function (Blueprint $table) { + $table->dropColumn('is_buildtime_only'); + }); + } + + /** + * Reverse the migrations. + */ + public function down(): void + { + Schema::table('environment_variables', function (Blueprint $table) { + // Re-add the is_buildtime_only column + $table->boolean('is_buildtime_only')->default(false)->after('is_preview'); + }); + + // Restore data to is_buildtime_only based on new fields + DB::table('environment_variables') + ->where('is_runtime', false) + ->where('is_buildtime', true) + ->update(['is_buildtime_only' => true]); + + DB::table('environment_variables') + ->where('is_runtime', true) + ->update(['is_buildtime_only' => false]); + + // Remove new columns + Schema::table('environment_variables', function (Blueprint $table) { + $table->dropColumn(['is_runtime', 'is_buildtime']); + }); + } +}; diff --git a/database/seeders/DatabaseSeeder.php b/database/seeders/DatabaseSeeder.php index e0e7a3ba5..57ccab4ae 100644 --- a/database/seeders/DatabaseSeeder.php +++ b/database/seeders/DatabaseSeeder.php @@ -29,6 +29,7 @@ public function run(): void DisableTwoStepConfirmationSeeder::class, SentinelSeeder::class, CaSslCertSeeder::class, + PersonalAccessTokenSeeder::class, ]); } } diff --git a/database/seeders/PersonalAccessTokenSeeder.php b/database/seeders/PersonalAccessTokenSeeder.php new file mode 100644 index 000000000..38a45219c --- /dev/null +++ b/database/seeders/PersonalAccessTokenSeeder.php @@ -0,0 +1,115 @@ +<?php + +namespace Database\Seeders; + +use App\Models\PersonalAccessToken; +use App\Models\Team; +use App\Models\User; +use Illuminate\Database\Seeder; +use Illuminate\Support\Facades\Hash; + +class PersonalAccessTokenSeeder extends Seeder +{ + /** + * Run the database seeds. + */ + public function run(): void + { + // Only run in development environment + if (app()->environment('production')) { + $this->command->warn('Skipping PersonalAccessTokenSeeder in production environment'); + + return; + } + + // Get the first user (usually the admin user created during setup) + $user = User::find(0); + + if (! $user) { + $this->command->warn('No user found. Please run UserSeeder first.'); + + return; + } + + // Get the user's first team + $team = $user->teams()->first(); + + if (! $team) { + $this->command->warn('No team found for user. Cannot create API tokens.'); + + return; + } + + // Define test tokens with different scopes + $testTokens = [ + [ + 'name' => 'Development Root Token', + 'token' => 'root', + 'abilities' => ['root'], + ], + [ + 'name' => 'Development Read Token', + 'token' => 'read', + 'abilities' => ['read'], + ], + [ + 'name' => 'Development Read Sensitive Token', + 'token' => 'read-sensitive', + 'abilities' => ['read', 'read:sensitive'], + ], + [ + 'name' => 'Development Write Token', + 'token' => 'write', + 'abilities' => ['write'], + ], + [ + 'name' => 'Development Write Sensitive Token', + 'token' => 'write-sensitive', + 'abilities' => ['write', 'write:sensitive'], + ], + [ + 'name' => 'Development Deploy Token', + 'token' => 'deploy', + 'abilities' => ['deploy'], + ], + ]; + + // First, remove all existing development tokens for this user + $deletedCount = PersonalAccessToken::where('tokenable_id', $user->id) + ->where('tokenable_type', get_class($user)) + ->whereIn('name', array_column($testTokens, 'name')) + ->delete(); + + if ($deletedCount > 0) { + $this->command->info("Removed {$deletedCount} existing development token(s)."); + } + + // Now create fresh tokens + foreach ($testTokens as $tokenData) { + // Create the token with a simple format: Bearer {scope} + // The token format in the database is the hash of the plain text token + $plainTextToken = $tokenData['token']; + + PersonalAccessToken::create([ + 'tokenable_type' => get_class($user), + 'tokenable_id' => $user->id, + 'name' => $tokenData['name'], + 'token' => hash('sha256', $plainTextToken), + 'abilities' => $tokenData['abilities'], + 'team_id' => $team->id, + ]); + + $this->command->info("Created token '{$tokenData['name']}' with Bearer token: {$plainTextToken}"); + } + + $this->command->info(''); + $this->command->info('Test API tokens created successfully!'); + $this->command->info('You can use these tokens in development as:'); + $this->command->info(' Bearer root - Root access'); + $this->command->info(' Bearer read - Read only access'); + $this->command->info(' Bearer read-sensitive - Read with sensitive data access'); + $this->command->info(' Bearer write - Write access'); + $this->command->info(' Bearer write-sensitive - Write with sensitive data access'); + $this->command->info(' Bearer deploy - Deploy access'); + } +} diff --git a/docker/coolify-helper/Dockerfile b/docker/coolify-helper/Dockerfile index c66b8d67e..212703798 100644 --- a/docker/coolify-helper/Dockerfile +++ b/docker/coolify-helper/Dockerfile @@ -10,9 +10,10 @@ ARG DOCKER_BUILDX_VERSION=0.25.0 # https://github.com/buildpacks/pack/releases ARG PACK_VERSION=0.38.2 # https://github.com/railwayapp/nixpacks/releases -ARG NIXPACKS_VERSION=1.39.0 +ARG NIXPACKS_VERSION=1.40.0 # https://github.com/minio/mc/releases -ARG MINIO_VERSION=RELEASE.2025-03-12T17-29-24Z +ARG MINIO_VERSION=RELEASE.2025-08-13T08-35-41Z + FROM minio/mc:${MINIO_VERSION} AS minio-client diff --git a/docker/coolify-realtime/package-lock.json b/docker/coolify-realtime/package-lock.json index 49907cbd4..c445c972c 100644 --- a/docker/coolify-realtime/package-lock.json +++ b/docker/coolify-realtime/package-lock.json @@ -7,7 +7,7 @@ "dependencies": { "@xterm/addon-fit": "0.10.0", "@xterm/xterm": "5.5.0", - "axios": "1.8.4", + "axios": "1.12.0", "cookie": "1.0.2", "dotenv": "16.5.0", "node-pty": "1.0.0", @@ -36,13 +36,13 @@ "license": "MIT" }, "node_modules/axios": { - "version": "1.8.4", - "resolved": "https://registry.npmjs.org/axios/-/axios-1.8.4.tgz", - "integrity": "sha512-eBSYY4Y68NNlHbHBMdeDmKNtDgXWhQsJcGqzO3iLUM0GraQFSS9cVgPX5I9b3lbdFKyYoAEGAZF1DwhTaljNAw==", + "version": "1.12.0", + "resolved": "https://registry.npmjs.org/axios/-/axios-1.12.0.tgz", + "integrity": "sha512-oXTDccv8PcfjZmPGlWsPSwtOJCZ/b6W5jAMCNcfwJbCzDckwG0jrYJFaWH1yvivfCXjVzV/SPDEhMB3Q+DSurg==", "license": "MIT", "dependencies": { "follow-redirects": "^1.15.6", - "form-data": "^4.0.0", + "form-data": "^4.0.4", "proxy-from-env": "^1.1.0" } }, diff --git a/docker/coolify-realtime/package.json b/docker/coolify-realtime/package.json index 7851d7f4d..aec3dbe3d 100644 --- a/docker/coolify-realtime/package.json +++ b/docker/coolify-realtime/package.json @@ -5,7 +5,7 @@ "@xterm/addon-fit": "0.10.0", "@xterm/xterm": "5.5.0", "cookie": "1.0.2", - "axios": "1.8.4", + "axios": "1.12.0", "dotenv": "16.5.0", "node-pty": "1.0.0", "ws": "8.18.1" diff --git a/docker/production/Dockerfile b/docker/production/Dockerfile index 6c9628a81..628fb5054 100644 --- a/docker/production/Dockerfile +++ b/docker/production/Dockerfile @@ -72,6 +72,7 @@ RUN apk add --no-cache gnupg && \ curl -fSsL https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor > /usr/share/keyrings/postgresql.gpg # Install system dependencies +RUN apk upgrade RUN apk add --no-cache \ postgresql${POSTGRES_VERSION}-client \ openssh-client \ diff --git a/hooks/pre-commit b/hooks/pre-commit index 029f67917..fc96e9766 100644 --- a/hooks/pre-commit +++ b/hooks/pre-commit @@ -4,6 +4,19 @@ if sh -c ": >/dev/tty" >/dev/null 2>/dev/null; then exec </dev/tty fi +# Generate service templates and OpenAPI documentation +echo "🔄 Generating service templates..." +php artisan generate:services + +echo "📚 Generating OpenAPI documentation..." +php artisan generate:openapi + +# Add the generated files to the commit +git add templates/service-templates*.json +git add openapi.json openapi.yaml + +echo "✅ Generated files have been added to the commit" + # Get list of stashed PHP files stashed_files=$(git diff --cached --name-only --diff-filter=ACM -- '*.php') diff --git a/openapi.json b/openapi.json index ad20633c4..901741dd0 100644 --- a/openapi.json +++ b/openapi.json @@ -2773,10 +2773,6 @@ "type": "boolean", "description": "The flag to indicate if the environment variable is used in preview deployments." }, - "is_build_time": { - "type": "boolean", - "description": "The flag to indicate if the environment variable is used in build time." - }, "is_literal": { "type": "boolean", "description": "The flag to indicate if the environment variable is a literal, nothing espaced." @@ -2870,10 +2866,6 @@ "type": "boolean", "description": "The flag to indicate if the environment variable is used in preview deployments." }, - "is_build_time": { - "type": "boolean", - "description": "The flag to indicate if the environment variable is used in build time." - }, "is_literal": { "type": "boolean", "description": "The flag to indicate if the environment variable is a literal, nothing espaced." @@ -2972,10 +2964,6 @@ "type": "boolean", "description": "The flag to indicate if the environment variable is used in preview deployments." }, - "is_build_time": { - "type": "boolean", - "description": "The flag to indicate if the environment variable is used in build time." - }, "is_literal": { "type": "boolean", "description": "The flag to indicate if the environment variable is a literal, nothing espaced." @@ -3321,6 +3309,55 @@ ] } }, + "\/databases\/{uuid}\/backups": { + "get": { + "tags": [ + "Databases" + ], + "summary": "Get", + "description": "Get backups details by database UUID.", + "operationId": "get-database-backups-by-uuid", + "parameters": [ + { + "name": "uuid", + "in": "path", + "description": "UUID of the database.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + } + ], + "responses": { + "200": { + "description": "Get all backups for a database", + "content": { + "application\/json": { + "schema": { + "type": "string" + }, + "example": "Content is very complex. Will be implemented later." + } + } + }, + "401": { + "$ref": "#\/components\/responses\/401" + }, + "400": { + "$ref": "#\/components\/responses\/400" + }, + "404": { + "$ref": "#\/components\/responses\/404" + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, "\/databases\/{uuid}": { "get": { "tags": [ @@ -3670,6 +3707,200 @@ ] } }, + "\/databases\/{uuid}\/backups\/{scheduled_backup_uuid}": { + "delete": { + "tags": [ + "Databases" + ], + "summary": "Delete backup configuration", + "description": "Deletes a backup configuration and all its executions.", + "operationId": "delete-backup-configuration-by-uuid", + "parameters": [ + { + "name": "uuid", + "in": "path", + "description": "UUID of the database", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "scheduled_backup_uuid", + "in": "path", + "description": "UUID of the backup configuration to delete", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "name": "delete_s3", + "in": "query", + "description": "Whether to delete all backup files from S3", + "required": false, + "schema": { + "type": "boolean", + "default": false + } + } + ], + "responses": { + "200": { + "description": "Backup configuration deleted.", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "string", + "example": "Backup configuration and all executions deleted." + } + }, + "type": "object" + } + } + } + }, + "404": { + "description": "Backup configuration not found.", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "string", + "example": "Backup configuration not found." + } + }, + "type": "object" + } + } + } + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + }, + "patch": { + "tags": [ + "Databases" + ], + "summary": "Update", + "description": "Update a specific backup configuration for a given database, identified by its UUID and the backup ID", + "operationId": "update-database-backup", + "parameters": [ + { + "name": "uuid", + "in": "path", + "description": "UUID of the database.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "name": "scheduled_backup_uuid", + "in": "path", + "description": "UUID of the backup configuration.", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + } + ], + "requestBody": { + "description": "Database backup configuration data", + "required": true, + "content": { + "application\/json": { + "schema": { + "properties": { + "save_s3": { + "type": "boolean", + "description": "Whether data is saved in s3 or not" + }, + "s3_storage_uuid": { + "type": "string", + "description": "S3 storage UUID" + }, + "backup_now": { + "type": "boolean", + "description": "Whether to take a backup now or not" + }, + "enabled": { + "type": "boolean", + "description": "Whether the backup is enabled or not" + }, + "databases_to_backup": { + "type": "string", + "description": "Comma separated list of databases to backup" + }, + "dump_all": { + "type": "boolean", + "description": "Whether all databases are dumped or not" + }, + "frequency": { + "type": "string", + "description": "Frequency of the backup" + }, + "database_backup_retention_amount_locally": { + "type": "integer", + "description": "Retention amount of the backup locally" + }, + "database_backup_retention_days_locally": { + "type": "integer", + "description": "Retention days of the backup locally" + }, + "database_backup_retention_max_storage_locally": { + "type": "integer", + "description": "Max storage of the backup locally" + }, + "database_backup_retention_amount_s3": { + "type": "integer", + "description": "Retention amount of the backup in s3" + }, + "database_backup_retention_days_s3": { + "type": "integer", + "description": "Retention days of the backup in s3" + }, + "database_backup_retention_max_storage_s3": { + "type": "integer", + "description": "Max storage of the backup in S3" + } + }, + "type": "object" + } + } + } + }, + "responses": { + "200": { + "description": "Database backup configuration updated" + }, + "401": { + "$ref": "#\/components\/responses\/401" + }, + "400": { + "$ref": "#\/components\/responses\/400" + }, + "404": { + "$ref": "#\/components\/responses\/404" + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, "\/databases\/postgresql": { "post": { "tags": [ @@ -4706,6 +4937,175 @@ ] } }, + "\/databases\/{uuid}\/backups\/{scheduled_backup_uuid}\/executions\/{execution_uuid}": { + "delete": { + "tags": [ + "Databases" + ], + "summary": "Delete backup execution", + "description": "Deletes a specific backup execution.", + "operationId": "delete-backup-execution-by-uuid", + "parameters": [ + { + "name": "uuid", + "in": "path", + "description": "UUID of the database", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "scheduled_backup_uuid", + "in": "path", + "description": "UUID of the backup configuration", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "name": "execution_uuid", + "in": "path", + "description": "UUID of the backup execution to delete", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + }, + { + "name": "delete_s3", + "in": "query", + "description": "Whether to delete the backup from S3", + "required": false, + "schema": { + "type": "boolean", + "default": false + } + } + ], + "responses": { + "200": { + "description": "Backup execution deleted.", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "string", + "example": "Backup execution deleted." + } + }, + "type": "object" + } + } + } + }, + "404": { + "description": "Backup execution not found.", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "string", + "example": "Backup execution not found." + } + }, + "type": "object" + } + } + } + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, + "\/databases\/{uuid}\/backups\/{scheduled_backup_uuid}\/executions": { + "get": { + "tags": [ + "Databases" + ], + "summary": "List backup executions", + "description": "Get all executions for a specific backup configuration.", + "operationId": "list-backup-executions", + "parameters": [ + { + "name": "uuid", + "in": "path", + "description": "UUID of the database", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "scheduled_backup_uuid", + "in": "path", + "description": "UUID of the backup configuration", + "required": true, + "schema": { + "type": "string", + "format": "uuid" + } + } + ], + "responses": { + "200": { + "description": "List of backup executions", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "array", + "items": { + "properties": { + "uuid": { + "type": "string" + }, + "filename": { + "type": "string" + }, + "size": { + "type": "integer" + }, + "created_at": { + "type": "string" + }, + "message": { + "type": "string" + }, + "status": { + "type": "string" + } + }, + "type": "object" + } + } + }, + "type": "object" + } + } + } + }, + "404": { + "description": "Backup configuration not found." + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, "\/databases\/{uuid}\/start": { "get": { "tags": [ @@ -5107,6 +5507,477 @@ ] } }, + "\/github-apps": { + "post": { + "tags": [ + "GitHub Apps" + ], + "summary": "Create GitHub App", + "description": "Create a new GitHub app.", + "operationId": "create-github-app", + "requestBody": { + "description": "GitHub app creation payload.", + "required": true, + "content": { + "application\/json": { + "schema": { + "required": [ + "name", + "api_url", + "html_url", + "app_id", + "installation_id", + "client_id", + "client_secret", + "private_key_uuid" + ], + "properties": { + "name": { + "type": "string", + "description": "Name of the GitHub app." + }, + "organization": { + "type": "string", + "nullable": true, + "description": "Organization to associate the app with." + }, + "api_url": { + "type": "string", + "description": "API URL for the GitHub app (e.g., https:\/\/api.github.com)." + }, + "html_url": { + "type": "string", + "description": "HTML URL for the GitHub app (e.g., https:\/\/github.com)." + }, + "custom_user": { + "type": "string", + "description": "Custom user for SSH access (default: git)." + }, + "custom_port": { + "type": "integer", + "description": "Custom port for SSH access (default: 22)." + }, + "app_id": { + "type": "integer", + "description": "GitHub App ID from GitHub." + }, + "installation_id": { + "type": "integer", + "description": "GitHub Installation ID." + }, + "client_id": { + "type": "string", + "description": "GitHub OAuth App Client ID." + }, + "client_secret": { + "type": "string", + "description": "GitHub OAuth App Client Secret." + }, + "webhook_secret": { + "type": "string", + "description": "Webhook secret for GitHub webhooks." + }, + "private_key_uuid": { + "type": "string", + "description": "UUID of an existing private key for GitHub App authentication." + }, + "is_system_wide": { + "type": "boolean", + "description": "Is this app system-wide (cloud only)." + } + }, + "type": "object" + } + } + } + }, + "responses": { + "201": { + "description": "GitHub app created successfully.", + "content": { + "application\/json": { + "schema": { + "properties": { + "id": { + "type": "integer" + }, + "uuid": { + "type": "string" + }, + "name": { + "type": "string" + }, + "organization": { + "type": "string", + "nullable": true + }, + "api_url": { + "type": "string" + }, + "html_url": { + "type": "string" + }, + "custom_user": { + "type": "string" + }, + "custom_port": { + "type": "integer" + }, + "app_id": { + "type": "integer" + }, + "installation_id": { + "type": "integer" + }, + "client_id": { + "type": "string" + }, + "private_key_id": { + "type": "integer" + }, + "is_system_wide": { + "type": "boolean" + }, + "team_id": { + "type": "integer" + } + }, + "type": "object" + } + } + } + }, + "400": { + "$ref": "#\/components\/responses\/400" + }, + "401": { + "$ref": "#\/components\/responses\/401" + }, + "422": { + "$ref": "#\/components\/responses\/422" + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, + "\/github-apps\/{github_app_id}\/repositories": { + "get": { + "tags": [ + "GitHub Apps" + ], + "summary": "Load Repositories for a GitHub App", + "description": "Fetch repositories from GitHub for a given GitHub app.", + "operationId": "load-repositories", + "parameters": [ + { + "name": "github_app_id", + "in": "path", + "description": "GitHub App ID", + "required": true, + "schema": { + "type": "integer" + } + } + ], + "responses": { + "200": { + "description": "Repositories loaded successfully.", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "array", + "items": { + "type": "object" + } + } + }, + "type": "object" + } + } + } + }, + "400": { + "$ref": "#\/components\/responses\/400" + }, + "401": { + "$ref": "#\/components\/responses\/401" + }, + "404": { + "$ref": "#\/components\/responses\/404" + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, + "\/github-apps\/{github_app_id}\/repositories\/{owner}\/{repo}\/branches": { + "get": { + "tags": [ + "GitHub Apps" + ], + "summary": "Load Branches for a GitHub Repository", + "description": "Fetch branches from GitHub for a given repository.", + "operationId": "load-branches", + "parameters": [ + { + "name": "github_app_id", + "in": "path", + "description": "GitHub App ID", + "required": true, + "schema": { + "type": "integer" + } + }, + { + "name": "owner", + "in": "path", + "description": "Repository owner", + "required": true, + "schema": { + "type": "string" + } + }, + { + "name": "repo", + "in": "path", + "description": "Repository name", + "required": true, + "schema": { + "type": "string" + } + } + ], + "responses": { + "200": { + "description": "Branches loaded successfully.", + "content": { + "application\/json": { + "schema": { + "properties": { + "": { + "type": "array", + "items": { + "type": "object" + } + } + }, + "type": "object" + } + } + } + }, + "400": { + "$ref": "#\/components\/responses\/400" + }, + "401": { + "$ref": "#\/components\/responses\/401" + }, + "404": { + "$ref": "#\/components\/responses\/404" + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, + "\/github-apps\/{github_app_id}": { + "delete": { + "tags": [ + "GitHub Apps" + ], + "summary": "Delete GitHub App", + "description": "Delete a GitHub app if it's not being used by any applications.", + "operationId": "deleteGithubApp", + "parameters": [ + { + "name": "github_app_id", + "in": "path", + "description": "GitHub App ID", + "required": true, + "schema": { + "type": "integer" + } + } + ], + "responses": { + "200": { + "description": "GitHub app deleted successfully", + "content": { + "application\/json": { + "schema": { + "properties": { + "message": { + "type": "string", + "example": "GitHub app deleted successfully" + } + }, + "type": "object" + } + } + } + }, + "401": { + "description": "Unauthorized" + }, + "404": { + "description": "GitHub app not found" + }, + "409": { + "description": "Conflict - GitHub app is in use", + "content": { + "application\/json": { + "schema": { + "properties": { + "message": { + "type": "string", + "example": "This GitHub app is being used by 5 application(s). Please delete all applications first." + } + }, + "type": "object" + } + } + } + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + }, + "patch": { + "tags": [ + "GitHub Apps" + ], + "summary": "Update GitHub App", + "description": "Update an existing GitHub app.", + "operationId": "updateGithubApp", + "parameters": [ + { + "name": "github_app_id", + "in": "path", + "description": "GitHub App ID", + "required": true, + "schema": { + "type": "integer" + } + } + ], + "requestBody": { + "required": true, + "content": { + "application\/json": { + "schema": { + "properties": { + "name": { + "type": "string", + "description": "GitHub App name" + }, + "organization": { + "type": "string", + "nullable": true, + "description": "GitHub organization" + }, + "api_url": { + "type": "string", + "description": "GitHub API URL" + }, + "html_url": { + "type": "string", + "description": "GitHub HTML URL" + }, + "custom_user": { + "type": "string", + "description": "Custom user for SSH" + }, + "custom_port": { + "type": "integer", + "description": "Custom port for SSH" + }, + "app_id": { + "type": "integer", + "description": "GitHub App ID" + }, + "installation_id": { + "type": "integer", + "description": "GitHub Installation ID" + }, + "client_id": { + "type": "string", + "description": "GitHub Client ID" + }, + "client_secret": { + "type": "string", + "description": "GitHub Client Secret" + }, + "webhook_secret": { + "type": "string", + "description": "GitHub Webhook Secret" + }, + "private_key_uuid": { + "type": "string", + "description": "Private key UUID" + }, + "is_system_wide": { + "type": "boolean", + "description": "Is system wide (non-cloud instances only)" + } + }, + "type": "object" + } + } + } + }, + "responses": { + "200": { + "description": "GitHub app updated successfully", + "content": { + "application\/json": { + "schema": { + "properties": { + "message": { + "type": "string", + "example": "GitHub app updated successfully" + }, + "data": { + "type": "object", + "description": "Updated GitHub app data" + } + }, + "type": "object" + } + } + } + }, + "401": { + "description": "Unauthorized" + }, + "404": { + "description": "GitHub app not found" + }, + "422": { + "description": "Validation error" + } + }, + "security": [ + { + "bearerAuth": [] + } + ] + } + }, "\/version": { "get": { "summary": "Version", @@ -7179,10 +8050,6 @@ "type": "boolean", "description": "The flag to indicate if the environment variable is used in preview deployments." }, - "is_build_time": { - "type": "boolean", - "description": "The flag to indicate if the environment variable is used in build time." - }, "is_literal": { "type": "boolean", "description": "The flag to indicate if the environment variable is a literal, nothing espaced." @@ -7276,10 +8143,6 @@ "type": "boolean", "description": "The flag to indicate if the environment variable is used in preview deployments." }, - "is_build_time": { - "type": "boolean", - "description": "The flag to indicate if the environment variable is used in build time." - }, "is_literal": { "type": "boolean", "description": "The flag to indicate if the environment variable is a literal, nothing espaced." @@ -7378,10 +8241,6 @@ "type": "boolean", "description": "The flag to indicate if the environment variable is used in preview deployments." }, - "is_build_time": { - "type": "boolean", - "description": "The flag to indicate if the environment variable is used in build time." - }, "is_literal": { "type": "boolean", "description": "The flag to indicate if the environment variable is a literal, nothing espaced." @@ -8375,9 +9234,6 @@ "resourceable_id": { "type": "integer" }, - "is_build_time": { - "type": "boolean" - }, "is_literal": { "type": "boolean" }, @@ -8387,6 +9243,12 @@ "is_preview": { "type": "boolean" }, + "is_runtime": { + "type": "boolean" + }, + "is_buildtime": { + "type": "boolean" + }, "is_shared": { "type": "boolean" }, @@ -8911,6 +9773,10 @@ "name": "Deployments", "description": "Deployments" }, + { + "name": "GitHub Apps", + "description": "GitHub Apps" + }, { "name": "Projects", "description": "Projects" diff --git a/openapi.yaml b/openapi.yaml index ddd814e32..3e39c5d36 100644 --- a/openapi.yaml +++ b/openapi.yaml @@ -1778,9 +1778,6 @@ paths: is_preview: type: boolean description: 'The flag to indicate if the environment variable is used in preview deployments.' - is_build_time: - type: boolean - description: 'The flag to indicate if the environment variable is used in build time.' is_literal: type: boolean description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' @@ -1843,9 +1840,6 @@ paths: is_preview: type: boolean description: 'The flag to indicate if the environment variable is used in preview deployments.' - is_build_time: - type: boolean - description: 'The flag to indicate if the environment variable is used in build time.' is_literal: type: boolean description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' @@ -1901,7 +1895,7 @@ paths: properties: data: type: array - items: { properties: { key: { type: string, description: 'The key of the environment variable.' }, value: { type: string, description: 'The value of the environment variable.' }, is_preview: { type: boolean, description: 'The flag to indicate if the environment variable is used in preview deployments.' }, is_build_time: { type: boolean, description: 'The flag to indicate if the environment variable is used in build time.' }, is_literal: { type: boolean, description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' }, is_multiline: { type: boolean, description: 'The flag to indicate if the environment variable is multiline.' }, is_shown_once: { type: boolean, description: "The flag to indicate if the environment variable's value is shown on the UI." } }, type: object } + items: { properties: { key: { type: string, description: 'The key of the environment variable.' }, value: { type: string, description: 'The value of the environment variable.' }, is_preview: { type: boolean, description: 'The flag to indicate if the environment variable is used in preview deployments.' }, is_literal: { type: boolean, description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' }, is_multiline: { type: boolean, description: 'The flag to indicate if the environment variable is multiline.' }, is_shown_once: { type: boolean, description: "The flag to indicate if the environment variable's value is shown on the UI." } }, type: object } type: object responses: '201': @@ -2103,6 +2097,39 @@ paths: security: - bearerAuth: [] + '/databases/{uuid}/backups': + get: + tags: + - Databases + summary: Get + description: 'Get backups details by database UUID.' + operationId: get-database-backups-by-uuid + parameters: + - + name: uuid + in: path + description: 'UUID of the database.' + required: true + schema: + type: string + format: uuid + responses: + '200': + description: 'Get all backups for a database' + content: + application/json: + schema: + type: string + example: 'Content is very complex. Will be implemented later.' + '401': + $ref: '#/components/responses/401' + '400': + $ref: '#/components/responses/400' + '404': + $ref: '#/components/responses/404' + security: + - + bearerAuth: [] '/databases/{uuid}': get: tags: @@ -2353,6 +2380,139 @@ paths: security: - bearerAuth: [] + '/databases/{uuid}/backups/{scheduled_backup_uuid}': + delete: + tags: + - Databases + summary: 'Delete backup configuration' + description: 'Deletes a backup configuration and all its executions.' + operationId: delete-backup-configuration-by-uuid + parameters: + - + name: uuid + in: path + description: 'UUID of the database' + required: true + schema: + type: string + - + name: scheduled_backup_uuid + in: path + description: 'UUID of the backup configuration to delete' + required: true + schema: + type: string + format: uuid + - + name: delete_s3 + in: query + description: 'Whether to delete all backup files from S3' + required: false + schema: + type: boolean + default: false + responses: + '200': + description: 'Backup configuration deleted.' + content: + application/json: + schema: + properties: + '': { type: string, example: 'Backup configuration and all executions deleted.' } + type: object + '404': + description: 'Backup configuration not found.' + content: + application/json: + schema: + properties: + '': { type: string, example: 'Backup configuration not found.' } + type: object + security: + - + bearerAuth: [] + patch: + tags: + - Databases + summary: Update + description: 'Update a specific backup configuration for a given database, identified by its UUID and the backup ID' + operationId: update-database-backup + parameters: + - + name: uuid + in: path + description: 'UUID of the database.' + required: true + schema: + type: string + format: uuid + - + name: scheduled_backup_uuid + in: path + description: 'UUID of the backup configuration.' + required: true + schema: + type: string + format: uuid + requestBody: + description: 'Database backup configuration data' + required: true + content: + application/json: + schema: + properties: + save_s3: + type: boolean + description: 'Whether data is saved in s3 or not' + s3_storage_uuid: + type: string + description: 'S3 storage UUID' + backup_now: + type: boolean + description: 'Whether to take a backup now or not' + enabled: + type: boolean + description: 'Whether the backup is enabled or not' + databases_to_backup: + type: string + description: 'Comma separated list of databases to backup' + dump_all: + type: boolean + description: 'Whether all databases are dumped or not' + frequency: + type: string + description: 'Frequency of the backup' + database_backup_retention_amount_locally: + type: integer + description: 'Retention amount of the backup locally' + database_backup_retention_days_locally: + type: integer + description: 'Retention days of the backup locally' + database_backup_retention_max_storage_locally: + type: integer + description: 'Max storage of the backup locally' + database_backup_retention_amount_s3: + type: integer + description: 'Retention amount of the backup in s3' + database_backup_retention_days_s3: + type: integer + description: 'Retention days of the backup in s3' + database_backup_retention_max_storage_s3: + type: integer + description: 'Max storage of the backup in S3' + type: object + responses: + '200': + description: 'Database backup configuration updated' + '401': + $ref: '#/components/responses/401' + '400': + $ref: '#/components/responses/400' + '404': + $ref: '#/components/responses/404' + security: + - + bearerAuth: [] /databases/postgresql: post: tags: @@ -3100,6 +3260,102 @@ paths: security: - bearerAuth: [] + '/databases/{uuid}/backups/{scheduled_backup_uuid}/executions/{execution_uuid}': + delete: + tags: + - Databases + summary: 'Delete backup execution' + description: 'Deletes a specific backup execution.' + operationId: delete-backup-execution-by-uuid + parameters: + - + name: uuid + in: path + description: 'UUID of the database' + required: true + schema: + type: string + - + name: scheduled_backup_uuid + in: path + description: 'UUID of the backup configuration' + required: true + schema: + type: string + format: uuid + - + name: execution_uuid + in: path + description: 'UUID of the backup execution to delete' + required: true + schema: + type: string + format: uuid + - + name: delete_s3 + in: query + description: 'Whether to delete the backup from S3' + required: false + schema: + type: boolean + default: false + responses: + '200': + description: 'Backup execution deleted.' + content: + application/json: + schema: + properties: + '': { type: string, example: 'Backup execution deleted.' } + type: object + '404': + description: 'Backup execution not found.' + content: + application/json: + schema: + properties: + '': { type: string, example: 'Backup execution not found.' } + type: object + security: + - + bearerAuth: [] + '/databases/{uuid}/backups/{scheduled_backup_uuid}/executions': + get: + tags: + - Databases + summary: 'List backup executions' + description: 'Get all executions for a specific backup configuration.' + operationId: list-backup-executions + parameters: + - + name: uuid + in: path + description: 'UUID of the database' + required: true + schema: + type: string + - + name: scheduled_backup_uuid + in: path + description: 'UUID of the backup configuration' + required: true + schema: + type: string + format: uuid + responses: + '200': + description: 'List of backup executions' + content: + application/json: + schema: + properties: + '': { type: array, items: { properties: { uuid: { type: string }, filename: { type: string }, size: { type: integer }, created_at: { type: string }, message: { type: string }, status: { type: string } }, type: object } } + type: object + '404': + description: 'Backup configuration not found.' + security: + - + bearerAuth: [] '/databases/{uuid}/start': get: tags: @@ -3354,6 +3610,300 @@ paths: security: - bearerAuth: [] + /github-apps: + post: + tags: + - 'GitHub Apps' + summary: 'Create GitHub App' + description: 'Create a new GitHub app.' + operationId: create-github-app + requestBody: + description: 'GitHub app creation payload.' + required: true + content: + application/json: + schema: + required: + - name + - api_url + - html_url + - app_id + - installation_id + - client_id + - client_secret + - private_key_uuid + properties: + name: + type: string + description: 'Name of the GitHub app.' + organization: + type: string + nullable: true + description: 'Organization to associate the app with.' + api_url: + type: string + description: 'API URL for the GitHub app (e.g., https://api.github.com).' + html_url: + type: string + description: 'HTML URL for the GitHub app (e.g., https://github.com).' + custom_user: + type: string + description: 'Custom user for SSH access (default: git).' + custom_port: + type: integer + description: 'Custom port for SSH access (default: 22).' + app_id: + type: integer + description: 'GitHub App ID from GitHub.' + installation_id: + type: integer + description: 'GitHub Installation ID.' + client_id: + type: string + description: 'GitHub OAuth App Client ID.' + client_secret: + type: string + description: 'GitHub OAuth App Client Secret.' + webhook_secret: + type: string + description: 'Webhook secret for GitHub webhooks.' + private_key_uuid: + type: string + description: 'UUID of an existing private key for GitHub App authentication.' + is_system_wide: + type: boolean + description: 'Is this app system-wide (cloud only).' + type: object + responses: + '201': + description: 'GitHub app created successfully.' + content: + application/json: + schema: + properties: + id: { type: integer } + uuid: { type: string } + name: { type: string } + organization: { type: string, nullable: true } + api_url: { type: string } + html_url: { type: string } + custom_user: { type: string } + custom_port: { type: integer } + app_id: { type: integer } + installation_id: { type: integer } + client_id: { type: string } + private_key_id: { type: integer } + is_system_wide: { type: boolean } + team_id: { type: integer } + type: object + '400': + $ref: '#/components/responses/400' + '401': + $ref: '#/components/responses/401' + '422': + $ref: '#/components/responses/422' + security: + - + bearerAuth: [] + '/github-apps/{github_app_id}/repositories': + get: + tags: + - 'GitHub Apps' + summary: 'Load Repositories for a GitHub App' + description: 'Fetch repositories from GitHub for a given GitHub app.' + operationId: load-repositories + parameters: + - + name: github_app_id + in: path + description: 'GitHub App ID' + required: true + schema: + type: integer + responses: + '200': + description: 'Repositories loaded successfully.' + content: + application/json: + schema: + properties: + '': { type: array, items: { type: object } } + type: object + '400': + $ref: '#/components/responses/400' + '401': + $ref: '#/components/responses/401' + '404': + $ref: '#/components/responses/404' + security: + - + bearerAuth: [] + '/github-apps/{github_app_id}/repositories/{owner}/{repo}/branches': + get: + tags: + - 'GitHub Apps' + summary: 'Load Branches for a GitHub Repository' + description: 'Fetch branches from GitHub for a given repository.' + operationId: load-branches + parameters: + - + name: github_app_id + in: path + description: 'GitHub App ID' + required: true + schema: + type: integer + - + name: owner + in: path + description: 'Repository owner' + required: true + schema: + type: string + - + name: repo + in: path + description: 'Repository name' + required: true + schema: + type: string + responses: + '200': + description: 'Branches loaded successfully.' + content: + application/json: + schema: + properties: + '': { type: array, items: { type: object } } + type: object + '400': + $ref: '#/components/responses/400' + '401': + $ref: '#/components/responses/401' + '404': + $ref: '#/components/responses/404' + security: + - + bearerAuth: [] + '/github-apps/{github_app_id}': + delete: + tags: + - 'GitHub Apps' + summary: 'Delete GitHub App' + description: "Delete a GitHub app if it's not being used by any applications." + operationId: deleteGithubApp + parameters: + - + name: github_app_id + in: path + description: 'GitHub App ID' + required: true + schema: + type: integer + responses: + '200': + description: 'GitHub app deleted successfully' + content: + application/json: + schema: + properties: + message: { type: string, example: 'GitHub app deleted successfully' } + type: object + '401': + description: Unauthorized + '404': + description: 'GitHub app not found' + '409': + description: 'Conflict - GitHub app is in use' + content: + application/json: + schema: + properties: + message: { type: string, example: 'This GitHub app is being used by 5 application(s). Please delete all applications first.' } + type: object + security: + - + bearerAuth: [] + patch: + tags: + - 'GitHub Apps' + summary: 'Update GitHub App' + description: 'Update an existing GitHub app.' + operationId: updateGithubApp + parameters: + - + name: github_app_id + in: path + description: 'GitHub App ID' + required: true + schema: + type: integer + requestBody: + required: true + content: + application/json: + schema: + properties: + name: + type: string + description: 'GitHub App name' + organization: + type: string + nullable: true + description: 'GitHub organization' + api_url: + type: string + description: 'GitHub API URL' + html_url: + type: string + description: 'GitHub HTML URL' + custom_user: + type: string + description: 'Custom user for SSH' + custom_port: + type: integer + description: 'Custom port for SSH' + app_id: + type: integer + description: 'GitHub App ID' + installation_id: + type: integer + description: 'GitHub Installation ID' + client_id: + type: string + description: 'GitHub Client ID' + client_secret: + type: string + description: 'GitHub Client Secret' + webhook_secret: + type: string + description: 'GitHub Webhook Secret' + private_key_uuid: + type: string + description: 'Private key UUID' + is_system_wide: + type: boolean + description: 'Is system wide (non-cloud instances only)' + type: object + responses: + '200': + description: 'GitHub app updated successfully' + content: + application/json: + schema: + properties: + message: { type: string, example: 'GitHub app updated successfully' } + data: { type: object, description: 'Updated GitHub app data' } + type: object + '401': + description: Unauthorized + '404': + description: 'GitHub app not found' + '422': + description: 'Validation error' + security: + - + bearerAuth: [] /version: get: summary: Version @@ -4615,9 +5165,6 @@ paths: is_preview: type: boolean description: 'The flag to indicate if the environment variable is used in preview deployments.' - is_build_time: - type: boolean - description: 'The flag to indicate if the environment variable is used in build time.' is_literal: type: boolean description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' @@ -4680,9 +5227,6 @@ paths: is_preview: type: boolean description: 'The flag to indicate if the environment variable is used in preview deployments.' - is_build_time: - type: boolean - description: 'The flag to indicate if the environment variable is used in build time.' is_literal: type: boolean description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' @@ -4738,7 +5282,7 @@ paths: properties: data: type: array - items: { properties: { key: { type: string, description: 'The key of the environment variable.' }, value: { type: string, description: 'The value of the environment variable.' }, is_preview: { type: boolean, description: 'The flag to indicate if the environment variable is used in preview deployments.' }, is_build_time: { type: boolean, description: 'The flag to indicate if the environment variable is used in build time.' }, is_literal: { type: boolean, description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' }, is_multiline: { type: boolean, description: 'The flag to indicate if the environment variable is multiline.' }, is_shown_once: { type: boolean, description: "The flag to indicate if the environment variable's value is shown on the UI." } }, type: object } + items: { properties: { key: { type: string, description: 'The key of the environment variable.' }, value: { type: string, description: 'The value of the environment variable.' }, is_preview: { type: boolean, description: 'The flag to indicate if the environment variable is used in preview deployments.' }, is_literal: { type: boolean, description: 'The flag to indicate if the environment variable is a literal, nothing espaced.' }, is_multiline: { type: boolean, description: 'The flag to indicate if the environment variable is multiline.' }, is_shown_once: { type: boolean, description: "The flag to indicate if the environment variable's value is shown on the UI." } }, type: object } type: object responses: '201': @@ -5417,14 +5961,16 @@ components: type: string resourceable_id: type: integer - is_build_time: - type: boolean is_literal: type: boolean is_multiline: type: boolean is_preview: type: boolean + is_runtime: + type: boolean + is_buildtime: + type: boolean is_shared: type: boolean is_shown_once: @@ -5791,6 +6337,9 @@ tags: - name: Deployments description: Deployments + - + name: 'GitHub Apps' + description: 'GitHub Apps' - name: Projects description: Projects diff --git a/other/nightly/docker-compose.prod.yml b/other/nightly/docker-compose.prod.yml index 57f062202..b90f126a2 100644 --- a/other/nightly/docker-compose.prod.yml +++ b/other/nightly/docker-compose.prod.yml @@ -61,7 +61,7 @@ services: retries: 10 timeout: 2s soketi: - image: '${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-realtime:1.0.9' + image: '${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-realtime:1.0.10' ports: - "${SOKETI_PORT:-6001}:6001" - "6002:6002" diff --git a/other/nightly/docker-compose.windows.yml b/other/nightly/docker-compose.windows.yml index e19ec961f..09ce3ead3 100644 --- a/other/nightly/docker-compose.windows.yml +++ b/other/nightly/docker-compose.windows.yml @@ -103,7 +103,7 @@ services: retries: 10 timeout: 2s soketi: - image: 'ghcr.io/coollabsio/coolify-realtime:1.0.0' + image: 'ghcr.io/coollabsio/coolify-realtime:1.0.10' pull_policy: always container_name: coolify-realtime restart: always diff --git a/other/nightly/install.sh b/other/nightly/install.sh index 92ad12302..bcd37e71f 100755 --- a/other/nightly/install.sh +++ b/other/nightly/install.sh @@ -20,7 +20,6 @@ DATE=$(date +"%Y%m%d-%H%M%S") OS_TYPE=$(grep -w "ID" /etc/os-release | cut -d "=" -f 2 | tr -d '"') ENV_FILE="/data/coolify/source/.env" -VERSION="21" DOCKER_VERSION="27.0" # TODO: Ask for a user CURRENT_USER=$USER @@ -32,7 +31,7 @@ fi echo -e "Welcome to Coolify Installer!" echo -e "This script will install everything for you. Sit back and relax." -echo -e "Source code: https://github.com/coollabsio/coolify/blob/main/scripts/install.sh\n" +echo -e "Source code: https://github.com/coollabsio/coolify/blob/v4.x/scripts/install.sh" # Predefined root user ROOT_USERNAME=${ROOT_USERNAME:-} @@ -711,84 +710,80 @@ curl -fsSL $CDN/docker-compose.prod.yml -o /data/coolify/source/docker-compose.p curl -fsSL $CDN/.env.production -o /data/coolify/source/.env.production curl -fsSL $CDN/upgrade.sh -o /data/coolify/source/upgrade.sh -echo -e "6. Make backup of .env to .env-$DATE" +echo -e "6. Setting up environment variable file" -# Copy .env.example if .env does not exist -if [ -f $ENV_FILE ]; then - cp $ENV_FILE $ENV_FILE-$DATE +if [ -f "$ENV_FILE" ]; then + # If .env exists, create backup + echo " - Creating backup of existing .env file to .env-$DATE" + cp "$ENV_FILE" "$ENV_FILE-$DATE" + # Merge .env.production values into .env + echo " - Merging .env.production values into .env" + awk -F '=' '!seen[$1]++' "$ENV_FILE" "/data/coolify/source/.env.production" > "$ENV_FILE.tmp" && mv "$ENV_FILE.tmp" "$ENV_FILE" + echo " - .env file merged successfully" else - echo " - File does not exist: $ENV_FILE" - echo " - Copying .env.production to .env-$DATE" - cp /data/coolify/source/.env.production $ENV_FILE-$DATE - # Generate a secure APP_ID and APP_KEY - sed -i "s|^APP_ID=.*|APP_ID=$(openssl rand -hex 16)|" "$ENV_FILE-$DATE" - sed -i "s|^APP_KEY=.*|APP_KEY=base64:$(openssl rand -base64 32)|" "$ENV_FILE-$DATE" - - # Generate a secure Postgres DB username and password - # Causes issues: database "random-user" does not exist - # sed -i "s|^DB_USERNAME=.*|DB_USERNAME=$(openssl rand -hex 16)|" "$ENV_FILE-$DATE" - sed -i "s|^DB_PASSWORD=.*|DB_PASSWORD=$(openssl rand -base64 32)|" "$ENV_FILE-$DATE" - - # Generate a secure Redis password - sed -i "s|^REDIS_PASSWORD=.*|REDIS_PASSWORD=$(openssl rand -base64 32)|" "$ENV_FILE-$DATE" - - # Generate secure Pusher credentials - sed -i "s|^PUSHER_APP_ID=.*|PUSHER_APP_ID=$(openssl rand -hex 32)|" "$ENV_FILE-$DATE" - sed -i "s|^PUSHER_APP_KEY=.*|PUSHER_APP_KEY=$(openssl rand -hex 32)|" "$ENV_FILE-$DATE" - sed -i "s|^PUSHER_APP_SECRET=.*|PUSHER_APP_SECRET=$(openssl rand -hex 32)|" "$ENV_FILE-$DATE" + # If no .env exists, copy .env.production to .env + echo " - No .env file found, copying .env.production to .env" + cp "/data/coolify/source/.env.production" "$ENV_FILE" fi +echo -e "7. Checking and updating environment variables if necessary..." + +update_env_var() { + local key="$1" + local value="$2" + + # If variable "key=" exists but has no value, update the value of the existing line + if grep -q "^${key}=$" "$ENV_FILE"; then + sed -i "s|^${key}=$|${key}=${value}|" "$ENV_FILE" + echo " - Updated value of ${key} as the current value was empty" + # If variable "key=" doesn't exist, append it to the file with value + elif ! grep -q "^${key}=" "$ENV_FILE"; then + printf '%s=%s\n' "$key" "$value" >>"$ENV_FILE" + echo " - Added ${key} and it's value as the variable was missing" + fi +} + +update_env_var "APP_ID" "$(openssl rand -hex 16)" +update_env_var "APP_KEY" "base64:$(openssl rand -base64 32)" +# update_env_var "DB_USERNAME" "$(openssl rand -hex 16)" # Causes issues: database "random-user" does not exist +update_env_var "DB_PASSWORD" "$(openssl rand -base64 32)" +update_env_var "REDIS_PASSWORD" "$(openssl rand -base64 32)" +update_env_var "PUSHER_APP_ID" "$(openssl rand -hex 32)" +update_env_var "PUSHER_APP_KEY" "$(openssl rand -hex 32)" +update_env_var "PUSHER_APP_SECRET" "$(openssl rand -hex 32)" + # Add default root user credentials from environment variables if [ -n "$ROOT_USERNAME" ] && [ -n "$ROOT_USER_EMAIL" ] && [ -n "$ROOT_USER_PASSWORD" ]; then - if grep -q "^ROOT_USERNAME=" "$ENV_FILE-$DATE"; then - sed -i "s|^ROOT_USERNAME=.*|ROOT_USERNAME=$ROOT_USERNAME|" "$ENV_FILE-$DATE" - fi - if grep -q "^ROOT_USER_EMAIL=" "$ENV_FILE-$DATE"; then - sed -i "s|^ROOT_USER_EMAIL=.*|ROOT_USER_EMAIL=$ROOT_USER_EMAIL|" "$ENV_FILE-$DATE" - fi - if grep -q "^ROOT_USER_PASSWORD=" "$ENV_FILE-$DATE"; then - sed -i "s|^ROOT_USER_PASSWORD=.*|ROOT_USER_PASSWORD=$ROOT_USER_PASSWORD|" "$ENV_FILE-$DATE" - fi + echo " - Setting predefined root user credentials from environment" + update_env_var "ROOT_USERNAME" "$ROOT_USERNAME" + update_env_var "ROOT_USER_EMAIL" "$ROOT_USER_EMAIL" + update_env_var "ROOT_USER_PASSWORD" "$ROOT_USER_PASSWORD" fi -# Add registry URL to .env file if [ -n "${REGISTRY_URL+x}" ]; then # Only update if REGISTRY_URL was explicitly provided - if grep -q "^REGISTRY_URL=" "$ENV_FILE-$DATE"; then - sed -i "s|^REGISTRY_URL=.*|REGISTRY_URL=$REGISTRY_URL|" "$ENV_FILE-$DATE" - else - echo "REGISTRY_URL=$REGISTRY_URL" >>"$ENV_FILE-$DATE" - fi + update_env_var "REGISTRY_URL" "$REGISTRY_URL" fi -# Merge .env and .env.production. New values will be added to .env -echo -e "7. Propagating .env with new values - if necessary." -awk -F '=' '!seen[$1]++' "$ENV_FILE-$DATE" /data/coolify/source/.env.production >$ENV_FILE - if [ "$AUTOUPDATE" = "false" ]; then - if ! grep -q "AUTOUPDATE=" /data/coolify/source/.env; then - echo "AUTOUPDATE=false" >>/data/coolify/source/.env - else - sed -i "s|AUTOUPDATE=.*|AUTOUPDATE=false|g" /data/coolify/source/.env + update_env_var "AUTOUPDATE" "false" +fi + +if [ "$DOCKER_POOL_BASE_PROVIDED" = true ]; then + update_env_var "DOCKER_ADDRESS_POOL_BASE" "$DOCKER_ADDRESS_POOL_BASE" +else + # Add with default value if missing + if ! grep -q "^DOCKER_ADDRESS_POOL_BASE=" "$ENV_FILE"; then + update_env_var "DOCKER_ADDRESS_POOL_BASE" "$DOCKER_ADDRESS_POOL_BASE" fi fi -# Save Docker address pool configuration to .env file -if ! grep -q "DOCKER_ADDRESS_POOL_BASE=" /data/coolify/source/.env; then - echo "DOCKER_ADDRESS_POOL_BASE=$DOCKER_ADDRESS_POOL_BASE" >>/data/coolify/source/.env +if [ "$DOCKER_POOL_SIZE_PROVIDED" = true ]; then + update_env_var "DOCKER_ADDRESS_POOL_SIZE" "$DOCKER_ADDRESS_POOL_SIZE" else - # Only update if explicitly provided - if [ "$DOCKER_POOL_BASE_PROVIDED" = true ]; then - sed -i "s|DOCKER_ADDRESS_POOL_BASE=.*|DOCKER_ADDRESS_POOL_BASE=$DOCKER_ADDRESS_POOL_BASE|g" /data/coolify/source/.env - fi -fi - -if ! grep -q "DOCKER_ADDRESS_POOL_SIZE=" /data/coolify/source/.env; then - echo "DOCKER_ADDRESS_POOL_SIZE=$DOCKER_ADDRESS_POOL_SIZE" >>/data/coolify/source/.env -else - # Only update if explicitly provided - if [ "$DOCKER_POOL_SIZE_PROVIDED" = true ]; then - sed -i "s|DOCKER_ADDRESS_POOL_SIZE=.*|DOCKER_ADDRESS_POOL_SIZE=$DOCKER_ADDRESS_POOL_SIZE|g" /data/coolify/source/.env + # Add with default value if missing + if ! grep -q "^DOCKER_ADDRESS_POOL_SIZE=" "$ENV_FILE"; then + update_env_var "DOCKER_ADDRESS_POOL_SIZE" "$DOCKER_ADDRESS_POOL_SIZE" fi fi @@ -824,14 +819,13 @@ echo -e " - Please wait." getAJoke if [[ $- == *x* ]]; then - bash -x /data/coolify/source/upgrade.sh "${LATEST_VERSION:-latest}" "${LATEST_HELPER_VERSION:-latest}" "${REGISTRY_URL:-ghcr.io}" + bash -x /data/coolify/source/upgrade.sh "${LATEST_VERSION:-latest}" "${LATEST_HELPER_VERSION:-latest}" "${REGISTRY_URL:-ghcr.io}" "true" else - bash /data/coolify/source/upgrade.sh "${LATEST_VERSION:-latest}" "${LATEST_HELPER_VERSION:-latest}" "${REGISTRY_URL:-ghcr.io}" + bash /data/coolify/source/upgrade.sh "${LATEST_VERSION:-latest}" "${LATEST_HELPER_VERSION:-latest}" "${REGISTRY_URL:-ghcr.io}" "true" fi echo " - Coolify installed successfully." -rm -f $ENV_FILE-$DATE -echo " - Waiting for 20 seconds for Coolify (database migrations) to be ready." +echo " - Waiting 20 seconds for Coolify database migrations to complete." getAJoke sleep 20 @@ -868,5 +862,5 @@ if [ -n "$PRIVATE_IPS" ]; then fi done fi + echo -e "\nWARNING: It is highly recommended to backup your Environment variables file (/data/coolify/source/.env) to a safe location, outside of this server (e.g. into a Password Manager).\n" -cp /data/coolify/source/.env /data/coolify/source/.env.backup diff --git a/other/nightly/upgrade.sh b/other/nightly/upgrade.sh index 0b031ca75..14eede4ee 100644 --- a/other/nightly/upgrade.sh +++ b/other/nightly/upgrade.sh @@ -1,11 +1,12 @@ #!/bin/bash ## Do not modify this file. You will lose the ability to autoupdate! -VERSION="15" CDN="https://cdn.coollabs.io/coolify-nightly" LATEST_IMAGE=${1:-latest} LATEST_HELPER_VERSION=${2:-latest} REGISTRY_URL=${3:-ghcr.io} +SKIP_BACKUP=${4:-false} +ENV_FILE="/data/coolify/source/.env" DATE=$(date +%Y-%m-%d-%H-%M-%S) LOGFILE="/data/coolify/source/upgrade-${DATE}.log" @@ -14,20 +15,39 @@ curl -fsSL $CDN/docker-compose.yml -o /data/coolify/source/docker-compose.yml curl -fsSL $CDN/docker-compose.prod.yml -o /data/coolify/source/docker-compose.prod.yml curl -fsSL $CDN/.env.production -o /data/coolify/source/.env.production -# Merge .env and .env.production. New values will be added to .env -awk -F '=' '!seen[$1]++' /data/coolify/source/.env /data/coolify/source/.env.production >/data/coolify/source/.env.tmp && mv /data/coolify/source/.env.tmp /data/coolify/source/.env -# Check if PUSHER_APP_ID or PUSHER_APP_KEY or PUSHER_APP_SECRET is empty in /data/coolify/source/.env -if grep -q "PUSHER_APP_ID=$" /data/coolify/source/.env; then - sed -i "s|PUSHER_APP_ID=.*|PUSHER_APP_ID=$(openssl rand -hex 32)|g" /data/coolify/source/.env +# Backup existing .env file before making any changes +if [ "$SKIP_BACKUP" != "true" ]; then + if [ -f "$ENV_FILE" ]; then + echo "Creating backup of existing .env file to .env-$DATE" >>"$LOGFILE" + cp "$ENV_FILE" "$ENV_FILE-$DATE" + else + echo "No existing .env file found to backup" >>"$LOGFILE" + fi fi -if grep -q "PUSHER_APP_KEY=$" /data/coolify/source/.env; then - sed -i "s|PUSHER_APP_KEY=.*|PUSHER_APP_KEY=$(openssl rand -hex 32)|g" /data/coolify/source/.env -fi +echo "Merging .env.production values into .env" >>"$LOGFILE" +awk -F '=' '!seen[$1]++' "$ENV_FILE" /data/coolify/source/.env.production > "$ENV_FILE.tmp" && mv "$ENV_FILE.tmp" "$ENV_FILE" +echo ".env file merged successfully" >>"$LOGFILE" -if grep -q "PUSHER_APP_SECRET=$" /data/coolify/source/.env; then - sed -i "s|PUSHER_APP_SECRET=.*|PUSHER_APP_SECRET=$(openssl rand -hex 32)|g" /data/coolify/source/.env -fi +update_env_var() { + local key="$1" + local value="$2" + + # If variable "key=" exists but has no value, update the value of the existing line + if grep -q "^${key}=$" "$ENV_FILE"; then + sed -i "s|^${key}=$|${key}=${value}|" "$ENV_FILE" + echo " - Updated value of ${key} as the current value was empty" >>"$LOGFILE" + # If variable "key=" doesn't exist, append it to the file with value + elif ! grep -q "^${key}=" "$ENV_FILE"; then + printf '%s=%s\n' "$key" "$value" >>"$ENV_FILE" + echo " - Added ${key} with default value as the variable was missing" >>"$LOGFILE" + fi +} + +echo "Checking and updating environment variables if necessary..." >>"$LOGFILE" +update_env_var "PUSHER_APP_ID" "$(openssl rand -hex 32)" +update_env_var "PUSHER_APP_KEY" "$(openssl rand -hex 32)" +update_env_var "PUSHER_APP_SECRET" "$(openssl rand -hex 32)" # Make sure coolify network exists # It is created when starting Coolify with docker compose @@ -37,11 +57,16 @@ if ! docker network inspect coolify >/dev/null 2>&1; then docker network create --attachable coolify 2>/dev/null fi fi -# docker network create --attachable --driver=overlay coolify-overlay 2>/dev/null + +# Check if Docker config file exists +DOCKER_CONFIG_MOUNT="" +if [ -f /root/.docker/config.json ]; then + DOCKER_CONFIG_MOUNT="-v /root/.docker/config.json:/root/.docker/config.json" +fi if [ -f /data/coolify/source/docker-compose.custom.yml ]; then - echo "docker-compose.custom.yml detected." >>$LOGFILE - docker run -v /data/coolify/source:/data/coolify/source -v /var/run/docker.sock:/var/run/docker.sock --rm ${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-helper:${LATEST_HELPER_VERSION} bash -c "LATEST_IMAGE=${LATEST_IMAGE} docker compose --env-file /data/coolify/source/.env -f /data/coolify/source/docker-compose.yml -f /data/coolify/source/docker-compose.prod.yml -f /data/coolify/source/docker-compose.custom.yml up -d --remove-orphans --force-recreate --wait --wait-timeout 60" >>$LOGFILE 2>&1 + echo "docker-compose.custom.yml detected." >>"$LOGFILE" + docker run -v /data/coolify/source:/data/coolify/source -v /var/run/docker.sock:/var/run/docker.sock ${DOCKER_CONFIG_MOUNT} --rm ${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-helper:${LATEST_HELPER_VERSION} bash -c "LATEST_IMAGE=${LATEST_IMAGE} docker compose --env-file /data/coolify/source/.env -f /data/coolify/source/docker-compose.yml -f /data/coolify/source/docker-compose.prod.yml -f /data/coolify/source/docker-compose.custom.yml up -d --remove-orphans --force-recreate --wait --wait-timeout 60" >>"$LOGFILE" 2>&1 else - docker run -v /data/coolify/source:/data/coolify/source -v /var/run/docker.sock:/var/run/docker.sock --rm ${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-helper:${LATEST_HELPER_VERSION} bash -c "LATEST_IMAGE=${LATEST_IMAGE} docker compose --env-file /data/coolify/source/.env -f /data/coolify/source/docker-compose.yml -f /data/coolify/source/docker-compose.prod.yml up -d --remove-orphans --force-recreate --wait --wait-timeout 60" >>$LOGFILE 2>&1 + docker run -v /data/coolify/source:/data/coolify/source -v /var/run/docker.sock:/var/run/docker.sock ${DOCKER_CONFIG_MOUNT} --rm ${REGISTRY_URL:-ghcr.io}/coollabsio/coolify-helper:${LATEST_HELPER_VERSION} bash -c "LATEST_IMAGE=${LATEST_IMAGE} docker compose --env-file /data/coolify/source/.env -f /data/coolify/source/docker-compose.yml -f /data/coolify/source/docker-compose.prod.yml up -d --remove-orphans --force-recreate --wait --wait-timeout 60" >>"$LOGFILE" 2>&1 fi diff --git a/other/nightly/versions.json b/other/nightly/versions.json index 8697712a8..b5cf3360a 100644 --- a/other/nightly/versions.json +++ b/other/nightly/versions.json @@ -1,19 +1,19 @@ { "coolify": { "v4": { - "version": "4.0.0-beta.423" + "version": "4.0.0-beta.433" }, "nightly": { - "version": "4.0.0-beta.424" + "version": "4.0.0-beta.434" }, "helper": { - "version": "1.0.10" + "version": "1.0.11" }, "realtime": { "version": "1.0.10" }, "sentinel": { - "version": "0.0.15" + "version": "0.0.16" } } } \ No newline at end of file diff --git a/package-lock.json b/package-lock.json index 34b2c1dd5..56e48288c 100644 --- a/package-lock.json +++ b/package-lock.json @@ -22,7 +22,7 @@ "pusher-js": "8.4.0", "tailwind-scrollbar": "4.0.2", "tailwindcss": "4.1.10", - "vite": "6.3.5", + "vite": "6.3.6", "vue": "3.5.16" } }, @@ -1131,6 +1131,66 @@ "node": ">=14.0.0" } }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/core": { + "version": "1.4.3", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/wasi-threads": "1.0.2", + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/runtime": { + "version": "1.4.3", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/wasi-threads": { + "version": "1.0.2", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@napi-rs/wasm-runtime": { + "version": "0.2.10", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/core": "^1.4.3", + "@emnapi/runtime": "^1.4.3", + "@tybys/wasm-util": "^0.9.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@tybys/wasm-util": { + "version": "0.9.0", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/tslib": { + "version": "2.8.0", + "dev": true, + "inBundle": true, + "license": "0BSD", + "optional": true + }, "node_modules/@tailwindcss/oxide-win32-arm64-msvc": { "version": "4.1.10", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.1.10.tgz", @@ -2635,9 +2695,9 @@ "license": "MIT" }, "node_modules/vite": { - "version": "6.3.5", - "resolved": "https://registry.npmjs.org/vite/-/vite-6.3.5.tgz", - "integrity": "sha512-cZn6NDFE7wdTpINgs++ZJ4N49W2vRp8LCKrn3Ob1kYNtOo21vfDoaV5GzBfLU4MovSAB8uNRm4jgzVQZ+mBzPQ==", + "version": "6.3.6", + "resolved": "https://registry.npmjs.org/vite/-/vite-6.3.6.tgz", + "integrity": "sha512-0msEVHJEScQbhkbVTb/4iHZdJ6SXp/AvxL2sjwYQFfBqleHtnCqv1J3sa9zbWz/6kW1m9Tfzn92vW+kZ1WV6QA==", "dev": true, "license": "MIT", "dependencies": { diff --git a/package.json b/package.json index 10ec71415..e29c5e8e6 100644 --- a/package.json +++ b/package.json @@ -16,7 +16,7 @@ "pusher-js": "8.4.0", "tailwind-scrollbar": "4.0.2", "tailwindcss": "4.1.10", - "vite": "6.3.5", + "vite": "6.3.6", "vue": "3.5.16" }, "dependencies": { diff --git a/public/coolify-logo-dev-transparent.png b/public/coolify-logo-dev-transparent.png index 9beeb9ba3..4e65e8b72 100644 Binary files a/public/coolify-logo-dev-transparent.png and b/public/coolify-logo-dev-transparent.png differ diff --git a/public/coolify-logo-dev-transparent.svg b/public/coolify-logo-dev-transparent.svg new file mode 100644 index 000000000..a4159154f --- /dev/null +++ b/public/coolify-logo-dev-transparent.svg @@ -0,0 +1 @@ +<svg role="img" viewBox="0 0 352 352" xmlns="http://www.w3.org/2000/svg"><title>Coolify \ No newline at end of file diff --git a/public/coolify-logo-monochrome.png b/public/coolify-logo-monochrome.png new file mode 100644 index 000000000..48605e8fd Binary files /dev/null and b/public/coolify-logo-monochrome.png differ diff --git a/public/coolify-logo-monochrome.svg b/public/coolify-logo-monochrome.svg new file mode 100644 index 000000000..f60f33f97 --- /dev/null +++ b/public/coolify-logo-monochrome.svg @@ -0,0 +1 @@ +Coolify \ No newline at end of file diff --git a/public/coolify-logo-red.png b/public/coolify-logo-red.png new file mode 100644 index 000000000..b3f7d2b6c Binary files /dev/null and b/public/coolify-logo-red.png differ diff --git a/public/coolify-logo-red.svg b/public/coolify-logo-red.svg new file mode 100644 index 000000000..4cbfef43f --- /dev/null +++ b/public/coolify-logo-red.svg @@ -0,0 +1 @@ +Coolify \ No newline at end of file diff --git a/public/coolify-logo.svg b/public/coolify-logo.svg index 6f4f641f5..bff8f6b40 100644 --- a/public/coolify-logo.svg +++ b/public/coolify-logo.svg @@ -1,9 +1 @@ - - - - - - - - - +Coolify \ No newline at end of file diff --git a/public/coolify-transparent.png b/public/coolify-transparent.png index 96fc0db36..99a56acbe 100644 Binary files a/public/coolify-transparent.png and b/public/coolify-transparent.png differ diff --git a/resources/css/app.css b/resources/css/app.css index 77fa2d66b..c1dc7e56d 100644 --- a/resources/css/app.css +++ b/resources/css/app.css @@ -20,8 +20,11 @@ @theme { --color-warning: #fcd452; --color-success: #16a34a; --color-error: #dc2626; + --color-coollabs-50: #f5f0ff; --color-coollabs: #6b16ed; --color-coollabs-100: #7317ff; + --color-coollabs-200: #5a12c7; + --color-coollabs-300: #4a0fa3; --color-coolgray-100: #181818; --color-coolgray-200: #202020; --color-coolgray-300: #242424; @@ -91,11 +94,11 @@ option { } button[isError]:not(:disabled) { - @apply text-white bg-red-600 hover:bg-red-700; + @apply text-red-800 dark:text-red-300 bg-red-50 dark:bg-red-900/30 border-red-300 dark:border-red-800 hover:bg-red-300 hover:text-white dark:hover:bg-red-800 dark:hover:text-white; } button[isHighlighted]:not(:disabled) { - @apply text-white bg-coollabs hover:bg-coollabs-100; + @apply text-coollabs-200 dark:text-white bg-coollabs-50 dark:bg-coollabs/20 border-coollabs dark:border-coollabs-100 hover:bg-coollabs hover:text-white dark:hover:bg-coollabs-100 dark:hover:text-white; } h1 { @@ -118,6 +121,11 @@ a { @apply hover:text-black dark:hover:text-white; } +button:focus-visible, +a:focus-visible { + @apply outline-none ring-2 ring-coollabs dark:ring-warning ring-offset-2 dark:ring-offset-coolgray-100; +} + label { @apply dark:text-neutral-400; } diff --git a/resources/css/utilities.css b/resources/css/utilities.css index d09d7f49c..bedfb51bc 100644 --- a/resources/css/utilities.css +++ b/resources/css/utilities.css @@ -6,10 +6,31 @@ @utility apexcharts-tooltip-title { @apply hidden!; } +@utility apexcharts-grid-borders { + @apply dark:hidden!; +} + @utility apexcharts-xaxistooltip { @apply hidden!; } +@utility apexcharts-tooltip-custom { + @apply bg-white dark:bg-coolgray-100 border border-neutral-200 dark:border-coolgray-300 rounded-lg shadow-lg p-3 text-sm; + min-width: 160px; +} + +@utility apexcharts-tooltip-custom-value { + @apply text-neutral-700 dark:text-neutral-300 mb-1; +} + +@utility apexcharts-tooltip-value-bold { + @apply font-bold text-black dark:text-white; +} + +@utility apexcharts-tooltip-custom-title { + @apply text-xs text-neutral-500 dark:text-neutral-400 font-medium; +} + @utility input-sticky { @apply block py-1.5 w-full text-sm text-black rounded-sm border-0 ring-1 ring-inset dark:bg-coolgray-100 dark:text-white ring-neutral-200 dark:ring-coolgray-300 focus:ring-2 focus:ring-neutral-400 dark:focus:ring-coolgray-300; } @@ -42,7 +63,7 @@ @utility select { } @utility button { - @apply flex gap-2 justify-center items-center px-2 py-1 text-sm text-black normal-case rounded-sm border outline-0 cursor-pointer bg-neutral-200/50 border-neutral-300 hover:bg-neutral-300 dark:bg-coolgray-200 dark:text-white dark:hover:text-white dark:hover:bg-coolgray-500 dark:border-coolgray-300 hover:text-black disabled:cursor-not-allowed min-w-fit dark:disabled:text-neutral-600 disabled:border-transparent disabled:hover:bg-transparent disabled:bg-transparent disabled:text-neutral-300; + @apply flex gap-2 justify-center items-center px-2 h-8 text-sm text-black normal-case rounded-sm border-2 outline-0 cursor-pointer font-medium bg-white border-neutral-200 hover:bg-neutral-100 dark:bg-coolgray-100 dark:text-white dark:hover:text-white dark:hover:bg-coolgray-200 dark:border-coolgray-300 hover:text-black disabled:cursor-not-allowed min-w-fit dark:disabled:text-neutral-600 disabled:border-transparent disabled:hover:bg-transparent disabled:bg-transparent disabled:text-neutral-300 focus-visible:ring-2 focus-visible:ring-coollabs dark:focus-visible:ring-warning focus-visible:ring-offset-2 dark:focus-visible:ring-offset-coolgray-100; } @utility alert-success { @@ -62,11 +83,11 @@ @utility add-tag { } @utility dropdown-item { - @apply flex relative gap-2 justify-start items-center py-1 pr-4 pl-2 w-full text-xs transition-colors cursor-pointer select-none dark:text-white hover:bg-neutral-100 dark:hover:bg-coollabs outline-none data-disabled:pointer-events-none data-disabled:opacity-50; + @apply flex relative gap-2 justify-start items-center py-1 pr-4 pl-2 w-full text-xs transition-colors cursor-pointer select-none dark:text-white hover:bg-neutral-100 dark:hover:bg-coollabs outline-none data-disabled:pointer-events-none data-disabled:opacity-50 focus-visible:bg-neutral-100 dark:focus-visible:bg-coollabs; } @utility dropdown-item-no-padding { - @apply flex relative gap-2 justify-start items-center py-1 w-full text-xs transition-colors cursor-pointer select-none dark:text-white hover:bg-neutral-100 dark:hover:bg-coollabs outline-none data-disabled:pointer-events-none data-disabled:opacity-50; + @apply flex relative gap-2 justify-start items-center py-1 w-full text-xs transition-colors cursor-pointer select-none dark:text-white hover:bg-neutral-100 dark:hover:bg-coollabs outline-none data-disabled:pointer-events-none data-disabled:opacity-50 focus-visible:bg-neutral-100 dark:focus-visible:bg-coollabs; } @utility badge { @@ -134,15 +155,15 @@ @utility kbd-custom { } @utility box { - @apply relative flex lg:flex-row flex-col p-2 transition-colors cursor-pointer min-h-[4rem] dark:bg-coolgray-100 shadow-sm bg-white border text-black dark:text-white hover:text-black border-neutral-200 dark:border-black hover:bg-neutral-100 dark:hover:bg-coollabs-100 dark:hover:text-white hover:no-underline; + @apply relative flex lg:flex-row flex-col p-2 transition-colors cursor-pointer min-h-[4rem] dark:bg-coolgray-100 shadow-sm bg-white border text-black dark:text-white hover:text-black border-neutral-200 dark:border-coolgray-300 hover:bg-neutral-100 dark:hover:bg-coollabs-100 dark:hover:text-white hover:no-underline rounded-sm; } @utility box-boarding { - @apply flex lg:flex-row flex-col p-2 transition-colors cursor-pointer min-h-[4rem] dark:bg-coolgray-100 dark:text-white bg-neutral-50 border border-neutral-200 dark:border-black hover:bg-neutral-100 dark:hover:bg-coollabs-100 dark:hover:text-white hover:text-black hover:no-underline text-black; + @apply flex lg:flex-row flex-col p-2 transition-colors cursor-pointer min-h-[4rem] dark:bg-coolgray-100 dark:text-white bg-neutral-50 border border-neutral-200 dark:border-coolgray-300 hover:bg-neutral-100 dark:hover:bg-coollabs-100 dark:hover:text-white hover:text-black hover:no-underline text-black rounded-sm; } @utility box-without-bg { - @apply flex p-2 transition-colors dark:hover:text-white hover:no-underline min-h-[4rem] border border-neutral-200 dark:border-black; + @apply flex p-2 transition-colors dark:hover:text-white hover:no-underline min-h-[4rem] border border-neutral-200 dark:border-coolgray-300 rounded-sm; } @utility box-without-bg-without-border { @@ -178,7 +199,7 @@ @utility info-helper { } @utility info-helper-popup { - @apply hidden absolute z-40 text-xs rounded-sm text-neutral-700 group-hover:block dark:border-coolgray-500 border-neutral-900 dark:bg-coolgray-400 bg-neutral-200 dark:text-neutral-300; + @apply hidden absolute z-40 text-xs rounded-sm text-neutral-700 group-hover:block dark:border-coolgray-500 border-neutral-900 dark:bg-coolgray-400 bg-neutral-200 dark:text-neutral-300 max-w-xs whitespace-normal break-words; } @utility buyme { diff --git a/resources/views/components/applications/advanced.blade.php b/resources/views/components/applications/advanced.blade.php index 46ea54e99..e36583741 100644 --- a/resources/views/components/applications/advanced.blade.php +++ b/resources/views/components/applications/advanced.blade.php @@ -19,7 +19,7 @@ @else
- + + The following domain(s) are already in use by other resources. Using the same domain for + multiple resources can cause routing conflicts and unpredictable behavior. +
-

Conflicting Resources:

    @foreach ($conflicts as $conflict)
  • @@ -58,9 +56,7 @@ class="underline hover:text-red-400">
- +
-
+
{{ $slot }}
diff --git a/resources/views/components/environment-variable-warning.blade.php b/resources/views/components/environment-variable-warning.blade.php new file mode 100644 index 000000000..1ad05ed46 --- /dev/null +++ b/resources/views/components/environment-variable-warning.blade.php @@ -0,0 +1,37 @@ +@props(['problematicVariables' => []]) + + diff --git a/resources/views/components/modal-confirmation.blade.php b/resources/views/components/modal-confirmation.blade.php index 0d185782f..1a3c88f80 100644 --- a/resources/views/components/modal-confirmation.blade.php +++ b/resources/views/components/modal-confirmation.blade.php @@ -11,6 +11,7 @@ 'content' => null, 'checkboxes' => [], 'actions' => [], + 'warningMessage' => null, 'confirmWithText' => true, 'confirmationText' => 'Confirm Deletion', 'confirmationLabel' => 'Please confirm the execution of the actions by entering the Name below', @@ -200,9 +201,6 @@ class="flex absolute top-2 right-2 justify-center items-center w-8 h-8 rounded-f @if (!empty($checkboxes))
-
-

Actions

-
@foreach ($checkboxes as $index => $checkbox)
- + + {!! $warningMessage ?: 'This operation is permanent and cannot be undone. Please think again before proceeding!' !!} +
The following actions will be performed:
    @foreach ($actions as $action) @@ -324,10 +320,9 @@ class="w-auto" isError @if (!$disableTwoStepConfirmation)
    - + + Please enter your password to confirm this destructive action. +
    @php $passwordConfirm = Str::uuid(); diff --git a/resources/views/components/modal-input.blade.php b/resources/views/components/modal-input.blade.php index 71959d66d..c15985d03 100644 --- a/resources/views/components/modal-input.blade.php +++ b/resources/views/components/modal-input.blade.php @@ -28,6 +28,7 @@ class="relative w-auto h-auto" wire:ignore> @endif