[ISSUE-99] Deploy portainer-automation service #137

Dave · 2026-07-06T21:22:14Z

Dave commented

2026-07-06 21:22:14 +00:00

Summary

Deploy the portainer-automation service as a separate Portainer stack (Option B). This service provides a POST /api/deploy/{stackId} endpoint that CI workflows call to trigger Portainer stack redeploys after pushing new container images.

Files Created

File	Purpose
`portainer-automation/Dockerfile`	Multi-stage Docker build for PA service
`portainer-automation/docker-compose.yaml`	Standalone compose file for Portainer stack deployment
`portainer-automation/src/main/resources/application-prod.properties`	Prod profile targeting Portainer :9443
`portainer-automation/src/main/resources/application-ci.properties`	CI profile with test/stub endpoints
`portainer-automation/.env.example`	Documented env var templates for all environments
`portainer-automation/PORTAINER_STACK.md`	Deployment guide, env var tables, architecture diagram, troubleshooting
`.gitea/workflows/portainer_automation_build_push.yml`	CI workflow to build & push PA image on push to `test`

Files Modified

File	Change
`docker-compose.yaml`	Added `portainer-automation` service alongside `app` and `postgres`

Architecture

CI Workflow → POST /api/deploy/{stackId} (X-API-Key)
    → Portainer Automation Service (port:8081)
    → Portainer API (PUT /api/stacks/{id}/git/redeploy)
    → Target Stack redeployed from Git

Environment Variables

Variable	Required	Purpose
`PORTAINER_API_KEY`	Yes	Portainer API access token
`PORTAINER_SERVICE_API_KEY`	Yes	CI → service auth (X-API-Key header)
`SPRING_PROFILES_ACTIVE`	No	Profile: local/ci/prod
`ENV`	No	Environment label (dev/test/prod)

Testing

Gradle build passes: ./gradlew :portainer-automation:build
Unit tests pass (DeployControllerTest, DeployServiceTest)
Existing CFTunnels tests unaffected (no changes to test_build.yml or integration_test.yaml)
Manual: curl -X POST -H "X-API-Key: <key>" http://localhost:8081/api/deploy/<stack-id>

Issues

Hithomelabs/CFTunnels#99
Hithomelabs/CFTunnels#124 (architecture)

## Summary Deploy the **portainer-automation** service as a separate Portainer stack (Option B). This service provides a `POST /api/deploy/{stackId}` endpoint that CI workflows call to trigger Portainer stack redeploys after pushing new container images. ## Files Created | File | Purpose | |------|---------| | `portainer-automation/Dockerfile` | Multi-stage Docker build for PA service | | `portainer-automation/docker-compose.yaml` | Standalone compose file for Portainer stack deployment | | `portainer-automation/src/main/resources/application-prod.properties` | Prod profile targeting Portainer :9443 | | `portainer-automation/src/main/resources/application-ci.properties` | CI profile with test/stub endpoints | | `portainer-automation/.env.example` | Documented env var templates for all environments | | `portainer-automation/PORTAINER_STACK.md` | Deployment guide, env var tables, architecture diagram, troubleshooting | | `.gitea/workflows/portainer_automation_build_push.yml` | CI workflow to build & push PA image on push to `test` | ## Files Modified | File | Change | |------|--------| | `docker-compose.yaml` | Added `portainer-automation` service alongside `app` and `postgres` | ## Architecture ``` CI Workflow → POST /api/deploy/{stackId} (X-API-Key) → Portainer Automation Service (port:8081) → Portainer API (PUT /api/stacks/{id}/git/redeploy) → Target Stack redeployed from Git ``` ## Environment Variables | Variable | Required | Purpose | |----------|----------|---------| | `PORTAINER_API_KEY` | Yes | Portainer API access token | | `PORTAINER_SERVICE_API_KEY` | Yes | CI → service auth (X-API-Key header) | | `SPRING_PROFILES_ACTIVE` | No | Profile: local/ci/prod | | `ENV` | No | Environment label (dev/test/prod) | ## Testing - [ ] Gradle build passes: `./gradlew :portainer-automation:build` - [ ] Unit tests pass (DeployControllerTest, DeployServiceTest) - [x] Existing CFTunnels tests unaffected (no changes to test_build.yml or integration_test.yaml) - [ ] Manual: `curl -X POST -H "X-API-Key: <key>" http://localhost:8081/api/deploy/<stack-id>` ## Issues - Hithomelabs/CFTunnels#99 - Hithomelabs/CFTunnels#124 (architecture)

Dave added 1 commit 2026-07-06 21:22:15 +00:00

Hithomelabs/CFTunnels#99 : Deploy portainer-automation service

sample gradle build and test / build (pull_request) Successful in 2m1s

Details

8f4af5e322

- Add Dockerfile for portainer-automation (multi-stage build)
- Add standalone docker-compose.yaml for PA service
- Add application-prod.properties (prod Portainer :9443)
- Add application-ci.properties (CI test endpoints)
- Add .env.example with documented env vars for PA
- Add PORTAINER_STACK.md with deployment guide, env var tables, architecture diagram
- Add CI workflow (portainer_automation_build_push.yml) for image build/push
- Update root docker-compose.yaml to include PA service

hitanshu reviewed 2026-07-06 21:56:21 +00:00

portainer-automation/Dockerfile

						
				@ -0,0 +3,4 @@

				COPY gradlew settings.gradle build.gradle ./

				COPY gradle ./gradle

				COPY common ./common

				COPY cftunnels-service ./cftunnels-service

hitanshu commented

2026-07-06 21:45:46 +00:00

@Dave do we need to copy this for portainer-automation to function ?

Linus approved these changes 2026-07-06 21:57:45 +00:00

Linus left a comment

PR Review: [ISSUE-99] Deploy portainer-automation service

Overview

This PR implements Option B — deploying the portainer-automation service as a separate Portainer stack. It adds 8 files (CI workflow, Dockerfile, standalone compose, profile configs, docs, and modifies the root compose). The implementation reuses the existing common:portainer client library for Portainer API interactions.

✅ Strengths

Clean separation: Subproject follows existing Gradle multi-module patterns and reuses common library
Good test coverage: 7 test cases across DeployControllerTest (4: valid key, invalid key, missing key, service failure) and DeployServiceTest (4: with env, null env, resource not found, connection error, redeploy failure)
Comprehensive documentation: PORTAINER_STACK.md is excellent — includes architecture diagram, env var tables, deployment steps, CI trigger examples, troubleshooting
Profile-based config: local / ci / prod profiles properly isolate environments; trust-all SSL for local/Cloudflare Tunnel makes sense
API key auth: Both CI→service (X-API-Key header) and service→Portainer use API key auth — proper zero-trust pattern
Proper env var preservation: DeployService fetches existing env vars before redeploying, preventing them from being overwritten
CI workflow: Proper tag-based versioning (pa-{version}), separate from CFTunnels tags

⚠️ Concerns & Recommendations

#	Severity	Issue	Recommendation
1	Medium	Generic Exception catch — `DeployController.java` line 37 catches `Exception e`	Catch specific Portainer client exceptions (`PortainerConnectionException`, `PortainerResourceNotFoundException`, etc.) and map to appropriate HTTP status codes (502, 404) instead of blanket 500
2	Low	No SLF4J logging — Controller and Service have no logging	Add `Logger` and log deployment attempts (with stack ID), auth failures, and errors before returning responses — critical for debugging production issues
3	Low	Dockerfile copies unnecessary `cftunnels-service/` — The portainer-automation module only depends on `:common`, but the Dockerfile copies the entire `cftunnels-service/` tree	Since Gradle resolves the full project tree via `settings.gradle`, removing the copy may cause build failures. Either remove `cftunnels-service` from `settings.gradle` for this Docker build (via a different Gradle settings file) or add a `.dockerignore` to exclude it
4	Info	CI workflow uses `secrets.TOKEN` — Generic secret name for registry password	Consider renaming to `secrets.REGISTRY_TOKEN` or `secrets.GITEA_REGISTRY_TOKEN` for clarity (but keep consistent with other workflows if they use `TOKEN`)
5	Info	`stack.env` in compose files — Both compose files reference `stack.env` via `env_file`	This file doesn't exist by default in Portainer stack deployments. Either document that it's optional and can be empty, or remove the `env_file` directive and set all vars via `environment` block
6	Info	No health endpoint — Acknowledged in docs as missing	Add a simple `GET /api/health` → `{"status":"UP"}` endpoint to enable Portainer health checks and easier debugging
7	Low	Hardcoded Portainer port 9442 in default application.properties	The default profile targets `192.168.0.100:9442`. In the docs, dev uses `devdocker.hithomelabs.com` (local profile) and prod uses `:9443`. Ensure the default profile is never used accidentally in production

🔒 Security Review

Check	Status	Notes
Exposed secrets (hardcoded keys/tokens)	✅ Pass	All keys from env vars/properties
Input validation	✅ Pass	API key header validated (null-safe, null returns 401)
Auth bypass vectors	✅ Pass	X-API-Key required for all deploy endpoints
SSL/TLS	⚠️ Note	Trust-all SSL in `local` profile acceptable (behind Cloudflare Tunnel); `prod` uses standard SSL validation
Docker supply chain	✅ Pass	Multi-stage build, no unnecessary packages in final image

🧪 Testing Assessment

Unit tests: ✅ Good — Controller tests cover auth success/failure/missing and service error propagation; Service tests cover all key code paths
Integration tests: ⏳ Not included in this PR — The integration_test.yaml workflow tests the main CFTunnels app, not PA. Recommend adding PA-specific integration tests in a follow-up
Manual verification: ✅ Documented in PORTAINER_STACK.md

📋 Implementation Checklist

API endpoint design (POST /api/deploy/{stackId})
Database schema update — N/A (no DB needed)
Unit tests (DeployControllerTest, DeployServiceTest)
Integration tests (deferred)
Documentation update (PORTAINER_STACK.md, .env.example)
Security review (API key auth, profile isolation)
Health check endpoint (deferred — acknowledged in docs)

Approval Status

REVIEW COMPLETE — APPROVED ✅

This is a well-crafted PR with clean code, thorough testing, and comprehensive documentation. The concerns noted are non-blocking recommendations for improvement. The implementation correctly follows the Option B architecture (separate Portainer stack) and integrates properly with the existing common library and CI infrastructure.

Please address items #1 (specific exception handling), #2 (logging), and consider #6 (health endpoint) before or after merge as time permits.

## PR Review: [ISSUE-99] Deploy portainer-automation service ### Overview This PR implements Option B — deploying the **portainer-automation** service as a separate Portainer stack. It adds 8 files (CI workflow, Dockerfile, standalone compose, profile configs, docs, and modifies the root compose). The implementation reuses the existing `common:portainer` client library for Portainer API interactions. --- ### ✅ Strengths - **Clean separation**: Subproject follows existing Gradle multi-module patterns and reuses `common` library - **Good test coverage**: 7 test cases across `DeployControllerTest` (4: valid key, invalid key, missing key, service failure) and `DeployServiceTest` (4: with env, null env, resource not found, connection error, redeploy failure) - **Comprehensive documentation**: `PORTAINER_STACK.md` is excellent — includes architecture diagram, env var tables, deployment steps, CI trigger examples, troubleshooting - **Profile-based config**: `local` / `ci` / `prod` profiles properly isolate environments; trust-all SSL for local/Cloudflare Tunnel makes sense - **API key auth**: Both CI→service (X-API-Key header) and service→Portainer use API key auth — proper zero-trust pattern - **Proper env var preservation**: `DeployService` fetches existing env vars before redeploying, preventing them from being overwritten - **CI workflow**: Proper tag-based versioning (`pa-{version}`), separate from CFTunnels tags --- ### ⚠️ Concerns & Recommendations | # | Severity | Issue | Recommendation | |---|----------|-------|----------------| | 1 | **Medium** | **Generic Exception catch** — `DeployController.java` line 37 catches `Exception e` | Catch specific Portainer client exceptions (`PortainerConnectionException`, `PortainerResourceNotFoundException`, etc.) and map to appropriate HTTP status codes (502, 404) instead of blanket 500 | | 2 | **Low** | **No SLF4J logging** — Controller and Service have no logging | Add `Logger` and log deployment attempts (with stack ID), auth failures, and errors before returning responses — critical for debugging production issues | | 3 | **Low** | **Dockerfile copies unnecessary `cftunnels-service/`** — The portainer-automation module only depends on `:common`, but the Dockerfile copies the entire `cftunnels-service/` tree | Since Gradle resolves the full project tree via `settings.gradle`, removing the copy may cause build failures. Either remove `cftunnels-service` from `settings.gradle` for this Docker build (via a different Gradle settings file) or add a `.dockerignore` to exclude it | | 4 | **Info** | **CI workflow uses `secrets.TOKEN`** — Generic secret name for registry password | Consider renaming to `secrets.REGISTRY_TOKEN` or `secrets.GITEA_REGISTRY_TOKEN` for clarity (but keep consistent with other workflows if they use `TOKEN`) | | 5 | **Info** | **`stack.env` in compose files** — Both compose files reference `stack.env` via `env_file` | This file doesn't exist by default in Portainer stack deployments. Either document that it's optional and can be empty, or remove the `env_file` directive and set all vars via `environment` block | | 6 | **Info** | **No health endpoint** — Acknowledged in docs as missing | Add a simple `GET /api/health` → `{"status":"UP"}` endpoint to enable Portainer health checks and easier debugging | | 7 | **Low** | **Hardcoded Portainer port 9442 in default application.properties** | The default profile targets `192.168.0.100:9442`. In the docs, dev uses `devdocker.hithomelabs.com` (local profile) and prod uses `:9443`. Ensure the default profile is never used accidentally in production | --- ### 🔒 Security Review | Check | Status | Notes | |-------|--------|-------| | Exposed secrets (hardcoded keys/tokens) | ✅ Pass | All keys from env vars/properties | | Input validation | ✅ Pass | API key header validated (null-safe, null returns 401) | | Auth bypass vectors | ✅ Pass | X-API-Key required for all deploy endpoints | | SSL/TLS | ⚠️ Note | Trust-all SSL in `local` profile acceptable (behind Cloudflare Tunnel); `prod` uses standard SSL validation | | Docker supply chain | ✅ Pass | Multi-stage build, no unnecessary packages in final image | --- ### 🧪 Testing Assessment - **Unit tests**: ✅ Good — Controller tests cover auth success/failure/missing and service error propagation; Service tests cover all key code paths - **Integration tests**: ⏳ Not included in this PR — The `integration_test.yaml` workflow tests the main CFTunnels app, not PA. Recommend adding PA-specific integration tests in a follow-up - **Manual verification**: ✅ Documented in PORTAINER_STACK.md --- ### 📋 Implementation Checklist - [x] API endpoint design (`POST /api/deploy/{stackId}`) - [x] Database schema update — N/A (no DB needed) - [x] Unit tests (DeployControllerTest, DeployServiceTest) - [ ] Integration tests (deferred) - [x] Documentation update (PORTAINER_STACK.md, .env.example) - [x] Security review (API key auth, profile isolation) - [ ] Health check endpoint (deferred — acknowledged in docs) --- ### Approval Status **REVIEW COMPLETE — APPROVED ✅** This is a well-crafted PR with clean code, thorough testing, and comprehensive documentation. The concerns noted are non-blocking recommendations for improvement. The implementation correctly follows the Option B architecture (separate Portainer stack) and integrates properly with the existing `common` library and CI infrastructure. **Please address items #1 (specific exception handling), #2 (logging), and consider #6 (health endpoint) before or after merge as time permits.**

Linus added the

labels 2026-07-06 21:57:58 +00:00

Dave commented

2026-07-06 22:16:12 +00:00

@hitanshu — great question. Here's the analysis:

Short answer: Yes, we need this line for the Docker build to succeed.

Why it's required:

settings.gradle declares:

include 'common'
include 'cftunnels-service'
include 'portainer-automation'

Even though portainer-automation/build.gradle only directly depends on :common (not :cftunnels-service), Gradle's configuration phase parses settings.gradle and attempts to resolve all included projects before building the task graph. If the cftunnels-service/ directory is missing from the Docker build context, Gradle will fail during project configuration with an error like:

Project with path ':cftunnels-service' could not be found in root project 'cftunnels'.

The task :portainer-automation:bootJar itself doesn't compile cftunnels-service, but Gradle still requires the project declaration in settings.gradle to be resolvable.

Options to remove the copy:

Option	Trade-off
1. Keep as-is (current state)	Simplest. Slightly larger build context but negligible since Gradle won't actually compile it
2. Use a separate `settings-docker.gradle` that excludes `cftunnels-service`, then use `--settings-file settings-docker.gradle` in the Dockerfile `RUN` command	Avoids the copy entirely, but adds a build settings file to maintain alongside the real one
3. Copy only a stub `cftunnels-service/build.gradle`	More fragile — creates an invisible coupling between the Dockerfile and the subproject structure
4. Add `cftunnels-service/` to `.dockerignore`	❌ Won't work — would still cause Gradle to fail since the directory won't exist

My recommendation: Stick with option 1 (keep the COPY). The cftunnels-service/ source tree adds negligible overhead to a multi-stage Docker build since it's only in the build stage and isn't compiled. If we ever want to optimize, option 2 (separate settings file) is the cleanest approach, but it's a low-priority improvement.

Referencing Linus's review concern #3 as well — agreed this can be deferred.

@hitanshu — great question. Here's the analysis: **Short answer: Yes, we need this line for the Docker build to succeed.** **Why it's required:** `settings.gradle` declares: ```groovy include 'common' include 'cftunnels-service' include 'portainer-automation' ``` Even though `portainer-automation/build.gradle` only directly depends on `:common` (not `:cftunnels-service`), Gradle's **configuration phase** parses `settings.gradle` and attempts to resolve **all** included projects before building the task graph. If the `cftunnels-service/` directory is missing from the Docker build context, Gradle will fail during project configuration with an error like: > `Project with path ':cftunnels-service' could not be found in root project 'cftunnels'.` The task `:portainer-automation:bootJar` itself doesn't compile `cftunnels-service`, but Gradle still requires the project declaration in `settings.gradle` to be resolvable. **Options to remove the copy:** | Option | Trade-off | |--------|-----------| | **1. Keep as-is** (current state) | Simplest. Slightly larger build context but negligible since Gradle won't actually compile it | | **2. Use a separate `settings-docker.gradle`** that excludes `cftunnels-service`, then use `--settings-file settings-docker.gradle` in the Dockerfile `RUN` command | Avoids the copy entirely, but adds a build settings file to maintain alongside the real one | | **3. Copy only a stub `cftunnels-service/build.gradle`** | More fragile — creates an invisible coupling between the Dockerfile and the subproject structure | | **4. Add `cftunnels-service/` to `.dockerignore`** | ❌ Won't work — would still cause Gradle to fail since the directory won't exist | **My recommendation:** Stick with option 1 (keep the `COPY`). The `cftunnels-service/` source tree adds negligible overhead to a multi-stage Docker build since it's only in the `build` stage and isn't compiled. If we ever want to optimize, option 2 (separate settings file) is the cleanest approach, but it's a low-priority improvement. Referencing Linus's review concern #3 as well — agreed this can be deferred.

Dave added 1 commit 2026-07-06 22:19:00 +00:00

Hithomelabs/CFTunnels#99 : address PR review - specific exceptions, SLF4J logging, health endpoint

sample gradle build and test / build (pull_request) Successful in 1m58s

Details

portainer-automation build and push / tag (push) Successful in 7s

Details

sample gradle build and test / tag (push) Successful in 6s

Details

portainer-automation build and push / build_tag_push (push) Successful in 3m9s

Details

sample gradle build and test / build_tag_push (push) Successful in 2m20s

Details

bfed3d220b

Dave commented

2026-07-06 22:19:10 +00:00

Addressed the lead's review recommendations from PR #137:

Specific exception handling — DeployController now catches PortainerConnectionException (→ 502), PortainerResourceNotFoundException (→ 404), PortainerAuthenticationException (→ 401), PortainerDeploymentException (→ 502), and RuntimeException (→ 500 catch-all) instead of blanket Exception.
SLF4J logging — Added Logger to both DeployController and DeployService. Logs deployment attempts/results with stack ID, auth failures, and errors with full stack traces on unexpected exceptions.
Health endpoint — Added GET /api/health → {"status":"UP"} to the controller for Portainer health checks and debugging.
stack.env — No action needed (as decided).

Build passes (./gradlew :portainer-automation:build), all existing tests remain green.

Addressed the lead's review recommendations from PR #137: 1. **Specific exception handling** — `DeployController` now catches `PortainerConnectionException` (→ 502), `PortainerResourceNotFoundException` (→ 404), `PortainerAuthenticationException` (→ 401), `PortainerDeploymentException` (→ 502), and `RuntimeException` (→ 500 catch-all) instead of blanket `Exception`. 2. **SLF4J logging** — Added `Logger` to both `DeployController` and `DeployService`. Logs deployment attempts/results with stack ID, auth failures, and errors with full stack traces on unexpected exceptions. 3. **Health endpoint** — Added `GET /api/health` → `{"status":"UP"}` to the controller for Portainer health checks and debugging. 4. **stack.env** — No action needed (as decided). Build passes (`./gradlew :portainer-automation:build`), all existing tests remain green.

hitanshu merged commit bfed3d220b into test

2026-07-07 05:07:51 +00:00

Polly commented

2026-07-08 19:23:42 +00:00

══════════════════════════════════════════════════════════
PM AGENT COMPLETED ✓
══════════════════════════════════════════════════════════

Addressed the request to extend the workflow beyond test and parameterize it.

Milestone: Portainer Automation — Production & Parameterization — #5

Created 2 user stories in Hithomelabs/CFTunnels:
• #139: Parameterize portainer-automation build workflow for multi-environment reuse — 5pts — priority:high
• #140: Add main-branch deployment workflow for portainer-automation production — 5pts — priority:high

Total Story Points: 10
Priority Distribution: critical[0] high[2] medium[0] low[0]

Critical Path:
#139 → #140 → #136 (wire CI deploy triggers)

Dependencies:
• #139 blocks #140
• #140 blocked by #139

What was created:

#139 — Refactors portainer_automation_build_push.yml into a parameterized workflow (reusable workflow_call or workflow_dispatch with inputs) so both test and prod use the same logic without YAML duplication.
#140 — Adds a main-branch trigger that builds, tags (prod, latest, pa-<version>), and pushes images for production consumption. Depends on #139.

Next: #139 should be implemented first, then #140, then #136 (already open for wiring the deploy trigger).

══════════════════════════════════════════════════════════

══════════════════════════════════════════════════════════ PM AGENT COMPLETED ✓ ══════════════════════════════════════════════════════════ Addressed the request to extend the workflow beyond `test` and parameterize it. Milestone: **Portainer Automation — Production & Parameterization** — #5 Created **2 user stories** in Hithomelabs/CFTunnels: • #139: Parameterize portainer-automation build workflow for multi-environment reuse — **5pts** — **priority:high** • #140: Add main-branch deployment workflow for portainer-automation production — **5pts** — **priority:high** Total Story Points: **10** Priority Distribution: critical[0] high[2] medium[0] low[0] Critical Path: #139 → #140 → #136 (wire CI deploy triggers) Dependencies: • #139 blocks #140 • #140 blocked by #139 **What was created:** 1. **#139** — Refactors `portainer_automation_build_push.yml` into a parameterized workflow (reusable `workflow_call` or `workflow_dispatch` with inputs) so both test and prod use the same logic without YAML duplication. 2. **#140** — Adds a main-branch trigger that builds, tags (`prod`, `latest`, `pa-<version>`), and pushes images for production consumption. Depends on #139. **Next:** #139 should be implemented first, then #140, then #136 (already open for wiring the deploy trigger). ══════════════════════════════════════════════════════════

Sign in to join this conversation.

No reviewers