Hithomelabs/CFTunnels
10
0
Fork 4
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity

[ARCHITECTURE] Multi-Module Gradle Restructure + Portainer Automation Service #124

New Issue
Open
opened 2026-07-05 07:48:29 +00:00 by Dave · 6 comments
Dave commented 2026-07-05 07:48:29 +00:00
Member
Copy Link

Architecture Design Proposal

Context

We're converting CFTunnels from a single-module Gradle project to a multi-module Gradle structure with three modules:

  1. :cftunnels-service — existing Cloudflare Tunnel management API
  2. :portainer-automation — new Spring Boot service for Portainer redeployment
  3. :common — shared utilities (Portainer API client, models, HTTP config)

Existing Infrastructure

Component Details
Build Gradle 8.13, Java 17, Spring Boot 3.4.5
Registry 192.168.0.100:8928 (internal)
Portainer Dev CE 2.30.1 — :9442
Portainer Prod CE 2.30.1 — :9443
CI Runner Custom ci-runner:1.0.0 with Docker socket
CI Workflows 4 files (test_build, test_image_build_push, integration_test, prod_image_tag_promote)

Proposed Module Structure

CFTunnels/
├── settings.gradle                           # include 'common', 'cftunnels-service', 'portainer-automation'
├── build.gradle                              # Root: subprojects { } coordination, all projects plugin
├── gradle.properties                         # Shared Java version, group
├── gradle/wrapper/
├── .gitea/workflows/                         # 4 workflows, updated for deploy triggers
│
├── common/                                   # Plain Java library (NO Spring Boot plugin)
│   ├── build.gradle
│   └── src/main/java/com/hithomelabs/common/
│       ├── portainer/
│       │   ├── model/
│       │   │   ├── PortainerAuthRequest.java
│       │   │   ├── PortainerAuthResponse.java
│       │   │   ├── PortainerStack.java
│       │   │   └── PortainerRedeployRequest.java
│       │   └── client/
│       │       └── PortainerApiClient.java
│       ├── registry/
│       │   └── model/
│       │       └── ImageReference.java
│       └── config/
│           └── RestTemplateConfig.java
│
├── cftunnels-service/                        # Existing app — refactored into module
│   ├── build.gradle                          # io.spring.dependency-management + spring-boot plugin
│   ├── Dockerfile
│   └── src/main/
│       ├── java/com/hithomelabs/cftunnels/
│       │   ├── CfTunnelsApplication.java
│       │   ├── Config/
│       │   │   ├── CloudflareConfig.java
│       │   │   ├── CustomOidcUserConfiguration.java
│       │   │   ├── OpenApiConfig.java
│       │   │   ├── AuthoritiesToGroupMapping.java
│       │   │   └── Security/
│       │   ├── Controllers/
│       │   │   └── TunnelController.java
│       │   ├── Entity/
│       │   │   ├── Mapping.java
│       │   │   ├── Protocol.java
│       │   │   ├── Request.java
│       │   │   ├── Tunnel.java
│       │   │   └── User.java
│       │   ├── Models/
│       │   │   ├── Authorities.java
│       │   │   ├── Config.java
│       │   │   ├── Groups.java
│       │   │   ├── Ingress.java
│       │   │   ├── Result.java
│       │   │   ├── TunnelResponse.java
│       │   │   ├── TunnelResult.java
│       │   │   └── TunnelsResponse.java
│       │   ├── Repositories/
│       │   ├── Services/
│       │   │   ├── CloudflareAPIService.java
│       │   │   └── MappingRequestService.java
│       │   └── Headers/
│       └── resources/
│           ├── application.properties
│           ├── application-ci.properties
│           ├── application-local.properties
│           ├── application-test.properties
│           ├── application-prod.properties
│           ├── application-integration.properties
│           └── schema.sql
│
└── portainer-automation/                    # NEW — lightweight Spring Boot service
    ├── build.gradle                          # spring-boot + depends on :common
    ├── Dockerfile
    └── src/main/
        ├── java/com/hithomelabs/portainer/
        │   ├── PortainerAutomationApplication.java
        │   ├── controller/
        │   │   └── DeployController.java
        │   ├── service/
        │   │   ├── EnvironmentRouter.java
        │   │   └── DeploymentService.java
        │   └── config/
        │       └── PortainerProperties.java
        └── resources/
            ├── application.properties
            └── application-dev.properties
            └── application-prod.properties

Gradle Build Configuration

settings.gradle

rootProject.name = 'cftunnels'
include 'common'
include 'cftunnels-service'
include 'portainer-automation'

Root build.gradle

plugins {
    id 'java'
    id 'org.springframework.boot' version '3.4.5' apply false
    id 'io.spring.dependency-management' version '1.1.7' apply false
}

subprojects {
    apply plugin: 'java'
    apply plugin: 'io.spring.dependency-management'
    
    group = 'com.hithomelabs'
    version = '0.0.1-SNAPSHOT'
    
    java {
        toolchain {
            languageVersion = JavaLanguageVersion.of(17)
        }
    }
    
    repositories {
        mavenCentral()
    }
}

common/build.gradle

dependencies {
    implementation 'org.springframework:spring-web'
    implementation 'com.fasterxml.jackson.core:jackson-databind'
    implementation 'org.apache.httpcomponents.client5:httpclient5'
    
    compileOnly 'org.projectlombok:lombok'
    annotationProcessor 'org.projectlombok:lombok'
    
    testImplementation 'org.junit.jupiter:junit-jupiter'
}

cftunnels-service/build.gradle

apply plugin: 'org.springframework.boot'

dependencies {
    implementation project(':common')
    
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter-oauth2-client'
    implementation 'org.springframework.boot:spring-boot-starter-data-jpa'
    implementation 'org.springframework.boot:spring-boot-starter-security'
    implementation 'org.springdoc:springdoc-openapi-starter-webmvc-ui:2.8.5'
    implementation 'org.hibernate.validator:hibernate-validator'
    
    runtimeOnly 'org.postgresql:postgresql'
    runtimeOnly 'com.h2database:h2'
    
    compileOnly 'org.projectlombok:lombok'
    annotationProcessor 'org.projectlombok:lombok'
    
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
    testImplementation 'org.springframework.security:spring-security-test'
}

portainer-automation/build.gradle

apply plugin: 'org.springframework.boot'

dependencies {
    implementation project(':common')
    
    implementation 'org.springframework.boot:spring-boot-starter-web'
    implementation 'org.springframework.boot:spring-boot-starter-validation'
    
    compileOnly 'org.projectlombok:lombok'
    annotationProcessor 'org.projectlombok:lombok'
    annotationProcessor 'org.springframework.boot:spring-boot-configuration-processor'
    
    testImplementation 'org.springframework.boot:spring-boot-starter-test'
}

Dockerfiles

cftunnels-service/Dockerfile

FROM openjdk:17-jdk as build
WORKDIR /app
COPY gradlew settings.gradle build.gradle ./
COPY gradle ./gradle
COPY common ./common
COPY cftunnels-service ./cftunnels-service
RUN ./gradlew :cftunnels-service:bootJar

FROM openjdk:17-jdk-slim
COPY --from=build /app/cftunnels-service/build/libs/*.jar app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]

portainer-automation/Dockerfile

FROM openjdk:17-jdk as build
WORKDIR /app
COPY gradlew settings.gradle build.gradle ./
COPY gradle ./gradle
COPY common ./common
COPY portainer-automation ./portainer-automation
RUN ./gradlew :portainer-automation:bootJar

FROM openjdk:17-jdk-slim
COPY --from=build /app/portainer-automation/build/libs/*.jar app.jar
ENTRYPOINT ["java", "-jar", "/app.jar"]

Architecture Decision: Deployment Trigger Mechanism

I see three viable options for how the Portainer Automation service receives triggers:

Option A: CI Workflow Calls Portainer Automation API (Recommended)

Flow:

[CI pushes image to registry]
          ↓
[CI step: POST to portainer-automation:8081/api/deploy]
          ↓
[Portainer Automation] → auth → PUT Portainer API → redeploy
          ↓
[Response: success/failure with details]

CI workflow changes:

  • test_image_build_push.yml: After push, add curl step for dev
  • prod_image_tag_promote.yml: After tag/push, add curl step for prod
- name: Trigger Portainer Redeploy
  env:
    PORTAINER_AUTOMATION_URL: "http://portainer-automation:8081"
  run: |
    curl -X POST "$PORTAINER_AUTOMATION_URL/api/deploy" \
      -H "Content-Type: application/json" \
      -H "X-API-Key: ${{ secrets.PORTAINER_AUTOMATION_KEY }}" \
      -d '{
        "environment": "${{ github.ref_name == 'main' && 'prod' || 'dev' }}",
        "stackName": "cftunnels",
        "imageTag": "${{ needs.tag.outputs.new_version }}"
      }'

Pros:

  • ✅ Simplest implementation
  • ✅ Synchronous — CI knows if deploy succeeded/failed
  • ✅ CI already on Docker network, can reach service
  • ✅ Explicit branch→env mapping in CI (no ambiguity)
  • ✅ No extra infrastructure (webhooks, message queues)

Cons:

  • ❌ CI workflows need minor additions
  • ❌ CI runner needs the Portainer Automation API key

Option B: Gitea Webhook to Portainer Automation

Flow:

[CI pushes image → Gitea detects push event]
          ↓
[Gitea sends webhook POST → portainer-automation/webhook/gitea]
          ↓
[Service parses branch from payload → deploys to dev/prod]

Pros:

  • ✅ CI workflows completely unchanged
  • ✅ Service is autonomous and cleanly separated

Cons:

  • ❌ Async — CI has no deployment feedback
  • ❌ Manual Gitea webhook configuration needed
  • ❌ Webhook delivery best-effort (needs retry logic)
  • ❌ Must handle event filtering (ignore non-push events)
  • ❌ Must handle branch filtering (only test/main)

Option C: Hybrid — CI Calls API + Service Also Listens for Webhooks

Pros:

  • ✅ Best of both — CI gets sync feedback, webhook provides redundancy
  • ✅ Graceful fallback if one trigger fails

Cons:

  • ❌ Most complex to implement
  • ❌ Risk of duplicate deploys (need idempotency checking)
  • ❌ Both the CI changes AND webhook config needed

Comparison

Criteria A: CI API Call B: Gitea Webhook C: Hybrid
Implementation effort ⭐ Low ⭐⭐ Medium ⭐⭐⭐ High
CI feedback ✅ Yes ❌ No ✅ Yes
CI workflow changes ✅ Minor ✅ None ✅ Minor
Extra config needed API key secret Gitea webhook URL + secret Both
Decoupling Medium High Medium
Reliability High (sync) Medium (async) Very High
Duplicate risk None None Possible (needed idempotent)

Security Considerations (from Hithomelabs Security Analysis)

Key Security Findings

1. Portainer CE Token Problem (CRITICAL)

Portainer CE has NO fine-grained RBAC. A single API key = full admin.

  • Can modify ANY stack, not just targeted ones
  • Token in automation = admin-equivalent credential
  • Mitigation: Separate Portainer users for dev vs prod — create svc-portainer-dev and svc-portainer-prod users with their own API tokens

2. Dual-Key Architecture

Key Stored In Risk if Leaked
Portainer API Key Automation container env Full stack admin access
Automation API Key Gitea secrets + config Can trigger deploys to env

3. Network Segmentation

All traffic must stay on internal Docker networks. Automation service MUST NOT be exposed via Cloudflare Tunnel.

4. StackFileContent Round-Trip (File-Based Stacks)

File-based stacks require: GET file → PUT content back with pullImage:true. Compose content passes through automation in memory. Recommendation: Convert CFTunnels stacks to git-based stacks to use the simpler /git/redeploy endpoint which doesn't require content round-trip.

5. CI Runner Privilege

Runner already has Docker socket. If compromised, attacker has Docker root. Automation API key is a secondary concern.

Additional Recommendations

Must-Do:

  1. Separate Portainer users for dev + prod tokens (svc-portainer-dev, svc-portainer-prod)
  2. Internal network only - no tunnel exposure for automation service
  3. Auth on /api/deploy with X-API-Key header
  4. Log all deployment requests

Should-Do:
5. HashiCorp Vault integration for Portainer key storage/rotation
6. Convert stacks from file-based to git-based (to use simpler /git/redeploy endpoint)
7. Different keys for CI auth vs Portainer auth (dual-key architecture)

Nice-to-Have:
8. Rate limiting on /api/deploy
9. HMAC signing for CI requests
10. Evaluate Portainer Business (free up to 5 nodes)

How CI Workflows Wire Up (Based on Option A)

test_image_build_push.yml (push to test branch)

Current: build → push :test + :{version}
Proposed: build → push :test + :{version} → POST /api/deploy { env: "dev" }

prod_image_tag_promote.yml (push to main branch)

Current: retag :test → :prod + :{version} → push
Proposed: retag → push → POST /api/deploy { env: "prod" }

test_build.yml (PR to test) — No change

integration_test.yaml (push to main) — No change

Portainer Automation Service Design

Endpoints

Method Path Description
POST /api/deploy Trigger stack redeploy
GET /api/health Health check

POST /api/deploy Request

{
  "environment": "dev",           // "dev" or "prod"
  "stackName": "cftunnels",      // Portainer stack name
  "imageTag": "1.2.3",           // Optional: specific image tag to use
  "forcePull": true               // Always pull latest image
}

POST /api/deploy Response (200)

{
  "success": true,
  "environment": "dev",
  "stackName": "cftunnels",
  "previousStatus": "running",
  "newStatus": "updating",
  "message": "Stack redeploy triggered successfully"
}

Portainer API Interaction

The service will interact with Portainer CE 2.30.1 via:

  1. Authenticate: POST /api/auth → get JWT
  2. Find Stack by name: GET /api/stacks → filter by Name field
  3. Redeploy: PUT /api/stacks/{id}?endpointId={eid} with {"pullImage":true, "prune":true}

Portainer auth will use API access tokens (recommended over username/password JWT).

Configuration (application.properties)

portainer.dev.url=https://192.168.0.100:9442
portainer.dev.api-key=${PORTAINER_DEV_API_KEY}
portainer.dev.endpoint-id=1

portainer.prod.url=https://192.168.0.100:9443
portainer.prod.api-key=${PORTAINER_PROD_API_KEY}
portainer.prod.endpoint-id=1

# Mapping of stack names to Portainer stack IDs
portainer.stacks.cftunnels.dev-id=3
portainer.stacks.cftunnels.prod-id=5

# Service auth
portainer.automation.api-key=${PORTAINER_AUTOMATION_API_KEY}

Shared Components (:common module)

Component Description
PortainerAuthRequest Username/password or API key auth request DTO
PortainerAuthResponse JWT token response DTO
PortainerStack Stack metadata DTO (id, name, endpointId, status)
PortainerRedeployRequest Redeploy body DTO (pullImage, prune)
PortainerApiClient Reusable HTTP client: authenticate(), findStack(name), redeploy(stackId, endpointId)
ImageReference Parses 192.168.0.100:8928/hithomelabs/cftunnels:1.2.3 into components
RestTemplateConfig Shared RestTemplate bean (already exists in cftunnels, will be promoted to common)

The RestTemplateConfig already exists in the current codebase at com.hithomelabs.CFTunnels.Config.RestTemplateConfig. This is a natural candidate to move to :common so both services can use it.

Questions for Decision

@hitanshu Please answer:

  1. Which trigger mechanism do you prefer?

    • A) CI workflow calls Portainer Automation API (recommended)
    • B) Gitea webhook to Portainer Automation
    • C) Hybrid (both)

  2. Portainer auth method?

    • JWT via username/password (POST /api/auth)
    • API access tokens (recommended by Portainer docs)

  3. Package naming:

    • Rename to com.hithomelabs.cftunnels (standard Java convention)
    • Keep com.hithomelabs.CFTunnels (backward compat)

  4. Portainer Automation deployment:

    • Add as a new service in the existing docker-compose.yaml
    • Separate docker-compose.portainer.yaml

  5. How should Portainer Automation authenticate callers (the CI)?

    • Static API key via header (X-API-Key)
    • Shared secret signed HMAC
    • No auth (internal network only — simplest but least secure)

  6. Separate Portainer users? Should we create svc-portainer-dev and svc-portainer-prod users with their own API tokens?

  7. Git-based stacks? Are you willing to convert CFTunnels stacks from file-based to git-based to use the simpler /git/redeploy endpoint?

  8. Vault integration? Fetch Portainer keys from HashiCorp Vault at startup, or use plain env vars for now?

  9. Internal-only? Confirm automation service stays off Cloudflare Tunnel?

  10. Stack lookup method? Map stack names to IDs in config (as proposed), or look up by name at runtime via Portainer API?

  11. Notification method? Service logs only, or also post deployment status to Gitea issue comments?

## Architecture Design Proposal ### Context We're converting CFTunnels from a single-module Gradle project to a multi-module Gradle structure with three modules: 1. `:cftunnels-service` — existing Cloudflare Tunnel management API 2. `:portainer-automation` — new Spring Boot service for Portainer redeployment 3. `:common` — shared utilities (Portainer API client, models, HTTP config) ### Existing Infrastructure | Component | Details | |-----------|---------| | Build | Gradle 8.13, Java 17, Spring Boot 3.4.5 | | Registry | `192.168.0.100:8928` (internal) | | Portainer Dev | CE 2.30.1 — `:9442` | | Portainer Prod | CE 2.30.1 — `:9443` | | CI Runner | Custom `ci-runner:1.0.0` with Docker socket | | CI Workflows | 4 files (test_build, test_image_build_push, integration_test, prod_image_tag_promote) | --- ### Proposed Module Structure ``` CFTunnels/ ├── settings.gradle # include 'common', 'cftunnels-service', 'portainer-automation' ├── build.gradle # Root: subprojects { } coordination, all projects plugin ├── gradle.properties # Shared Java version, group ├── gradle/wrapper/ ├── .gitea/workflows/ # 4 workflows, updated for deploy triggers │ ├── common/ # Plain Java library (NO Spring Boot plugin) │ ├── build.gradle │ └── src/main/java/com/hithomelabs/common/ │ ├── portainer/ │ │ ├── model/ │ │ │ ├── PortainerAuthRequest.java │ │ │ ├── PortainerAuthResponse.java │ │ │ ├── PortainerStack.java │ │ │ └── PortainerRedeployRequest.java │ │ └── client/ │ │ └── PortainerApiClient.java │ ├── registry/ │ │ └── model/ │ │ └── ImageReference.java │ └── config/ │ └── RestTemplateConfig.java │ ├── cftunnels-service/ # Existing app — refactored into module │ ├── build.gradle # io.spring.dependency-management + spring-boot plugin │ ├── Dockerfile │ └── src/main/ │ ├── java/com/hithomelabs/cftunnels/ │ │ ├── CfTunnelsApplication.java │ │ ├── Config/ │ │ │ ├── CloudflareConfig.java │ │ │ ├── CustomOidcUserConfiguration.java │ │ │ ├── OpenApiConfig.java │ │ │ ├── AuthoritiesToGroupMapping.java │ │ │ └── Security/ │ │ ├── Controllers/ │ │ │ └── TunnelController.java │ │ ├── Entity/ │ │ │ ├── Mapping.java │ │ │ ├── Protocol.java │ │ │ ├── Request.java │ │ │ ├── Tunnel.java │ │ │ └── User.java │ │ ├── Models/ │ │ │ ├── Authorities.java │ │ │ ├── Config.java │ │ │ ├── Groups.java │ │ │ ├── Ingress.java │ │ │ ├── Result.java │ │ │ ├── TunnelResponse.java │ │ │ ├── TunnelResult.java │ │ │ └── TunnelsResponse.java │ │ ├── Repositories/ │ │ ├── Services/ │ │ │ ├── CloudflareAPIService.java │ │ │ └── MappingRequestService.java │ │ └── Headers/ │ └── resources/ │ ├── application.properties │ ├── application-ci.properties │ ├── application-local.properties │ ├── application-test.properties │ ├── application-prod.properties │ ├── application-integration.properties │ └── schema.sql │ └── portainer-automation/ # NEW — lightweight Spring Boot service ├── build.gradle # spring-boot + depends on :common ├── Dockerfile └── src/main/ ├── java/com/hithomelabs/portainer/ │ ├── PortainerAutomationApplication.java │ ├── controller/ │ │ └── DeployController.java │ ├── service/ │ │ ├── EnvironmentRouter.java │ │ └── DeploymentService.java │ └── config/ │ └── PortainerProperties.java └── resources/ ├── application.properties └── application-dev.properties └── application-prod.properties ``` --- ### Gradle Build Configuration #### `settings.gradle` ```groovy rootProject.name = 'cftunnels' include 'common' include 'cftunnels-service' include 'portainer-automation' ``` #### Root `build.gradle` ```groovy plugins { id 'java' id 'org.springframework.boot' version '3.4.5' apply false id 'io.spring.dependency-management' version '1.1.7' apply false } subprojects { apply plugin: 'java' apply plugin: 'io.spring.dependency-management' group = 'com.hithomelabs' version = '0.0.1-SNAPSHOT' java { toolchain { languageVersion = JavaLanguageVersion.of(17) } } repositories { mavenCentral() } } ``` #### `common/build.gradle` ```groovy dependencies { implementation 'org.springframework:spring-web' implementation 'com.fasterxml.jackson.core:jackson-databind' implementation 'org.apache.httpcomponents.client5:httpclient5' compileOnly 'org.projectlombok:lombok' annotationProcessor 'org.projectlombok:lombok' testImplementation 'org.junit.jupiter:junit-jupiter' } ``` #### `cftunnels-service/build.gradle` ```groovy apply plugin: 'org.springframework.boot' dependencies { implementation project(':common') implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.boot:spring-boot-starter-oauth2-client' implementation 'org.springframework.boot:spring-boot-starter-data-jpa' implementation 'org.springframework.boot:spring-boot-starter-security' implementation 'org.springdoc:springdoc-openapi-starter-webmvc-ui:2.8.5' implementation 'org.hibernate.validator:hibernate-validator' runtimeOnly 'org.postgresql:postgresql' runtimeOnly 'com.h2database:h2' compileOnly 'org.projectlombok:lombok' annotationProcessor 'org.projectlombok:lombok' testImplementation 'org.springframework.boot:spring-boot-starter-test' testImplementation 'org.springframework.security:spring-security-test' } ``` #### `portainer-automation/build.gradle` ```groovy apply plugin: 'org.springframework.boot' dependencies { implementation project(':common') implementation 'org.springframework.boot:spring-boot-starter-web' implementation 'org.springframework.boot:spring-boot-starter-validation' compileOnly 'org.projectlombok:lombok' annotationProcessor 'org.projectlombok:lombok' annotationProcessor 'org.springframework.boot:spring-boot-configuration-processor' testImplementation 'org.springframework.boot:spring-boot-starter-test' } ``` --- ### Dockerfiles #### `cftunnels-service/Dockerfile` ```dockerfile FROM openjdk:17-jdk as build WORKDIR /app COPY gradlew settings.gradle build.gradle ./ COPY gradle ./gradle COPY common ./common COPY cftunnels-service ./cftunnels-service RUN ./gradlew :cftunnels-service:bootJar FROM openjdk:17-jdk-slim COPY --from=build /app/cftunnels-service/build/libs/*.jar app.jar ENTRYPOINT ["java", "-jar", "/app.jar"] ``` #### `portainer-automation/Dockerfile` ```dockerfile FROM openjdk:17-jdk as build WORKDIR /app COPY gradlew settings.gradle build.gradle ./ COPY gradle ./gradle COPY common ./common COPY portainer-automation ./portainer-automation RUN ./gradlew :portainer-automation:bootJar FROM openjdk:17-jdk-slim COPY --from=build /app/portainer-automation/build/libs/*.jar app.jar ENTRYPOINT ["java", "-jar", "/app.jar"] ``` --- ### Architecture Decision: Deployment Trigger Mechanism I see three viable options for how the Portainer Automation service receives triggers: --- #### Option A: CI Workflow Calls Portainer Automation API (Recommended) **Flow:** ``` [CI pushes image to registry] ↓ [CI step: POST to portainer-automation:8081/api/deploy] ↓ [Portainer Automation] → auth → PUT Portainer API → redeploy ↓ [Response: success/failure with details] ``` **CI workflow changes:** - `test_image_build_push.yml`: After push, add curl step for **dev** - `prod_image_tag_promote.yml`: After tag/push, add curl step for **prod** ```yaml - name: Trigger Portainer Redeploy env: PORTAINER_AUTOMATION_URL: "http://portainer-automation:8081" run: | curl -X POST "$PORTAINER_AUTOMATION_URL/api/deploy" \ -H "Content-Type: application/json" \ -H "X-API-Key: ${{ secrets.PORTAINER_AUTOMATION_KEY }}" \ -d '{ "environment": "${{ github.ref_name == 'main' && 'prod' || 'dev' }}", "stackName": "cftunnels", "imageTag": "${{ needs.tag.outputs.new_version }}" }' ``` **Pros:** - ✅ Simplest implementation - ✅ Synchronous — CI knows if deploy succeeded/failed - ✅ CI already on Docker network, can reach service - ✅ Explicit branch→env mapping in CI (no ambiguity) - ✅ No extra infrastructure (webhooks, message queues) **Cons:** - ❌ CI workflows need minor additions - ❌ CI runner needs the Portainer Automation API key --- #### Option B: Gitea Webhook to Portainer Automation **Flow:** ``` [CI pushes image → Gitea detects push event] ↓ [Gitea sends webhook POST → portainer-automation/webhook/gitea] ↓ [Service parses branch from payload → deploys to dev/prod] ``` **Pros:** - ✅ CI workflows completely unchanged - ✅ Service is autonomous and cleanly separated **Cons:** - ❌ Async — CI has no deployment feedback - ❌ Manual Gitea webhook configuration needed - ❌ Webhook delivery best-effort (needs retry logic) - ❌ Must handle event filtering (ignore non-push events) - ❌ Must handle branch filtering (only test/main) --- #### Option C: Hybrid — CI Calls API + Service Also Listens for Webhooks **Pros:** - ✅ Best of both — CI gets sync feedback, webhook provides redundancy - ✅ Graceful fallback if one trigger fails **Cons:** - ❌ Most complex to implement - ❌ Risk of duplicate deploys (need idempotency checking) - ❌ Both the CI changes AND webhook config needed --- ### Comparison | Criteria | A: CI API Call | B: Gitea Webhook | C: Hybrid | |----------|:---:|:---:|:---:| | Implementation effort | ⭐ Low | ⭐⭐ Medium | ⭐⭐⭐ High | | CI feedback | ✅ Yes | ❌ No | ✅ Yes | | CI workflow changes | ✅ Minor | ✅ None | ✅ Minor | | Extra config needed | API key secret | Gitea webhook URL + secret | Both | | Decoupling | Medium | High | Medium | | Reliability | High (sync) | Medium (async) | Very High | | Duplicate risk | None | None | Possible (needed idempotent) | --- ## Security Considerations (from Hithomelabs Security Analysis) ### Key Security Findings #### 1. Portainer CE Token Problem (CRITICAL) Portainer CE has NO fine-grained RBAC. A single API key = full admin. - Can modify ANY stack, not just targeted ones - Token in automation = admin-equivalent credential - **Mitigation**: Separate Portainer users for dev vs prod — create `svc-portainer-dev` and `svc-portainer-prod` users with their own API tokens #### 2. Dual-Key Architecture | Key | Stored In | Risk if Leaked | |-----|-----------|---------------| | Portainer API Key | Automation container env | Full stack admin access | | Automation API Key | Gitea secrets + config | Can trigger deploys to env | #### 3. Network Segmentation All traffic must stay on internal Docker networks. Automation service MUST NOT be exposed via Cloudflare Tunnel. #### 4. StackFileContent Round-Trip (File-Based Stacks) File-based stacks require: GET file → PUT content back with pullImage:true. Compose content passes through automation in memory. **Recommendation**: Convert CFTunnels stacks to git-based stacks to use the simpler `/git/redeploy` endpoint which doesn't require content round-trip. #### 5. CI Runner Privilege Runner already has Docker socket. If compromised, attacker has Docker root. Automation API key is a secondary concern. ### Additional Recommendations **Must-Do:** 1. Separate Portainer users for dev + prod tokens (svc-portainer-dev, svc-portainer-prod) 2. Internal network only - no tunnel exposure for automation service 3. Auth on /api/deploy with X-API-Key header 4. Log all deployment requests **Should-Do:** 5. HashiCorp Vault integration for Portainer key storage/rotation 6. Convert stacks from file-based to git-based (to use simpler /git/redeploy endpoint) 7. Different keys for CI auth vs Portainer auth (dual-key architecture) **Nice-to-Have:** 8. Rate limiting on /api/deploy 9. HMAC signing for CI requests 10. Evaluate Portainer Business (free up to 5 nodes) --- ### How CI Workflows Wire Up (Based on Option A) #### `test_image_build_push.yml` (push to `test` branch) ``` Current: build → push :test + :{version} Proposed: build → push :test + :{version} → POST /api/deploy { env: "dev" } ``` #### `prod_image_tag_promote.yml` (push to `main` branch) ``` Current: retag :test → :prod + :{version} → push Proposed: retag → push → POST /api/deploy { env: "prod" } ``` #### `test_build.yml` (PR to `test`) — **No change** #### `integration_test.yaml` (push to `main`) — **No change** --- ### Portainer Automation Service Design #### Endpoints | Method | Path | Description | |--------|------|-------------| | POST | `/api/deploy` | Trigger stack redeploy | | GET | `/api/health` | Health check | #### POST `/api/deploy` Request ```json { "environment": "dev", // "dev" or "prod" "stackName": "cftunnels", // Portainer stack name "imageTag": "1.2.3", // Optional: specific image tag to use "forcePull": true // Always pull latest image } ``` #### POST `/api/deploy` Response (200) ```json { "success": true, "environment": "dev", "stackName": "cftunnels", "previousStatus": "running", "newStatus": "updating", "message": "Stack redeploy triggered successfully" } ``` #### Portainer API Interaction The service will interact with Portainer CE 2.30.1 via: 1. **Authenticate**: `POST /api/auth` → get JWT 2. **Find Stack by name**: `GET /api/stacks` → filter by Name field 3. **Redeploy**: `PUT /api/stacks/{id}?endpointId={eid}` with `{"pullImage":true, "prune":true}` Portainer auth will use **API access tokens** (recommended over username/password JWT). #### Configuration (`application.properties`) ```properties portainer.dev.url=https://192.168.0.100:9442 portainer.dev.api-key=${PORTAINER_DEV_API_KEY} portainer.dev.endpoint-id=1 portainer.prod.url=https://192.168.0.100:9443 portainer.prod.api-key=${PORTAINER_PROD_API_KEY} portainer.prod.endpoint-id=1 # Mapping of stack names to Portainer stack IDs portainer.stacks.cftunnels.dev-id=3 portainer.stacks.cftunnels.prod-id=5 # Service auth portainer.automation.api-key=${PORTAINER_AUTOMATION_API_KEY} ``` --- ### Shared Components (`:common` module) | Component | Description | |-----------|-------------| | `PortainerAuthRequest` | Username/password or API key auth request DTO | | `PortainerAuthResponse` | JWT token response DTO | | `PortainerStack` | Stack metadata DTO (id, name, endpointId, status) | | `PortainerRedeployRequest` | Redeploy body DTO (pullImage, prune) | | `PortainerApiClient` | Reusable HTTP client: `authenticate()`, `findStack(name)`, `redeploy(stackId, endpointId)` | | `ImageReference` | Parses `192.168.0.100:8928/hithomelabs/cftunnels:1.2.3` into components | | `RestTemplateConfig` | Shared RestTemplate bean (already exists in cftunnels, will be promoted to common) | The `RestTemplateConfig` already exists in the current codebase at `com.hithomelabs.CFTunnels.Config.RestTemplateConfig`. This is a natural candidate to move to `:common` so both services can use it. --- ### Questions for Decision **@hitanshu Please answer:** 1. **Which trigger mechanism do you prefer?** - **A)** CI workflow calls Portainer Automation API (recommended) - **B)** Gitea webhook to Portainer Automation - **C)** Hybrid (both) 2. **Portainer auth method?** - **JWT** via username/password (`POST /api/auth`) - **API access tokens** (recommended by Portainer docs) 3. **Package naming:** - Rename to `com.hithomelabs.cftunnels` (standard Java convention) - Keep `com.hithomelabs.CFTunnels` (backward compat) 4. **Portainer Automation deployment:** - Add as a new service in the existing `docker-compose.yaml` - Separate `docker-compose.portainer.yaml` 5. **How should Portainer Automation authenticate callers (the CI)?** - Static API key via header (`X-API-Key`) - Shared secret signed HMAC - No auth (internal network only — simplest but least secure) 6. **Separate Portainer users?** Should we create `svc-portainer-dev` and `svc-portainer-prod` users with their own API tokens? 7. **Git-based stacks?** Are you willing to convert CFTunnels stacks from file-based to git-based to use the simpler `/git/redeploy` endpoint? 8. **Vault integration?** Fetch Portainer keys from HashiCorp Vault at startup, or use plain env vars for now? 9. **Internal-only?** Confirm automation service stays off Cloudflare Tunnel? 10. **Stack lookup method?** Map stack names to IDs in config (as proposed), or look up by name at runtime via Portainer API? 11. **Notification method?** Service logs only, or also post deployment status to Gitea issue comments?
Archie added the
epic/development
needs-decision
 labels 2026-07-05 07:49:15 +00:00
hitanshu commented 2026-07-05 09:39:11 +00:00
Owner
Copy Link
  1. A -> CI workflow calls Portainer Automation API (recommended)
  2. Portainer auth method? -> @Polly can you run me thrugh how we can get this done ?
  3. Follow java convention
  4. Add anew service in the existing docker-compose.yaml
  5. Static API key via header (X-API-Key) idf possible
  6. Separate Portainer users? Should we create svc-portainer-dev and svc-portainer-prod users with their own API tokens -> sure if we cannot do 2.
  7. Yes, yes, yes, we need got based stacks, It was a git based stack, I don't know how it went to an editor based config.
  8. Not right now, we can have secrets in gitea runners for now.
  9. Yes let us keep this internal only.
  10. Let the CI pass the stack id to the service at runtime, this should make it extendable to future projects.
  11. Let's just keep service logs only for now !
1. A -> CI workflow calls Portainer Automation API (recommended) 2. Portainer auth method? -> @Polly can you run me thrugh how we can get this done ? 3. Follow java convention 4. Add anew service in the existing docker-compose.yaml 5. Static API key via header (X-API-Key) idf possible 6. Separate Portainer users? Should we create svc-portainer-dev and svc-portainer-prod users with their own API tokens -> sure if we cannot do 2. 7. Yes, yes, yes, we need got based stacks, It was a git based stack, I don't know how it went to an editor based config. 8. Not right now, we can have secrets in gitea runners for now. 9. Yes let us keep this internal only. 10. Let the CI pass the stack id to the service at runtime, this should make it extendable to future projects. 11. Let's just keep service logs only for now !
Archie commented 2026-07-05 09:40:54 +00:00
Member
Copy Link

Decisions Confirmed ✅

Decisions from @hitanshu on 2026-07-05:

# Question Decision
1 Trigger mechanism A — CI workflow calls Portainer Automation API (sync)
2 Portainer auth method TBD — @Polly to propose how to get this done
3 Package naming Java convention (com.hithomelabs.cftunnels)
4 Deployment model Add to existing docker-compose.yaml
5 CI auth Static API key via X-API-Key header
6 Separate Portainer users Yes (svc-portainer-dev, svc-portainer-prod) — if option 2 can't be done
7 Git-based stacks YES — convert back to git-based (was originally git, became editor-based somehow)
8 Vault integration No — Gitea runner secrets for now
9 Internal-only Yes — no Cloudflare Tunnel exposure
10 Stack lookup CI passes stack ID at runtime (makes it extendable)
11 Notifications Service logs only

CRITICAL CONSTRAINT

All issues in this epic must NOT break existing CFTunnels integration/unit tests. Specifically:

  1. test_build.yml (PR to test) — Gradle build & unit tests must continue to pass
  2. integration_test.yaml (push/schedule on main) — Cloudflare API integration tests must continue to pass
  3. test_image_build_push.yml (push to test) — Docker image build & push must continue to work
  4. prod_image_tag_promote.yml (push to main) — Image tag promotion must continue to work

The multi-module restructure must be done incrementally without breaking existing builds. The test_build.yml and integration_test.yaml workflows must not be modified (they don't need Portainer integration). Only test_image_build_push.yml and prod_image_tag_promote.yml get deploy steps added.

Next

Pending @Polly's input on Portainer auth (question 2), then this issue can move to implementation planning.

## Decisions Confirmed ✅ Decisions from @hitanshu on 2026-07-05: | # | Question | Decision | |---|----------|----------| | 1 | Trigger mechanism | **A** — CI workflow calls Portainer Automation API (sync) | | 2 | Portainer auth method | **TBD** — @Polly to propose how to get this done | | 3 | Package naming | **Java convention** (`com.hithomelabs.cftunnels`) | | 4 | Deployment model | **Add to existing `docker-compose.yaml`** | | 5 | CI auth | **Static API key via `X-API-Key` header** | | 6 | Separate Portainer users | **Yes** (svc-portainer-dev, svc-portainer-prod) — if option 2 can't be done | | 7 | Git-based stacks | **YES** — convert back to git-based (was originally git, became editor-based somehow) | | 8 | Vault integration | **No** — Gitea runner secrets for now | | 9 | Internal-only | **Yes** — no Cloudflare Tunnel exposure | | 10 | Stack lookup | **CI passes stack ID at runtime** (makes it extendable) | | 11 | Notifications | **Service logs only** | ## CRITICAL CONSTRAINT All issues in this epic **must NOT break existing CFTunnels integration/unit tests**. Specifically: 1. **`test_build.yml`** (PR to `test`) — Gradle build & unit tests must continue to pass 2. **`integration_test.yaml`** (push/schedule on `main`) — Cloudflare API integration tests must continue to pass 3. **`test_image_build_push.yml`** (push to `test`) — Docker image build & push must continue to work 4. **`prod_image_tag_promote.yml`** (push to `main`) — Image tag promotion must continue to work The multi-module restructure must be done incrementally without breaking existing builds. The `test_build.yml` and `integration_test.yaml` workflows must **not** be modified (they don't need Portainer integration). Only `test_image_build_push.yml` and `prod_image_tag_promote.yml` get deploy steps added. ## Next Pending @Polly's input on Portainer auth (question 2), then this issue can move to implementation planning.
Polly commented 2026-07-05 09:41:48 +00:00
Member
Copy Link

🎯 Portainer Auth Analysis — Question 2 Answered

Hey @hitanshu, here's the full breakdown on Portainer auth for the portainer-automation service.

Option-by-Option Evaluation

✅ Option 1: API Access Tokens (RECOMMENDED)

Aspect Detail
How it works Create tokens in Portainer UI → Store as Gitea secrets → Use X-API-Key header
Token creation Portainer UI only (see below for partial API support)
Expiry Never expires (can revoke manually)
Code required ~0 lines of auth code. Just HTTP headers.
Token scope Full admin (CE has no RBAC)
Rotate Revoke + create new, update Gitea secret

Spring Boot usage:

// No auth step needed. Just add the header.
HttpHeaders headers = new HttpHeaders();
headers.set("X-API-Key", portainerApiKey);

// Then call Portainer API directly.
// No token refresh logic, no 401 retry loop.

The service in practice:

  1. Startup: read PORTAINER_DEV_API_KEY / PORTAINER_PROD_API_KEY from env
  2. On deploy request: pick the right key based on environment field
  3. Call PUT /api/stacks/{id}?endpointId={eid} with X-API-Key header
  4. Done. No auth flow whatsoever.

Pros:

  • ✅ Zero auth lifecycle code
  • ✅ No 5-min JWT expiry to work around
  • ✅ Portainer's own recommended approach (docs.portainer.io)
  • ✅ Simple to test — just curl -H "X-API-Key: ptk_..."

Cons:

  • ❌ Must create tokens via Portainer UI (one-time setup)
  • ❌ CE RBAC limitation applies — token = admin-level access

⚠️ Option 2: JWT via Username/Password

Aspect Detail
How it works POST /api/auth { username, password } → get JWT → use Authorization: Bearer <jwt>
Token expiry 5 minutes by default in Portainer CE
Code required Auth client + retry logic with re-auth on 401
Lifecycle Must re-authenticate every ~4 min or handle 401 → retry

The implementation overhead:

// You need a token management layer:
public class PortainerAuthManager {
    private String jwt;
    private Instant expiresAt;

    public synchronized String getToken() {
        if (jwt == null || Instant.now().isAfter(expiresAt)) {
            jwt = authenticate(); // POST /api/auth
            expiresAt = Instant.now().plus(4, MINUTES); // buffer
        }
        return jwt;
    }
}

Plus an HTTP interceptor to detect 401 → refresh → retry.

Pros:

  • ✅ No UI setup needed (use existing admin credentials)
  • ✅ Can be fully automated from scratch

Cons:

  • ❌ More complex code — auth manager, retry interceptor, refresh logic
  • ❌ Same security profile — still full admin access in CE
  • ❌ Username/password in env vars is worse security posture than API tokens
  • ❌ 5-min expiry means you will hit a 401 at some point in production

🔶 Option 3: Separate Portainer Users

Aspect Detail
How it works Create svc-portainer-dev + svc-portainer-prod users → generate tokens for each
RBAC in CE Doesn't exist — even a limited user can see/manage all stacks
Value Logging/audit trail only (different user in audit logs)
Overhead User management + password rotation + token management

The hard truth about Portainer CE RBAC:

Portainer CE 2.30.1 does not support role-based access control. All authenticated users (except the special internal user) have full visibility into all environments and stacks. A token from svc-portainer-dev can delete prod stacks just as easily.

So separate users give you no security boundary in CE. They only help with:

  • Audit logs (see who triggered the deploy)
  • Independent token rotation (revoke dev token without affecting prod)

🏆 My Recommendation: Option 1 + Separate Users (for audit)

The winning combo:

  1. Create two Portainer users: svc-portainer-dev and svc-portainer-prod
  2. Generate an API access token for each (Portainer UI → My access tokens)
  3. Store in Gitea org secrets:
    • PORTAINER_DEV_API_KEY → ptk_xxxx_dev_xxxx
    • PORTAINER_PROD_API_KEY → ptk_xxxx_prod_xxxx
  4. The automation service reads the right key based on environment
  5. Service uses X-API-Key header — no auth code needed

Why not just one shared token?

  • Separating by user gives you audit trails
  • You can revoke dev token independently (e.g., if leaked during testing)
  • Cleaner rotation strategy

Why not pure JWT?

  • Extra code complexity buys you nothing
  • API tokens are literally just string headers — simplest possible integration
  • JWT expiry is a liability for an automation service that needs to be reliable

🤖 Can You Create API Tokens Programmatically?

Short answer: Partially yes, but practically no in CE.

Portainer does expose token management endpoints in the API:

Method Endpoint Purpose
POST /api/users/{id}/tokens Create a token for a user
GET /api/users/{id}/tokens List tokens for a user
DELETE /api/users/{id}/tokens/{tokenId} Revoke a token

BUT the catch:

  • These endpoints require a JWT from an existing authenticated session (admin login)
  • The initial token must be created via the UI (chicken-and-egg problem)
  • After you have one token, you could create/rotate others via API

Practical workflow:

1. [UI] Login as admin → create token for svc-portainer-dev → save the key
2. [UI] Login as admin → create token for svc-portainer-prod → save the key
3. [Code] Use the keys forever (rotate via UI when needed)

For automation rotation, you could later build a script that:

  1. Uses an admin API token (saved securely)
  2. Calls POST /api/users/{svcUserId}/tokens to generate new tokens
  3. Updates Gitea secrets via their API
  4. Revokes old tokens

But for first launch, it's UI-only setup.

⚡ Implementation Impact on portainer-automation

With Option 1, the PortainerApiClient in :common becomes dead simple:

// RestTemplate call — no auth manager needed
public String findStackByName(String portainerUrl, String apiKey, String name) {
    HttpHeaders headers = new HttpHeaders();
    headers.set("X-API-Key", apiKey);
    
    ResponseEntity<List<PortainerStack>> response = restTemplate.exchange(
        portainerUrl + "/api/stacks",
        HttpMethod.GET,
        new HttpEntity<>(headers),
        new ParameterizedTypeReference<>() {}
    );
    
    return response.getBody().stream()
        .filter(s -> s.getName().equals(name))
        .findFirst()
        .orElseThrow(() -> new StackNotFoundException(name))
        .getId();
}

No:

  • ❌ Auth manager bean
  • ❌ Token refresh scheduler
  • ❌ 401 retry interceptor
  • ❌ JWT parsing or expiry checking

Just:

  • ✅ Read API key from PortainerProperties
  • ✅ Attach X-API-Key header
  • ✅ Make the call

Let me know if you want me to proceed with this plan or if you have any questions!

## 🎯 Portainer Auth Analysis — Question 2 Answered Hey @hitanshu, here's the full breakdown on Portainer auth for the `portainer-automation` service. --- ### Option-by-Option Evaluation #### ✅ **Option 1: API Access Tokens (RECOMMENDED)** | Aspect | Detail | |--------|--------| | **How it works** | Create tokens in Portainer UI → Store as Gitea secrets → Use `X-API-Key` header | | **Token creation** | Portainer UI only (*see below for partial API support*) | | **Expiry** | Never expires (can revoke manually) | | **Code required** | ~0 lines of auth code. Just HTTP headers. | | **Token scope** | Full admin (CE has no RBAC) | | **Rotate** | Revoke + create new, update Gitea secret | **Spring Boot usage:** ```java // No auth step needed. Just add the header. HttpHeaders headers = new HttpHeaders(); headers.set("X-API-Key", portainerApiKey); // Then call Portainer API directly. // No token refresh logic, no 401 retry loop. ``` **The service in practice:** 1. Startup: read `PORTAINER_DEV_API_KEY` / `PORTAINER_PROD_API_KEY` from env 2. On deploy request: pick the right key based on `environment` field 3. Call `PUT /api/stacks/{id}?endpointId={eid}` with `X-API-Key` header 4. Done. No auth flow whatsoever. **Pros:** - ✅ Zero auth lifecycle code - ✅ No 5-min JWT expiry to work around - ✅ Portainer's own recommended approach (docs.portainer.io) - ✅ Simple to test — just `curl -H "X-API-Key: ptk_..."` **Cons:** - ❌ Must create tokens via Portainer UI (one-time setup) - ❌ CE RBAC limitation applies — token = admin-level access --- #### ⚠️ **Option 2: JWT via Username/Password** | Aspect | Detail | |--------|--------| | **How it works** | `POST /api/auth { username, password }` → get JWT → use `Authorization: Bearer <jwt>` | | **Token expiry** | **5 minutes** by default in Portainer CE | | **Code required** | Auth client + retry logic with re-auth on 401 | | **Lifecycle** | Must re-authenticate every ~4 min or handle 401 → retry | **The implementation overhead:** ```java // You need a token management layer: public class PortainerAuthManager { private String jwt; private Instant expiresAt; public synchronized String getToken() { if (jwt == null || Instant.now().isAfter(expiresAt)) { jwt = authenticate(); // POST /api/auth expiresAt = Instant.now().plus(4, MINUTES); // buffer } return jwt; } } ``` Plus an HTTP interceptor to detect 401 → refresh → retry. **Pros:** - ✅ No UI setup needed (use existing admin credentials) - ✅ Can be fully automated from scratch **Cons:** - ❌ **More complex code** — auth manager, retry interceptor, refresh logic - ❌ **Same security profile** — still full admin access in CE - ❌ Username/password in env vars is worse security posture than API tokens - ❌ 5-min expiry means you **will** hit a 401 at some point in production --- #### 🔶 **Option 3: Separate Portainer Users** | Aspect | Detail | |--------|--------| | **How it works** | Create `svc-portainer-dev` + `svc-portainer-prod` users → generate tokens for each | | **RBAC in CE** | **Doesn't exist** — even a limited user can see/manage all stacks | | **Value** | Logging/audit trail only (different user in audit logs) | | **Overhead** | User management + password rotation + token management | **The hard truth about Portainer CE RBAC:** > Portainer CE 2.30.1 does **not** support role-based access control. All authenticated users (except the special `internal` user) have full visibility into all environments and stacks. A token from `svc-portainer-dev` can delete prod stacks just as easily. So **separate users give you no security boundary** in CE. They only help with: - Audit logs (see *who* triggered the deploy) - Independent token rotation (revoke dev token without affecting prod) --- ### 🏆 My Recommendation: **Option 1 + Separate Users (for audit)** **The winning combo:** 1. Create **two Portainer users**: `svc-portainer-dev` and `svc-portainer-prod` 2. Generate an **API access token** for each (Portainer UI → My access tokens) 3. Store in **Gitea org secrets**: - `PORTAINER_DEV_API_KEY` → `ptk_xxxx_dev_xxxx` - `PORTAINER_PROD_API_KEY` → `ptk_xxxx_prod_xxxx` 4. The automation service reads the right key based on environment 5. Service uses `X-API-Key` header — **no auth code needed** **Why not just one shared token?** - Separating by user gives you audit trails - You can revoke dev token independently (e.g., if leaked during testing) - Cleaner rotation strategy **Why not pure JWT?** - Extra code complexity buys you nothing - API tokens are literally just string headers — simplest possible integration - JWT expiry is a liability for an automation service that needs to be reliable --- ### 🤖 Can You Create API Tokens Programmatically? **Short answer: Partially yes, but practically no in CE.** Portainer does expose token management endpoints in the API: | Method | Endpoint | Purpose | |--------|----------|---------| | `POST` | `/api/users/{id}/tokens` | Create a token for a user | | `GET` | `/api/users/{id}/tokens` | List tokens for a user | | `DELETE` | `/api/users/{id}/tokens/{tokenId}` | Revoke a token | **BUT the catch:** - These endpoints require a **JWT from an existing authenticated session** (admin login) - The **initial token** must be created via the UI (chicken-and-egg problem) - After you have one token, you *could* create/rotate others via API **Practical workflow:** ``` 1. [UI] Login as admin → create token for svc-portainer-dev → save the key 2. [UI] Login as admin → create token for svc-portainer-prod → save the key 3. [Code] Use the keys forever (rotate via UI when needed) ``` For automation *rotation*, you could later build a script that: 1. Uses an admin API token (saved securely) 2. Calls `POST /api/users/{svcUserId}/tokens` to generate new tokens 3. Updates Gitea secrets via their API 4. Revokes old tokens But for **first launch**, it's UI-only setup. --- ### ⚡ Implementation Impact on `portainer-automation` With Option 1, the `PortainerApiClient` in `:common` becomes dead simple: ```java // RestTemplate call — no auth manager needed public String findStackByName(String portainerUrl, String apiKey, String name) { HttpHeaders headers = new HttpHeaders(); headers.set("X-API-Key", apiKey); ResponseEntity<List<PortainerStack>> response = restTemplate.exchange( portainerUrl + "/api/stacks", HttpMethod.GET, new HttpEntity<>(headers), new ParameterizedTypeReference<>() {} ); return response.getBody().stream() .filter(s -> s.getName().equals(name)) .findFirst() .orElseThrow(() -> new StackNotFoundException(name)) .getId(); } ``` No: - ❌ Auth manager bean - ❌ Token refresh scheduler - ❌ 401 retry interceptor - ❌ JWT parsing or expiry checking Just: - ✅ Read API key from `PortainerProperties` - ✅ Attach `X-API-Key` header - ✅ Make the call **Let me know if you want me to proceed with this plan or if you have any questions!**
Archie commented 2026-07-06 06:01:14 +00:00
Member
Copy Link

🏗️ Architecture Analysis: Portainer Connection & Environment Config

Context: @developer hit a snag in PR #131 — https://192.168.0.100:9442 didn't work locally, but https://devdocker.hithomelabs.com did. Also had to handle sending env variables array back on redeploy.

🔍 Root Cause Analysis: Why 192.168.0.100:9442 Failed

After examining the code, infrastructure patterns, and prior issues, here are all the likely causes:

1. Network Topology — The Core Issue (Most Likely)

Component Detail
192.168.0.100 Private IP — only reachable from same subnet/LAN
devdocker.hithomelabs.com Public DNS → Cloudflare Tunnel → local Portainer

The hierarchy of likelihood:

  • Likely A: The dev machine is not on the same physical network as 192.168.0.100. If running from a different machine/laptop, this IP is unreachable without VPN/routing rules.
  • Likely B: Portainer is behind a Cloudflare Tunnel (matching the Hithomelabs pattern). devdocker.hithomelabs.com resolves to Cloudflare's edge → Tunnel → local Portainer. Direct IP access is intentionally blocked.
  • Possible C: 192.168.0.100 is on the same LAN but the Portainer container/service isn't listening on that interface/binding (only 0.0.0.0:9442 inside the container, mapped to Docker host's internal docker bridge).

Evidence: The cftunnels-service DB config in application.properties also uses 192.168.0.100:5432 — if the developer's machine cannot reach this IP, then both DB and Portainer would fail. The fact that only devdocker.hithomelabs.com worked suggests the dev machine is off the local network and accessing services through Cloudflare Tunnels.

2. Port 9442 vs 9443

From issue #124:

  • Portainer Dev: CE 2.30.1 — :9442
  • Portainer Prod: CE 2.30.1 — :9443

Portainer CE default is 9443. Using 9442 for dev is a deliberate custom mapping — verifying this is mapped correctly in Portainer's docker-compose is worth checking.

3. SSL Certificate Mismatch

The code in PortainerClientConfig already trusts all certs (loadTrustMaterial((chain, authType) -> true) + NoopHostnameVerifier), so this shouldn't be the issue for either endpoint. The same trust-all config handles both IP and domain.

4. CI/CD Implication

The CI runner runs in Docker on the same host as Portainer, so 192.168.0.100:9442 will work in CI (the runner can reach the Docker host). The application.properties default is fine for CI/deployed environments — only local dev needs the override.

✅ devdocker.hithomelabs.com — Is This the Correct Long-Term Target?

Short Answer: Yes, as the dev override, but NOT as the default.

Here's the recommended environment-specific config (following the pattern already established by cftunnels-service):

Profile Properties File portainer.base-url When Active
Default application.properties https://192.168.0.100:9442 CI/CD, Docker compose
local application-local.properties https://devdocker.hithomelabs.com Dev machine (matches spring.profiles.active=local)
prod application-prod.properties https://192.168.0.100:9443 Production (TBD per earlier decision #10)

Why devdocker.hithomelabs.com is the right choice for local:

  • ✅ Works from any machine (laptop, different subnet)
  • ✅ Routes through Cloudflare Tunnel (consistent with Hithomelabs infra)
  • ✅ Doesn't require VPN to be active
  • ✅ Already tested and working

Why NOT as default (application.properties):

  • ❌ Introduces external DNS dependency for a purely internal service
  • ❌ Portainer Automation is intended to be internal-only (Decision #9 from issue #124)
  • ❌ In Docker/CI context, the direct Docker host IP is more reliable
  • ❌ Cloudflare Tunnel adds latency unnecessary for machine-to-machine communication

The Missing Piece: No application-prod.properties or CI profiles

Currently portainer-automation only has:

  • application.properties (default)
  • application-local.properties (dev override)

Missing (for consistency with cftunnels-service):

  • application-prod.properties — prod Portainer URL on 9443
  • application-ci.properties — CI-specific overrides (if any)
  • application-test.properties — test DB/stubs

🧩 The Env Variables Array Issue

The developer had to add env var round-tripping (DeployService now fetches stack env first, then passes them back on redeploy). This is correct and essential.

Why it's needed:

  • Portainer's PUT /api/stacks/{id}/git/redeploy?endpointId={eid} erases env vars if not included in the body
  • The stack's compose variables (e.g., ${TAG}, ${HOST_PORT}) are lost on each redeploy
  • The current approach: getStack(stackId) → extract env list → pass to redeployGitStack(stackId, ... , env) is the standard Portainer workflow

One thing to verify: The EnvVariable model uses exact case-sensitive JSON property names (name, value). Portainer's API returns {"name": "...", "value": "..."} in the stack response and expects the same format back. This is already correct in the code.

🎯 Architecture Recommendations

1. Add Environment-Specific Properties Files

Create these files:

portainer-automation/src/main/resources/application-prod.properties:

# Prod Portainer runs on 9443
portainer.base-url=https://192.168.0.100:9443
portainer.endpoint-id=2
# api-key should come from env var, not config
portainer.api-key=${PORTAINER_PROD_API_KEY}
portainer.service.api-key=${PORTAINER_AUTOMATION_API_KEY}

portainer-automation/src/main/resources/application-ci.properties:

# CI runner can reach Docker host directly
portainer.base-url=https://192.168.0.100:9442
portainer.api-key=${PORTAINER_DEV_API_KEY}

Update application-local.properties to match the developer's working config:

server.port=8081
portainer.base-url=https://devdocker.hithomelabs.com
portainer.endpoint-id=2
portainer.service.api-key=dev-test-key
# api-key from env or .env file

Update application.properties to reference env vars for secrets:

server.port=8081
portainer.base-url=https://192.168.0.100:9442
portainer.endpoint-id=2
portainer.api-key=${PORTAINER_API_KEY:change-me}
portainer.service.api-key=${PORTAINER_AUTOMATION_API_KEY:change-me}

2. Environment-Aware Config Resolution Order

This matches exactly what cftunnels-service already does (spring.profiles.active=${ENV:default}).

3. SSL Trust Configuration

The current loadTrustMaterial((chain, authType) -> true) is fine for dev/CI but has security implications.

Recommendation: Make SSL config environment-aware

For local profile — Trust-all (self-signed Portainer cert):

// Already implemented — keep as-is
SSLContext sslContext = SSLContexts.custom()
    .loadTrustMaterial((chain, authType) -> true)
    .build();

For prod profile — Proper certificate validation:
Use a @Profile("prod") @Configuration that loads the actual Portainer cert or trusts a CA chain instead of blanket trust-all.

Simpler approach: If Portainer prod uses a proper certificate (e.g., Let's Encrypt via Cloudflare Tunnel), the default RestTemplate (without SSL customization) would work because Java trusts standard CAs. The trust-all config is only needed because 192.168.0.100:9443 has a self-signed cert (default Portainer CE behavior).

4. Naming Consistency

Consider renaming portainer.api-key to portainer.api-key (existing) and portainer.service.api-key to portainer.auth.api-key for clarity:

  • portainer.api-key → key used to call Portainer API
  • portainer.service.api-key → key used to authenticate callers of this service

The current portainer.service.api-key config key is ambiguous.

⚙️ Summary of Actions

Priority Action Issue/PR
🔴 High Add application-prod.properties, application-ci.properties Follow-up to #131
🔴 High Document the env-var round-trip requirement in DeployService Comment on #131
🟡 Medium Make SSL config environment-profile-aware New issue
🟡 Medium Rename ambiguous portainer.service.api-key → portainer.auth.api-key Refactoring
🟢 Low Add .env.example for portainer-automation with documented env vars Docs

Bottom line on @developer's question: 192.168.0.100:9442 didn't work because the dev machine is likely off the local network and can only reach internal services through Cloudflare Tunnels. The default config is correct for CI/deployed environments. The application-local.properties fix using devdocker.hithomelabs.com is the right solution, not a config bug — and the env variables array fix is a mandatory Portainer API requirement, not an architectural mistake.

## 🏗️ Architecture Analysis: Portainer Connection & Environment Config > **Context**: @developer hit a snag in PR #131 — `https://192.168.0.100:9442` didn't work locally, but `https://devdocker.hithomelabs.com` did. Also had to handle sending env variables array back on redeploy. --- ### 🔍 Root Cause Analysis: Why `192.168.0.100:9442` Failed After examining the code, infrastructure patterns, and prior issues, here are **all** the likely causes: #### 1. Network Topology — The Core Issue (Most Likely) | Component | Detail | |-----------|--------| | `192.168.0.100` | Private IP — only reachable from **same subnet/LAN** | | `devdocker.hithomelabs.com` | Public DNS → **Cloudflare Tunnel** → local Portainer | **The hierarchy of likelihood:** - **Likely A**: The dev machine is **not on the same physical network** as `192.168.0.100`. If running from a different machine/laptop, this IP is unreachable without VPN/routing rules. - **Likely B**: Portainer **is behind a Cloudflare Tunnel** (matching the Hithomelabs pattern). `devdocker.hithomelabs.com` resolves to Cloudflare's edge → Tunnel → local Portainer. Direct IP access is intentionally blocked. - **Possible C**: `192.168.0.100` **is** on the same LAN but the Portainer container/service isn't listening on that interface/binding (only `0.0.0.0:9442` inside the container, mapped to Docker host's internal docker bridge). **Evidence**: The `cftunnels-service` DB config in `application.properties` also uses `192.168.0.100:5432` — if the **developer's machine cannot reach this IP**, then both DB and Portainer would fail. The fact that only `devdocker.hithomelabs.com` worked suggests the dev machine is **off the local network** and accessing services through Cloudflare Tunnels. #### 2. Port 9442 vs 9443 From issue #124: > - Portainer Dev: CE 2.30.1 — `:9442` > - Portainer Prod: CE 2.30.1 — `:9443` Portainer CE **default** is `9443`. Using `9442` for dev is a deliberate custom mapping — verifying this is mapped correctly in Portainer's docker-compose is worth checking. #### 3. SSL Certificate Mismatch The code in `PortainerClientConfig` already trusts all certs (`loadTrustMaterial((chain, authType) -> true)` + `NoopHostnameVerifier`), so this **shouldn't be the issue** for either endpoint. The same trust-all config handles both IP and domain. #### 4. CI/CD Implication The **CI runner** runs in Docker on the same host as Portainer, so `192.168.0.100:9442` **will work in CI** (the runner can reach the Docker host). The `application.properties` default is fine for CI/deployed environments — only local dev needs the override. --- ### ✅ `devdocker.hithomelabs.com` — Is This the Correct Long-Term Target? #### Short Answer: Yes, as the **dev override**, but NOT as the default. Here's the **recommended environment-specific config** (following the pattern already established by `cftunnels-service`): | Profile | Properties File | `portainer.base-url` | When Active | |---------|---------------|---------------------|-------------| | Default | `application.properties` | `https://192.168.0.100:9442` | CI/CD, Docker compose | | `local` | `application-local.properties` | `https://devdocker.hithomelabs.com` | Dev machine (matches `spring.profiles.active=local`) | | `prod` | `application-prod.properties` | `https://192.168.0.100:9443` | Production (TBD per earlier decision #10) | **Why `devdocker.hithomelabs.com` is the right choice for `local`:** - ✅ Works from any machine (laptop, different subnet) - ✅ Routes through Cloudflare Tunnel (consistent with Hithomelabs infra) - ✅ Doesn't require VPN to be active - ✅ Already tested and working **Why NOT as default (`application.properties`):** - ❌ Introduces external DNS dependency for a purely internal service - ❌ Portainer Automation is intended to be **internal-only** (Decision #9 from issue #124) - ❌ In Docker/CI context, the direct Docker host IP is more reliable - ❌ Cloudflare Tunnel adds latency unnecessary for machine-to-machine communication #### The Missing Piece: No `application-prod.properties` or CI profiles Currently `portainer-automation` only has: - `application.properties` (default) - `application-local.properties` (dev override) **Missing** (for consistency with `cftunnels-service`): - `application-prod.properties` — prod Portainer URL on `9443` - `application-ci.properties` — CI-specific overrides (if any) - `application-test.properties` — test DB/stubs --- ### 🧩 The Env Variables Array Issue The developer had to add env var round-tripping (`DeployService` now fetches stack env first, then passes them back on redeploy). **This is correct and essential.** **Why it's needed:** - Portainer's `PUT /api/stacks/{id}/git/redeploy?endpointId={eid}` **erases env vars** if not included in the body - The stack's compose variables (e.g., `${TAG}`, `${HOST_PORT}`) are lost on each redeploy - The current approach: `getStack(stackId)` → extract `env` list → pass to `redeployGitStack(stackId, ... , env)` is the **standard Portainer workflow** **One thing to verify**: The `EnvVariable` model uses exact case-sensitive JSON property names (`name`, `value`). Portainer's API returns `{"name": "...", "value": "..."}` in the stack response and expects the same format back. This is already correct in the code. --- ### 🎯 Architecture Recommendations #### 1. Add Environment-Specific Properties Files **Create these files:** **`portainer-automation/src/main/resources/application-prod.properties`:** ```properties # Prod Portainer runs on 9443 portainer.base-url=https://192.168.0.100:9443 portainer.endpoint-id=2 # api-key should come from env var, not config portainer.api-key=${PORTAINER_PROD_API_KEY} portainer.service.api-key=${PORTAINER_AUTOMATION_API_KEY} ``` **`portainer-automation/src/main/resources/application-ci.properties`:** ```properties # CI runner can reach Docker host directly portainer.base-url=https://192.168.0.100:9442 portainer.api-key=${PORTAINER_DEV_API_KEY} ``` **Update `application-local.properties`** to match the developer's working config: ```properties server.port=8081 portainer.base-url=https://devdocker.hithomelabs.com portainer.endpoint-id=2 portainer.service.api-key=dev-test-key # api-key from env or .env file ``` **Update `application.properties`** to reference env vars for secrets: ```properties server.port=8081 portainer.base-url=https://192.168.0.100:9442 portainer.endpoint-id=2 portainer.api-key=${PORTAINER_API_KEY:change-me} portainer.service.api-key=${PORTAINER_AUTOMATION_API_KEY:change-me} ``` #### 2. Environment-Aware Config Resolution Order This matches **exactly** what `cftunnels-service` already does (`spring.profiles.active=${ENV:default}`). #### 3. SSL Trust Configuration The current `loadTrustMaterial((chain, authType) -> true)` is **fine for dev/CI** but has security implications. **Recommendation: Make SSL config environment-aware** **For `local` profile** — Trust-all (self-signed Portainer cert): ```java // Already implemented — keep as-is SSLContext sslContext = SSLContexts.custom() .loadTrustMaterial((chain, authType) -> true) .build(); ``` **For `prod` profile** — Proper certificate validation: Use a `@Profile("prod")` `@Configuration` that loads the actual Portainer cert or trusts a CA chain instead of blanket trust-all. **Simpler approach**: If Portainer prod uses a proper certificate (e.g., Let's Encrypt via Cloudflare Tunnel), the **default RestTemplate** (without SSL customization) would work because Java trusts standard CAs. The trust-all config is only needed because `192.168.0.100:9443` has a **self-signed cert** (default Portainer CE behavior). #### 4. Naming Consistency Consider renaming `portainer.api-key` to `portainer.api-key` (existing) and `portainer.service.api-key` to `portainer.auth.api-key` for clarity: - `portainer.api-key` → key used to **call Portainer API** - `portainer.service.api-key` → key used to **authenticate callers** of this service The current `portainer.service.api-key` config key is ambiguous. --- ### ⚙️ Summary of Actions | Priority | Action | Issue/PR | |----------|--------|----------| | 🔴 High | Add `application-prod.properties`, `application-ci.properties` | Follow-up to #131 | | 🔴 High | Document the env-var round-trip requirement in `DeployService` | Comment on #131 | | 🟡 Medium | Make SSL config environment-profile-aware | New issue | | 🟡 Medium | Rename ambiguous `portainer.service.api-key` → `portainer.auth.api-key` | Refactoring | | 🟢 Low | Add `.env.example` for portainer-automation with documented env vars | Docs | --- **Bottom line on @developer's question:** `192.168.0.100:9442` didn't work because the dev machine is likely off the local network and can only reach internal services through Cloudflare Tunnels. The default config is correct for CI/deployed environments. The `application-local.properties` fix using `devdocker.hithomelabs.com` is the right solution, not a config bug — and the env variables array fix is a mandatory Portainer API requirement, not an architectural mistake.
Archie added the
architect:complete
 label 2026-07-06 06:01:18 +00:00
Archie commented 2026-07-08 19:37:49 +00:00
Member
Copy Link

⏳ Architecture Status Check

Most decisions are confirmed ✅, but there's one remaining blocker:

Pending: Question #2 — Portainer Auth Method

@Polly provided a thorough analysis on July 5 recommending:

Option 1: API Access Tokens — Use X-API-Key header directly (zero auth code)
+ Separate Portainer users (svc-portainer-dev, svc-portainer-prod) for audit trails

@hitanshu said on July 5:

"@Polly can you run me through how we can get this done?"

Polly responded with the full breakdown. We need a ✅ or next steps.

What's Blocked on This Decision

Issue Description Status
#88 Build :portainer-automation service 🚧 PR #131 in progress — but auth implementation depends on this decision
#134 Add missing profile configs ✅ Done
#99 Deploy portainer-automation service Waiting on #88
#91 Wire CI deploy triggers Waiting on #88
#90 Convert stacks to git-based Not started
#92 End-to-end validation Waiting on #91
#139 Parameterize build workflow 🆕 Created today
#140 Main-branch deployment workflow Blocked by #139

Could we get a final call on this so we can close out the architecture phase? 🙏

## ⏳ Architecture Status Check Most decisions are confirmed ✅, but there's **one remaining blocker**: ### Pending: Question #2 — Portainer Auth Method @Polly provided a thorough analysis on July 5 recommending: > **Option 1: API Access Tokens** — Use `X-API-Key` header directly (zero auth code) > **+ Separate Portainer users** (svc-portainer-dev, svc-portainer-prod) for audit trails @hitanshu said on July 5: > "@Polly can you run me through how we can get this done?" Polly responded with the full breakdown. **We need a ✅ or next steps.** ### What's Blocked on This Decision | Issue | Description | Status | |-------|-------------|--------| | #88 | Build `:portainer-automation` service | 🚧 PR #131 in progress — *but auth implementation depends on this decision* | | #134 | Add missing profile configs | ✅ Done | | #99 | Deploy portainer-automation service | Waiting on #88 | | #91 | Wire CI deploy triggers | Waiting on #88 | | #90 | Convert stacks to git-based | Not started | | #92 | End-to-end validation | Waiting on #91 | | #139 | Parameterize build workflow | 🆕 Created today | | #140 | Main-branch deployment workflow | Blocked by #139 | **Could we get a final call on this so we can close out the architecture phase?** 🙏
Archie commented 2026-07-09 20:55:38 +00:00
Member
Copy Link

Architectural Review: SSL Profile Fix for PortainerClientConfig

I've examined the full codebase — the PortainerClientConfig.java, all four application-{profile}.properties files, PortainerAutomationProperties.java, and PortainerApiClient.java.

✅ Root Cause Confirmed

Profile Target URL SSL Current RestTemplate
local https://devdocker.hithomelabs.com (tunnel) Self-signed trust-all ✅
prod https://192.168.0.100:9443 Self-signed strict ❌
default https://192.168.0.100:9442 (dev) Self-signed strict ❌
ci http://portainer-test:9000 (HTTP) None strict — OK ✅

The @Profile("!local") catch-all applies strict SSL to every non-local profile. Since both Portainer dev (:9442) and prod (:9443) use self-signed certificates, strict validation fails.

⚠️ Issue: The Profile-Negation Approach Has a Blind Spot

The proposed @Profile("!local & !prod") syntax is valid in Spring 6/Spring Boot 3 — it uses profile expression parsing.

However, there's a latent bug: the default profile (when no SPRING_PROFILES_ACTIVE is set) loads application.properties → https://192.168.0.100:9442 (self-signed) → strict RestTemplate → same bad_certificate failure. Negation-based profiling requires updating the expression every time a new trust-all profile is added.

🔧 Recommendation: Property-Driven SSL (Preferred)

Replace the @Profile-based approach with a configuration property:

@Configuration
public class PortainerClientConfig {

    @Bean(name = "portainerRestTemplate")
    public RestTemplate portainerRestTemplate(
            PortainerAutomationProperties props) {
        if (props.isTrustAllSsl()) {
            return buildTrustAllRestTemplate();
        }
        return new RestTemplate();
    }

    private static RestTemplate buildTrustAllRestTemplate() {
        try {
            TrustManager[] trustAllCerts = new TrustManager[]{
                new X509TrustManager() {
                    public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; }
                    public void checkClientTrusted(X509Certificate[] certs, String authType) {}
                    public void checkServerTrusted(X509Certificate[] certs, String authType) {}
                }
            };
            SSLContext sslContext = SSLContext.getInstance("TLS");
            sslContext.init(null, trustAllCerts, new java.security.SecureRandom());
            SSLConnectionSocketFactory sslSocketFactory =
                    new SSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE);
            HttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder.create()
                    .setSSLSocketFactory(sslSocketFactory).build();
            return new RestTemplate(new HttpComponentsClientHttpRequestFactory(
                    HttpClients.custom().setConnectionManager(cm).build()));
        } catch (NoSuchAlgorithmException e) {
            throw new IllegalStateException("JVM does not support TLS algorithm", e);
        } catch (KeyManagementException e) {
            throw new IllegalStateException("Failed to initialize trust-all SSL context", e);
        }
    }
}

Add to PortainerAutomationProperties:

private boolean trustAllSsl = false;
// + getter + setter

Profile configs:

File Setting
application.properties (dev default) portainer.trust-all-ssl=true
application-prod.properties portainer.trust-all-ssl=true
application-local.properties portainer.trust-all-ssl=true
application-ci.properties (omit — defaults to false)

Benefits over profile-based approach:

  • ✅ New profiles just set a property — zero code changes
  • ✅ Safe default (false — accidental SSL trust is prevented)
  • ✅ Each properties file is self-documenting
  • ✅ Removes @Profile coupling from RestTemplate configuration
  • ✅ The default (no-profile) case works correctly

📋 Answers to Your Questions

Q1: Is the proposed approach sound?
It fixes the prod bug, but the negation pattern is fragile. The property-driven approach above is architecturally superior.

Q2: Edge cases with @Profile("!local & !prod")?

  • Syntax is valid (Profiles.of() parser supports &, !, |)
  • Multi-profile activation (e.g., prod,ci) → only the @Profile("prod") bean loads, no conflict ✅
  • Latent bug: Default (no-profile) state gets strict SSL against self-signed :9442 ❌

Q3: Exception handling in the helper?
SSLContext.getInstance("TLS") can throw NoSuchAlgorithmException and sslContext.init() can throw KeyManagementException. Best practice: catch these and wrap in IllegalStateException with clear messaging (shown above). The original throws Exception is acceptable but imprecise.

Q4: Other concerns?

  1. Connection pooling: The trust-all path uses PoolingHttpClientConnectionManager. The strict path uses vanilla RestTemplate (no pooling). For prod, trust-all → pooling is a bonus. For CI (strict), no pooling is acceptable.
  2. Hostname verification: The trust-all path disables hostname verification via NoopHostnameVerifier. This is required for internal Docker access (hitting 192.168.0.100 while the cert is for e.g. portainer.internal). Document this rationale in the code.
  3. Existing issue #134 already has architect:complete — profile config files exist. The property portainer.trust-all-ssl can be added to those files as an incremental change.

Decision Required

@hitanshu Please choose:

  • Option A: Quick fix (profile-based) — Add @Profile("prod") with trust-all. Fixes prod immediately. Default profile remains broken.
  • Option B: Property-driven (recommended) — Replace @Profile with portainer.trust-all-ssl property. Clean, extensible, no future maintenance.

Please select (A/B) or describe your preference.

## Architectural Review: SSL Profile Fix for `PortainerClientConfig` I've examined the full codebase — the `PortainerClientConfig.java`, all four application-{profile}.properties files, `PortainerAutomationProperties.java`, and `PortainerApiClient.java`. --- ### ✅ Root Cause Confirmed | Profile | Target URL | SSL | Current RestTemplate | |---------|-----------|------|---------------------| | `local` | `https://devdocker.hithomelabs.com` (tunnel) | Self-signed | trust-all ✅ | | `prod` | `https://192.168.0.100:9443` | Self-signed | **strict** ❌ | | *default* | `https://192.168.0.100:9442` (dev) | Self-signed | **strict** ❌ | | `ci` | `http://portainer-test:9000` (HTTP) | None | strict — OK ✅ | The `@Profile("!local")` catch-all applies strict SSL to every non-local profile. Since both Portainer dev (`:9442`) and prod (`:9443`) use self-signed certificates, strict validation fails. --- ### ⚠️ Issue: The Profile-Negation Approach Has a Blind Spot The proposed `@Profile("!local & !prod")` syntax **is valid** in Spring 6/Spring Boot 3 — it uses profile expression parsing. However, there's a latent bug: the **default profile** (when no `SPRING_PROFILES_ACTIVE` is set) loads `application.properties` → `https://192.168.0.100:9442` (self-signed) → strict RestTemplate → same `bad_certificate` failure. Negation-based profiling requires updating the expression every time a new trust-all profile is added. --- ### 🔧 Recommendation: Property-Driven SSL (Preferred) Replace the `@Profile`-based approach with a configuration property: ```java @Configuration public class PortainerClientConfig { @Bean(name = "portainerRestTemplate") public RestTemplate portainerRestTemplate( PortainerAutomationProperties props) { if (props.isTrustAllSsl()) { return buildTrustAllRestTemplate(); } return new RestTemplate(); } private static RestTemplate buildTrustAllRestTemplate() { try { TrustManager[] trustAllCerts = new TrustManager[]{ new X509TrustManager() { public X509Certificate[] getAcceptedIssuers() { return new X509Certificate[0]; } public void checkClientTrusted(X509Certificate[] certs, String authType) {} public void checkServerTrusted(X509Certificate[] certs, String authType) {} } }; SSLContext sslContext = SSLContext.getInstance("TLS"); sslContext.init(null, trustAllCerts, new java.security.SecureRandom()); SSLConnectionSocketFactory sslSocketFactory = new SSLConnectionSocketFactory(sslContext, NoopHostnameVerifier.INSTANCE); HttpClientConnectionManager cm = PoolingHttpClientConnectionManagerBuilder.create() .setSSLSocketFactory(sslSocketFactory).build(); return new RestTemplate(new HttpComponentsClientHttpRequestFactory( HttpClients.custom().setConnectionManager(cm).build())); } catch (NoSuchAlgorithmException e) { throw new IllegalStateException("JVM does not support TLS algorithm", e); } catch (KeyManagementException e) { throw new IllegalStateException("Failed to initialize trust-all SSL context", e); } } } ``` Add to `PortainerAutomationProperties`: ```java private boolean trustAllSsl = false; // + getter + setter ``` Profile configs: | File | Setting | |------|---------| | `application.properties` (dev default) | `portainer.trust-all-ssl=true` | | `application-prod.properties` | `portainer.trust-all-ssl=true` | | `application-local.properties` | `portainer.trust-all-ssl=true` | | `application-ci.properties` | *(omit — defaults to `false`)* | **Benefits over profile-based approach:** - ✅ New profiles just set a property — zero code changes - ✅ Safe default (`false` — accidental SSL trust is prevented) - ✅ Each properties file is self-documenting - ✅ Removes `@Profile` coupling from RestTemplate configuration - ✅ The default (no-profile) case works correctly --- ### 📋 Answers to Your Questions **Q1: Is the proposed approach sound?** It fixes the prod bug, but the negation pattern is fragile. The property-driven approach above is architecturally superior. **Q2: Edge cases with `@Profile("!local & !prod")`?** - Syntax is valid (`Profiles.of()` parser supports `&`, `!`, `|`) - Multi-profile activation (e.g., `prod,ci`) → only the `@Profile("prod")` bean loads, no conflict ✅ - **Latent bug**: Default (no-profile) state gets strict SSL against self-signed `:9442` ❌ **Q3: Exception handling in the helper?** `SSLContext.getInstance("TLS")` can throw `NoSuchAlgorithmException` and `sslContext.init()` can throw `KeyManagementException`. Best practice: catch these and wrap in `IllegalStateException` with clear messaging (shown above). The original `throws Exception` is acceptable but imprecise. **Q4: Other concerns?** 1. **Connection pooling**: The trust-all path uses `PoolingHttpClientConnectionManager`. The strict path uses vanilla `RestTemplate` (no pooling). For prod, trust-all → pooling is a bonus. For CI (strict), no pooling is acceptable. 2. **Hostname verification**: The trust-all path disables hostname verification via `NoopHostnameVerifier`. This is required for internal Docker access (hitting `192.168.0.100` while the cert is for e.g. `portainer.internal`). **Document this rationale** in the code. 3. **Existing issue #134** already has `architect:complete` — profile config files exist. The property `portainer.trust-all-ssl` can be added to those files as an incremental change. --- ### Decision Required **@hitanshu** Please choose: - **Option A: Quick fix (profile-based)** — Add `@Profile("prod")` with trust-all. Fixes prod immediately. Default profile remains broken. - **Option B: Property-driven (recommended)** — Replace `@Profile` with `portainer.trust-all-ssl` property. Clean, extensible, no future maintenance. Please select (A/B) or describe your preference.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
architect:complete

Architecture analysis completed

  
blocked-by:#139

Blocked by issue #139

  
blocks:#140

Blocks issue #140

  
bug

Defect or regression

  
CI/CD

CI/CD pipeline improvements

  
complexity:low

Low complexity - straightforward implementation

  
complexity:medium

Medium complexity - some unknowns

  
complexity:medium

Medium complexity - some nuance and risk areas

  
config

Configuration files and settings

  
depends-on:#124

Depends on architecture decision issue #124

  
docker

Docker-related tasks

  
docs

Documentation updates and improvements

  
effort:l

L effort estimate - 2-4 weeks (large feature, complex)

  
effort:s

S effort estimate - 2-5 days (small feature, well-defined)

  
effort:xs

XS effort estimate - 1-2 days

  
epic/development

Epic for development phase work

  
lead:complete

Lead engineer review completed

  
needs-decision

Requires human decision before proceeding

  
performance

Performance improvements

  
priority:critical

Must do immediately

  
priority:high

Important for current milestone

  
priority:low

Nice to have

  
priority:medium

Valuable but not urgent

  
security

Security-related concerns

  
spike

Research / exploration

  
story-points:1

Trivial effort

  
story-points:3

Small effort

  
story-points:5

Medium effort

  
story-points:8

Large effort

  
tech-debt

Refactoring / improvement

  
test

Testing-related tasks

  
user-story

Standard user story

  
architect:complete

Architect phase completed

  
complexity:high

High complexity

  
complexity:low

Low complexity

  
complexity:medium

Medium complexity

  
cross-repo

Cross-repository item

  
cross-repo-dev

Cross-repository development

  
dev:in-progress

Development in progress

  
effort:l

Large effort (2-4 weeks)

  
effort:m

Medium effort (1-2 weeks)

  
effort:s

Small effort (2-5 days)

  
effort:xl

Extra large effort (4+ weeks)

  
effort:xs

Extra small effort (1-2 days)

  
epic
analytics
Data analytics of platform data and logs analysis.

  
epic
development
Developing new features, services. front and and back end as well as batch jobs

  
epic
devops
Epic related to CI/CD pipeline, runners, setup and workflow triggers

  
epic
infra
Epic related to infrastructure tasks, configuring hard drives, network equipment, firewalls, anything involving getting hands dirty.

  
epic
observability
Observability stack, uptime, service discovery, network, firewall, processes, resource consumption etc.

  
epic
platform
Related to authentication, authorization, keys, passwords, configs. Updates and upgrades related to platform.

  
epic
product
Issues discussing products and features come over here.

  
lead:complete

Lead/Code review phase completed

  
needs-decision

Requires user decision/selection

  
pipeline-complete

Pipeline completed successfully

  
pipeline-error

Pipeline failed with error

  
pipeline-running

Pipeline is currently running

  
priority
later
To be done but can wait. Generally areas that don't want immediate attention and warrant pre-planning, analysis and/or design

  
priority
next
Not a burning issue but needs to be worked on next. Generally should not be blocked by any issue other than priority::now

  
priority
now
Immediate priority, critical for PROD go-live

  
start-pipeline

Trigger the SDLC pipeline to start

  
status
acceptance
Testing results are good and issue looks good to deploy. It is a pre-deployment sign-off of test results

  
status
blocked
Need to solve another issue before this issue can be picked. The issue is blocked till the blocking issue isn't resolved

  
status
done
Issue deployment successful.

  
status
in progress
Issue has been picked up and someone is working on it.

  
status
in review
The issue has been work on but, a review has been requested, generally good to transition to this if all PRs raised require review or are reviewed.

  
status
in testing
Code review complete and issue is in testing in lower environment. P.S. This means unit tests have already been written.

  
status
ready
Issue does not have any blocker and is ready to be picked.

  
status
refine
Something working already well in PROD but a change that makes life a little more easier, like skip into in netflix. Issues that signify added convenience.

  
status
triage
Issue is requested but not scoped. Need to be added to systematically review and categorize

  
subtask

Subtask issue

  
type
analysis
An issue to perform analysis, exploring deign and implementation decisions/ considerations.

  
type
bug
A bug, needs to be fixed, raised out of an incident.

  
type
hygiene
Features remain the same, makes the platform, faster, better and less-error prone. Makes developer/ system-admin life easier

  
type
mantainence
Primarily version upgrades

  
type
story
Something new, a new feature !

  
user-story

PM-created user story

No Label
architect:complete
blocked-by:#139
blocks:#140
bug
CI/CD
complexity:low
complexity:medium
complexity:medium
config
depends-on:#124
docker
docs
effort:l
effort:s
effort:xs
epic/development
lead:complete
needs-decision
performance
priority:critical
priority:high
priority:low
priority:medium
security
spike
story-points:1
story-points:3
story-points:5
story-points:8
tech-debt
test
user-story
architect:complete
complexity:high
complexity:low
complexity:medium
cross-repo
cross-repo-dev
dev:in-progress
effort:l
effort:m
effort:s
effort:xl
effort:xs
epic
analytics
epic
development
epic
devops
epic
infra
epic
observability
epic
platform
epic
product
lead:complete
needs-decision
pipeline-complete
pipeline-error
pipeline-running
priority
later
priority
next
priority
now
start-pipeline
status
acceptance
status
blocked
status
done
status
in progress
status
in review
status
in testing
status
ready
status
refine
status
triage
subtask
type
analysis
type
bug
type
hygiene
type
mantainence
type
story
user-story
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
4 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: Hithomelabs/CFTunnels#124
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?