Add multi-site Caddy helpers and document usage

- add startup/shutdown scripts that render a Caddyfile from JSON config and run health checks

- add Python utilities and a sample sites.json for declarative multi-site configuration

- document the workflow and ignore generated Caddy state artifacts

- normalize double-quote style across challenge workflow controllers, nodes, and tests
This commit is contained in:
Joey Yakimowich-Payne 2025-10-15 22:03:56 -06:00
commit 6038fc25f5
No known key found for this signature in database
GPG key ID: 6BFE655FA5ABD1E1
26 changed files with 1018 additions and 168 deletions

1
docker/caddy/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
state/

99
docker/caddy/README.md Normal file
View file

@ -0,0 +1,99 @@
# Caddy Multi-Site Configuration
The Caddy helpers under `docker/` can now generate a single Caddyfile that serves multiple services. Instead of hard-coding upstreams in the shell script, each site to be exposed is described declaratively inside `docker/caddy/sites.json`.
## Layout
```
docker/
startup-caddy.sh # Generates Caddyfile + runs caddy start --config ...
shutdown-caddy.sh # Stops the running Caddy instance
caddy/
sites.json # Default site definitions (Dify + HackAPrompt example)
state/ # Generated Caddyfile, logs, pid files
```
Provide an alternative configuration file with `--config` or `CADDY_SITE_CONFIG` if you want a different set of sites.
```
# Use a staging configuration
./docker/startup-caddy.sh --config /path/to/staging-sites.json --regenerate
```
## Config File Format
The configuration file is JSON with a top-level `sites` array. Each site object supports the keys below (all optional unless marked **required**):
| Key | Type | Description |
| --- | ---- | ----------- |
| `name` | string | Display name used in logs; defaults to `site`. |
| `address` | string **required** | Caddy address such as `example.com` or `:8080`. |
| `auto_https` | bool/string | Disable auto HTTPS when serving plain HTTP (default: `true`). |
| `https_redirect` | bool/string | Emit an HTTP→HTTPS redirect block when a hostname is present. |
| `acme_challenge` | bool/string | Adds a handler for `/.well-known/acme-challenge/*`. |
| `headers` | object | Key/value pairs emitted inside a `header { ... }` block. |
| `health_path` | string | Path exposed as a simple `respond` handler (default `/health`). |
| `health_check` | object | `{ "url": "https://...", "skip": false }` controls post-start checks. |
| `app_url` | string | Logged after startup for operator convenience. |
| `log_file` | string | Custom log path; defaults to `$LOG_DIR/<name>-access.log`. |
| `api_routes` | array | Each entry `{ "path": "/api/*", "upstream": "host:port" }` creates a `handle` block with `reverse_proxy`. |
| `static_routes` | array | Each entry `{ "path": "/images/*", "root": "/var/www" }`; add `"browse": true` to enable directory listing. |
| `frontend` | object | `{"type":"reverse_proxy","upstream":"host:port"}` or `{"type":"static","root":"/dir","try_files":[...]} `. |
| `cache_static` | object | `{ "paths": ["*.js", ...], "header": "public, max-age=..." }`. |
All string values are expanded with `os.path.expandvars`, which means you can reference environment variables—`"${LOG_DIR}/dify-access.log"`, `"${DIFY_API_UPSTREAM:-127.0.0.1:5001}"`, etc.
## Example
```
{
"sites": [
{
"name": "dify",
"address": "${DIFY_ADDRESS:-:80}",
"https_redirect": "${DIFY_REDIRECT:-false}",
"api_routes": [
{ "path": "/api/*", "upstream": "${DIFY_API_UPSTREAM:-127.0.0.1:5001}" }
],
"frontend": {
"type": "reverse_proxy",
"upstream": "${DIFY_FRONTEND_UPSTREAM:-127.0.0.1:3000}"
},
"health_check": {
"url": "${DIFY_HEALTH_URL:-http://127.0.0.1/health}",
"skip": "${DIFY_SKIP_HEALTHCHECK:-false}"
},
"app_url": "${DIFY_APP_URL:-http://127.0.0.1}"
},
{
"name": "hackaprompt",
"address": "${HACKAPROMPT_ADDRESS:-:8080}",
"frontend": {
"type": "static",
"root": "${HACKAPROMPT_FRONTEND_ROOT:-/opt/hackaprompt-chat-viewer/frontend}",
"try_files": ["{path}", "{path}/", "/index.html"]
},
"api_routes": [
{ "path": "/api/*", "upstream": "${HACKAPROMPT_API_UPSTREAM:-127.0.0.1:5002}" }
],
"static_routes": [
{ "path": "/images/*", "root": "${HACKAPROMPT_APP_ROOT:-/opt/hackaprompt-chat-viewer}" }
]
}
]
}
```
## Health Checks & Logs
After Caddy starts, the script runs a curl-based health check for each site unless `--skip-healthcheck` is passed globally or the site entry sets `"skip": true`. Each check waits up to 30 seconds.
Logs are written to `docker/caddy/state/logs/` by default. Adjust `log_file` per site if you want a different location.
## Stopping Caddy
```
./docker/shutdown-caddy.sh
```
The shutdown helper remains unchanged; it simply reads the PID file and stops the running Caddy instance.

11
docker/caddy/__init__.py Normal file
View file

@ -0,0 +1,11 @@
"""Utilities for generating multi-site Caddy configurations."""
from .render_caddy import SiteMetadata, render_sites, render_metadata_lines
from .load_metadata import load_metadata
__all__ = [
"SiteMetadata",
"render_sites",
"render_metadata_lines",
"load_metadata",
]

View file

@ -0,0 +1,29 @@
"""Load site metadata without rewriting the Caddyfile."""
from __future__ import annotations
import json
import os
from pathlib import Path
from typing import List
from .render_caddy import SiteMetadata, _expand, _to_bool
def load_metadata(config_path: Path) -> List[SiteMetadata]:
with config_path.open("r", encoding="utf-8") as fh:
data = json.load(fh)
sites = data.get("sites", [])
metadata: List[SiteMetadata] = []
for raw_site in sites:
site = _expand(raw_site)
health_cfg = site.get("health_check") or {}
metadata.append(
SiteMetadata(
name=site.get("name", "site"),
health_url=health_cfg.get("url", ""),
skip_healthcheck=_to_bool(health_cfg.get("skip"), default=False),
app_url=site.get("app_url", ""),
)
)
return metadata

View file

@ -0,0 +1,296 @@
"""Render a multi-site Caddyfile from JSON configuration.
This module is invoked by docker/startup-caddy.sh. It reads a JSON config,
expands environment variables, and writes out a Caddyfile plus site metadata
used for health checks and logging.
"""
from __future__ import annotations
import json
import os
import re
from dataclasses import dataclass
from pathlib import Path
from typing import Iterable, Iterator, List, Tuple
from urllib.parse import urlparse
@dataclass
class SiteMetadata:
"""Compact representation of site health/check details."""
name: str
health_url: str
skip_healthcheck: bool
app_url: str
def serialize(self) -> str:
return "|".join(
[
self.name,
self.health_url,
"true" if self.skip_healthcheck else "false",
self.app_url,
]
)
DEFAULT_PATTERN = re.compile(r"\${([^}:]+):-([^}]*)}")
def _expand_string(raw: str) -> str:
def replace(match: re.Match[str]) -> str:
var, default = match.group(1), match.group(2)
current = os.environ.get(var)
return current if current not in (None, "") else default
substituted = DEFAULT_PATTERN.sub(replace, raw)
return os.path.expandvars(substituted)
def _expand(value): # type: ignore[no-untyped-def]
if isinstance(value, str):
return _expand_string(value)
if isinstance(value, list):
return [_expand(v) for v in value]
if isinstance(value, dict):
return {k: _expand(v) for k, v in value.items()}
return value
def _to_bool(value, default: bool = False) -> bool:
if value is None:
return default
if isinstance(value, bool):
return value
if isinstance(value, (int, float)):
return bool(value)
value_str = str(value).strip().lower()
if value_str in {"true", "1", "yes", "y", "on"}:
return True
if value_str in {"false", "0", "no", "n", "off"}:
return False
return default
def _iter_headers(value) -> Iterator[Tuple[str, str]]: # type: ignore[no-untyped-def]
if isinstance(value, dict):
for key, val in value.items():
if key and val is not None:
yield str(key), str(val)
return
if isinstance(value, (list, tuple)):
for item in value:
if isinstance(item, dict):
name = item.get("name") or item.get("header")
val = item.get("value")
if name and val is not None:
yield str(name), str(val)
elif isinstance(item, (list, tuple)) and len(item) == 2:
name, val = item
if name and val is not None:
yield str(name), str(val)
elif value is not None:
raise ValueError("Unsupported header specification: %r" % (value,))
def render_sites(config_path: Path, log_dir: Path) -> tuple[str, List[SiteMetadata]]:
with config_path.open("r", encoding="utf-8") as fh:
data = json.load(fh)
raw_sites = data.get("sites", [])
if not raw_sites:
raise ValueError("No sites defined in configuration file")
log_dir.mkdir(parents=True, exist_ok=True)
blocks: List[str] = []
metadata: List[SiteMetadata] = []
for raw_site in raw_sites:
site = _expand(raw_site)
name = site.get("name") or "site"
address = site.get("address")
if not address:
raise ValueError(f"Site '{name}' is missing the required 'address' field")
https_redirect = _to_bool(site.get("https_redirect"), default=False)
acme_challenge = _to_bool(site.get("acme_challenge"), default=False)
address = address.strip()
log_file = site.get("log_file")
log_path = Path(log_file) if log_file else log_dir / f"{name}-access.log"
if not log_path.is_absolute():
log_path = log_dir / log_path
log_path.parent.mkdir(parents=True, exist_ok=True)
lines: List[str] = [f"{address} {{"]
lines.append(" encode gzip")
headers = site.get("headers") or {}
if headers:
lines.append(" header {")
for key, value in headers.items():
lines.append(f' {key} "{value}"')
lines.append(" }")
health_path = site.get("health_path", "/health")
if health_path:
lines.append(f' respond {health_path} "OK" 200')
if acme_challenge:
lines.extend(
[
" handle /.well-known/acme-challenge/* {",
" file_server",
" }",
]
)
for route in site.get("api_routes") or []:
path = route.get("path")
upstream = route.get("upstream")
if not path or not upstream:
continue
headers_up = list(_iter_headers(route.get("headers_up")))
headers_down = list(_iter_headers(route.get("headers_down")))
lines.append(f" handle {path} {{")
if headers_up or headers_down:
lines.append(f" reverse_proxy {upstream} {{")
for key, val in headers_up:
lines.append(f" header_up {key} {val}")
for key, val in headers_down:
lines.append(f" header_down {key} {val}")
lines.append(" }")
else:
lines.append(f" reverse_proxy {upstream}")
lines.append(" }")
for route in site.get("static_routes") or []:
path = route.get("path")
root = route.get("root")
if not path or not root:
continue
browse = _to_bool(route.get("browse"))
lines.append(f" handle {path} {{")
lines.append(f" root * {root}")
lines.append(" file_server browse" if browse else " file_server")
lines.append(" }")
if site.get("explore_route"):
explore = site["explore_route"]
upstream = explore.get("upstream")
if upstream:
lines.append(" handle /explore* {")
lines.append(f" reverse_proxy {upstream}")
lines.append(" }")
if site.get("hooks_route"):
hook = site["hooks_route"]
path = hook.get("path", "/e/*")
upstream = hook.get("upstream")
if upstream:
headers_up = list(_iter_headers(hook.get("headers_up")))
headers_down = list(_iter_headers(hook.get("headers_down")))
lines.append(f" handle {path} {{")
if headers_up or headers_down:
lines.append(f" reverse_proxy {upstream} {{")
for key, val in headers_up:
lines.append(f" header_up {key} {val}")
for key, val in headers_down:
lines.append(f" header_down {key} {val}")
lines.append(" }")
else:
lines.append(f" reverse_proxy {upstream}")
lines.append(" }")
frontend = site.get("frontend") or {}
frontend_type = frontend.get("type", "reverse_proxy")
if frontend_type == "reverse_proxy":
upstream = frontend.get("upstream")
if upstream:
rp_lines = [" handle {"]
headers_up = list(_iter_headers(frontend.get("headers_up")))
headers_down = list(_iter_headers(frontend.get("headers_down")))
if headers_up or headers_down:
rp_lines.append(f" reverse_proxy {upstream} {{")
for key, val in headers_up:
rp_lines.append(f" header_up {key} {val}")
for key, val in headers_down:
rp_lines.append(f" header_down {key} {val}")
rp_lines.append(" }")
else:
rp_lines.append(f" reverse_proxy {upstream}")
rp_lines.append(" }")
lines.extend(rp_lines)
elif frontend_type == "static":
root = frontend.get("root")
if root:
lines.append(" handle {")
lines.append(f" root * {root}")
try_files = frontend.get("try_files") or []
if try_files:
lines.append(" try_files " + " ".join(try_files))
lines.append(" file_server")
lines.append(" }")
cache_static = site.get("cache_static") or {}
cache_paths = cache_static.get("paths") or []
cache_header = cache_static.get("header")
if cache_paths and cache_header:
lines.append(" @static {")
lines.append(" path " + " ".join(cache_paths))
lines.append(" }")
lines.append(f' header @static Cache-Control "{cache_header}"')
lines.extend(
[
" log {",
f" output file {log_path} {{",
" roll_size 100mb",
" roll_keep 10",
" roll_keep_for 720h",
" }",
" format json",
" }",
"}",
]
)
blocks.append("\n".join(lines))
host_for_redirect = ""
redirect_port = None
if not address.startswith(":"):
parsed_address = urlparse(address if "://" in address else f"https://{address}")
host_for_redirect = parsed_address.hostname or ""
redirect_port = parsed_address.port
if https_redirect and host_for_redirect:
port_segment = f":{redirect_port}" if redirect_port and redirect_port != 443 else ""
blocks.append(
"\n".join(
[
f"http://{host_for_redirect}{port_segment} {{",
f" redir https://{host_for_redirect}{{uri}}",
"}",
]
)
)
health_cfg = site.get("health_check") or {}
metadata.append(
SiteMetadata(
name=name,
health_url=health_cfg.get("url", ""),
skip_healthcheck=_to_bool(health_cfg.get("skip"), default=False),
app_url=site.get("app_url", ""),
)
)
return "\n\n".join(blocks) + "\n", metadata
def render_metadata_lines(entries: Iterable[SiteMetadata]) -> List[str]:
return [entry.serialize() for entry in entries]

137
docker/caddy/sites.json Normal file
View file

@ -0,0 +1,137 @@
{
"sites": [
{
"name": "dify",
"address": "${DIFY_ADDRESS:-dify.jojomaw.com}",
"https_redirect": "${DIFY_REDIRECT:-true}",
"auto_https": "${DIFY_AUTO_HTTPS:-true}",
"health_path": "/health",
"health_check": {
"url": "${DIFY_HEALTH_URL:-http://127.0.0.1/health}",
"skip": "${DIFY_SKIP_HEALTHCHECK:-false}"
},
"app_url": "${DIFY_APP_URL:-http://127.0.0.1}",
"log_file": "${LOG_DIR}/dify-access.log",
"acme_challenge": "${DIFY_ACME_CHALLENGE:-false}",
"headers": {
"X-Frame-Options": "SAMEORIGIN",
"X-Content-Type-Options": "nosniff",
"Referrer-Policy": "strict-origin-when-cross-origin",
"X-XSS-Protection": "1; mode=block",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains"
},
"api_routes": [
{
"path": "/api/*",
"upstream": "${DIFY_API_UPSTREAM:-127.0.0.1:5001}"
},
{
"path": "/v1/*",
"upstream": "${DIFY_API_UPSTREAM:-127.0.0.1:5001}"
},
{
"path": "/console/api/*",
"upstream": "${DIFY_API_UPSTREAM:-127.0.0.1:5001}"
},
{
"path": "/files/*",
"upstream": "${DIFY_FILES_UPSTREAM:-127.0.0.1:5001}"
},
{
"path": "/mcp/*",
"upstream": "${DIFY_MCP_UPSTREAM:-127.0.0.1:5001}"
}
],
"static_routes": [
{
"path": "/explore/*",
"root": "${DIFY_EXPLORE_ROOT:-/app/web}",
"browse": false
}
],
"frontend": {
"type": "reverse_proxy",
"upstream": "${DIFY_FRONTEND_UPSTREAM:-127.0.0.1:3000}"
},
"hooks_route": {
"path": "/e/*",
"upstream": "${DIFY_PLUGIN_DAEMON_UPSTREAM:-127.0.0.1:5002}",
"headers_up": [
{"name": "Dify-Hook-Url", "value": "${DIFY_HOOK_URL_HEADER:-{scheme}://{host}{uri}}"}
]
},
"cache_static": {
"paths": [
"*.js",
"*.css",
"*.png",
"*.jpg",
"*.jpeg",
"*.gif",
"*.ico",
"*.svg",
"*.woff",
"*.woff2",
"*.ttf",
"*.eot"
],
"header": "public, max-age=31536000, immutable"
}
},
{
"name": "hackaprompt",
"address": "${HACKAPROMPT_ADDRESS:-chat.jojomaw.com}",
"https_redirect": "${HACKAPROMPT_REDIRECT:-true}",
"auto_https": "${HACKAPROMPT_AUTO_HTTPS:-true}",
"health_path": "/health",
"health_check": {
"url": "${HACKAPROMPT_HEALTH_URL:-http://127.0.0.1:8080/health}",
"skip": "${HACKAPROMPT_SKIP_HEALTHCHECK:-false}"
},
"app_url": "${HACKAPROMPT_APP_URL:-http://127.0.0.1:8080}",
"log_file": "${LOG_DIR}/hackaprompt-access.log",
"acme_challenge": "${HACKAPROMPT_ACME_CHALLENGE:-false}",
"headers": {
"X-Frame-Options": "SAMEORIGIN",
"X-Content-Type-Options": "nosniff",
"Referrer-Policy": "strict-origin-when-cross-origin",
"X-XSS-Protection": "1; mode=block",
"Strict-Transport-Security": "max-age=31536000; includeSubDomains"
},
"api_routes": [
{
"path": "/api/*",
"upstream": "${HACKAPROMPT_API_UPSTREAM:-127.0.0.1:5501}"
}
],
"static_routes": [
{
"path": "/images/*",
"root": "${HACKAPROMPT_APP_ROOT:-/opt/hackaprompt-chat-viewer}"
}
],
"frontend": {
"type": "static",
"root": "${HACKAPROMPT_FRONTEND_ROOT:-/opt/hackaprompt-chat-viewer/frontend}",
"try_files": ["{path}", "{path}/", "/index.html"]
},
"cache_static": {
"paths": [
"*.js",
"*.css",
"*.png",
"*.jpg",
"*.jpeg",
"*.gif",
"*.ico",
"*.svg",
"*.woff",
"*.woff2",
"*.ttf",
"*.eot"
],
"header": "public, max-age=31536000, immutable"
}
}
]
}