diff --git a/GUI_PLAN.md b/GUI_PLAN.md new file mode 100644 index 0000000..1f98413 --- /dev/null +++ b/GUI_PLAN.md @@ -0,0 +1,487 @@ +# Warppipe GUI Plan (Qt6 Node-Based Audio Router) + +## Overview +A Qt6-based node editor GUI for warppipe using the QtNodes (nodeeditor) library. Visualizes PipeWire audio nodes, ports, and links as draggable nodes with connection lines. Supports creating virtual sinks/sources via context menu and displays ephemeral sources with visual fade when inactive. + +--- + +## Milestones + +- [ ] Milestone 0 - Qt6 Project Setup + - [ ] Create `gui/` subdirectory in warppipe project + - [ ] Add Qt6 + QtNodes to CMakeLists.txt (FetchContent for nodeeditor from github.com/paceholder/nodeeditor) + - [ ] Create `warppipe-gui` target with Qt6::Widgets and QtNodes dependencies + - [ ] Enable CMAKE_AUTOMOC, CMAKE_AUTORCC, CMAKE_AUTOUIC + - [ ] Create minimal main.cpp with QApplication + QMainWindow + - [ ] Verify GUI launches and shows empty window + +- [ ] Milestone 1 - Core Model Integration + - [ ] Create `WarpGraphModel : public QtNodes::AbstractGraphModel` + - [ ] Implement AbstractGraphModel interface (newNodeId, allNodeIds, nodeData, portData, etc.) + - [ ] Add `warppipe::Client*` member, connect to PipeWire on construction + - [ ] Map `warppipe::NodeInfo` to QtNodes NodeId via internal maps (m_nodes, m_pwToNode) + - [ ] Implement node refresh: call Client::ListNodes() and sync graph + - [ ] Create `GraphEditorWidget : public QWidget` + - [ ] Instantiate WarpGraphModel, QtNodes::BasicGraphicsScene, QtNodes::GraphicsView + - [ ] Lay out view in widget + - [ ] Connect model signals to refresh handlers + - [ ] Synthesize display title from NodeInfo: + - [ ] If `application_name` is non-empty and differs from `name`, use `application_name` as title + - [ ] Otherwise use `name` field + - [ ] Store synthesized title in `nodeData(NodeRole::Caption)` + - [ ] Map warppipe ports to QtNodes ports: + - [ ] Input ports (is_input=true) appear on LEFT side of node (QtNodes PortType::In) + - [ ] Output ports (is_input=false) appear on RIGHT side of node (QtNodes PortType::Out) + - [ ] Use port name from PortInfo as port label + - [ ] Verify nodes appear in graph view with correct titles and ports + +- [ ] Milestone 2 - Visual Styling and Node Types + - [ ] Define node type classification based on `media_class`: + - [ ] Sink → "Hardware Sink" (blue-gray base color) + - [ ] Source → "Hardware Source" (blue-gray base color) + - [ ] Virtual sinks created by warppipe → "Virtual Sink" (green base color) + - [ ] Virtual sources created by warppipe → "Virtual Source" (green base color) + - [ ] Application audio streams (ephemeral) → "Application" (brown/orange base color) + - [ ] Implement custom NodeStyle via `nodeData(NodeRole::Style)`: + - [ ] Return QtNodes::NodeStyle::toJson().toVariantMap() + - [ ] Set GradientColor0-3, NormalBoundaryColor, FontColor based on node type + - [ ] Reference potato's `nodeStyleVariant()` function for color scheme + - [ ] Detect ephemeral (application) nodes: + - [ ] Track node appearance/disappearance via Client poll or registry events + - [ ] Mark node as "inactive" if it disappears (no audio playing) + - [ ] Persist inactive nodes in graph model (do NOT remove from visual graph) + - [ ] Apply "ghost" styling to inactive nodes: + - [ ] Set `Opacity = 0.6f` (vs 1.0f for active) + - [ ] Darken gradient colors (use `.darker(150-180)`) + - [ ] Fade font color (lighter gray) + - [ ] Keep connections visible with faded style + - [ ] Verify: Application nodes appear vibrant when active, fade when inactive, never disappear + +- [ ] Milestone 3 - Link Visualization and Drag-Connect + - [ ] Implement connection mapping: + - [ ] Call `Client::ListLinks()` to get existing PipeWire links + - [ ] For each Link, find corresponding NodeId and PortIndex for output/input + - [ ] Create QtNodes::ConnectionId from (outNodeId, outPortType, outPortIndex, inNodeId, inPortType, inPortIndex) + - [ ] Store in model's m_connections set + - [ ] Implement `addConnection(ConnectionId)`: + - [ ] Extract output port and input port from ConnectionId + - [ ] Call `Client::CreateLink(outputPortId, inputPortId, LinkOptions{})` + - [ ] If successful, add connection to m_connections + - [ ] If failed, emit error and do NOT add to graph + - [ ] Implement `deleteConnection(ConnectionId)`: + - [ ] Find corresponding warppipe LinkId from connection + - [ ] Call `Client::RemoveLink(linkId)` + - [ ] Remove from m_connections + - [ ] Verify: Drag connection from output port to input port creates PipeWire link; delete removes it + +- [ ] Milestone 4 - Context Menu and Virtual Node Creation + - [ ] Add context menu to GraphEditorWidget: + - [ ] Right-click on canvas (not on node) shows menu + - [ ] Menu items: "Create Virtual Sink", "Create Virtual Source" + - [ ] Implement "Create Virtual Sink": + - [ ] Prompt user for name (QInputDialog or inline text field) + - [ ] Call `Client::CreateVirtualSink(name, VirtualNodeOptions{})` with default options + - [ ] On success, node appears in graph at context menu position + - [ ] Implement "Create Virtual Source": + - [ ] Same as sink but call `Client::CreateVirtualSource()` + - [ ] Add context menu on nodes: + - [ ] Right-click on virtual node shows "Delete Node" option + - [ ] Call `Client::RemoveNode(nodeId)` and remove from graph + - [ ] Verify: Can create/delete virtual sinks and sources via right-click + +- [ ] Milestone 5 - Layout Persistence and Polish + - [ ] Implement layout save/load: + - [ ] Save node positions to JSON file in `~/.config/warppipe-gui/layout.json` + - [ ] Store by stable ID (use NodeInfo.name as stable key) + - [ ] Save on position change (debounced) + - [ ] Load on startup and restore positions + - [ ] Implement auto-arrange: + - [ ] Menu or button to auto-layout nodes (left-to-right: sources → sinks) + - [ ] Use simple grid or layered layout algorithm + - [ ] Add visual polish: + - [ ] Connection lines styled (color, width, curvature) + - [ ] Highlight connections on hover + - [ ] Port connection points visible and responsive + - [ ] Add status bar: + - [ ] Show connection status to PipeWire daemon + - [ ] Show count of nodes, links + - [ ] Verify: Layout persists across sessions, UI feels responsive and polished + +- [ ] Milestone 6 - Screenshot Infrastructure (AI-Assisted Debugging) + - [ ] Add CLI flags to main.cpp via QCommandLineParser: + - [ ] `--screenshot ` / `-s `: Capture window to PNG and exit + - [ ] `--quit-after-screenshot` / `-q`: Explicit quit flag (redundant with -s but conventional) + - [ ] `--screenshot-delay `: Configurable render delay before capture (default 800ms) + - [ ] `--debug-screenshot-dir `: Continuous mode — save timestamped screenshot on every graph state change (node add/remove, connection change, ghost toggle) + - [ ] Implement two-tier QPixmap capture (from potato pattern): + - [ ] Primary: `window.grab()` (renders widget tree to pixmap) + - [ ] Fallback: `screen->grabWindow(window.winId())` if .grab() returns null + - [ ] Exit code 3 on capture failure + - [ ] Add F12 hotkey for interactive screenshot: + - [ ] Save to `$XDG_PICTURES_DIR/warppipe/warppipe_YYYYMMDD_HHmmss.png` + - [ ] Auto-create directory via QDir::mkpath() + - [ ] Implement "render complete" signal: + - [ ] GraphEditorWidget emits `graphReady()` after initial node sync completes + - [ ] Use signal instead of hardcoded delay for --screenshot when possible + - [ ] Fall back to --screenshot-delay if signal doesn't fire within timeout + - [ ] Support headless rendering for CI/AI: + - [ ] Document `QT_QPA_PLATFORM=offscreen` environment variable for headless capture + - [ ] Verify screenshots render correctly without a display server + - [ ] Add `--offscreen` convenience flag that sets QT_QPA_PLATFORM=offscreen internally via `qputenv()` + - [ ] Implement debug screenshot naming convention: + - [ ] Format: `warppipe__.png` (e.g., `warppipe_20260129_143052_node_added.png`) + - [ ] In --debug-screenshot-dir mode, capture on: initial load, node add, node remove, node ghost/unghost, connection add, connection remove, context menu open + - [ ] Verify: `warppipe-gui --screenshot /tmp/test.png` produces a valid PNG with visible nodes; headless mode works with QT_QPA_PLATFORM=offscreen + +- [ ] Milestone 7 - GUI Tests + - [ ] Create `tests/gui/` directory and `warppipe_gui_tests.cpp` test file + - [ ] Add `warppipe-gui-tests` CMake target linking warppipe, Qt6::Widgets, Qt6::Test, QtNodes, Catch2 + - [ ] Model unit tests (no display server needed, pure logic): + - [ ] WarpGraphModel: inject nodes via WARPPIPE_TESTING helpers → verify allNodeIds(), nodeData(Caption), nodeData(Style) + - [ ] WarpGraphModel: inject ports → verify portData(PortCount), portData(Caption) for correct port labels + - [ ] WarpGraphModel: inject links → verify allConnectionIds(), connectionExists() + - [ ] WarpGraphModel: ghost state tracking — mark node ghost → verify Opacity=0.6 in style, mark unghost → verify Opacity=1.0 + - [ ] WarpGraphModel: title synthesis — node with application_name="Firefox" → caption="Firefox"; node with empty application_name → caption=name + - [ ] WarpGraphModel: port orientation — is_input=true ports map to PortType::In (left); is_input=false → PortType::Out (right) + - [ ] WarpGraphModel: node removal doesn't crash when connections exist + - [ ] WarpGraphModel: duplicate node ID handling (update vs reject) + - [ ] Connection logic tests: + - [ ] addConnection() with valid ports → succeeds, stored in model + - [ ] addConnection() with mismatched types (output→output) → connectionPossible() returns false + - [ ] deleteConnection() removes from model + - [ ] Ghost connections: connection to ghost node remains in model, isGhostConnection() returns true + - [ ] Screenshot smoke tests (require QT_QPA_PLATFORM=offscreen): + - [ ] Launch warppipe-gui with --screenshot → exit code 0, PNG file exists, file size > 0 + - [ ] Launch with WARPPIPE_TESTING injected nodes → screenshot contains non-trivial content (file size > 10KB as heuristic) + - [ ] Launch with --debug-screenshot-dir → directory populated after state changes + - [ ] Integration tests with warppipe test harness: + - [ ] Create Client with WARPPIPE_TESTING → inject nodes/ports/links → construct WarpGraphModel → verify graph state matches injected data + - [ ] Inject node, then remove → verify ghost state in model + - [ ] Inject node, add rule, trigger policy check → verify model reflects auto-linked connections + - [ ] Add CTest integration: + - [ ] Model tests run without display server (always) + - [ ] Screenshot tests gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF) + - [ ] `ctest --test-dir build` runs model tests; `ctest --test-dir build -L visual` runs screenshot tests + +- [ ] Milestone 8 (Optional) - Advanced Features + - [ ] Add routing rule UI (separate panel or dialog) + - [ ] List existing rules from `Client::ListRouteRules()` + - [ ] Add/remove rules with RuleMatch fields + - [ ] Show which nodes are affected by rules + - [ ] Add volume/mute controls (if warppipe adds port parameters API) + - [ ] Add audio level meters (requires PipeWire param monitoring) + - [ ] Add config save/load UI (call `Client::SaveConfig()/LoadConfig()`) + - [ ] Add presets system (save/restore full node+link+rule configuration) + +--- + +## Architecture + +``` +warppipe/ +├── CMakeLists.txt # Add Qt6 + QtNodes, warppipe-gui + test targets +├── gui/ +│ ├── main.cpp # QApplication entry, CLI flags (--screenshot, --debug-screenshot-dir, --offscreen) +│ ├── WarpGraphModel.h # QtNodes::AbstractGraphModel implementation +│ ├── WarpGraphModel.cpp # Model logic, warppipe::Client integration +│ ├── GraphEditorWidget.h # Main UI widget with scene + view +│ ├── GraphEditorWidget.cpp # Event handlers, context menus, refresh logic, graphReady() signal +│ ├── NodeStyleHelper.h/cpp # Node styling functions (colors, ghost mode) +│ └── ScreenshotHelper.h/cpp # QPixmap capture, debug dir, naming conventions +├── tests/ +│ ├── warppipe_tests.cpp # Existing library tests +│ └── gui/ +│ └── warppipe_gui_tests.cpp # Model unit tests + screenshot smoke tests +├── include/warppipe/ +│ └── warppipe.hpp # No changes needed +└── src/ + └── warppipe.cpp # No changes needed +``` + +--- + +## Design Notes + +### Node Title Synthesis +warppipe::NodeInfo does not have a `description` field. Derive display title: +- **Application nodes**: Use `application_name` if non-empty (e.g., "Firefox", "Spotify") +- **Hardware/Virtual nodes**: Use `name` field (e.g., "alsa_output.pci-0000_00_1f.3.analog-stereo") +- Fallback to `name` if `application_name` is empty + +### Port Orientation +- **Input ports** (is_input=true): LEFT side of node (QtNodes::PortType::In) +- **Output ports** (is_input=false): RIGHT side of node (QtNodes::PortType::Out) + +### Ephemeral Node Handling +Application audio streams are ephemeral — they appear when an app plays audio and can disappear when stopped. +- **Do NOT remove from visual graph** when inactive +- **Mark as "ghost"** and apply faded styling (opacity 0.6, darker colors) +- **Persist connections** visually even when node is inactive +- **Re-activate** styling when node reappears (audio resumes) + +Tracking strategy: +1. Poll `Client::ListNodes()` periodically (e.g., every 500ms) +2. Compare current list to previous list +3. Nodes that disappeared → mark as ghost, keep in graph +4. Nodes that reappeared → restore active styling + +### Node Type Colors (inspired by potato) +| Type | Base Color | Description | +|------|------------|-------------| +| Hardware Sink/Source | Blue-gray (72, 94, 118) | Physical audio devices | +| Virtual Sink/Source | Green (62, 122, 104) | Virtual nodes created by warppipe | +| Application | Brown/Orange (138, 104, 72) | Ephemeral app audio streams | + +Active vs Ghost: +- **Active**: `Opacity = 1.0f`, lighter gradient, bright font +- **Ghost**: `Opacity = 0.6f`, darker gradient (`.darker(150-180)`), faded font + +### Connection Creation Flow +1. User drags from output port to input port in UI +2. QtNodes calls `WarpGraphModel::addConnection(connectionId)` +3. Model extracts port IDs from connectionId +4. Model calls `warppipe::Client::CreateLink(outputPortId, inputPortId, LinkOptions{})` +5. If success: add to m_connections, connection appears +6. If failure: show error, do NOT add connection to graph + +### Context Menu Actions +- **On canvas**: "Create Virtual Sink", "Create Virtual Source" +- **On virtual node**: "Delete Node" +- **Future**: "Add Routing Rule", "Set as Default", "Edit Properties" + +### Layout Persistence +- Save to `~/.config/warppipe-gui/layout.json` (XDG_CONFIG_HOME) +- Format: + ```json + { + "version": 1, + "nodes": [ + { + "stable_id": "alsa_output.pci-0000_00_1f.3.analog-stereo", + "position": {"x": 100.0, "y": 200.0} + } + ], + "view": { + "scale": 1.0, + "center": {"x": 0.0, "y": 0.0} + } + } + ``` +- Use `NodeInfo.name` as stable_id (unique across sessions) + +--- + +## Qt/CMake Integration + +### CMakeLists.txt additions +```cmake +# After existing warppipe library target: + +option(BUILD_GUI "Build Qt6 GUI application" ON) + +if(BUILD_GUI) + find_package(Qt6 6.2 REQUIRED COMPONENTS Core Widgets) + + set(CMAKE_AUTOMOC ON) + set(CMAKE_AUTORCC ON) + set(CMAKE_AUTOUIC ON) + + include(FetchContent) + FetchContent_Declare( + QtNodes + GIT_REPOSITORY https://github.com/paceholder/nodeeditor + GIT_TAG master + ) + FetchContent_MakeAvailable(QtNodes) + + add_executable(warppipe-gui + gui/main.cpp + gui/WarpGraphModel.cpp + gui/GraphEditorWidget.cpp + gui/NodeStyleHelper.cpp + ) + + target_link_libraries(warppipe-gui PRIVATE + warppipe + Qt6::Widgets + QtNodes + ) + + install(TARGETS warppipe-gui RUNTIME DESTINATION bin) +endif() +``` + +### GUI test target +```cmake +if(BUILD_GUI) + # Model unit tests (no display needed) + add_executable(warppipe-gui-tests + tests/gui/warppipe_gui_tests.cpp + gui/WarpGraphModel.cpp + gui/NodeStyleHelper.cpp + ) + + target_compile_definitions(warppipe-gui-tests PRIVATE WARPPIPE_TESTING) + + target_link_libraries(warppipe-gui-tests PRIVATE + warppipe + Qt6::Widgets + QtNodes + Catch2::Catch2WithMain + ) + + # Screenshot smoke tests (opt-in, need Qt runtime) + option(WARPPIPE_GUI_VISUAL_TESTS "Enable screenshot-based visual tests" OFF) + + if(WARPPIPE_GUI_VISUAL_TESTS) + add_test(NAME gui_screenshot_smoke + COMMAND ${CMAKE_COMMAND} -E env QT_QPA_PLATFORM=offscreen + $ --screenshot ${CMAKE_BINARY_DIR}/test_screenshot.png + ) + set_tests_properties(gui_screenshot_smoke PROPERTIES + LABELS "visual" + TIMEOUT 10 + ) + endif() +endif() +``` + +### Dependencies +- Qt6 >= 6.2 (Core, Widgets) +- Qt6::Test (for GUI test target) +- QtNodes (nodeeditor) — fetched via CMake FetchContent +- Catch2 v3 — fetched via CMake FetchContent (shared with existing tests) +- warppipe library (existing) + +--- + +## Key QtNodes API Usage + +### AbstractGraphModel Interface +```cpp +class WarpGraphModel : public QtNodes::AbstractGraphModel { + Q_OBJECT +public: + // Node management + QtNodes::NodeId newNodeId() override; + std::unordered_set allNodeIds() const override; + bool nodeExists(QtNodes::NodeId) const override; + QtNodes::NodeId addNode(QString const nodeType) override; + bool deleteNode(QtNodes::NodeId) override; + + // Node data (caption, style, position, widget) + QVariant nodeData(QtNodes::NodeId, QtNodes::NodeRole) const override; + bool setNodeData(QtNodes::NodeId, QtNodes::NodeRole, QVariant) override; + + // Port data (count, label, type, connection policy) + QVariant portData(QtNodes::NodeId, QtNodes::PortType, QtNodes::PortIndex, QtNodes::PortRole) const override; + + // Connection management + std::unordered_set allConnectionIds(QtNodes::NodeId) const override; + bool connectionPossible(QtNodes::ConnectionId) const override; + void addConnection(QtNodes::ConnectionId) override; + bool deleteConnection(QtNodes::ConnectionId) override; + + // Custom methods + void refreshFromClient(); // Poll warppipe::Client and sync graph + void markNodeAsGhost(QtNodes::NodeId, bool isGhost); +}; +``` + +### Scene and View Setup +```cpp +// In GraphEditorWidget constructor: +m_model = new WarpGraphModel(client, this); +m_scene = new QtNodes::BasicGraphicsScene(*m_model, this); +m_view = new QtNodes::GraphicsView(m_scene); + +QVBoxLayout *layout = new QVBoxLayout(this); +layout->addWidget(m_view); + +// Connect signals +connect(m_scene, &QtNodes::BasicGraphicsScene::connectionCreated, + this, &GraphEditorWidget::onConnectionCreated); +connect(m_scene, &QtNodes::BasicGraphicsScene::connectionDeleted, + this, &GraphEditorWidget::onConnectionDeleted); +``` + +--- + +## Testing Strategy + +### Three test tiers + +**Tier 1 — Model Unit Tests (no display server)** +Pure logic tests for WarpGraphModel. Use warppipe's WARPPIPE_TESTING helpers to inject fake nodes/ports/links into a Client, then construct the model and assert state. These tests run everywhere (CI, headless, local). + +Coverage targets: +- Node mapping (PipeWire ID → QtNodes NodeId, caption synthesis, style by type) +- Port mapping (is_input orientation, port count, labels) +- Connection mapping (link → ConnectionId, add/delete roundtrip) +- Ghost state (inactive toggle, opacity values, ghost connections) +- Edge cases (duplicate IDs, remove with active connections, empty graph) + +**Tier 2 — Screenshot Smoke Tests (headless, QT_QPA_PLATFORM=offscreen)** +Launch warppipe-gui with `--screenshot` in offscreen mode. Verify: +- Exit code 0 and PNG file produced +- File size heuristic (>10KB = non-trivial content rendered) +- Debug screenshot dir populates on state changes + +Gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF) since they need Qt runtime. + +**Tier 3 — Manual QA with AI-Assisted Debugging** +Run warppipe-gui with a live PipeWire daemon. Use `--debug-screenshot-dir` to capture every state transition. When something looks wrong: +1. Run `warppipe-gui --debug-screenshot-dir /tmp/warppipe-debug/` +2. Reproduce the issue +3. Hand the screenshot directory to the AI for visual analysis + +Checklist: +- All audio nodes appear with correct titles +- Ports on correct sides (input=left, output=right) +- Dragging connections creates PipeWire links +- Virtual sink/source creation via context menu +- Ephemeral nodes fade when inactive, connections persist +- Layout persists across restarts + +### AI Debugging Workflow + +The screenshot infrastructure in Milestone 6 is specifically designed so the AI can debug visual issues: + +```bash +# Quick single screenshot for AI analysis +QT_QPA_PLATFORM=offscreen warppipe-gui --screenshot /tmp/gui-state.png + +# Or use convenience flag +warppipe-gui --offscreen --screenshot /tmp/gui-state.png + +# Continuous debug capture (every state change gets a timestamped screenshot) +warppipe-gui --debug-screenshot-dir /tmp/warppipe-debug/ + +# Then hand to AI: +# "Here are the screenshots from /tmp/warppipe-debug/, the ghost nodes +# aren't fading correctly — can you see what's wrong?" +``` + +The AI can then use its multimodal capabilities to examine the PNGs, compare expected vs actual visual state, and identify styling or layout bugs without needing a live display. + +--- + +## Future Enhancements + +- **Routing rule editor**: Visual panel to add/edit/remove per-app routing rules +- **Audio level meters**: Real-time level monitoring (requires PipeWire param API) +- **Volume/mute controls**: Inline controls on nodes (requires warppipe volume API) +- **Presets**: Save/load full configurations (nodes + links + rules) +- **Search/filter**: Filter nodes by type, name, connection status +- **Minimap**: Overview of entire graph in large setups +- **Themes**: Light/dark mode, custom color schemes +- **Keyboard shortcuts**: Create nodes, delete selection, undo/redo + +--- + +## References + +- **potato project**: ~/Projects/potato/ (reference implementation using QtNodes + PipeWire) +- **QtNodes library**: https://github.com/paceholder/nodeeditor +- **warppipe API**: include/warppipe/warppipe.hpp, docs/api.md +- **Qt6 Widgets**: https://doc.qt.io/qt-6/qtwidgets-index.html