604 lines
30 KiB
Markdown
604 lines
30 KiB
Markdown
# Warppipe GUI Plan (Qt6 Node-Based Audio Router)
|
||
|
||
## Overview
|
||
A Qt6-based node editor GUI for warppipe using the QtNodes (nodeeditor) library. Visualizes PipeWire audio nodes, ports, and links as draggable nodes with connection lines. Supports creating virtual sinks/sources via context menu and displays ephemeral sources with visual fade when inactive.
|
||
|
||
---
|
||
|
||
## Milestones
|
||
|
||
- [x] Milestone 0 - Qt6 Project Setup
|
||
- [x] Create `gui/` subdirectory in warppipe project
|
||
- [x] Add Qt6 + QtNodes to CMakeLists.txt (FetchContent for nodeeditor from github.com/paceholder/nodeeditor)
|
||
- [x] Create `warppipe-gui` target with Qt6::Widgets and QtNodes dependencies
|
||
- [x] Enable CMAKE_AUTOMOC, CMAKE_AUTORCC, CMAKE_AUTOUIC
|
||
- [x] Create minimal main.cpp with QApplication + QMainWindow
|
||
- [x] Verify GUI launches and shows empty window
|
||
|
||
- [x] Milestone 1 - Core Model Integration
|
||
- [x] Create `WarpGraphModel : public QtNodes::AbstractGraphModel`
|
||
- [x] Implement AbstractGraphModel interface (newNodeId, allNodeIds, nodeData, portData, etc.)
|
||
- [x] Add `warppipe::Client*` member, connect to PipeWire on construction
|
||
- [x] Map `warppipe::NodeInfo` to QtNodes NodeId via internal maps (m_nodes, m_pwToNode)
|
||
- [x] Implement node refresh: call Client::ListNodes() and sync graph
|
||
- [x] Create `GraphEditorWidget : public QWidget`
|
||
- [x] Instantiate WarpGraphModel, QtNodes::BasicGraphicsScene, QtNodes::GraphicsView
|
||
- [x] Lay out view in widget
|
||
- [x] Connect model signals to refresh handlers
|
||
- [x] Synthesize display title from NodeInfo:
|
||
- [x] If `application_name` is non-empty and differs from `name`, use `application_name` as title
|
||
- [x] Otherwise use `name` field
|
||
- [x] Store synthesized title in `nodeData(NodeRole::Caption)`
|
||
- [x] Map warppipe ports to QtNodes ports:
|
||
- [x] Input ports (is_input=true) appear on LEFT side of node (QtNodes PortType::In)
|
||
- [x] Output ports (is_input=false) appear on RIGHT side of node (QtNodes PortType::Out)
|
||
- [x] Use port name from PortInfo as port label
|
||
- [x] Verify nodes appear in graph view with correct titles and ports
|
||
|
||
- [x] Milestone 2 - Visual Styling and Node Types
|
||
- [x] Define node type classification based on `media_class`:
|
||
- [x] Sink → "Hardware Sink" (blue-gray base color)
|
||
- [x] Source → "Hardware Source" (blue-gray base color)
|
||
- [x] Virtual sinks created by warppipe → "Virtual Sink" (green base color)
|
||
- [x] Virtual sources created by warppipe → "Virtual Source" (green base color)
|
||
- [x] Application audio streams (ephemeral) → "Application" (brown/orange base color)
|
||
- [x] Implement custom NodeStyle via `nodeData(NodeRole::Style)`:
|
||
- [x] Return QtNodes::NodeStyle::toJson().toVariantMap()
|
||
- [x] Set GradientColor0-3, NormalBoundaryColor, FontColor based on node type
|
||
- [x] Reference potato's `nodeStyleVariant()` function for color scheme
|
||
- [x] Detect ephemeral (application) nodes:
|
||
- [x] Track node appearance/disappearance via Client poll or registry events
|
||
- [x] Mark node as "inactive" if it disappears (no audio playing)
|
||
- [x] Persist inactive nodes in graph model (do NOT remove from visual graph)
|
||
- [x] Apply "ghost" styling to inactive nodes:
|
||
- [x] Set `Opacity = 0.6f` (vs 1.0f for active)
|
||
- [x] Darken gradient colors (use `.darker(150-180)`)
|
||
- [x] Fade font color (lighter gray)
|
||
- [x] Keep connections visible with faded style
|
||
- [x] Verify: Application nodes appear vibrant when active, fade when inactive, never disappear
|
||
|
||
- [x] Milestone 3 - Link Visualization and Drag-Connect
|
||
- [x] Implement connection mapping:
|
||
- [x] Call `Client::ListLinks()` to get existing PipeWire links
|
||
- [x] For each Link, find corresponding NodeId and PortIndex for output/input
|
||
- [x] Create QtNodes::ConnectionId from (outNodeId, outPortType, outPortIndex, inNodeId, inPortType, inPortIndex)
|
||
- [x] Store in model's m_connections set
|
||
- [x] Implement `addConnection(ConnectionId)`:
|
||
- [x] Extract output port and input port from ConnectionId
|
||
- [x] Call `Client::CreateLink(outputPortId, inputPortId, LinkOptions{})`
|
||
- [x] If successful, add connection to m_connections
|
||
- [x] If failed, emit error and do NOT add to graph
|
||
- [x] Implement `deleteConnection(ConnectionId)`:
|
||
- [x] Find corresponding warppipe LinkId from connection
|
||
- [x] Call `Client::RemoveLink(linkId)`
|
||
- [x] Remove from m_connections
|
||
- [x] Verify: Drag connection from output port to input port creates PipeWire link; delete removes it
|
||
|
||
- [x] Milestone 4 - Context Menu and Virtual Node Creation
|
||
- [x] Add context menu to GraphEditorWidget:
|
||
- [x] Right-click on canvas (not on node) shows menu
|
||
- [x] Menu items: "Create Virtual Sink", "Create Virtual Source"
|
||
- [x] Implement "Create Virtual Sink":
|
||
- [x] Prompt user for name (QInputDialog or inline text field)
|
||
- [x] Call `Client::CreateVirtualSink(name, VirtualNodeOptions{})` with default options
|
||
- [x] On success, node appears in graph at context menu position
|
||
- [x] Implement "Create Virtual Source":
|
||
- [x] Same as sink but call `Client::CreateVirtualSource()`
|
||
- [x] Add context menu on nodes:
|
||
- [x] Right-click on virtual node shows "Delete Node" option
|
||
- [x] Call `Client::RemoveNode(nodeId)` and remove from graph
|
||
- [x] Verify: Can create/delete virtual sinks and sources via right-click
|
||
|
||
- [x] Milestone 5 - Layout Persistence and Polish
|
||
- [x] Implement layout save/load:
|
||
- [x] Save node positions to JSON file in `~/.config/warppipe-gui/layout.json`
|
||
- [x] Store by stable ID (use NodeInfo.name as stable key)
|
||
- [x] Save on position change (debounced)
|
||
- [x] Load on startup and restore positions
|
||
- [x] Implement auto-arrange:
|
||
- [x] Menu or button to auto-layout nodes (left-to-right: sources → sinks)
|
||
- [x] Use simple grid or layered layout algorithm
|
||
- [x] Add visual polish:
|
||
- [x] Connection lines styled (color, width, curvature)
|
||
- [x] Highlight connections on hover
|
||
- [x] Port connection points visible and responsive
|
||
- [x] Add status bar:
|
||
- [x] Show connection status to PipeWire daemon
|
||
- [x] Show count of nodes, links
|
||
- [x] Verify: Layout persists across sessions, UI feels responsive and polished
|
||
|
||
- [x] Milestone 6 - Screenshot Infrastructure (AI-Assisted Debugging)
|
||
- [x] Add CLI flags to main.cpp via QCommandLineParser:
|
||
- [x] `--screenshot <path>` / `-s <path>`: Capture window to PNG and exit
|
||
- [x] `--screenshot-delay <ms>`: Configurable render delay before capture (default 800ms)
|
||
- [x] `--debug-screenshot-dir <dir>`: Continuous mode — save timestamped screenshot on every graph state change (node add/remove, connection change, ghost toggle)
|
||
- [x] Implement two-tier QPixmap capture (from potato pattern):
|
||
- [x] Primary: `window.grab()` (renders widget tree to pixmap)
|
||
- [x] Fallback: `screen->grabWindow(window.winId())` if .grab() returns null
|
||
- [x] Exit code 3 on capture failure
|
||
- [x] Add F12 hotkey for interactive screenshot:
|
||
- [x] Save to `$XDG_PICTURES_DIR/warppipe/warppipe_YYYYMMDD_HHmmss.png`
|
||
- [x] Auto-create directory via QDir::mkpath()
|
||
- [x] Implement "render complete" signal:
|
||
- [x] GraphEditorWidget emits `graphReady()` after initial node sync completes
|
||
- [x] Use signal instead of hardcoded delay for --screenshot when possible
|
||
- [x] Fall back to --screenshot-delay if signal doesn't fire within timeout
|
||
- [x] Support headless rendering for CI/AI:
|
||
- [x] Verify screenshots render correctly without a display server
|
||
- [x] Add `--offscreen` convenience flag that sets QT_QPA_PLATFORM=offscreen internally via `qputenv()`
|
||
- [x] Implement debug screenshot naming convention:
|
||
- [x] Format: `warppipe_<timestamp>_<event>.png` (e.g., `warppipe_20260129_143052_node_added.png`)
|
||
- [x] In --debug-screenshot-dir mode, capture on: initial load, node add, node remove, node ghost/unghost, connection add, connection remove
|
||
- [x] Verify: `warppipe-gui --screenshot /tmp/test.png` produces a valid PNG with visible nodes; headless mode works with QT_QPA_PLATFORM=offscreen
|
||
|
||
- [x] Milestone 7 - GUI Tests
|
||
- [x] Create `tests/gui/` directory and `warppipe_gui_tests.cpp` test file
|
||
- [x] Add `warppipe-gui-tests` CMake target linking warppipe, Qt6::Widgets, QtNodes, Catch2
|
||
- [x] Model unit tests (no display server needed, pure logic):
|
||
- [x] WarpGraphModel: inject nodes via WARPPIPE_TESTING helpers → verify allNodeIds(), nodeData(Caption), nodeData(Style)
|
||
- [x] WarpGraphModel: inject ports → verify portData(PortCount), portData(Caption) for correct port labels
|
||
- [x] WarpGraphModel: inject links → verify allConnectionIds(), connectionExists()
|
||
- [x] WarpGraphModel: ghost state tracking — mark node ghost → verify isGhost, mark unghost → verify !isGhost
|
||
- [x] WarpGraphModel: title synthesis — node with description → caption=description; node with application_name="Firefox" → caption="Firefox"; fallback to name
|
||
- [x] WarpGraphModel: port orientation — is_input=true ports map to PortType::In; is_input=false → PortType::Out
|
||
- [x] WarpGraphModel: node removal doesn't crash when connections exist
|
||
- [x] WarpGraphModel: volume meter stream filtering (empty name + app_name skipped)
|
||
- [x] Connection logic tests:
|
||
- [x] Links from client appear as connections in model
|
||
- [x] connectionPossible() rejects invalid port indices and nonexistent nodes
|
||
- [x] deleteConnection() removes from model
|
||
- [x] Screenshot smoke tests (require QT_QPA_PLATFORM=offscreen):
|
||
- [x] Gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF)
|
||
- [x] Launch warppipe-gui with --screenshot → CTest verifies exit code 0
|
||
- [x] Integration tests with warppipe test harness:
|
||
- [x] Create Client with WARPPIPE_TESTING → inject nodes/ports/links → construct WarpGraphModel → verify graph state matches injected data
|
||
- [x] Inject node, then remove → verify ghost state in model
|
||
- [x] Inject node, remove, re-insert with same name → verify ghost reactivation
|
||
- [x] Add CTest integration:
|
||
- [x] Model tests run without display server (always)
|
||
- [x] Screenshot tests gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF)
|
||
- [x] `ctest --test-dir build` runs model + GUI tests
|
||
|
||
- [x] Milestone 8a - Undo/Redo, Clipboard, and Keyboard Shortcuts
|
||
- [x] Integrate `QUndoStack` via `BasicGraphicsScene::undoStack()`
|
||
- [x] Undo/Redo already works for connection create/delete (built-in QtNodes `ConnectCommand`/`DisconnectCommand`)
|
||
- [x] Verify Ctrl+Z / Ctrl+Shift+Z (or Ctrl+Y) work out of the box for connections
|
||
- [x] Implement `DeleteVirtualNodeCommand : QUndoCommand`
|
||
- [x] `redo()`: destroy virtual node via `Client::RemoveNode()`
|
||
- [x] `undo()`: re-create virtual node via `Client::CreateVirtualSink/Source()` with same name/channels/rate
|
||
- [x] Store node position and restore on undo
|
||
- [x] Implement `deleteSelection()` for Del key
|
||
- [x] Collect selected `NodeGraphicsObject` items from `m_scene->selectedItems()`
|
||
- [x] Virtual nodes → push `DeleteVirtualNodeCommand` onto undo stack
|
||
- [x] Non-virtual nodes → push `QtNodes::DeleteCommand` (removes from graph only, not PipeWire)
|
||
- [x] Connection-only selection → push `QtNodes::DeleteCommand`
|
||
- [x] Implement `copySelection()` (Ctrl+C)
|
||
- [x] Serialize selected virtual nodes to JSON: stable_id, name, media_class, channels, rate, position
|
||
- [x] Include links between selected nodes (source stable_id:port_name → target stable_id:port_name)
|
||
- [x] Set `QClipboard` with custom MIME type `application/warppipe-virtual-graph`
|
||
- [x] Implement `pasteSelection()` (Ctrl+V)
|
||
- [x] Parse clipboard JSON, create new virtual nodes with " Copy" name suffix
|
||
- [x] Position pasted nodes at offset from originals
|
||
- [x] Deferred link resolution via `PendingPasteLink` queue (nodes may not exist yet)
|
||
- [x] `tryResolvePendingLinks()` called on node add to wire up deferred links
|
||
- [x] Implement `duplicateSelection()` (Ctrl+D) — copy + paste with (40, 40) offset
|
||
- [x] Register keyboard shortcuts on `m_view`:
|
||
- [x] Del → `deleteSelection()`
|
||
- [x] Ctrl+C → `copySelection()`
|
||
- [x] Ctrl+V → `pasteSelection()`
|
||
- [x] Ctrl+D → `duplicateSelection()`
|
||
- [x] Ctrl+L → auto-arrange + zoom fit
|
||
- [x] Remove default QtNodes copy/paste actions to avoid conflicts
|
||
- [x] Add tests for undo/redo command state (push command → undo → verify node re-created → redo → verify deleted)
|
||
- [ ] Milestone 8b - View and Layout Enhancements
|
||
- [ ] Add "Zoom Fit All" context menu action → `m_view->zoomFitAll()`
|
||
- [ ] Add "Zoom Fit Selected" context menu action → `m_view->zoomFitSelected()`
|
||
- [ ] Add "Save Layout As..." context menu action
|
||
- [ ] `QFileDialog::getSaveFileName()` → save layout JSON to custom path
|
||
- [ ] Reuse existing `saveLayout()` serialization, write to chosen path
|
||
- [ ] Add "Reset Layout" context menu action
|
||
- [ ] Clear saved positions, run `autoArrange()`, save, zoom fit
|
||
- [ ] Add "Refresh Graph" context menu action
|
||
- [ ] Reset model, re-sync from client, zoom fit
|
||
- [ ] Persist view state in layout JSON:
|
||
- [ ] Save view scale + center position (`m_view->getScale()`, `m_view->mapToScene(viewport center)`)
|
||
- [ ] Restore on load: `m_view->setupScale()` + `m_view->centerOn()`
|
||
- [ ] Fallback to `zoomFitAll()` when no saved view state
|
||
- [ ] Persist ghost nodes in layout JSON:
|
||
- [ ] Serialize ghost node stable_id, name, description, input/output ports (id + name), position
|
||
- [ ] Serialize ghost connections (out_stable_id, out_port_index, in_stable_id, in_port_index)
|
||
- [ ] Restore ghosts from layout on load (before live sync)
|
||
- [ ] Add middle-click center: `eventFilter` on viewport catches `MiddleButton` → `m_view->centerOn(mapToScene(pos))`
|
||
- [ ] Add tests for view state save/load round-trip and ghost persistence
|
||
- [ ] Milestone 8c - Sidebar and Preset System
|
||
- [ ] Add `QSplitter` between graph view and sidebar panel
|
||
- [ ] Graph view (stretch factor 1) on left, sidebar (stretch factor 0) on right
|
||
- [ ] Persist splitter sizes in layout JSON, restore on load
|
||
- [ ] Default sizes: graph 1200, sidebar 320
|
||
- [ ] Add `QTabWidget` sidebar with styled tabs (dark theme)
|
||
- [ ] Tab styling: dark background, selected tab has accent underline
|
||
- [ ] Initially one tab: "PRESETS" (meters/mixer tabs added in M8d/M8e)
|
||
- [ ] Implement `PresetManager` class:
|
||
- [ ] `savePreset(path)` → serialize to JSON:
|
||
- [ ] Virtual devices: name, description, media_class, channels, rate
|
||
- [ ] Routing: links by stable_id:port_name pairs
|
||
- [ ] UI layout: node positions, view state
|
||
- [ ] `loadPreset(path)` → apply from JSON:
|
||
- [ ] Create missing virtual devices
|
||
- [ ] Re-create links from routing entries
|
||
- [ ] Apply layout positions
|
||
- [ ] Save on quit via `QCoreApplication::aboutToQuit` signal
|
||
- [ ] Add "Save Preset..." context menu action → `QFileDialog::getSaveFileName()`
|
||
- [ ] Add "Load Preset..." context menu action → `QFileDialog::getOpenFileName()`
|
||
- [ ] Add tests for preset save/load round-trip
|
||
- [ ] Milestone 8d - Volume/Mute Controls (requires core API: `SetNodeVolume()`)
|
||
- [ ] Add `NodeVolumeState` struct: `{ float volume; bool mute; }`
|
||
- [ ] Add `ClickSlider : QSlider` — click jumps to position instead of page-stepping
|
||
- [ ] Add inline volume widget per node via `nodeData(NodeRole::Widget)`:
|
||
- [ ] Horizontal `ClickSlider` (0-100) + mute `QToolButton`
|
||
- [ ] Calls `Client::SetNodeVolume(nodeId, volume, mute)` on change
|
||
- [ ] Styled: dark background, green slider fill, rounded mute button
|
||
- [ ] Implement `VolumeChangeCommand : QUndoCommand`
|
||
- [ ] Stores previous + next `NodeVolumeState`, node ID
|
||
- [ ] `undo()` → apply previous state; `redo()` → apply next state
|
||
- [ ] Push on slider release or mute toggle (not during drag)
|
||
- [ ] Track volume states in model: `QHash<uint32_t, NodeVolumeState> m_nodeVolumeState`
|
||
- [ ] `setNodeVolumeState()` — update state + sync inline widget
|
||
- [ ] `nodeVolumeState()` — read current state
|
||
- [ ] Emit `nodeVolumeChanged(nodeId, previous, current)` signal
|
||
- [ ] Add "MIXER" tab to sidebar `QTabWidget`:
|
||
- [ ] `QScrollArea` with horizontal layout of channel strips
|
||
- [ ] Per-node strip: `AudioLevelMeter` + vertical `ClickSlider` (fader) + Mute (M) + Solo (S) buttons + node label
|
||
- [ ] Solo logic: when any node is soloed, all non-soloed nodes are muted
|
||
- [ ] Volume fader changes push `VolumeChangeCommand` onto undo stack
|
||
- [ ] `refreshMixerStrip()` — create strip when node appears
|
||
- [ ] `removeMixerStrip()` — destroy strip when node removed
|
||
- [ ] `updateMixerState()` — sync fader/mute from model state
|
||
- [ ] Include volume/mute states in preset save/load (`persistent_volumes`, `persistent_mutes`)
|
||
- [ ] Add tests for VolumeChangeCommand undo/redo and mixer strip lifecycle
|
||
- [ ] Milestone 8e - Audio Level Meters (requires core API: `MeterPeak()`, `NodeMeterPeak()`, `EnsureNodeMeter()`)
|
||
- [ ] Implement `AudioLevelMeter : QWidget`
|
||
- [ ] Custom `paintEvent`: vertical bar from bottom, background `(24,24,28)`
|
||
- [ ] Color thresholds: green (0-0.7), yellow (0.7-0.9), red (0.9-1.0)
|
||
- [ ] Peak hold indicator: white horizontal line, holds 6 frames then decays at 0.02/frame
|
||
- [ ] `setLevel(float)` — clamp 0-1, update hold, call `update()`
|
||
- [ ] `sizeHint()` → 40×160
|
||
- [ ] Add "METERS" tab to sidebar `QTabWidget`:
|
||
- [ ] "MASTER OUTPUT" label + master `AudioLevelMeter`
|
||
- [ ] "NODE METERS" label + scrollable list of per-node meter rows
|
||
- [ ] Per-node row: elided label + compact `AudioLevelMeter` (fixed 26px wide, min 70px tall)
|
||
- [ ] Add 30fps meter update timer (33ms, `Qt::PreciseTimer`)
|
||
- [ ] Poll `Client::MeterPeak()` → master meter
|
||
- [ ] Poll `Client::NodeMeterPeak(nodeId)` → per-node meters + mixer meters
|
||
- [ ] Skip updates when widget is not visible (`isVisible()` check)
|
||
- [ ] Auto-manage per-node meters:
|
||
- [ ] Create meter when node has active links (`ensureNodeMeter()`)
|
||
- [ ] Remove meter when node removed or all links removed (`removeNodeMeter()`)
|
||
- [ ] Skip meter nodes (filter by name prefix)
|
||
- [ ] Add tests for AudioLevelMeter level clamping, hold/decay logic
|
||
- [ ] Milestone 8f (Optional) - Architecture and Routing Rules
|
||
- [ ] Event-driven updates: replace 500ms polling with signal/slot if core adds registry callbacks
|
||
- [ ] `nodeAdded(NodeInfo)`, `nodeRemoved(uint32_t)`, `nodeChanged(NodeInfo)`
|
||
- [ ] `linkAdded(LinkInfo)`, `linkRemoved(uint32_t)`
|
||
- [ ] Keep polling as fallback if signals not available
|
||
- [ ] Link intent system: remember intended links by stable key, restore when nodes reappear
|
||
- [ ] `rememberLinkIntent(LinkInfo)` — store stable_id:port_name pairs
|
||
- [ ] `tryRestoreLinks()` — called on node add, resolves stored intents
|
||
- [ ] Persist link intents in layout JSON
|
||
- [ ] Add routing rule UI (separate panel or dialog)
|
||
- [ ] List existing rules from `Client::ListRouteRules()`
|
||
- [ ] Add/remove rules with RuleMatch fields
|
||
- [ ] Show which nodes are affected by rules
|
||
|
||
---
|
||
|
||
## Architecture
|
||
|
||
```
|
||
warppipe/
|
||
├── CMakeLists.txt # Add Qt6 + QtNodes, warppipe-gui + test targets
|
||
├── gui/
|
||
│ ├── main.cpp # QApplication entry, CLI flags (--screenshot, --debug-screenshot-dir, --offscreen)
|
||
│ ├── WarpGraphModel.h # QtNodes::AbstractGraphModel implementation
|
||
│ ├── WarpGraphModel.cpp # Model logic, warppipe::Client integration
|
||
│ ├── GraphEditorWidget.h # Main UI widget with scene + view
|
||
│ ├── GraphEditorWidget.cpp # Event handlers, context menus, refresh logic, graphReady() signal
|
||
│ ├── NodeStyleHelper.h/cpp # Node styling functions (colors, ghost mode)
|
||
│ └── ScreenshotHelper.h/cpp # QPixmap capture, debug dir, naming conventions
|
||
├── tests/
|
||
│ ├── warppipe_tests.cpp # Existing library tests
|
||
│ └── gui/
|
||
│ └── warppipe_gui_tests.cpp # Model unit tests + screenshot smoke tests
|
||
├── include/warppipe/
|
||
│ └── warppipe.hpp # No changes needed
|
||
└── src/
|
||
└── warppipe.cpp # No changes needed
|
||
```
|
||
|
||
---
|
||
|
||
## Design Notes
|
||
|
||
### Node Title Synthesis
|
||
Display title priority: `description` > `application_name` > `name`
|
||
- **Hardware/Virtual nodes**: Use `description` (PW_KEY_NODE_DESCRIPTION), e.g., "Speakers", "Headphones"
|
||
- **Application nodes**: Use `application_name` if non-empty (e.g., "Firefox", "Spotify")
|
||
- Fallback to `name` if both are empty
|
||
|
||
### Port Orientation
|
||
- **Input ports** (is_input=true): LEFT side of node (QtNodes::PortType::In)
|
||
- **Output ports** (is_input=false): RIGHT side of node (QtNodes::PortType::Out)
|
||
|
||
### Ephemeral Node Handling
|
||
Application audio streams are ephemeral — they appear when an app plays audio and can disappear when stopped.
|
||
- **Do NOT remove from visual graph** when inactive
|
||
- **Mark as "ghost"** and apply faded styling (opacity 0.6, darker colors)
|
||
- **Persist connections** visually even when node is inactive
|
||
- **Re-activate** styling when node reappears (audio resumes)
|
||
|
||
Tracking strategy:
|
||
1. Poll `Client::ListNodes()` periodically (e.g., every 500ms)
|
||
2. Compare current list to previous list
|
||
3. Nodes that disappeared → mark as ghost, keep in graph
|
||
4. Nodes that reappeared → restore active styling
|
||
|
||
### Node Type Colors (inspired by potato)
|
||
| Type | Base Color | Description |
|
||
|------|------------|-------------|
|
||
| Hardware Sink/Source | Blue-gray (72, 94, 118) | Physical audio devices |
|
||
| Virtual Sink/Source | Green (62, 122, 104) | Virtual nodes created by warppipe |
|
||
| Application | Brown/Orange (138, 104, 72) | Ephemeral app audio streams |
|
||
|
||
Active vs Ghost:
|
||
- **Active**: `Opacity = 1.0f`, lighter gradient, bright font
|
||
- **Ghost**: `Opacity = 0.6f`, darker gradient (`.darker(150-180)`), faded font
|
||
|
||
### Connection Creation Flow
|
||
1. User drags from output port to input port in UI
|
||
2. QtNodes calls `WarpGraphModel::addConnection(connectionId)`
|
||
3. Model extracts port IDs from connectionId
|
||
4. Model calls `warppipe::Client::CreateLink(outputPortId, inputPortId, LinkOptions{})`
|
||
5. If success: add to m_connections, connection appears
|
||
6. If failure: show error, do NOT add connection to graph
|
||
|
||
### Context Menu Actions
|
||
- **On canvas**: "Create Virtual Sink", "Create Virtual Source"
|
||
- **On virtual node**: "Delete Node"
|
||
- **Future**: "Add Routing Rule", "Set as Default", "Edit Properties"
|
||
|
||
### Layout Persistence
|
||
- Save to `~/.config/warppipe-gui/layout.json` (XDG_CONFIG_HOME)
|
||
- Format:
|
||
```json
|
||
{
|
||
"version": 1,
|
||
"nodes": [
|
||
{
|
||
"stable_id": "alsa_output.pci-0000_00_1f.3.analog-stereo",
|
||
"position": {"x": 100.0, "y": 200.0}
|
||
}
|
||
],
|
||
"view": {
|
||
"scale": 1.0,
|
||
"center": {"x": 0.0, "y": 0.0}
|
||
}
|
||
}
|
||
```
|
||
- Use `NodeInfo.name` as stable_id (unique across sessions)
|
||
|
||
---
|
||
|
||
## Qt/CMake Integration
|
||
|
||
### CMakeLists.txt additions
|
||
```cmake
|
||
# After existing warppipe library target:
|
||
|
||
option(BUILD_GUI "Build Qt6 GUI application" ON)
|
||
|
||
if(BUILD_GUI)
|
||
find_package(Qt6 6.2 REQUIRED COMPONENTS Core Widgets)
|
||
|
||
set(CMAKE_AUTOMOC ON)
|
||
set(CMAKE_AUTORCC ON)
|
||
set(CMAKE_AUTOUIC ON)
|
||
|
||
include(FetchContent)
|
||
FetchContent_Declare(
|
||
QtNodes
|
||
GIT_REPOSITORY https://github.com/paceholder/nodeeditor
|
||
GIT_TAG master
|
||
)
|
||
FetchContent_MakeAvailable(QtNodes)
|
||
|
||
add_executable(warppipe-gui
|
||
gui/main.cpp
|
||
gui/WarpGraphModel.cpp
|
||
gui/GraphEditorWidget.cpp
|
||
gui/NodeStyleHelper.cpp
|
||
)
|
||
|
||
target_link_libraries(warppipe-gui PRIVATE
|
||
warppipe
|
||
Qt6::Widgets
|
||
QtNodes
|
||
)
|
||
|
||
install(TARGETS warppipe-gui RUNTIME DESTINATION bin)
|
||
endif()
|
||
```
|
||
|
||
### GUI test target
|
||
```cmake
|
||
if(BUILD_GUI)
|
||
# Model unit tests (no display needed)
|
||
add_executable(warppipe-gui-tests
|
||
tests/gui/warppipe_gui_tests.cpp
|
||
gui/WarpGraphModel.cpp
|
||
gui/NodeStyleHelper.cpp
|
||
)
|
||
|
||
target_compile_definitions(warppipe-gui-tests PRIVATE WARPPIPE_TESTING)
|
||
|
||
target_link_libraries(warppipe-gui-tests PRIVATE
|
||
warppipe
|
||
Qt6::Widgets
|
||
QtNodes
|
||
Catch2::Catch2WithMain
|
||
)
|
||
|
||
# Screenshot smoke tests (opt-in, need Qt runtime)
|
||
option(WARPPIPE_GUI_VISUAL_TESTS "Enable screenshot-based visual tests" OFF)
|
||
|
||
if(WARPPIPE_GUI_VISUAL_TESTS)
|
||
add_test(NAME gui_screenshot_smoke
|
||
COMMAND ${CMAKE_COMMAND} -E env QT_QPA_PLATFORM=offscreen
|
||
$<TARGET_FILE:warppipe-gui> --screenshot ${CMAKE_BINARY_DIR}/test_screenshot.png
|
||
)
|
||
set_tests_properties(gui_screenshot_smoke PROPERTIES
|
||
LABELS "visual"
|
||
TIMEOUT 10
|
||
)
|
||
endif()
|
||
endif()
|
||
```
|
||
|
||
### Dependencies
|
||
- Qt6 >= 6.2 (Core, Widgets)
|
||
- Qt6::Test (for GUI test target)
|
||
- QtNodes (nodeeditor) — fetched via CMake FetchContent
|
||
- Catch2 v3 — fetched via CMake FetchContent (shared with existing tests)
|
||
- warppipe library (existing)
|
||
|
||
---
|
||
|
||
## Key QtNodes API Usage
|
||
|
||
### AbstractGraphModel Interface
|
||
```cpp
|
||
class WarpGraphModel : public QtNodes::AbstractGraphModel {
|
||
Q_OBJECT
|
||
public:
|
||
// Node management
|
||
QtNodes::NodeId newNodeId() override;
|
||
std::unordered_set<QtNodes::NodeId> allNodeIds() const override;
|
||
bool nodeExists(QtNodes::NodeId) const override;
|
||
QtNodes::NodeId addNode(QString const nodeType) override;
|
||
bool deleteNode(QtNodes::NodeId) override;
|
||
|
||
// Node data (caption, style, position, widget)
|
||
QVariant nodeData(QtNodes::NodeId, QtNodes::NodeRole) const override;
|
||
bool setNodeData(QtNodes::NodeId, QtNodes::NodeRole, QVariant) override;
|
||
|
||
// Port data (count, label, type, connection policy)
|
||
QVariant portData(QtNodes::NodeId, QtNodes::PortType, QtNodes::PortIndex, QtNodes::PortRole) const override;
|
||
|
||
// Connection management
|
||
std::unordered_set<QtNodes::ConnectionId> allConnectionIds(QtNodes::NodeId) const override;
|
||
bool connectionPossible(QtNodes::ConnectionId) const override;
|
||
void addConnection(QtNodes::ConnectionId) override;
|
||
bool deleteConnection(QtNodes::ConnectionId) override;
|
||
|
||
// Custom methods
|
||
void refreshFromClient(); // Poll warppipe::Client and sync graph
|
||
void markNodeAsGhost(QtNodes::NodeId, bool isGhost);
|
||
};
|
||
```
|
||
|
||
### Scene and View Setup
|
||
```cpp
|
||
// In GraphEditorWidget constructor:
|
||
m_model = new WarpGraphModel(client, this);
|
||
m_scene = new QtNodes::BasicGraphicsScene(*m_model, this);
|
||
m_view = new QtNodes::GraphicsView(m_scene);
|
||
|
||
QVBoxLayout *layout = new QVBoxLayout(this);
|
||
layout->addWidget(m_view);
|
||
|
||
// Connect signals
|
||
connect(m_scene, &QtNodes::BasicGraphicsScene::connectionCreated,
|
||
this, &GraphEditorWidget::onConnectionCreated);
|
||
connect(m_scene, &QtNodes::BasicGraphicsScene::connectionDeleted,
|
||
this, &GraphEditorWidget::onConnectionDeleted);
|
||
```
|
||
|
||
---
|
||
|
||
## Testing Strategy
|
||
|
||
### Three test tiers
|
||
|
||
**Tier 1 — Model Unit Tests (no display server)**
|
||
Pure logic tests for WarpGraphModel. Use warppipe's WARPPIPE_TESTING helpers to inject fake nodes/ports/links into a Client, then construct the model and assert state. These tests run everywhere (CI, headless, local).
|
||
|
||
Coverage targets:
|
||
- Node mapping (PipeWire ID → QtNodes NodeId, caption synthesis, style by type)
|
||
- Port mapping (is_input orientation, port count, labels)
|
||
- Connection mapping (link → ConnectionId, add/delete roundtrip)
|
||
- Ghost state (inactive toggle, opacity values, ghost connections)
|
||
- Edge cases (duplicate IDs, remove with active connections, empty graph)
|
||
|
||
**Tier 2 — Screenshot Smoke Tests (headless, QT_QPA_PLATFORM=offscreen)**
|
||
Launch warppipe-gui with `--screenshot` in offscreen mode. Verify:
|
||
- Exit code 0 and PNG file produced
|
||
- File size heuristic (>10KB = non-trivial content rendered)
|
||
- Debug screenshot dir populates on state changes
|
||
|
||
Gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF) since they need Qt runtime.
|
||
|
||
**Tier 3 — Manual QA with AI-Assisted Debugging**
|
||
Run warppipe-gui with a live PipeWire daemon. Use `--debug-screenshot-dir` to capture every state transition. When something looks wrong:
|
||
1. Run `warppipe-gui --debug-screenshot-dir /tmp/warppipe-debug/`
|
||
2. Reproduce the issue
|
||
3. Hand the screenshot directory to the AI for visual analysis
|
||
|
||
Checklist:
|
||
- All audio nodes appear with correct titles
|
||
- Ports on correct sides (input=left, output=right)
|
||
- Dragging connections creates PipeWire links
|
||
- Virtual sink/source creation via context menu
|
||
- Ephemeral nodes fade when inactive, connections persist
|
||
- Layout persists across restarts
|
||
|
||
### AI Debugging Workflow
|
||
|
||
The screenshot infrastructure in Milestone 6 is specifically designed so the AI can debug visual issues:
|
||
|
||
```bash
|
||
# Quick single screenshot for AI analysis
|
||
QT_QPA_PLATFORM=offscreen warppipe-gui --screenshot /tmp/gui-state.png
|
||
|
||
# Or use convenience flag
|
||
warppipe-gui --offscreen --screenshot /tmp/gui-state.png
|
||
|
||
# Continuous debug capture (every state change gets a timestamped screenshot)
|
||
warppipe-gui --debug-screenshot-dir /tmp/warppipe-debug/
|
||
|
||
# Then hand to AI:
|
||
# "Here are the screenshots from /tmp/warppipe-debug/, the ghost nodes
|
||
# aren't fading correctly — can you see what's wrong?"
|
||
```
|
||
|
||
The AI can then use its multimodal capabilities to examine the PNGs, compare expected vs actual visual state, and identify styling or layout bugs without needing a live display.
|
||
|
||
---
|
||
|
||
## Future Enhancements
|
||
|
||
- **Routing rule editor**: Visual panel to add/edit/remove per-app routing rules
|
||
- **Audio level meters**: Real-time level monitoring (requires PipeWire param API)
|
||
- **Volume/mute controls**: Inline controls on nodes (requires warppipe volume API)
|
||
- **Presets**: Save/load full configurations (nodes + links + rules)
|
||
- **Search/filter**: Filter nodes by type, name, connection status
|
||
- **Minimap**: Overview of entire graph in large setups
|
||
- **Themes**: Light/dark mode, custom color schemes
|
||
- **Keyboard shortcuts**: Create nodes, delete selection, undo/redo
|
||
|
||
---
|
||
|
||
## References
|
||
|
||
- **potato project**: ~/Projects/potato/ (reference implementation using QtNodes + PipeWire)
|
||
- **QtNodes library**: https://github.com/paceholder/nodeeditor
|
||
- **warppipe API**: include/warppipe/warppipe.hpp, docs/api.md
|
||
- **Qt6 Widgets**: https://doc.qt.io/qt-6/qtwidgets-index.html
|