warp-pipe/GUI_PLAN.md
2026-01-31 11:21:28 -07:00

603 lines
30 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# Warp Pipe GUI Plan (Qt6 Node-Based Audio Router)
## Overview
A Qt6-based node editor GUI for warppipe using the QtNodes (nodeeditor) library. Visualizes PipeWire audio nodes, ports, and links as draggable nodes with connection lines. Supports creating virtual sinks/sources via context menu and displays ephemeral sources with visual fade when inactive.
---
## Milestones
- [x] Milestone 0 - Qt6 Project Setup
- [x] Create `gui/` subdirectory in warppipe project
- [x] Add Qt6 + QtNodes to CMakeLists.txt (FetchContent for nodeeditor from github.com/paceholder/nodeeditor)
- [x] Create `warppipe-gui` target with Qt6::Widgets and QtNodes dependencies
- [x] Enable CMAKE_AUTOMOC, CMAKE_AUTORCC, CMAKE_AUTOUIC
- [x] Create minimal main.cpp with QApplication + QMainWindow
- [x] Verify GUI launches and shows empty window
- [x] Milestone 1 - Core Model Integration
- [x] Create `WarpGraphModel : public QtNodes::AbstractGraphModel`
- [x] Implement AbstractGraphModel interface (newNodeId, allNodeIds, nodeData, portData, etc.)
- [x] Add `warppipe::Client*` member, connect to PipeWire on construction
- [x] Map `warppipe::NodeInfo` to QtNodes NodeId via internal maps (m_nodes, m_pwToNode)
- [x] Implement node refresh: call Client::ListNodes() and sync graph
- [x] Create `GraphEditorWidget : public QWidget`
- [x] Instantiate WarpGraphModel, QtNodes::BasicGraphicsScene, QtNodes::GraphicsView
- [x] Lay out view in widget
- [x] Connect model signals to refresh handlers
- [x] Synthesize display title from NodeInfo:
- [x] If `application_name` is non-empty and differs from `name`, use `application_name` as title
- [x] Otherwise use `name` field
- [x] Store synthesized title in `nodeData(NodeRole::Caption)`
- [x] Map warppipe ports to QtNodes ports:
- [x] Input ports (is_input=true) appear on LEFT side of node (QtNodes PortType::In)
- [x] Output ports (is_input=false) appear on RIGHT side of node (QtNodes PortType::Out)
- [x] Use port name from PortInfo as port label
- [x] Verify nodes appear in graph view with correct titles and ports
- [x] Milestone 2 - Visual Styling and Node Types
- [x] Define node type classification based on `media_class`:
- [x] Sink → "Hardware Sink" (blue-gray base color)
- [x] Source → "Hardware Source" (blue-gray base color)
- [x] Virtual sinks created by warppipe → "Virtual Sink" (green base color)
- [x] Virtual sources created by warppipe → "Virtual Source" (green base color)
- [x] Application audio streams (ephemeral) → "Application" (brown/orange base color)
- [x] Implement custom NodeStyle via `nodeData(NodeRole::Style)`:
- [x] Return QtNodes::NodeStyle::toJson().toVariantMap()
- [x] Set GradientColor0-3, NormalBoundaryColor, FontColor based on node type
- [x] Reference potato's `nodeStyleVariant()` function for color scheme
- [x] Detect ephemeral (application) nodes:
- [x] Track node appearance/disappearance via Client poll or registry events
- [x] Mark node as "inactive" if it disappears (no audio playing)
- [x] Persist inactive nodes in graph model (do NOT remove from visual graph)
- [x] Apply "ghost" styling to inactive nodes:
- [x] Set `Opacity = 0.6f` (vs 1.0f for active)
- [x] Darken gradient colors (use `.darker(150-180)`)
- [x] Fade font color (lighter gray)
- [x] Keep connections visible with faded style
- [x] Verify: Application nodes appear vibrant when active, fade when inactive, never disappear
- [x] Milestone 3 - Link Visualization and Drag-Connect
- [x] Implement connection mapping:
- [x] Call `Client::ListLinks()` to get existing PipeWire links
- [x] For each Link, find corresponding NodeId and PortIndex for output/input
- [x] Create QtNodes::ConnectionId from (outNodeId, outPortType, outPortIndex, inNodeId, inPortType, inPortIndex)
- [x] Store in model's m_connections set
- [x] Implement `addConnection(ConnectionId)`:
- [x] Extract output port and input port from ConnectionId
- [x] Call `Client::CreateLink(outputPortId, inputPortId, LinkOptions{})`
- [x] If successful, add connection to m_connections
- [x] If failed, emit error and do NOT add to graph
- [x] Implement `deleteConnection(ConnectionId)`:
- [x] Find corresponding warppipe LinkId from connection
- [x] Call `Client::RemoveLink(linkId)`
- [x] Remove from m_connections
- [x] Verify: Drag connection from output port to input port creates PipeWire link; delete removes it
- [x] Milestone 4 - Context Menu and Virtual Node Creation
- [x] Add context menu to GraphEditorWidget:
- [x] Right-click on canvas (not on node) shows menu
- [x] Menu items: "Create Virtual Sink", "Create Virtual Source"
- [x] Implement "Create Virtual Sink":
- [x] Prompt user for name (QInputDialog or inline text field)
- [x] Call `Client::CreateVirtualSink(name, VirtualNodeOptions{})` with default options
- [x] On success, node appears in graph at context menu position
- [x] Implement "Create Virtual Source":
- [x] Same as sink but call `Client::CreateVirtualSource()`
- [x] Add context menu on nodes:
- [x] Right-click on virtual node shows "Delete Node" option
- [x] Call `Client::RemoveNode(nodeId)` and remove from graph
- [x] Verify: Can create/delete virtual sinks and sources via right-click
- [x] Milestone 5 - Layout Persistence and Polish
- [x] Implement layout save/load:
- [x] Save node positions to JSON file in `~/.config/warppipe-gui/layout.json`
- [x] Store by stable ID (use NodeInfo.name as stable key)
- [x] Save on position change (debounced)
- [x] Load on startup and restore positions
- [x] Implement auto-arrange:
- [x] Menu or button to auto-layout nodes (left-to-right: sources → sinks)
- [x] Use simple grid or layered layout algorithm
- [x] Add visual polish:
- [x] Connection lines styled (color, width, curvature)
- [x] Highlight connections on hover
- [x] Port connection points visible and responsive
- [x] Add status bar:
- [x] Show connection status to PipeWire daemon
- [x] Show count of nodes, links
- [x] Verify: Layout persists across sessions, UI feels responsive and polished
- [x] Milestone 6 - Screenshot Infrastructure (AI-Assisted Debugging)
- [x] Add CLI flags to main.cpp via QCommandLineParser:
- [x] `--screenshot <path>` / `-s <path>`: Capture window to PNG and exit
- [x] `--screenshot-delay <ms>`: Configurable render delay before capture (default 800ms)
- [x] `--debug-screenshot-dir <dir>`: Continuous mode — save timestamped screenshot on every graph state change (node add/remove, connection change, ghost toggle)
- [x] Implement two-tier QPixmap capture (from potato pattern):
- [x] Primary: `window.grab()` (renders widget tree to pixmap)
- [x] Fallback: `screen->grabWindow(window.winId())` if .grab() returns null
- [x] Exit code 3 on capture failure
- [x] Add F12 hotkey for interactive screenshot:
- [x] Save to `$XDG_PICTURES_DIR/warppipe/warppipe_YYYYMMDD_HHmmss.png`
- [x] Auto-create directory via QDir::mkpath()
- [x] Implement "render complete" signal:
- [x] GraphEditorWidget emits `graphReady()` after initial node sync completes
- [x] Use signal instead of hardcoded delay for --screenshot when possible
- [x] Fall back to --screenshot-delay if signal doesn't fire within timeout
- [x] Support headless rendering for CI/AI:
- [x] Verify screenshots render correctly without a display server
- [x] Add `--offscreen` convenience flag that sets QT_QPA_PLATFORM=offscreen internally via `qputenv()`
- [x] Implement debug screenshot naming convention:
- [x] Format: `warppipe_<timestamp>_<event>.png` (e.g., `warppipe_20260129_143052_node_added.png`)
- [x] In --debug-screenshot-dir mode, capture on: initial load, node add, node remove, node ghost/unghost, connection add, connection remove
- [x] Verify: `warppipe-gui --screenshot /tmp/test.png` produces a valid PNG with visible nodes; headless mode works with QT_QPA_PLATFORM=offscreen
- [x] Milestone 7 - GUI Tests
- [x] Create `tests/gui/` directory and `warppipe_gui_tests.cpp` test file
- [x] Add `warppipe-gui-tests` CMake target linking warppipe, Qt6::Widgets, QtNodes, Catch2
- [x] Model unit tests (no display server needed, pure logic):
- [x] WarpGraphModel: inject nodes via WARPPIPE_TESTING helpers → verify allNodeIds(), nodeData(Caption), nodeData(Style)
- [x] WarpGraphModel: inject ports → verify portData(PortCount), portData(Caption) for correct port labels
- [x] WarpGraphModel: inject links → verify allConnectionIds(), connectionExists()
- [x] WarpGraphModel: ghost state tracking — mark node ghost → verify isGhost, mark unghost → verify !isGhost
- [x] WarpGraphModel: title synthesis — node with description → caption=description; node with application_name="Firefox" → caption="Firefox"; fallback to name
- [x] WarpGraphModel: port orientation — is_input=true ports map to PortType::In; is_input=false → PortType::Out
- [x] WarpGraphModel: node removal doesn't crash when connections exist
- [x] WarpGraphModel: volume meter stream filtering (empty name + app_name skipped)
- [x] Connection logic tests:
- [x] Links from client appear as connections in model
- [x] connectionPossible() rejects invalid port indices and nonexistent nodes
- [x] deleteConnection() removes from model
- [x] Screenshot smoke tests (require QT_QPA_PLATFORM=offscreen):
- [x] Gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF)
- [x] Launch warppipe-gui with --screenshot → CTest verifies exit code 0
- [x] Integration tests with warppipe test harness:
- [x] Create Client with WARPPIPE_TESTING → inject nodes/ports/links → construct WarpGraphModel → verify graph state matches injected data
- [x] Inject node, then remove → verify ghost state in model
- [x] Inject node, remove, re-insert with same name → verify ghost reactivation
- [x] Add CTest integration:
- [x] Model tests run without display server (always)
- [x] Screenshot tests gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF)
- [x] `ctest --test-dir build` runs model + GUI tests
- [x] Milestone 8a - Undo/Redo, Clipboard, and Keyboard Shortcuts
- [x] Integrate `QUndoStack` via `BasicGraphicsScene::undoStack()`
- [x] Undo/Redo already works for connection create/delete (built-in QtNodes `ConnectCommand`/`DisconnectCommand`)
- [x] Verify Ctrl+Z / Ctrl+Shift+Z (or Ctrl+Y) work out of the box for connections
- [x] Implement `DeleteVirtualNodeCommand : QUndoCommand`
- [x] `redo()`: destroy virtual node via `Client::RemoveNode()`
- [x] `undo()`: re-create virtual node via `Client::CreateVirtualSink/Source()` with same name/channels/rate
- [x] Store node position and restore on undo
- [x] Implement `deleteSelection()` for Del key
- [x] Collect selected `NodeGraphicsObject` items from `m_scene->selectedItems()`
- [x] Virtual nodes → push `DeleteVirtualNodeCommand` onto undo stack
- [x] Non-virtual nodes → push `QtNodes::DeleteCommand` (removes from graph only, not PipeWire)
- [x] Connection-only selection → push `QtNodes::DeleteCommand`
- [x] Implement `copySelection()` (Ctrl+C)
- [x] Serialize selected virtual nodes to JSON: stable_id, name, media_class, channels, rate, position
- [x] Include links between selected nodes (source stable_id:port_name → target stable_id:port_name)
- [x] Set `QClipboard` with custom MIME type `application/warppipe-virtual-graph`
- [x] Implement `pasteSelection()` (Ctrl+V)
- [x] Parse clipboard JSON, create new virtual nodes with " Copy" name suffix
- [x] Position pasted nodes at offset from originals
- [x] Deferred link resolution via `PendingPasteLink` queue (nodes may not exist yet)
- [x] `tryResolvePendingLinks()` called on node add to wire up deferred links
- [x] Implement `duplicateSelection()` (Ctrl+D) — copy + paste with (40, 40) offset
- [x] Register keyboard shortcuts on `m_view`:
- [x] Del → `deleteSelection()`
- [x] Ctrl+C → `copySelection()`
- [x] Ctrl+V → `pasteSelection()`
- [x] Ctrl+D → `duplicateSelection()`
- [x] Ctrl+L → auto-arrange + zoom fit
- [x] Remove default QtNodes copy/paste actions to avoid conflicts
- [x] Add tests for undo/redo command state (push command → undo → verify node re-created → redo → verify deleted)
- [x] Milestone 8b - View and Layout Enhancements
- [x] Add "Zoom Fit All" context menu action → `m_view->zoomFitAll()`
- [x] Add "Zoom Fit Selected" context menu action → `m_view->zoomFitSelected()`
- [x] Add "Save Layout As..." context menu action
- [x] `QFileDialog::getSaveFileName()` → save layout JSON to custom path
- [x] Reuse existing `saveLayout()` serialization, write to chosen path
- [x] Add "Reset Layout" context menu action
- [x] Clear saved positions, run `autoArrange()`, save, zoom fit
- [x] Add "Refresh Graph" context menu action
- [x] Reset model, re-sync from client, zoom fit
- [x] Persist view state in layout JSON:
- [x] Save view scale + center position (`m_view->getScale()`, `m_view->mapToScene(viewport center)`)
- [x] Restore on load: `m_view->setupScale()` + `m_view->centerOn()`
- [x] Fallback to `zoomFitAll()` when no saved view state
- [x] Persist ghost nodes in layout JSON:
- [x] Serialize ghost node stable_id, name, description, input/output ports (id + name), position
- [x] Serialize ghost connections (out_stable_id, out_port_index, in_stable_id, in_port_index)
- [x] Restore ghosts from layout on load (before live sync)
- [x] Add middle-click center: `eventFilter` on viewport catches `MiddleButton``m_view->centerOn(mapToScene(pos))`
- [x] Add tests for view state save/load round-trip and ghost persistence
- [x] Milestone 8c - Sidebar and Preset System
- [x] Add `QSplitter` between graph view and sidebar panel
- [x] Graph view (stretch factor 1) on left, sidebar (stretch factor 0) on right
- [x] Persist splitter sizes in layout JSON, restore on load
- [x] Default sizes: graph 1200, sidebar 320
- [x] Add `QTabWidget` sidebar with styled tabs (dark theme)
- [x] Tab styling: dark background, selected tab has accent underline
- [x] Initially one tab: "PRESETS" (meters/mixer tabs added in M8d/M8e)
- [x] Implement `PresetManager` class:
- [x] `savePreset(path)` → serialize to JSON:
- [x] Virtual devices: name, description, media_class, channels, rate
- [x] Routing: links by stable_id:port_name pairs
- [x] UI layout: node positions, view state
- [x] `loadPreset(path)` → apply from JSON:
- [x] Create missing virtual devices
- [x] Re-create links from routing entries
- [x] Apply layout positions
- [x] Save on quit via `QCoreApplication::aboutToQuit` signal
- [x] Add "Save Preset..." context menu action → `QFileDialog::getSaveFileName()`
- [x] Add "Load Preset..." context menu action → `QFileDialog::getOpenFileName()`
- [x] Add tests for preset save/load round-trip
- [x] Milestone 8d - Volume/Mute Controls (requires core API: `SetNodeVolume()`)
- [x] Add `NodeVolumeState` struct: `{ float volume; bool mute; }`
- [x] Add `ClickSlider : QSlider` — click jumps to position instead of page-stepping
- [x] Add inline volume widget per node via `nodeData(NodeRole::Widget)`:
- [x] Horizontal `ClickSlider` (0-100) + mute `QToolButton`
- [x] Calls `Client::SetNodeVolume(nodeId, volume, mute)` on change
- [x] Styled: dark background, green slider fill, rounded mute button
- [x] Implement `VolumeChangeCommand : QUndoCommand`
- [x] Stores previous + next `NodeVolumeState`, node ID
- [x] `undo()` → apply previous state; `redo()` → apply next state
- [x] Push on slider release or mute toggle (not during drag)
- [x] Track volume states in model: `std::unordered_map<NodeId, NodeVolumeState> m_volumeStates`
- [x] `setNodeVolumeState()` — update state + sync inline widget + call Client API
- [x] `nodeVolumeState()` — read current state
- [x] Emit `nodeVolumeChanged(nodeId, previous, current)` signal
- [x] Add "MIXER" tab to sidebar `QTabWidget`:
- [x] `QScrollArea` with vertical layout of channel strips
- [x] Per-node strip: horizontal `ClickSlider` (fader) + Mute (M) button + node label
- [x] Volume fader changes push `VolumeChangeCommand` onto undo stack
- [x] `rebuildMixerStrips()` — create/remove strips when nodes appear/disappear
- [x] Mixer strips sync from model state via `nodeVolumeChanged` signal
- [x] Include volume/mute states in preset save/load (`volumes` array in JSON)
- [x] Add tests for volume state tracking, signal emission, widget sync, preset round-trip, cleanup on deletion
- [x] Milestone 8e - Audio Level Meters (requires core API: `MeterPeak()`, `NodeMeterPeak()`, `EnsureNodeMeter()`)
- [x] Implement `AudioLevelMeter : QWidget`
- [x] Custom `paintEvent`: vertical bar from bottom, background `(24,24,28)`
- [x] Color thresholds: green (0-0.7), yellow (0.7-0.9), red (0.9-1.0)
- [x] Peak hold indicator: white horizontal line, holds 6 frames then decays at 0.02/frame
- [x] `setLevel(float)` — clamp 0-1, update hold, call `update()`
- [x] `sizeHint()` → 40×160
- [x] Add "METERS" tab to sidebar `QTabWidget`:
- [x] "MASTER OUTPUT" label + master `AudioLevelMeter`
- [x] "NODE METERS" label + scrollable list of per-node meter rows
- [x] Per-node row: elided label + compact `AudioLevelMeter` (fixed 26px wide, min 70px tall)
- [x] Add 30fps meter update timer (33ms, `Qt::PreciseTimer`)
- [x] Poll `Client::MeterPeak()` → master meter
- [x] Poll `Client::NodeMeterPeak(nodeId)` → per-node meters
- [x] Auto-rebuild node meters on node create/delete
- [x] Auto-manage per-node meters:
- [x] Call `EnsureNodeMeter()` for each node during rebuild
- [x] Remove meter rows when nodes deleted
- [x] `rebuildNodeMeters()` wired to `nodeCreated`/`nodeDeleted` signals
- [x] Add tests for AudioLevelMeter level clamping, hold/decay logic, METERS tab existence, meter row creation
- [x] Milestone 8f - Architecture and Routing Rules
- [x] Event-driven updates: core `SetChangeCallback()` fires on registry changes, GUI debounces via 50ms QTimer + QueuedConnection marshal (2s polling kept as fallback)
- [x] `Client::SetChangeCallback(ChangeCallback)` — fires from PW thread on node/port/link add/remove
- [x] `NotifyChange()` uses dedicated `change_cb_mutex` (not cache_mutex) to avoid lock ordering issues
- [x] GUI marshals to Qt thread via `QMetaObject::invokeMethod(..., Qt::QueuedConnection)`
- [x] Link intent system: implemented via core `saved_links` + deferred `ProcessSavedLinks()`
- [x] `LoadConfig()` parses links into `saved_links` vector (stable node:port name pairs)
- [x] `ProcessSavedLinks()` resolves names → port IDs on each CoreDone, creates via `CreateSavedLinkAsync()`
- [x] Competing links from WirePlumber auto-removed after saved link creation
- [x] Persisted in config.json `links` array (not layout JSON — core owns link state)
- [x] Add routing rule UI (RULES sidebar tab)
- [x] List existing rules from `Client::ListRouteRules()` as styled cards
- [x] Add rules via dialog with Application Name, Process Binary, Media Role, Target Node fields
- [x] Delete rules via per-card ✕ button
---
## Architecture
```
warppipe/
├── CMakeLists.txt # Add Qt6 + QtNodes, warppipe-gui + test targets
├── gui/
│ ├── main.cpp # QApplication entry, CLI flags (--screenshot, --debug-screenshot-dir, --offscreen)
│ ├── WarpGraphModel.h # QtNodes::AbstractGraphModel implementation
│ ├── WarpGraphModel.cpp # Model logic, warppipe::Client integration
│ ├── GraphEditorWidget.h # Main UI widget with scene + view
│ ├── GraphEditorWidget.cpp # Event handlers, context menus, refresh logic, graphReady() signal
│ ├── NodeStyleHelper.h/cpp # Node styling functions (colors, ghost mode)
│ └── ScreenshotHelper.h/cpp # QPixmap capture, debug dir, naming conventions
├── tests/
│ ├── warppipe_tests.cpp # Existing library tests
│ └── gui/
│ └── warppipe_gui_tests.cpp # Model unit tests + screenshot smoke tests
├── include/warppipe/
│ └── warppipe.hpp # No changes needed
└── src/
└── warppipe.cpp # No changes needed
```
---
## Design Notes
### Node Title Synthesis
Display title priority: `description` > `application_name` > `name`
- **Hardware/Virtual nodes**: Use `description` (PW_KEY_NODE_DESCRIPTION), e.g., "Speakers", "Headphones"
- **Application nodes**: Use `application_name` if non-empty (e.g., "Firefox", "Spotify")
- Fallback to `name` if both are empty
### Port Orientation
- **Input ports** (is_input=true): LEFT side of node (QtNodes::PortType::In)
- **Output ports** (is_input=false): RIGHT side of node (QtNodes::PortType::Out)
### Ephemeral Node Handling
Application audio streams are ephemeral — they appear when an app plays audio and can disappear when stopped.
- **Do NOT remove from visual graph** when inactive
- **Mark as "ghost"** and apply faded styling (opacity 0.6, darker colors)
- **Persist connections** visually even when node is inactive
- **Re-activate** styling when node reappears (audio resumes)
Tracking strategy:
1. Poll `Client::ListNodes()` periodically (e.g., every 500ms)
2. Compare current list to previous list
3. Nodes that disappeared → mark as ghost, keep in graph
4. Nodes that reappeared → restore active styling
### Node Type Colors (inspired by potato)
| Type | Base Color | Description |
|------|------------|-------------|
| Hardware Sink/Source | Blue-gray (72, 94, 118) | Physical audio devices |
| Virtual Sink/Source | Green (62, 122, 104) | Virtual nodes created by warppipe |
| Application | Brown/Orange (138, 104, 72) | Ephemeral app audio streams |
Active vs Ghost:
- **Active**: `Opacity = 1.0f`, lighter gradient, bright font
- **Ghost**: `Opacity = 0.6f`, darker gradient (`.darker(150-180)`), faded font
### Connection Creation Flow
1. User drags from output port to input port in UI
2. QtNodes calls `WarpGraphModel::addConnection(connectionId)`
3. Model extracts port IDs from connectionId
4. Model calls `warppipe::Client::CreateLink(outputPortId, inputPortId, LinkOptions{})`
5. If success: add to m_connections, connection appears
6. If failure: show error, do NOT add connection to graph
### Context Menu Actions
- **On canvas**: "Create Virtual Sink", "Create Virtual Source"
- **On virtual node**: "Delete Node"
- **Future**: "Add Routing Rule", "Set as Default", "Edit Properties"
### Layout Persistence
- Save to `~/.config/warppipe-gui/layout.json` (XDG_CONFIG_HOME)
- Format:
```json
{
"version": 1,
"nodes": [
{
"stable_id": "alsa_output.pci-0000_00_1f.3.analog-stereo",
"position": {"x": 100.0, "y": 200.0}
}
],
"view": {
"scale": 1.0,
"center": {"x": 0.0, "y": 0.0}
}
}
```
- Use `NodeInfo.name` as stable_id (unique across sessions)
---
## Qt/CMake Integration
### CMakeLists.txt additions
```cmake
# After existing warppipe library target:
option(BUILD_GUI "Build Qt6 GUI application" ON)
if(WARPPIPE_BUILD_GUI)
find_package(Qt6 6.2 REQUIRED COMPONENTS Core Widgets)
set(CMAKE_AUTOMOC ON)
set(CMAKE_AUTORCC ON)
set(CMAKE_AUTOUIC ON)
include(FetchContent)
FetchContent_Declare(
QtNodes
GIT_REPOSITORY https://github.com/paceholder/nodeeditor
GIT_TAG master
)
FetchContent_MakeAvailable(QtNodes)
add_executable(warppipe-gui
gui/main.cpp
gui/WarpGraphModel.cpp
gui/GraphEditorWidget.cpp
gui/NodeStyleHelper.cpp
)
target_link_libraries(warppipe-gui PRIVATE
warppipe
Qt6::Widgets
QtNodes
)
install(TARGETS warppipe-gui RUNTIME DESTINATION bin)
endif()
```
### GUI test target
```cmake
if(BUILD_GUI)
# Model unit tests (no display needed)
add_executable(warppipe-gui-tests
tests/gui/warppipe_gui_tests.cpp
gui/WarpGraphModel.cpp
gui/NodeStyleHelper.cpp
)
target_compile_definitions(warppipe-gui-tests PRIVATE WARPPIPE_TESTING)
target_link_libraries(warppipe-gui-tests PRIVATE
warppipe
Qt6::Core
Qt6::Widgets
QtNodes
Catch2::Catch2WithMain
)
# Screenshot smoke tests (opt-in, need Qt runtime)
option(WARPPIPE_GUI_VISUAL_TESTS "Enable screenshot-based visual tests" OFF)
if(WARPPIPE_GUI_VISUAL_TESTS)
add_test(NAME gui_screenshot_smoke
COMMAND ${CMAKE_COMMAND} -E env QT_QPA_PLATFORM=offscreen
$<TARGET_FILE:warppipe-gui> --screenshot ${CMAKE_BINARY_DIR}/test_screenshot.png
)
set_tests_properties(gui_screenshot_smoke PROPERTIES
LABELS "visual"
TIMEOUT 10
)
endif()
endif()
```
### Dependencies
- Qt6 >= 6.2 (Core, Widgets)
- QtNodes (nodeeditor) — fetched via CMake FetchContent
- Catch2 v3 — required for GUI tests (via `find_package`)
- warppipe library (existing)
---
## Key QtNodes API Usage
### AbstractGraphModel Interface
```cpp
class WarpGraphModel : public QtNodes::AbstractGraphModel {
Q_OBJECT
public:
// Node management
QtNodes::NodeId newNodeId() override;
std::unordered_set<QtNodes::NodeId> allNodeIds() const override;
bool nodeExists(QtNodes::NodeId) const override;
QtNodes::NodeId addNode(QString const nodeType) override;
bool deleteNode(QtNodes::NodeId) override;
// Node data (caption, style, position, widget)
QVariant nodeData(QtNodes::NodeId, QtNodes::NodeRole) const override;
bool setNodeData(QtNodes::NodeId, QtNodes::NodeRole, QVariant) override;
// Port data (count, label, type, connection policy)
QVariant portData(QtNodes::NodeId, QtNodes::PortType, QtNodes::PortIndex, QtNodes::PortRole) const override;
// Connection management
std::unordered_set<QtNodes::ConnectionId> allConnectionIds(QtNodes::NodeId) const override;
bool connectionPossible(QtNodes::ConnectionId) const override;
void addConnection(QtNodes::ConnectionId) override;
bool deleteConnection(QtNodes::ConnectionId) override;
// Custom methods
void refreshFromClient(); // Poll warppipe::Client and sync graph
void markNodeAsGhost(QtNodes::NodeId, bool isGhost);
};
```
### Scene and View Setup
```cpp
// In GraphEditorWidget constructor:
m_model = new WarpGraphModel(client, this);
m_scene = new QtNodes::BasicGraphicsScene(*m_model, this);
m_view = new QtNodes::GraphicsView(m_scene);
QVBoxLayout *layout = new QVBoxLayout(this);
layout->addWidget(m_view);
// Connect signals
connect(m_scene, &QtNodes::BasicGraphicsScene::connectionCreated,
this, &GraphEditorWidget::onConnectionCreated);
connect(m_scene, &QtNodes::BasicGraphicsScene::connectionDeleted,
this, &GraphEditorWidget::onConnectionDeleted);
```
---
## Testing Strategy
### Three test tiers
**Tier 1 — Model Unit Tests (no display server)**
Pure logic tests for WarpGraphModel. Use warppipe's WARPPIPE_TESTING helpers to inject fake nodes/ports/links into a Client, then construct the model and assert state. These tests run everywhere (CI, headless, local).
Coverage targets:
- Node mapping (PipeWire ID → QtNodes NodeId, caption synthesis, style by type)
- Port mapping (is_input orientation, port count, labels)
- Connection mapping (link → ConnectionId, add/delete roundtrip)
- Ghost state (inactive toggle, opacity values, ghost connections)
- Edge cases (duplicate IDs, remove with active connections, empty graph)
**Tier 2 — Screenshot Smoke Tests (headless, QT_QPA_PLATFORM=offscreen)**
Launch warppipe-gui with `--screenshot` in offscreen mode. Verify:
- Exit code 0 and PNG file produced
- File size heuristic (>10KB = non-trivial content rendered)
- Debug screenshot dir populates on state changes
Gated behind `WARPPIPE_GUI_VISUAL_TESTS` CMake option (default OFF) since they need Qt runtime.
**Tier 3 — Manual QA with AI-Assisted Debugging**
Run warppipe-gui with a live PipeWire daemon. Use `--debug-screenshot-dir` to capture every state transition. When something looks wrong:
1. Run `warppipe-gui --debug-screenshot-dir /tmp/warppipe-debug/`
2. Reproduce the issue
3. Hand the screenshot directory to the AI for visual analysis
Checklist:
- All audio nodes appear with correct titles
- Ports on correct sides (input=left, output=right)
- Dragging connections creates PipeWire links
- Virtual sink/source creation via context menu
- Ephemeral nodes fade when inactive, connections persist
- Layout persists across restarts
### AI Debugging Workflow
The screenshot infrastructure in Milestone 6 is specifically designed so the AI can debug visual issues:
```bash
# Quick single screenshot for AI analysis
QT_QPA_PLATFORM=offscreen warppipe-gui --screenshot /tmp/gui-state.png
# Or use convenience flag
warppipe-gui --offscreen --screenshot /tmp/gui-state.png
# Continuous debug capture (every state change gets a timestamped screenshot)
warppipe-gui --debug-screenshot-dir /tmp/warppipe-debug/
# Then hand to AI:
# "Here are the screenshots from /tmp/warppipe-debug/, the ghost nodes
# aren't fading correctly — can you see what's wrong?"
```
The AI can then use its multimodal capabilities to examine the PNGs, compare expected vs actual visual state, and identify styling or layout bugs without needing a live display.
---
## Future Enhancements
- **Routing rule editor**: Visual panel to add/edit/remove per-app routing rules
- **Audio level meters**: Real-time level monitoring (requires PipeWire param API)
- **Volume/mute controls**: Inline controls on nodes (requires warppipe volume API)
- **Presets**: Save/load full configurations (nodes + links + rules)
- **Search/filter**: Filter nodes by type, name, connection status
- **Minimap**: Overview of entire graph in large setups
- **Themes**: Light/dark mode, custom color schemes
- **Keyboard shortcuts**: Create nodes, delete selection, undo/redo
---
## References
- **potato project**: ~/Projects/potato/ (reference implementation using QtNodes + PipeWire)
- **QtNodes library**: https://github.com/paceholder/nodeeditor
- **warppipe API**: include/warppipe/warppipe.hpp, docs/api.md
- **Qt6 Widgets**: https://doc.qt.io/qt-6/qtwidgets-index.html