feat: voice mode (#4642)
* WIP * works * stereo * ui v0 * unnecessary import * update steps in voice ws * [autofix.ci] apply automated fixes * unused * merge * [autofix.ci] apply automated fixes * cleanly handle missing OPENAI key * ruff * [autofix.ci] apply automated fixes * fix genericIconComponent path * update for recent async fixes * accidentally commited html file * better prompt and threading * client barge-in detection * fmt * error handling * fixed VAD with 24 to 16Hz resampling * comment out debug file * better_vad * lock * router * [autofix.ci] apply automated fixes * mcp fixes * timeout fix * global variable exception handling * don't close the websocket * fix double send bug * fix double send bug * response.output_item event type typo * voice_mode logging * 📝 (constants.py): Add "copy_field" attribute to FIELD_FORMAT_ATTRIBUTES list 📝 (webhook.py): Add "copy_field" attribute to MultilineInput component 📝 (input_mixin.py): Add "copy_field" attribute to BaseInputMixin class 📝 (inputs.py): Add "copy_field" attribute to StrInput class 📝 (template/field/base.py): Add "copy_field" attribute to Input class 🚀 (NodeDescription/index.tsx): Remove default placeholder text for emptyPlaceholder prop ✨ (copyFieldAreaComponent/index.tsx): Add new component for handling copy field functionality ♻️ (strRenderComponent/index.tsx): Refactor component to include CopyFieldAreaComponent when copy_field attribute is present in template data * ✨ (NodeDescription/index.tsx): refactor renderedDescription useMemo to improve readability and maintainability ♻️ (GenericNode/index.tsx): refactor code to improve readability and maintainability, and optimize rendering logic * 📝 (webhook.py): Add cURL field to WebhookComponent for better integration with external systems 📝 (graph/base.py): Add logging of vertex build information in Graph class for debugging purposes 📝 (NodeInputField/index.tsx): Add nodeInformationMetadata to NodeInputField for better tracking of node information 📝 (copyFieldAreaComponent/index.tsx): Refactor CopyFieldAreaComponent to handle different types of values, including webhooks 📝 (strRenderComponent/index.tsx): Add WebhookFieldComponent to handle webhook type in StrRenderComponent 📝 (tableNodeCellRender/index.tsx): Add nodeInformationMetadata to TableNodeCellRender for better tracking of node information 📝 (textAreaComponent/index.tsx): Add support for webhook format in TextAreaComponent for better integration with webhooks 📝 (webhookFieldComponent/index.tsx): Add WebhookFieldComponent to handle webhook type in ParameterRenderComponent 📝 (custom-parameter.tsx): Add nodeInformationMetadata to CustomParameterComponent for better tracking of node information 📝 (get-curl-code.tsx): Add support for different formats in getCurlWebhookCode for generating cURL commands 📝 (textAreaModal/index.tsx): Add onCloseModal callback to ComponentTextModal for better handling of modal closing 📝 (index.ts): Add type field to APIClassType for better typing of API classes * ✨ (index.tsx): Add a button to generate a token in the WebhookFieldComponent for improved user experience and functionality. Update the structure of the component to include the new button and styling adjustments. * [autofix.ci] apply automated fixes * ✨ (generate-token-dialog.tsx): add GenerateTokenDialog component to handle token generation in webhookFieldComponent 📝 (index.tsx): import and use GenerateTokenDialog component in WebhookFieldComponent for token generation functionality * ✨ (frontend): introduce new feature to create API keys with customizable modal properties 🔧 (frontend): add modalProps object to customize modal title, description, input label, input placeholder, button text, generated key message, and show icon flag * add pool interval variable and tests * 📝 (NodeOutputfield): Remove unused ScanEyeIcon component ✨ (validate-webhook.ts): Add function to validate webhook data before processing ♻️ (use-get-builds-pooling-mutation): Refactor to set flow pool based on current flow 🔧 (content-render.tsx): Add data-testid attribute to api key input element 🔧 (webhookComponent.spec.ts): Refactor test to use waitForRequest for monitoring build requests * [autofix.ci] apply automated fixes * 🔧 (backend): rename webhook_pooling_interval to webhook_polling_interval for consistency 🔧 (frontend): update references to webhook_pooling_interval to webhook_polling_interval for consistency * vad + dummy check * 📝 (frontend): Update import paths and remove unused imports for better code organization and maintainability 🔧 (frontend): Refactor background styles in components to use constants for consistency and easier theming 🚀 (frontend): Add custom SecretKeyModalButton component for better modularity and reusability * 📝 (use-get-api-keys.ts): add a TODO comment to request API key from DSLF endpoint for future implementation. * 📝 (input_mixin.py): Remove copy_field attribute from BaseInputMixin as it is no longer needed ♻️ (inputs.py): Remove copy_field attribute from StrInput class as it is no longer needed ♻️ (inputs.py): Set copy_field attribute to False in MultilineInput class to ensure consistency ♻️ (template/field/base.py): Remove copy_field attribute from Input class as it is no longer needed 📝 (textAreaComponent/index.tsx): Replace hardcoded value "CURL_WEBHOOK" with constant WEBHOOK_VALUE for better readability and maintainability * 🐛 (base.py): fix issue where flow_id could be None by defaulting to an empty string if flow_id is None * 🔧 (secret-key-modal.tsx): Remove unused SecretKeyModalButton component 🔧 (get-modal-props.tsx): Remove unused getModalPropsApiKey function and related imports and constants * 📝 (langflow): add noqa comments to suppress linting rule A005 for specific files in the io, logging, and socket modules * ✨ (frontend): Add voice assistant feature to chat input component 🔧 (frontend): Refactor import path for VoiceAssistant component 🔧 (frontend): Refactor class name for button in upload-file-button component 🔧 (frontend): Refactor class name for button in voice-button component 🔧 (frontend): Refactor class name for button in applies.css 🔧 (frontend): Refactor class name for button in styleUtils.ts * ✨ (api-key-popup.tsx): Add a new component ApiKeyPopup to enable users to enter their OpenAI API key for voice transcription ♻️ (index.tsx): Refactor VoiceAssistant component to check for the presence of OpenAI API key before starting voice transcription and show ApiKeyPopup component if key is missing 🔧 (apiKeyModal/index.tsx): Remove the obsolete APIKeyModal component as it is no longer needed after implementing ApiKeyPopup in the voice assistant feature * merge fix * ✨ (Voice Assistant): Introduce voice transcription feature powered by OpenAI. Add components and hooks for handling audio recording, processing, and WebSocket communication. Implement functionality to start, stop recording, play audio chunks, handle WebSocket messages, and initialize audio context. Add support for entering API key for OpenAI. * ✨ (voice-assistant.tsx): Add voice assistant feature to the chat input component for recording and processing audio input 🔧 (use-post-voice.tsx): Remove unused file use-post-voice.tsx from the project ♻️ (use-handle-websocket-message.ts): Refactor useHandleWebsocketMessage function to improve readability and remove unnecessary console logs ♻️ (use-initialize-audio.ts): Refactor useInitializeAudio function to handle audio context creation and resume more efficiently ✅ (use-interrupt-playback.ts): Add useInterruptPlayback function to handle interrupting audio playback ✅ (use-start-conversation.ts): Add useStartConversation function to initiate a conversation using a WebSocket connection 📝 (chat-input.tsx): Update import path for VoiceAssistant component to match the new file structure * ♻️ (use-interrupt-playback.ts): refactor useInterruptPlayback hook to clear audio queue, stop playback, and send stop message to audio processor if it exists * 🔧 (gitattributes): add *.raw file extension as binary to prevent git from modifying the file contents * lazy load components * TRACE logging * new voice_mode ws endpoint and elevenlabs mode * unique voice * text modality * stream elevenlabs * ws and sentence chunking * offload tts to new thread * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes * [autofix.ci] apply automated fixes (attempt 2/3) * add eleven labs changes on FE * 🐛 (use-handle-websocket-message.ts): Fix logic to handle error messages and show API key modal when api_key_missing error occurs ♻️ (use-handle-websocket-message.ts): Refactor error handling logic to improve readability and maintainability 📝 (voice-assistant.tsx): Add console log for showApiKeyModal variable to debug an issue * api_key_missing error includes key_name * ✨ (use-handle-websocket-message.ts): update status message when API key is missing to provide more context 🔧 (voice-assistant.tsx): add support for saving and handling API key using new API endpoints and alert messages 🔧 (voice-assistant.tsx): update logic to check for existing API key and handle saving new API key with appropriate error handling * 🔧 (gitignore): remove unnecessary whitespace at the end of the file 🔧 (output_audio.raw): delete output_audio.raw file as it is no longer needed * 🐛 (use-handle-websocket-message.ts): Fix logic to show API key modal only if hasOpenAIAPIKey is false 🐛 (voice-assistant.tsx): Fix missing hasOpenAIAPIKey parameter in VoiceAssistant component to prevent runtime errors * [autofix.ci] apply automated fixes * modalities * send all openai events and homogenize eleven_client name * 🔧 (voice_mode.py): Refactor get_or_create_elevenlabs_client function to accept user_id and session parameters for better flexibility and reusability 🔧 (voice-assistant.tsx): Add local storage utility functions to handle saving and retrieving audio settings for voice assistant 🔧 (voiceStore.ts): Create a new store to manage voice-related data such as available voices and providers 🔧 (audio-settings-dialog.tsx): Introduce a new component to manage audio settings for the voice assistant, including provider and voice selection 🔧 (settings-voice-button.tsx): Implement a button component to open the audio settings modal for the voice assistant * unnecessary barge in queue and elif fix * queue_service, remove old ws endpoint, and vad tweaks * traceback and better system prompt * [autofix.ci] apply automated fixes * merge * build_flow fix * fix flow_and_stream args * mcp flow_and_stream * build_flow_and_stream * retry prompt and conversation_id * [autofix.ci] apply automated fixes * prompt * switch to openai and elevenlabs & add token based auth * 📝 (voice_mode.py): Add import statements for new modules and classes 📝 (voice_mode.py): Add WebSocket endpoint parameter for session_id 📝 (voice_mode.py): Add logic to save message to database in WebSocket endpoint ✨ (use-get-messages-mutation.ts): Add function to fetch messages data from API ♻️ (use-start-conversation.ts): Update WebSocket URL to include session_id ♻️ (use-start-recording.ts): Add function to handle fetching messages after recording ♻️ (streamProcessor.ts): Add error handling for already registered processor ♻️ (voice-assistant.tsx): Add function to trigger fetching messages after recording 📝 (new-modal.tsx): Set and update current session ID in modal 📝 (utilityStore.ts): Add currentSessionId state and setter function 📝 (index.ts): Add type definition for currentSessionId in utility store * [autofix.ci] apply automated fixes * ✨ (voice_mode.py): add functionality to save user input messages to the database during a conversation in the voice mode feature. This allows for better tracking and analysis of user interactions. * [autofix.ci] apply automated fixes * continue to support ws without session_id * intercept and replace response.done when elevenlabs. and better logging * 🐛 (voice_mode.py): Fix issue where message was not being saved to the database in certain cases 🐛 (use-start-recording.ts): Fix bug where handleGetMessagesMutation was not being called in all scenarios * close openai ws when client ws closes * [autofix.ci] apply automated fixes * rename elevenlabs event type * move session to global, improve event error logging * [autofix.ci] apply automated fixes * session update merging * key check fix * fix session update forwarding to openai * [autofix.ci] apply automated fixes * 📝 (chat-view.tsx): Add chat history display functionality to show messages in the chat view component and handle file uploads and drag and drop functionality. * ✨ (inputGlobalComponent): Add GeneralDeleteConfirmationModal and GeneralGlobalVariableModal components for reusability and better code organization 📝 (audio-settings-dialog.tsx): Add userElevenLabsApiKey prop to SettingsVoiceModal component for ElevenLabs API key configuration 🔧 (delete-confirmation-modal.tsx): Create GeneralDeleteConfirmationModal component for reusability in delete confirmation modals 🔧 (global-variable-modal.tsx): Create GeneralGlobalVariableModal component for reusability in global variable modals * 🐛 (voice_mode.py): Fix error saving message to database and add transcript extraction function 🔧 (constants.ts): Update calculation for LANGFLOW_ACCESS_TOKEN_EXPIRE_SECONDS to use environment variable with fallback value ♻️ (audio-settings-dialog.tsx): Refactor imports and remove unused code, improve logic for setting voice provider and voice selection 🔧 (vite.config.mts): Add dotenv package for loading environment variables, update configuration to use environment variables for BACKEND_URL, ACCESS_TOKEN_EXPIRE_SECONDS, CI, and ELEVENLABS_API_KEY * [autofix.ci] apply automated fixes * handle BrokenResourceError * handle BrokenResourceError * voice UI/UX improvements * ✅ (voice-assistant.spec.ts): add test for voice assistant feature to ensure it is visible and interactive on the page * ✨ (frontend): add feature flag for enabling voice assistant functionality 📝 (input-wrapper.tsx): conditionally render VoiceButton based on ENABLE_VOICE_ASSISTANT flag to control visibility of voice assistant feature * ✨ (button.tsx): Add new button variant 'outlineAmber' for better visual consistency 🔧 (feature-flags.ts): Enable voice assistant feature by setting ENABLE_VOICE_ASSISTANT to true 🔧 (voice-assistant.tsx): Update logic to initialize audio only if hasOpenAIAPIKey is true 🔧 (voice-assistant.tsx): Update styling for voice assistant container and recording indicator 🔧 (voice-assistant.tsx): Update button variant to 'outlineAmber' for settings icon 📝 (index.css): Add new CSS variables for accent-amber and red-foreground colors for better theming 📝 (tailwind.config.mjs): Add 'red-foreground' color variable to Tailwind CSS config for consistency * 🐛 (feature-flags.ts): fix ENABLE_VOICE_ASSISTANT flag to be set to false instead of true * 🔧 (voice_mode.py): add global dictionary to store queues for each session and track active message processing tasks 🔧 (voice_mode.py): add message processing queue to ensure ordered processing of messages in the database * ✨ (frontend): enable voice assistant feature flag ♻️ (frontend): refactor code to use functional update pattern in useBarControls hook 🐛 (frontend): fix logic in VoiceAssistant component to properly handle recording and initialization of audio assistant * [autofix.ci] apply automated fixes * ✨ (audio-settings-dialog.tsx): add LanguageSelect component to allow users to select preferred language for speech recognition 📝 (audio-settings-dialog.tsx): add documentation for ALL_LANGUAGES constant and SettingsVoiceModalProps interface 🔧 (language-select.tsx): create LanguageSelect component for selecting preferred language for speech recognition 🔧 (use-start-recording.ts): add autoGainControl and sampleRate options for better audio recording quality 🔧 (use-start-recording.ts): set fftSize property on analyserRef for improved audio analysis 🔧 (use-start-recording.ts): pass preferredLanguage to input_audio_buffer.append event for language identification 🔧 (voice-assistant.tsx): add support for setting and saving preferred language for speech recognition * UI adjustments * Small fixes for client/LF session update merges * [autofix.ci] apply automated fixes * 🔧 (use-bar-controls.ts): Add useRef import to fix missing dependency in useBarControls hook 🔧 (use-bar-controls.ts): Add animationFrameRef and timeDataRef to manage animation and sound detection 🔧 (use-bar-controls.ts): Add support for sound detection using analyserRef and setSoundDetected 🔧 (use-bar-controls.ts): Update useEffect dependencies for better performance 🔧 (voice-assistant.tsx): Add soundDetected state to manage sound detection in VoiceAssistant component 🔧 (voice-assistant.tsx): Pass analyserRef and setSoundDetected to useBarControls hook in VoiceAssistant component 🔧 (chat-message.tsx): Update className logic to handle isAudioMessage condition in ChatMessage component * 🔧 (use-bar-controls.ts): refactor useBarControls hook to improve readability and maintainability by introducing useRef for baseHeights, lastRandomizeTime, and minHeight, and optimizing the sound detection logic. * ✨ (use-get-messages-polling.ts): Add new file for handling messages polling functionality ♻️ (use-bar-controls.ts): Refactor useBarControls hook to make setSoundDetected optional ♻️ (use-start-recording.ts): Remove handleGetMessagesMutation function call ♻️ (voice-assistant.tsx): Refactor VoiceAssistant component to use useGetMessagesPollingMutation instead of useGetMessagesMutation and manage soundDetected state using useVoiceStore 📝 (chat-message.tsx): Remove unused import and type ChatMessageType 🔧 (voiceStore.ts): Add soundDetected state and setSoundDetected function to voiceStore 🔧 (voice.types.ts): Add soundDetected boolean type and setSoundDetected function to VoiceStoreType * 🔧 (frontend): add Button component import to improve code organization 🔧 (frontend): add support for saving OpenAI API key in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponentType interface for flexibility 🔧 (frontend): add commandWidth prop to InputComponent in InputComponent index file 🔧 (frontend): add commandWidth prop to CustomInputPopover in popover index file 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in SettingsVoiceModal component 🔧 (frontend): add commandWidth prop to InputComponent in Settings * Update session instructions with more clear categories. Change how default LF session and client sessions are merged. * ✨ (chat-view.tsx): Add support for sound detection in chat view component to enhance user experience and interaction with the application. * [autofix.ci] apply automated fixes * ✨ (audio-settings-dialog.tsx): Add debounce functionality to handleOpenAIKeyChange to improve performance and reduce API calls 🔧 (audio-worklet-processor.js): Add audio worklet processor for voice activity detection to enhance voice assistant functionality ♻️ (use-start-recording.ts): Refactor useStartRecording hook to only send non-silent audio data to the server for processing 🔧 (voice-assistant.tsx): Remove unnecessary import and setShowSettingsModal call in handleClickSaveOpenAIApiKey to clean up code * 🐛 (audio-worklet-processor.js): Adjust noise and activation thresholds for better sensitivity and accuracy in speech detection 🐛 (audio-worklet-processor.js): Scale RMS volume to match use-bar-controls.ts scale for consistent threshold comparison 🐛 (use-bar-controls.ts): Update threshold for sound detection to align with scaled volume values for accurate detection * 🔧 (use-start-recording.ts): refactor useStartRecording hook to simplify audio data handling and sending process to WebSocket server * 🐛 (use-bar-controls.ts): adjust sound detection threshold to improve accuracy and responsiveness * [autofix.ci] apply automated fixes * 🐛 (voice-assistant.tsx): Update condition to initialize audio only if not recording, has API key, and settings modal is not shown 🐛 (chat-view.tsx): Fix syntax error in rendering new chat section in ChatView component * ♻️ (chat-view.tsx): remove unnecessary parentheses in ChatView component to improve code readability * 🐛 (voice-assistant.tsx): fix useEffect dependency array to include showSettingsModal to prevent unnecessary re-renders ♻️ (voice-assistant.tsx): remove duplicate onClick handler for showSettingsModal to improve code readability and maintainability * 🐛 (voice-assistant.tsx): fix voice assistant settings button not closing audio input when clicked * 🔧 (audio-settings-dialog.tsx): Remove unused 'open' prop from SettingsVoiceModal component ♻️ (voice-assistant.tsx): Refactor logic to handle audio recording and settings modal visibility 🔧 (voice-assistant.tsx): Replace AudioSettingsDialog component with SettingsVoiceModal component in VoiceAssistant component * 🐛 (audio-settings-dialog.tsx): reduce debounce timeout from 2000ms to 1000ms for faster response to user input 🐛 (audio-settings-dialog.tsx): adjust alignOffset value in DropdownMenuContent component to -54 for better alignment on the UI * ✨ (button.tsx): add medium size variant to button component 🔧 (audio-settings-dialog.tsx): add isEditingOpenAIKey boolean prop and setIsEditingOpenAIKey function prop to SettingsVoiceModal component 🔧 (voice-assistant.tsx): add isEditingOpenAIKey state and setIsEditingOpenAIKey function to VoiceAssistant component, update handleSaveApiKey function to handle editing OpenAI API key * 🐛 (audio-settings-dialog.tsx): Fix issue where the "Save" button text was not updating correctly based on editing state 📝 (voice-assistant.tsx): Add missing semicolon to the code to prevent syntax error * 🐛 (audio-settings-dialog.tsx): Fix issue with setIsEditingOpenAIKey function call ♻️ (use-bar-controls.ts): Refactor code to properly initialize and handle audio analyzer ✨ (voice-assistant.tsx): Introduce logic to properly initialize audio and start recording when voice assistant is activated * 🐛 (audio-settings-dialog.tsx): fix typo in setIsEditingOpenAIKey function call to properly close the editing of OpenAI key when modal is closed * 🔧 (audio-settings-dialog.tsx): change onClick handler function name from setOpen to onOpenChangeDropdownMenu for clarity and consistency * lint fixes * ruff fixes * [autofix.ci] apply automated fixes * ✨ (test_voice_mode.py): import pathlib module for improved file path handling ♻️ (test_voice_mode.py): refactor code to use newer numpy random Generator for better random number generation 📝 (test_voice_mode.py): update comments for clarity and consistency in test functions * 🔧 (components.py): refactor global variables to use a class for managing component cache 🔧 (components.py): replace os module with pathlib for file path operations 🔧 (components.py): refactor global variables to use a class for managing fully loaded components 🔧 (components.py): refactor function to write audio bytes to file using a helper function * 🔧 Refactor voice_utils.py: Improve error handling in write_audio_to_file function * ✨ (voice-select.tsx): add unique key prop to SelectItem component to avoid React warning and improve performance * [autofix.ci] apply automated fixes * ✨ (audio-settings-dialog.tsx): Update onClick function to setIsEditingOpenAIKey instead of onOpenChangeDropdownMenu for better clarity 📝 (microphone-select.tsx): Update text from "Input" to "Audio Input" for better user understanding ♻️ (voice-assistant.tsx): Refactor handleSaveApiKey function to handle both OpenAI and ElevenLabs API keys, improving code readability and maintainability. Remove unnecessary checks for already saved API keys. * 📝 (voice_utils.py): add docstring to resample_24k_to_16k function for clarity and documentation purposes 📝 (voice_utils.py): add input validation to ensure frame_24k_bytes is exactly 960 bytes before resampling 📝 (test_voice_mode.py): refactor test_webrtcvad_with_real_data to generate synthetic audio data for testing instead of reading from a file 📝 (test_voice_mode.py): update comments and assertions for clarity and accuracy in speech detection testing * [autofix.ci] apply automated fixes * 🐛 (feature-flags.ts): fix duplicate declaration of ENABLE_VOICE_ASSISTANT variable * cache config by session_id * [autofix.ci] apply automated fixes * ✨ (audio-settings-dialog.tsx): Add conditional rendering for MicrophoneSelect component based on the presence of microphones array to prevent rendering when no microphones are available 🐛 (microphone-select.tsx): Add optional chaining to navigator.mediaDevices calls to prevent errors when navigator or mediaDevices are null or undefined 🐛 (use-start-recording.ts): Add optional chaining to navigator.mediaDevices calls to prevent errors when navigator or mediaDevices are null or undefined * ♻️ (voice_mode.py): refactor openai_realtime_session attribute to explicitly define its type as a dictionary with string keys and any values for better code clarity and type safety * 📝 (voice_mode.py): add Optional import from typing module to improve type hinting ♻️ (voice_mode.py): make ElevenLabs package optional to avoid errors if not installed and provide a fallback mechanism * [autofix.ci] apply automated fixes * ♻️ (voice_mode.py): refactor ElevenLabs and ApiError classes to include type ignore comments to suppress redefinition warnings * [autofix.ci] apply automated fixes * merge * event logger to debug * 🐛 (language-select.tsx): Fix potential null reference error when accessing lang object properties 🐛 (voice-select.tsx): Fix potential null reference error when accessing voice object properties 🐛 (stringManipulation.ts): Fix potential null reference error when calling toLowerCase and toUpperCase functions * fix openai voices * error handling * frontend socket state * [autofix.ci] apply automated fixes * add session to event logging * add elevenlabs to base package * 🐛 (audio-settings-dialog.tsx): Fix rendering issue with MicrophoneSelect component 🐛 (microphone-select.tsx): Fix potential null pointer exception in rendering microphone label 🐛 (voice-assistant.tsx): Fix issue with toggling recording functionality to properly handle microphone stream and tracks * 🐛 (voice-assistant.tsx): fix optional chaining for microphoneRef to prevent potential null pointer errors * [autofix.ci] apply automated fixes * 🐛 (use-start-conversation.ts): fix variable name from sessionId to currentSessionId for clarity and consistency 🔧 (use-start-conversation.ts): refactor WebSocket connection URL to use current host, port, and protocol for better flexibility and compatibility * alternating messages * mypy * [autofix.ci] apply automated fixes * ✅ (userSettings.spec.ts): add additional wait times to improve stability and reliability of tests --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com> Co-authored-by: cristhianzl <cristhian.lousa@gmail.com> Co-authored-by: nfreybler <nfreybler@nvidia.com>
This commit is contained in:
Sebastián Estévez
GitHub
parent
68546e06e4
commit
5493ac7b0b
85 changed files with 5020 additions and 588 deletions
58
src/frontend/package-lock.json
generated
58
src/frontend/package-lock.json
generated
|
|
@ -1005,14 +1005,14 @@
|
|||
},
|
||||
"node_modules/@esbuild/darwin-arm64": {
|
||||
"version": "0.21.5",
|
||||
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.21.5.tgz",
|
||||
"integrity": "sha512-DwqXqZyuk5AiWWf3UfLiRDJ5EDd49zg6O9wclZ7kUMv2WRFr4HKjXp/5t8JZ11QbQfUS6/cRCKGwYhtNAY88kQ==",
|
||||
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.21.5.tgz",
|
||||
"integrity": "sha512-1rYdTpyv03iycF1+BhzrzQJCdOuAOtaqHTWJZCWvijKD2N5Xu0TtVC8/+1faWqcP9iBCWOmjmhoH94dH82BxPQ==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
"x64"
|
||||
],
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
|
|
@ -2007,14 +2007,14 @@
|
|||
},
|
||||
"node_modules/@million/lint/node_modules/@esbuild/darwin-arm64": {
|
||||
"version": "0.20.2",
|
||||
"resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.20.2.tgz",
|
||||
"integrity": "sha512-4J6IRT+10J3aJH3l1yzEg9y3wkTDgDk7TSDFX+wKFiWjqWp/iCfLIYzGyasx9l0SAFPT1HwSCR+0w/h1ES/MjA==",
|
||||
"resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.20.2.tgz",
|
||||
"integrity": "sha512-1MdwI6OOTsfQfek8sLwgyjOXAu+wKhLEoaOLTjbijk6E2WONYpH9ZU2mNtR+lZ2B4uwr+usqGuVfFT9tMtGvGw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
"x64"
|
||||
],
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=12"
|
||||
|
|
@ -2415,14 +2415,14 @@
|
|||
},
|
||||
"node_modules/@napi-rs/nice-darwin-arm64": {
|
||||
"version": "1.0.1",
|
||||
"resolved": "https://registry.npmjs.org/@napi-rs/nice-darwin-arm64/-/nice-darwin-arm64-1.0.1.tgz",
|
||||
"integrity": "sha512-91k3HEqUl2fsrz/sKkuEkscj6EAj3/eZNCLqzD2AA0TtVbkQi8nqxZCZDMkfklULmxLkMxuUdKe7RvG/T6s2AA==",
|
||||
"resolved": "https://registry.npmjs.org/@napi-rs/nice-linux-x64-gnu/-/nice-linux-x64-gnu-1.0.1.tgz",
|
||||
"integrity": "sha512-XQAJs7DRN2GpLN6Fb+ZdGFeYZDdGl2Fn3TmFlqEL5JorgWKrQGRUrpGKbgZ25UeZPILuTKJ+OowG2avN8mThBA==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
"x64"
|
||||
],
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">= 10"
|
||||
|
|
@ -4122,11 +4122,11 @@
|
|||
"resolved": "https://registry.npmjs.org/@rollup/rollup-darwin-arm64/-/rollup-darwin-arm64-4.36.0.tgz",
|
||||
"integrity": "sha512-JQ1Jk5G4bGrD4pWJQzWsD8I1n1mgPXq33+/vP4sk8j/z/C2siRuxZtaUA7yMTf71TCZTZl/4e1bfzwUmFb3+rw==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
"x64"
|
||||
],
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
"linux"
|
||||
]
|
||||
},
|
||||
"node_modules/@rollup/rollup-darwin-x64": {
|
||||
|
|
@ -4699,12 +4699,12 @@
|
|||
"resolved": "https://registry.npmjs.org/@swc/core-darwin-arm64/-/core-darwin-arm64-1.11.11.tgz",
|
||||
"integrity": "sha512-vJcjGVDB8cZH7zyOkC0AfpFYI/7GHKG0NSsH3tpuKrmoAXJyCYspKPGid7FT53EAlWreN7+Pew+bukYf5j+Fmg==",
|
||||
"cpu": [
|
||||
"arm64"
|
||||
"x64"
|
||||
],
|
||||
"dev": true,
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
"linux"
|
||||
],
|
||||
"engines": {
|
||||
"node": ">=10"
|
||||
|
|
@ -8449,19 +8449,6 @@
|
|||
"optional": true,
|
||||
"peer": true
|
||||
},
|
||||
"node_modules/fsevents": {
|
||||
"version": "2.3.2",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.2.tgz",
|
||||
"integrity": "sha512-xiqMQR4xAeHTuB9uWm+fFRcIOgKBMiOBP+eXiyT7jsgVCq1bkVygt00oASowB7EdtpOHaaPgKt812P9ab+DDKA==",
|
||||
"hasInstallScript": true,
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/function-bind": {
|
||||
"version": "1.1.2",
|
||||
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
|
||||
|
|
@ -15644,19 +15631,6 @@
|
|||
}
|
||||
}
|
||||
},
|
||||
"node_modules/vite/node_modules/fsevents": {
|
||||
"version": "2.3.3",
|
||||
"resolved": "https://registry.npmjs.org/fsevents/-/fsevents-2.3.3.tgz",
|
||||
"integrity": "sha512-5xoDfX+fL7faATnagmWPpbFtwh/R77WmMMqqHGS65C3vvB0YHrgF+B1YmZ3441tMj5n63k0212XNoJwzlhffQw==",
|
||||
"hasInstallScript": true,
|
||||
"optional": true,
|
||||
"os": [
|
||||
"darwin"
|
||||
],
|
||||
"engines": {
|
||||
"node": "^8.16.0 || ^10.6.0 || >=11.0.0"
|
||||
}
|
||||
},
|
||||
"node_modules/w3c-keyname": {
|
||||
"version": "2.2.8",
|
||||
"resolved": "https://registry.npmjs.org/w3c-keyname/-/w3c-keyname-2.2.8.tgz",
|
||||
|
|
|
|||
|
|
@ -166,7 +166,7 @@ export default function GlobalVariableModal({
|
|||
</Tabs>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<div className="space-y-2" id="global-variable-modal-inputs">
|
||||
<Label>Name*</Label>
|
||||
<Input
|
||||
value={key}
|
||||
|
|
|
|||
|
|
@ -56,13 +56,15 @@ const OptionBadge = ({
|
|||
className={cn("flex items-center gap-1 truncate", className)}
|
||||
>
|
||||
<div className="truncate">{option}</div>
|
||||
<X
|
||||
className="h-3 w-3 cursor-pointer bg-transparent hover:text-destructive"
|
||||
onClick={(e) =>
|
||||
onRemove(e as unknown as React.MouseEvent<HTMLButtonElement>)
|
||||
}
|
||||
data-testid="remove-icon-badge"
|
||||
/>
|
||||
<div>
|
||||
<X
|
||||
className="h-3 w-3 cursor-pointer bg-transparent hover:text-destructive"
|
||||
onClick={(e) =>
|
||||
onRemove(e as unknown as React.MouseEvent<HTMLButtonElement>)
|
||||
}
|
||||
data-testid="remove-icon-badge"
|
||||
/>
|
||||
</div>
|
||||
</Badge>
|
||||
);
|
||||
|
||||
|
|
@ -71,17 +73,24 @@ const CommandItemContent = ({
|
|||
isSelected,
|
||||
optionButton,
|
||||
nodeStyle,
|
||||
commandWidth,
|
||||
}: {
|
||||
option: string;
|
||||
isSelected: boolean;
|
||||
optionButton: (option: string) => ReactNode;
|
||||
nodeStyle?: string;
|
||||
commandWidth?: string;
|
||||
}) => (
|
||||
<div className="group flex w-full items-center justify-between">
|
||||
<div className="flex items-center justify-between">
|
||||
<SelectionIndicator isSelected={isSelected} />
|
||||
<ShadTooltip content={option} side="left">
|
||||
<div className={cn("truncate pr-2", nodeStyle ? "max-w-52" : "w-full")}>
|
||||
<div
|
||||
className={cn("w-full truncate pr-2", nodeStyle && "max-w-52")}
|
||||
style={{
|
||||
maxWidth: commandWidth,
|
||||
}}
|
||||
>
|
||||
<span>{option}</span>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
|
|
@ -119,6 +128,7 @@ const getInputClassName = (
|
|||
disabled: boolean,
|
||||
password: boolean,
|
||||
selectedOptions: string[],
|
||||
blockAddNewGlobalVariable: boolean = false,
|
||||
) => {
|
||||
return cn(
|
||||
"popover-input nodrag w-full truncate px-1 pr-4",
|
||||
|
|
@ -127,6 +137,7 @@ const getInputClassName = (
|
|||
disabled &&
|
||||
"disabled:text-muted disabled:opacity-100 placeholder:disabled:text-muted-foreground",
|
||||
password && "text-clip pr-14",
|
||||
blockAddNewGlobalVariable && "text-clip pr-8",
|
||||
selectedOptions?.length >= 0 && "cursor-default",
|
||||
);
|
||||
};
|
||||
|
|
@ -173,6 +184,8 @@ const CustomInputPopover = ({
|
|||
optionButton,
|
||||
autoFocus,
|
||||
popoverWidth,
|
||||
commandWidth,
|
||||
blockAddNewGlobalVariable,
|
||||
}) => {
|
||||
const [isFocused, setIsFocused] = useState(false);
|
||||
const memoizedOptions = useMemo(() => new Set<string>(options), [options]);
|
||||
|
|
@ -230,7 +243,11 @@ const CustomInputPopover = ({
|
|||
</div>
|
||||
) : selectedOption?.length > 0 ? (
|
||||
<ShadTooltip content={selectedOption} side="left">
|
||||
<div>
|
||||
<div
|
||||
style={{
|
||||
maxWidth: commandWidth,
|
||||
}}
|
||||
>
|
||||
<OptionBadge
|
||||
option={selectedOption}
|
||||
onRemove={(e) => handleRemoveOption(selectedOption, e)}
|
||||
|
|
@ -266,6 +283,7 @@ const CustomInputPopover = ({
|
|||
disabled,
|
||||
password,
|
||||
selectedOptions,
|
||||
blockAddNewGlobalVariable,
|
||||
)}
|
||||
placeholder={
|
||||
selectedOptions?.length > 0 || selectedOption ? "" : placeholder
|
||||
|
|
@ -318,6 +336,7 @@ const CustomInputPopover = ({
|
|||
}
|
||||
optionButton={optionButton}
|
||||
nodeStyle={nodeStyle}
|
||||
commandWidth={commandWidth}
|
||||
/>
|
||||
</CommandItem>
|
||||
))}
|
||||
|
|
|
|||
|
|
@ -40,6 +40,8 @@ export default function InputComponent({
|
|||
nodeStyle,
|
||||
isToolMode,
|
||||
popoverWidth,
|
||||
commandWidth,
|
||||
blockAddNewGlobalVariable = false,
|
||||
}: InputComponentType): JSX.Element {
|
||||
const [pwdVisible, setPwdVisible] = useState(false);
|
||||
const refInput = useRef<HTMLInputElement>(null);
|
||||
|
|
@ -151,54 +153,57 @@ export default function InputComponent({
|
|||
optionsPlaceholder={optionsPlaceholder}
|
||||
nodeStyle={nodeStyle}
|
||||
popoverWidth={popoverWidth}
|
||||
commandWidth={commandWidth}
|
||||
blockAddNewGlobalVariable={blockAddNewGlobalVariable}
|
||||
/>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
|
||||
{(setSelectedOption || setSelectedOptions) && (
|
||||
<span
|
||||
className={cn(
|
||||
password && selectedOption === "" ? "right-8" : "right-0",
|
||||
"absolute inset-y-0 flex items-center pr-2.5",
|
||||
disabled && "cursor-not-allowed opacity-50",
|
||||
)}
|
||||
>
|
||||
<button
|
||||
disabled={disabled}
|
||||
onClick={(e) => {
|
||||
if (disabled) return;
|
||||
setShowOptions(!showOptions);
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
}}
|
||||
{(setSelectedOption || setSelectedOptions) &&
|
||||
!blockAddNewGlobalVariable && (
|
||||
<span
|
||||
className={cn(
|
||||
onChange && setSelectedOption && selectedOption !== ""
|
||||
? "text-accent-emerald-foreground"
|
||||
: "text-placeholder-foreground",
|
||||
!disabled && "hover:text-foreground",
|
||||
password && selectedOption === "" ? "right-8" : "right-0",
|
||||
"absolute inset-y-0 flex items-center pr-2.5",
|
||||
disabled && "cursor-not-allowed opacity-50",
|
||||
)}
|
||||
>
|
||||
<ForwardedIconComponent
|
||||
name={
|
||||
getIconName(
|
||||
disabled!,
|
||||
selectedOption!,
|
||||
optionsIcon,
|
||||
nodeStyle!,
|
||||
isToolMode!,
|
||||
) || "ChevronsUpDown"
|
||||
}
|
||||
<button
|
||||
disabled={disabled}
|
||||
onClick={(e) => {
|
||||
if (disabled) return;
|
||||
setShowOptions(!showOptions);
|
||||
e.preventDefault();
|
||||
e.stopPropagation();
|
||||
}}
|
||||
className={cn(
|
||||
disabled ? "cursor-grab text-placeholder" : "cursor-pointer",
|
||||
"icon-size",
|
||||
onChange && setSelectedOption && selectedOption !== ""
|
||||
? "text-accent-emerald-foreground"
|
||||
: "text-placeholder-foreground",
|
||||
!disabled && "hover:text-foreground",
|
||||
)}
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
aria-hidden="true"
|
||||
/>
|
||||
</button>
|
||||
</span>
|
||||
)}
|
||||
>
|
||||
<ForwardedIconComponent
|
||||
name={
|
||||
getIconName(
|
||||
disabled!,
|
||||
selectedOption!,
|
||||
optionsIcon,
|
||||
nodeStyle!,
|
||||
isToolMode!,
|
||||
) || "ChevronsUpDown"
|
||||
}
|
||||
className={cn(
|
||||
disabled ? "cursor-grab text-placeholder" : "cursor-pointer",
|
||||
"icon-size",
|
||||
)}
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
aria-hidden="true"
|
||||
/>
|
||||
</button>
|
||||
</span>
|
||||
)}
|
||||
|
||||
{password && (!setSelectedOption || selectedOption === "") && (
|
||||
<button
|
||||
|
|
|
|||
|
|
@ -2,6 +2,8 @@ import {
|
|||
useDeleteGlobalVariables,
|
||||
useGetGlobalVariables,
|
||||
} from "@/controllers/API/queries/variables";
|
||||
import GeneralDeleteConfirmationModal from "@/shared/components/delete-confirmation-modal";
|
||||
import GeneralGlobalVariableModal from "@/shared/components/global-variable-modal";
|
||||
import { useGlobalVariablesStore } from "@/stores/globalVariablesStore/globalVariables";
|
||||
import { useEffect } from "react";
|
||||
import DeleteConfirmationModal from "../../../../../modals/deleteConfirmationModal";
|
||||
|
|
@ -26,10 +28,7 @@ export default function InputGlobalComponent({
|
|||
placeholder,
|
||||
isToolMode = false,
|
||||
}: InputProps<string, InputGlobalComponentType>): JSX.Element {
|
||||
const setErrorData = useAlertStore((state) => state.setErrorData);
|
||||
|
||||
const { data: globalVariables } = useGetGlobalVariables();
|
||||
const { mutate: mutateDeleteGlobalVariable } = useDeleteGlobalVariables();
|
||||
const unavailableFields = useGlobalVariablesStore(
|
||||
(state) => state.unavailableFields,
|
||||
);
|
||||
|
|
@ -59,31 +58,9 @@ export default function InputGlobalComponent({
|
|||
}
|
||||
}, [globalVariables, unavailableFields]);
|
||||
|
||||
async function handleDelete(key: string) {
|
||||
if (!globalVariables) return;
|
||||
const id = globalVariables.find((variable) => variable.name === key)?.id;
|
||||
if (id !== undefined) {
|
||||
mutateDeleteGlobalVariable(
|
||||
{ id },
|
||||
{
|
||||
onSuccess: () => {
|
||||
if (value === key && load_from_db) {
|
||||
handleOnNewValue({ value: "", load_from_db: false });
|
||||
}
|
||||
},
|
||||
onError: () => {
|
||||
setErrorData({
|
||||
title: "Error deleting variable",
|
||||
list: [cn("ID not found for variable: ", key)],
|
||||
});
|
||||
},
|
||||
},
|
||||
);
|
||||
} else {
|
||||
setErrorData({
|
||||
title: "Error deleting variable",
|
||||
list: [cn("ID not found for variable: ", key)],
|
||||
});
|
||||
function handleDelete(key: string) {
|
||||
if (value === key && load_from_db) {
|
||||
handleOnNewValue({ value: "", load_from_db: false });
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -100,43 +77,12 @@ export default function InputGlobalComponent({
|
|||
options={globalVariables?.map((variable) => variable.name) ?? []}
|
||||
optionsPlaceholder={"Global Variables"}
|
||||
optionsIcon="Globe"
|
||||
optionsButton={
|
||||
<GlobalVariableModal disabled={disabled}>
|
||||
<CommandItem value="doNotFilter-addNewVariable">
|
||||
<ForwardedIconComponent
|
||||
name="Plus"
|
||||
className={cn("mr-2 h-4 w-4 text-primary")}
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<span>Add New Variable</span>
|
||||
</CommandItem>
|
||||
</GlobalVariableModal>
|
||||
}
|
||||
optionsButton={<GeneralGlobalVariableModal />}
|
||||
optionButton={(option) => (
|
||||
<DeleteConfirmationModal
|
||||
onConfirm={(e) => {
|
||||
e.stopPropagation();
|
||||
e.preventDefault();
|
||||
handleDelete(option);
|
||||
}}
|
||||
description={'variable "' + option + '"'}
|
||||
asChild
|
||||
>
|
||||
<button
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
}}
|
||||
className="pr-1"
|
||||
>
|
||||
<ForwardedIconComponent
|
||||
name="Trash2"
|
||||
className={cn(
|
||||
"h-4 w-4 text-primary opacity-0 hover:text-status-red group-hover:opacity-100",
|
||||
)}
|
||||
aria-hidden="true"
|
||||
/>
|
||||
</button>
|
||||
</DeleteConfirmationModal>
|
||||
<GeneralDeleteConfirmationModal
|
||||
option={option}
|
||||
onConfirmDelete={() => handleDelete(option)}
|
||||
/>
|
||||
)}
|
||||
selectedOption={
|
||||
load_from_db &&
|
||||
|
|
|
|||
|
|
@ -14,6 +14,8 @@ const buttonVariants = cva(
|
|||
"bg-destructive text-destructive-foreground hover:bg-destructive/90",
|
||||
outline:
|
||||
"border border-input hover:bg-input hover:text-accent-foreground ",
|
||||
outlineAmber:
|
||||
"border border-accent-amber-foreground hover:border-accent-amber",
|
||||
primary:
|
||||
"border bg-background text-secondary-foreground hover:bg-muted hover:shadow-sm",
|
||||
warning:
|
||||
|
|
@ -31,6 +33,7 @@ const buttonVariants = cva(
|
|||
},
|
||||
size: {
|
||||
default: "h-10 py-2 px-4",
|
||||
md: "h-8 py-2 px-4",
|
||||
sm: "h-9 px-3 rounded-md",
|
||||
xs: "py-0.5 px-3 rounded-md",
|
||||
lg: "h-11 px-8 rounded-md",
|
||||
|
|
|
|||
|
|
@ -950,8 +950,8 @@ export const LANGFLOW_REFRESH_TOKEN = "refresh_token_lf";
|
|||
|
||||
export const LANGFLOW_ACCESS_TOKEN_EXPIRE_SECONDS = 60 * 60 - 60 * 60 * 0.1;
|
||||
export const LANGFLOW_ACCESS_TOKEN_EXPIRE_SECONDS_ENV =
|
||||
Number(process.env.ACCESS_TOKEN_EXPIRE_SECONDS) -
|
||||
Number(process.env.ACCESS_TOKEN_EXPIRE_SECONDS) * 0.1;
|
||||
Number(process.env?.ACCESS_TOKEN_EXPIRE_SECONDS ?? 60) -
|
||||
Number(process.env?.ACCESS_TOKEN_EXPIRE_SECONDS ?? 60) * 0.1;
|
||||
export const TEXT_FIELD_TYPES: string[] = ["str", "SecretStr"];
|
||||
export const NODE_WIDTH = 384;
|
||||
export const NODE_HEIGHT = NODE_WIDTH * 3;
|
||||
|
|
@ -1006,6 +1006,7 @@ export const DEFAULT_PLACEHOLDER = "Type something...";
|
|||
|
||||
export const DEFAULT_TOOLSET_PLACEHOLDER = "Used as a tool";
|
||||
|
||||
export const SAVE_API_KEY_ALERT = "API key saved successfully";
|
||||
export const PLAYGROUND_BUTTON_NAME = "Playground";
|
||||
export const POLLING_MESSAGES = {
|
||||
ENDPOINT_NOT_AVAILABLE: "Endpoint not available",
|
||||
|
|
|
|||
|
|
@ -24,6 +24,7 @@ export const URLs = {
|
|||
STARTER_PROJECTS: `starter-projects`,
|
||||
SIDEBAR_CATEGORIES: `sidebar_categories`,
|
||||
ALL: `all`,
|
||||
VOICE: `voice`,
|
||||
PUBLIC_FLOW: `/flows/public_flow`,
|
||||
} as const;
|
||||
|
||||
|
|
|
|||
|
|
@ -16,15 +16,13 @@ export const useGetTypes: useQueryFunctionType<undefined> = (options) => {
|
|||
`${getURL("ALL")}?force_refresh=true`,
|
||||
);
|
||||
const data = response?.data;
|
||||
console.log("[Types] Got types data:", data);
|
||||
setTypes(data);
|
||||
return data;
|
||||
} catch {
|
||||
(error) => {
|
||||
console.error("An error has occurred while fetching types.");
|
||||
console.log(error);
|
||||
setLoading(false);
|
||||
throw error;
|
||||
};
|
||||
} catch (error) {
|
||||
console.error("[Types] Error fetching types:", error);
|
||||
setLoading(false);
|
||||
throw error;
|
||||
}
|
||||
};
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,63 @@
|
|||
import { useMessagesStore } from "@/stores/messagesStore";
|
||||
import { UseMutationResult } from "@tanstack/react-query";
|
||||
import { ColDef, ColGroupDef } from "ag-grid-community";
|
||||
import { extractColumnsFromRows } from "../../../../utils/utils";
|
||||
import { api } from "../../api";
|
||||
import { getURL } from "../../helpers/constants";
|
||||
import { UseRequestProcessor } from "../../services/request-processor";
|
||||
|
||||
interface MessagesQueryParams {
|
||||
id?: string;
|
||||
mode: "intersection" | "union";
|
||||
excludedFields?: string[];
|
||||
params?: object;
|
||||
}
|
||||
|
||||
interface MessagesResponse {
|
||||
rows: Array<object>;
|
||||
columns: Array<ColDef | ColGroupDef>;
|
||||
}
|
||||
|
||||
export const useGetMessagesMutation = (
|
||||
options?: any,
|
||||
): UseMutationResult<
|
||||
MessagesResponse,
|
||||
unknown,
|
||||
MessagesQueryParams,
|
||||
unknown
|
||||
> => {
|
||||
const { mutate } = UseRequestProcessor();
|
||||
|
||||
const getMessagesFn = async (
|
||||
payload: MessagesQueryParams,
|
||||
): Promise<MessagesResponse> => {
|
||||
const { id, mode, excludedFields, params } = payload;
|
||||
const config = {};
|
||||
if (id) {
|
||||
config["params"] = { flow_id: id };
|
||||
}
|
||||
if (params) {
|
||||
config["params"] = { ...config["params"], ...params };
|
||||
}
|
||||
|
||||
const data = await api.get<any>(`${getURL("MESSAGES")}`, config);
|
||||
const columns = extractColumnsFromRows(data.data, mode, excludedFields);
|
||||
useMessagesStore.getState().setMessages(data.data);
|
||||
|
||||
return { rows: data.data, columns };
|
||||
};
|
||||
|
||||
// Cast the mutation to the correct type
|
||||
const mutation = mutate(
|
||||
["useGetMessagesMutation"],
|
||||
getMessagesFn,
|
||||
options,
|
||||
) as UseMutationResult<
|
||||
MessagesResponse,
|
||||
unknown,
|
||||
MessagesQueryParams,
|
||||
unknown
|
||||
>;
|
||||
|
||||
return mutation;
|
||||
};
|
||||
|
|
@ -0,0 +1,208 @@
|
|||
import { useMessagesStore } from "@/stores/messagesStore";
|
||||
import { UseMutationResult } from "@tanstack/react-query";
|
||||
import { ColDef, ColGroupDef } from "ag-grid-community";
|
||||
import { useEffect, useRef } from "react";
|
||||
import { extractColumnsFromRows } from "../../../../utils/utils";
|
||||
import { api } from "../../api";
|
||||
import { getURL } from "../../helpers/constants";
|
||||
import { UseRequestProcessor } from "../../services/request-processor";
|
||||
|
||||
interface MessagesQueryParams {
|
||||
id?: string;
|
||||
mode: "intersection" | "union";
|
||||
excludedFields?: string[];
|
||||
params?: object;
|
||||
onSuccess?: (data: MessagesResponse) => void;
|
||||
stopPollingOn?: (data: MessagesResponse) => boolean;
|
||||
}
|
||||
|
||||
interface MessagesResponse {
|
||||
rows: Array<object>;
|
||||
columns: Array<ColDef | ColGroupDef>;
|
||||
}
|
||||
|
||||
interface PollingItem {
|
||||
interval: NodeJS.Timeout;
|
||||
timestamp: number;
|
||||
id: string;
|
||||
callback: () => Promise<void>;
|
||||
}
|
||||
|
||||
const MessagesPollingManager = {
|
||||
pollingQueue: new Map<string, PollingItem[]>(),
|
||||
activePolls: new Map<string, PollingItem>(),
|
||||
|
||||
enqueuePolling(id: string, pollingItem: PollingItem) {
|
||||
if (!this.pollingQueue.has(id)) {
|
||||
this.pollingQueue.set(id, []);
|
||||
}
|
||||
this.pollingQueue.set(
|
||||
id,
|
||||
(this.pollingQueue.get(id) || []).filter(
|
||||
(item) => item.timestamp !== pollingItem.timestamp,
|
||||
),
|
||||
);
|
||||
this.pollingQueue.get(id)?.push(pollingItem);
|
||||
|
||||
if (!this.activePolls.has(id)) {
|
||||
this.startNextPolling(id);
|
||||
}
|
||||
},
|
||||
|
||||
startNextPolling(id: string) {
|
||||
const queue = this.pollingQueue.get(id) || [];
|
||||
if (queue.length === 0) {
|
||||
this.activePolls.delete(id);
|
||||
return;
|
||||
}
|
||||
|
||||
const nextPoll = queue[0];
|
||||
this.activePolls.set(id, nextPoll);
|
||||
nextPoll.callback();
|
||||
},
|
||||
|
||||
stopPoll(id: string) {
|
||||
const activePoll = this.activePolls.get(id);
|
||||
if (activePoll) {
|
||||
clearInterval(activePoll.interval);
|
||||
this.activePolls.delete(id);
|
||||
const queue = this.pollingQueue.get(id) || [];
|
||||
this.pollingQueue.set(
|
||||
id,
|
||||
queue.filter((item) => item.timestamp !== activePoll.timestamp),
|
||||
);
|
||||
this.startNextPolling(id);
|
||||
}
|
||||
},
|
||||
|
||||
stopAll() {
|
||||
this.activePolls.forEach((poll) => clearInterval(poll.interval));
|
||||
this.activePolls.clear();
|
||||
this.pollingQueue.clear();
|
||||
},
|
||||
|
||||
removeFromQueue(id: string, timestamp: number) {
|
||||
const queue = this.pollingQueue.get(id) || [];
|
||||
this.pollingQueue.set(
|
||||
id,
|
||||
queue.filter((item) => item.timestamp !== timestamp),
|
||||
);
|
||||
},
|
||||
};
|
||||
|
||||
export const useGetMessagesPollingMutation = (
|
||||
options?: any,
|
||||
): UseMutationResult<
|
||||
MessagesResponse,
|
||||
unknown,
|
||||
MessagesQueryParams,
|
||||
unknown
|
||||
> => {
|
||||
const { mutate } = UseRequestProcessor();
|
||||
const requestIdRef = useRef<string | null>(null);
|
||||
const requestInProgressRef = useRef<Record<string, boolean>>({});
|
||||
|
||||
// Default polling interval of 5 seconds (5000ms)
|
||||
const POLLING_INTERVAL = 5000;
|
||||
|
||||
const getMessagesFn = async (
|
||||
payload: MessagesQueryParams,
|
||||
): Promise<MessagesResponse> => {
|
||||
const requestId = payload.id || "default";
|
||||
|
||||
if (requestInProgressRef.current[requestId]) {
|
||||
return Promise.reject("Request already in progress");
|
||||
}
|
||||
|
||||
try {
|
||||
requestInProgressRef.current[requestId] = true;
|
||||
const { id, mode, excludedFields, params } = payload;
|
||||
const config = {};
|
||||
if (id) {
|
||||
config["params"] = { flow_id: id };
|
||||
}
|
||||
if (params) {
|
||||
config["params"] = { ...config["params"], ...params };
|
||||
}
|
||||
|
||||
const data = await api.get<any>(`${getURL("MESSAGES")}`, config);
|
||||
const columns = extractColumnsFromRows(data.data, mode, excludedFields);
|
||||
useMessagesStore.getState().setMessages(data.data);
|
||||
|
||||
return { rows: data.data, columns };
|
||||
} finally {
|
||||
requestInProgressRef.current[requestId] = false;
|
||||
}
|
||||
};
|
||||
|
||||
const startPolling = (payload: MessagesQueryParams) => {
|
||||
const requestId = payload.id || "default";
|
||||
|
||||
if (requestInProgressRef.current[requestId]) {
|
||||
return Promise.reject("Request already in progress");
|
||||
}
|
||||
|
||||
if (
|
||||
requestIdRef.current === requestId &&
|
||||
MessagesPollingManager.activePolls.has(requestId)
|
||||
) {
|
||||
return Promise.resolve({ rows: [], columns: [] });
|
||||
}
|
||||
|
||||
requestIdRef.current = requestId;
|
||||
|
||||
const timestamp = Date.now();
|
||||
const pollCallback = async () => {
|
||||
const data = await getMessagesFn(payload);
|
||||
payload.onSuccess?.(data);
|
||||
|
||||
if (payload.stopPollingOn?.(data)) {
|
||||
MessagesPollingManager.stopPoll(requestId);
|
||||
}
|
||||
};
|
||||
|
||||
const intervalId = setInterval(pollCallback, POLLING_INTERVAL);
|
||||
|
||||
const pollingItem: PollingItem = {
|
||||
interval: intervalId,
|
||||
timestamp,
|
||||
id: requestId,
|
||||
callback: pollCallback,
|
||||
};
|
||||
|
||||
MessagesPollingManager.enqueuePolling(requestId, pollingItem);
|
||||
|
||||
return getMessagesFn(payload).then((data) => {
|
||||
payload.onSuccess?.(data);
|
||||
if (payload.stopPollingOn?.(data)) {
|
||||
MessagesPollingManager.stopPoll(requestId);
|
||||
}
|
||||
return data;
|
||||
});
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
return () => {
|
||||
if (requestIdRef.current) {
|
||||
MessagesPollingManager.stopPoll(requestIdRef.current);
|
||||
}
|
||||
};
|
||||
}, []);
|
||||
|
||||
// Cast the mutation to the correct type
|
||||
const mutation = mutate(
|
||||
["useGetMessagesMutation"],
|
||||
(payload: MessagesQueryParams) =>
|
||||
startPolling(payload) ?? Promise.reject("Failed to start polling"),
|
||||
options,
|
||||
) as UseMutationResult<
|
||||
MessagesResponse,
|
||||
unknown,
|
||||
MessagesQueryParams,
|
||||
unknown
|
||||
>;
|
||||
|
||||
return mutation;
|
||||
};
|
||||
|
||||
export { MessagesPollingManager };
|
||||
|
|
@ -0,0 +1,49 @@
|
|||
import { useVoiceStore } from "@/stores/voiceStore";
|
||||
import { useQueryFunctionType } from "@/types/api";
|
||||
import { api } from "../../api";
|
||||
import { getURL } from "../../helpers/constants";
|
||||
import { UseRequestProcessor } from "../../services/request-processor";
|
||||
|
||||
export const useGetVoiceList: useQueryFunctionType<undefined, any> = (
|
||||
options,
|
||||
) => {
|
||||
const { query } = UseRequestProcessor();
|
||||
const setVoices = useVoiceStore((state) => state.setVoices);
|
||||
const voices = useVoiceStore((state) => state.voices);
|
||||
|
||||
const getVoiceListFn = async (): Promise<
|
||||
{
|
||||
name: string;
|
||||
voice_id: string;
|
||||
}[]
|
||||
> => {
|
||||
if (voices.length > 0) {
|
||||
return voices;
|
||||
}
|
||||
|
||||
const res = await api.get(`${getURL("VOICE")}/elevenlabs/voice_ids`);
|
||||
const data = res.data;
|
||||
|
||||
const voicesMapped = data.map((voice) => ({
|
||||
name: voice.name,
|
||||
value: voice.voice_id,
|
||||
}));
|
||||
|
||||
setVoices(voicesMapped);
|
||||
return voicesMapped;
|
||||
};
|
||||
|
||||
const defaultOptions = {
|
||||
refetchOnMount: false,
|
||||
refetchOnWindowFocus: false,
|
||||
staleTime: 1000 * 60 * 5,
|
||||
...options,
|
||||
};
|
||||
|
||||
const queryResult = query(
|
||||
["useGetVoiceList"],
|
||||
getVoiceListFn,
|
||||
defaultOptions,
|
||||
);
|
||||
return queryResult;
|
||||
};
|
||||
|
|
@ -11,3 +11,4 @@ export const ENABLE_DATASTAX_LANGFLOW = false;
|
|||
export const ENABLE_FILE_MANAGEMENT = true;
|
||||
export const ENABLE_PUBLISH = true;
|
||||
export const ENABLE_WIDGET = true;
|
||||
export const ENABLE_VOICE_ASSISTANT = true;
|
||||
|
|
|
|||
|
|
@ -4,7 +4,7 @@ import { Separator } from "@/components/ui/separator";
|
|||
import { cn } from "@/utils/utils";
|
||||
import IconComponent from "../../../components/common/genericIconComponent";
|
||||
import { ChatViewWrapperProps } from "../types/chat-view-wrapper";
|
||||
import ChatView from "./chatView/chat-view";
|
||||
import ChatView from "./chatView/components/chat-view";
|
||||
|
||||
export const ChatViewWrapper = ({
|
||||
selectedViewField,
|
||||
|
|
|
|||
|
|
@ -1,16 +1,13 @@
|
|||
import { Button } from "@/components/ui/button";
|
||||
import Loading from "@/components/ui/loading";
|
||||
import { usePostUploadFile } from "@/controllers/API/queries/files/use-post-upload-file";
|
||||
import useFileSizeValidator from "@/shared/hooks/use-file-size-validator";
|
||||
import useAlertStore from "@/stores/alertStore";
|
||||
import useFlowStore from "@/stores/flowStore";
|
||||
import { useUtilityStore } from "@/stores/utilityStore";
|
||||
import { useEffect, useRef } from "react";
|
||||
import { AnimatePresence, motion } from "framer-motion";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import ShortUniqueId from "short-unique-id";
|
||||
import {
|
||||
ALLOWED_IMAGE_INPUT_EXTENSIONS,
|
||||
CHAT_INPUT_PLACEHOLDER,
|
||||
CHAT_INPUT_PLACEHOLDER_SEND,
|
||||
FS_ERROR_TEXT,
|
||||
SN_ERROR_TEXT,
|
||||
} from "../../../../../constants/constants";
|
||||
|
|
@ -19,12 +16,12 @@ import {
|
|||
ChatInputType,
|
||||
FilePreviewType,
|
||||
} from "../../../../../types/components";
|
||||
import FilePreview from "../fileComponent/components/file-preview";
|
||||
import ButtonSendWrapper from "./components/button-send-wrapper";
|
||||
import TextAreaWrapper from "./components/text-area-wrapper";
|
||||
import UploadFileButton from "./components/upload-file-button";
|
||||
import InputWrapper from "./components/input-wrapper";
|
||||
import NoInputView from "./components/no-input";
|
||||
import { VoiceAssistant } from "./components/voice-assistant/voice-assistant";
|
||||
import useAutoResizeTextArea from "./hooks/use-auto-resize-text-area";
|
||||
import useFocusOnUnlock from "./hooks/use-focus-unlock";
|
||||
|
||||
export default function ChatInput({
|
||||
sendMessage,
|
||||
inputRef,
|
||||
|
|
@ -42,6 +39,8 @@ export default function ChatInput({
|
|||
const isBuilding = useFlowStore((state) => state.isBuilding);
|
||||
const chatValue = useUtilityStore((state) => state.chatValueStore);
|
||||
|
||||
const [showAudioInput, setShowAudioInput] = useState(false);
|
||||
|
||||
useFocusOnUnlock(isBuilding, inputRef);
|
||||
useAutoResizeTextArea(chatValue, inputRef);
|
||||
|
||||
|
|
@ -164,8 +163,6 @@ export default function ChatInput({
|
|||
);
|
||||
};
|
||||
|
||||
const classNameFilePreview = `flex w-full items-center gap-2 py-2 overflow-auto custom-scroll`;
|
||||
|
||||
const handleButtonClick = () => {
|
||||
fileInputRef.current!.click();
|
||||
};
|
||||
|
|
@ -177,99 +174,56 @@ export default function ChatInput({
|
|||
|
||||
if (noInput) {
|
||||
return (
|
||||
<div className="flex h-full w-full flex-col items-center justify-center">
|
||||
<div className="flex w-full flex-col items-center justify-center gap-3 rounded-md border border-input bg-muted p-2 py-4">
|
||||
{!isBuilding ? (
|
||||
<Button
|
||||
data-testid="button-send"
|
||||
className="font-semibold"
|
||||
onClick={() => {
|
||||
sendMessage({
|
||||
repeat: 1,
|
||||
});
|
||||
}}
|
||||
>
|
||||
Run Flow
|
||||
</Button>
|
||||
) : (
|
||||
<Button
|
||||
onClick={stopBuilding}
|
||||
data-testid="button-stop"
|
||||
unstyled
|
||||
className="form-modal-send-button cursor-pointer bg-muted text-foreground hover:bg-secondary-hover dark:hover:bg-input"
|
||||
>
|
||||
<div className="flex items-center gap-2 rounded-md text-[14px] font-medium">
|
||||
Stop
|
||||
<Loading className="h-[16px] w-[16px]" />
|
||||
</div>
|
||||
</Button>
|
||||
)}
|
||||
|
||||
<p className="text-muted-foreground">
|
||||
Add a{" "}
|
||||
<a
|
||||
className="underline underline-offset-4"
|
||||
target="_blank"
|
||||
href="https://docs.langflow.org/components-io#chat-input"
|
||||
>
|
||||
Chat Input
|
||||
</a>{" "}
|
||||
component to your flow to send messages.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
<NoInputView
|
||||
isBuilding={isBuilding}
|
||||
sendMessage={sendMessage}
|
||||
stopBuilding={stopBuilding}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex w-full flex-col-reverse">
|
||||
<div className="flex w-full flex-col rounded-md border border-input p-4 hover:border-muted-foreground focus:border-[1.75px] has-[:focus]:border-primary">
|
||||
<TextAreaWrapper
|
||||
isBuilding={isBuilding}
|
||||
checkSendingOk={checkSendingOk}
|
||||
send={send}
|
||||
noInput={noInput}
|
||||
chatValue={chatValue}
|
||||
CHAT_INPUT_PLACEHOLDER={CHAT_INPUT_PLACEHOLDER}
|
||||
CHAT_INPUT_PLACEHOLDER_SEND={CHAT_INPUT_PLACEHOLDER_SEND}
|
||||
inputRef={inputRef}
|
||||
files={files}
|
||||
isDragging={isDragging}
|
||||
/>
|
||||
<div className={classNameFilePreview}>
|
||||
{files.map((file) => (
|
||||
<FilePreview
|
||||
error={file.error}
|
||||
file={file.file}
|
||||
loading={file.loading}
|
||||
key={file.id}
|
||||
onDelete={() => {
|
||||
handleDeleteFile(file);
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
<div className="flex w-full items-end justify-between">
|
||||
{!playgroundPage && (
|
||||
<div className={isBuilding ? "cursor-not-allowed" : ""}>
|
||||
<UploadFileButton
|
||||
isBuilding={isBuilding}
|
||||
fileInputRef={fileInputRef}
|
||||
handleFileChange={handleFileChange}
|
||||
handleButtonClick={handleButtonClick}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<div className={playgroundPage ? "ml-auto" : ""}>
|
||||
<ButtonSendWrapper
|
||||
send={send}
|
||||
noInput={noInput}
|
||||
chatValue={chatValue}
|
||||
files={files}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<AnimatePresence mode="wait">
|
||||
{showAudioInput ? (
|
||||
<motion.div
|
||||
key="voice-assistant"
|
||||
initial={{ opacity: 0 }}
|
||||
animate={{ opacity: 1 }}
|
||||
exit={{ opacity: 0 }}
|
||||
transition={{ duration: 0.2 }}
|
||||
>
|
||||
<VoiceAssistant
|
||||
flowId={currentFlowId}
|
||||
setShowAudioInput={setShowAudioInput}
|
||||
/>
|
||||
</motion.div>
|
||||
) : (
|
||||
<motion.div
|
||||
key="input-wrapper"
|
||||
initial={{ opacity: 0 }}
|
||||
animate={{ opacity: 1 }}
|
||||
exit={{ opacity: 0 }}
|
||||
transition={{ duration: 0.2 }}
|
||||
>
|
||||
<InputWrapper
|
||||
isBuilding={isBuilding}
|
||||
checkSendingOk={checkSendingOk}
|
||||
send={send}
|
||||
noInput={noInput}
|
||||
chatValue={chatValue}
|
||||
inputRef={inputRef}
|
||||
files={files}
|
||||
isDragging={isDragging}
|
||||
handleDeleteFile={handleDeleteFile}
|
||||
fileInputRef={fileInputRef}
|
||||
handleFileChange={handleFileChange}
|
||||
handleButtonClick={handleButtonClick}
|
||||
setShowAudioInput={setShowAudioInput}
|
||||
currentFlowId={currentFlowId}
|
||||
playgroundPage={playgroundPage}
|
||||
/>
|
||||
</motion.div>
|
||||
)}
|
||||
</AnimatePresence>
|
||||
);
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,113 @@
|
|||
import { ENABLE_VOICE_ASSISTANT } from "@/customization/feature-flags";
|
||||
import { FilePreviewType } from "@/types/components";
|
||||
import React from "react";
|
||||
import {
|
||||
CHAT_INPUT_PLACEHOLDER,
|
||||
CHAT_INPUT_PLACEHOLDER_SEND,
|
||||
} from "../../../../../../constants/constants";
|
||||
import FilePreview from "../../fileComponent/components/file-preview";
|
||||
import ButtonSendWrapper from "./button-send-wrapper";
|
||||
import TextAreaWrapper from "./text-area-wrapper";
|
||||
import UploadFileButton from "./upload-file-button";
|
||||
import VoiceButton from "./voice-assistant/components/voice-button";
|
||||
|
||||
interface InputWrapperProps {
|
||||
isBuilding: boolean;
|
||||
checkSendingOk: (event: React.KeyboardEvent<HTMLTextAreaElement>) => boolean;
|
||||
send: () => void;
|
||||
noInput: boolean;
|
||||
chatValue: string;
|
||||
inputRef: React.RefObject<HTMLTextAreaElement>;
|
||||
files: FilePreviewType[];
|
||||
isDragging: boolean;
|
||||
handleDeleteFile: (file: FilePreviewType) => void;
|
||||
fileInputRef: React.RefObject<HTMLInputElement>;
|
||||
handleFileChange: (event: React.ChangeEvent<HTMLInputElement>) => void;
|
||||
handleButtonClick: () => void;
|
||||
setShowAudioInput: (value: boolean) => void;
|
||||
currentFlowId: string;
|
||||
playgroundPage: boolean;
|
||||
}
|
||||
|
||||
const InputWrapper: React.FC<InputWrapperProps> = ({
|
||||
isBuilding,
|
||||
checkSendingOk,
|
||||
send,
|
||||
noInput,
|
||||
chatValue,
|
||||
inputRef,
|
||||
files,
|
||||
isDragging,
|
||||
handleDeleteFile,
|
||||
fileInputRef,
|
||||
handleFileChange,
|
||||
handleButtonClick,
|
||||
setShowAudioInput,
|
||||
currentFlowId,
|
||||
playgroundPage,
|
||||
}) => {
|
||||
const classNameFilePreview = `flex w-full items-center gap-2 py-2 overflow-auto custom-scroll`;
|
||||
|
||||
return (
|
||||
<div className="flex w-full flex-col-reverse">
|
||||
<div
|
||||
data-testid="input-wrapper"
|
||||
className="flex w-full flex-col rounded-md border border-input p-4 hover:border-muted-foreground focus:border-[1.75px] has-[:focus]:border-primary"
|
||||
>
|
||||
<TextAreaWrapper
|
||||
isBuilding={isBuilding}
|
||||
checkSendingOk={checkSendingOk}
|
||||
send={send}
|
||||
noInput={noInput}
|
||||
chatValue={chatValue}
|
||||
CHAT_INPUT_PLACEHOLDER={CHAT_INPUT_PLACEHOLDER}
|
||||
CHAT_INPUT_PLACEHOLDER_SEND={CHAT_INPUT_PLACEHOLDER_SEND}
|
||||
inputRef={inputRef}
|
||||
files={files}
|
||||
isDragging={isDragging}
|
||||
/>
|
||||
<div className={classNameFilePreview}>
|
||||
{files.map((file) => (
|
||||
<FilePreview
|
||||
error={file.error}
|
||||
file={file.file}
|
||||
loading={file.loading}
|
||||
key={file.id}
|
||||
onDelete={() => {
|
||||
handleDeleteFile(file);
|
||||
}}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
<div className="flex w-full items-end justify-between">
|
||||
{!playgroundPage && (
|
||||
<div className={isBuilding ? "cursor-not-allowed" : ""}>
|
||||
<UploadFileButton
|
||||
isBuilding={isBuilding}
|
||||
fileInputRef={fileInputRef}
|
||||
handleFileChange={handleFileChange}
|
||||
handleButtonClick={handleButtonClick}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<div className="flex items-center gap-2">
|
||||
{ENABLE_VOICE_ASSISTANT && (
|
||||
<VoiceButton toggleRecording={() => setShowAudioInput(true)} />
|
||||
)}
|
||||
|
||||
<div className={playgroundPage ? "ml-auto" : ""}>
|
||||
<ButtonSendWrapper
|
||||
send={send}
|
||||
noInput={noInput}
|
||||
chatValue={chatValue}
|
||||
files={files}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default InputWrapper;
|
||||
|
|
@ -0,0 +1,64 @@
|
|||
import { Button } from "@/components/ui/button";
|
||||
import Loading from "@/components/ui/loading";
|
||||
import React, { useEffect, useRef, useState } from "react";
|
||||
import IconComponent from "../../../../../../components/common/genericIconComponent";
|
||||
import { ICON_STROKE_WIDTH } from "../../../../../../constants/constants";
|
||||
import { cn } from "../../../../../../utils/utils";
|
||||
|
||||
interface NoInputViewProps {
|
||||
isBuilding: boolean;
|
||||
sendMessage: (args: { repeat: number }) => void;
|
||||
stopBuilding: () => void;
|
||||
}
|
||||
|
||||
const NoInputView: React.FC<NoInputViewProps> = ({
|
||||
isBuilding,
|
||||
sendMessage,
|
||||
stopBuilding,
|
||||
}) => {
|
||||
return (
|
||||
<div className="flex h-full w-full flex-col items-center justify-center">
|
||||
<div className="flex w-full flex-col items-center justify-center gap-3 rounded-md border border-input bg-muted p-2 py-4">
|
||||
{!isBuilding ? (
|
||||
<Button
|
||||
data-testid="button-send"
|
||||
className="font-semibold"
|
||||
onClick={() => {
|
||||
sendMessage({
|
||||
repeat: 1,
|
||||
});
|
||||
}}
|
||||
>
|
||||
Run Flow
|
||||
</Button>
|
||||
) : (
|
||||
<Button
|
||||
onClick={stopBuilding}
|
||||
data-testid="button-stop"
|
||||
unstyled
|
||||
className="form-modal-send-button cursor-pointer bg-muted text-foreground hover:bg-secondary-hover dark:hover:bg-input"
|
||||
>
|
||||
<div className="flex items-center gap-2 rounded-md text-[14px] font-medium">
|
||||
Stop
|
||||
<Loading className="h-[16px] w-[16px]" />
|
||||
</div>
|
||||
</Button>
|
||||
)}
|
||||
|
||||
<p className="text-muted-foreground">
|
||||
Add a{" "}
|
||||
<a
|
||||
className="underline underline-offset-4"
|
||||
target="_blank"
|
||||
href="https://docs.langflow.org/components-io#chat-input"
|
||||
>
|
||||
Chat Input
|
||||
</a>{" "}
|
||||
component to your flow to send messages.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default NoInputView;
|
||||
|
|
@ -24,7 +24,7 @@ const UploadFileButton = ({
|
|||
/>
|
||||
<Button
|
||||
disabled={isBuilding}
|
||||
className={`flex h-[32px] w-[32px] items-center justify-center rounded-md bg-muted font-bold transition-all ${
|
||||
className={`btn-playground-actions ${
|
||||
isBuilding
|
||||
? "cursor-not-allowed"
|
||||
: "text-muted-foreground hover:text-primary"
|
||||
|
|
|
|||
|
|
@ -0,0 +1,431 @@
|
|||
import IconComponent from "@/components/common/genericIconComponent";
|
||||
import ShadTooltip from "@/components/common/shadTooltipComponent";
|
||||
import InputComponent from "@/components/core/parameterRenderComponent/components/inputComponent";
|
||||
import { getPlaceholder } from "@/components/core/parameterRenderComponent/helpers/get-placeholder-disabled";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import {
|
||||
DropdownMenu,
|
||||
DropdownMenuContent,
|
||||
DropdownMenuTrigger,
|
||||
} from "@/components/ui/dropdown-menu";
|
||||
import { Separator } from "@/components/ui/separator";
|
||||
import { useGetVoiceList } from "@/controllers/API/queries/voice/use-get-voice-list";
|
||||
import GeneralDeleteConfirmationModal from "@/shared/components/delete-confirmation-modal";
|
||||
import GeneralGlobalVariableModal from "@/shared/components/global-variable-modal";
|
||||
import { useGlobalVariablesStore } from "@/stores/globalVariablesStore/globalVariables";
|
||||
import { useVoiceStore } from "@/stores/voiceStore";
|
||||
import { getLocalStorage, setLocalStorage } from "@/utils/local-storage-util";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import AudioSettingsHeader from "./components/header";
|
||||
import LanguageSelect from "./components/language-select";
|
||||
import MicrophoneSelect from "./components/microphone-select";
|
||||
import VoiceSelect from "./components/voice-select";
|
||||
|
||||
const ALL_LANGUAGES = [
|
||||
{ value: "en-US", name: "English (US)" },
|
||||
{ value: "en-GB", name: "English (UK)" },
|
||||
{ value: "it-IT", name: "Italian" },
|
||||
{ value: "fr-FR", name: "French" },
|
||||
{ value: "es-ES", name: "Spanish" },
|
||||
{ value: "de-DE", name: "German" },
|
||||
{ value: "ja-JP", name: "Japanese" },
|
||||
{ value: "pt-BR", name: "Portuguese (Brazil)" },
|
||||
{ value: "zh-CN", name: "Chinese (Simplified)" },
|
||||
{ value: "ru-RU", name: "Russian" },
|
||||
{ value: "ar-SA", name: "Arabic" },
|
||||
{ value: "hi-IN", name: "Hindi" },
|
||||
];
|
||||
|
||||
interface SettingsVoiceModalProps {
|
||||
children?: React.ReactNode;
|
||||
userOpenaiApiKey?: string;
|
||||
userElevenLabsApiKey?: string;
|
||||
hasElevenLabsApiKeyEnv?: boolean;
|
||||
setShowSettingsModal: (
|
||||
open: boolean,
|
||||
openaiApiKey: string,
|
||||
elevenLabsApiKey: string,
|
||||
) => void;
|
||||
hasOpenAIAPIKey: boolean;
|
||||
language?: string;
|
||||
setLanguage?: (language: string) => void;
|
||||
handleClickSaveOpenAIApiKey: (openaiApiKey: string) => void;
|
||||
isEditingOpenAIKey: boolean;
|
||||
setIsEditingOpenAIKey: (isEditingOpenAIKey: boolean) => void;
|
||||
}
|
||||
|
||||
const SettingsVoiceModal = ({
|
||||
children,
|
||||
userOpenaiApiKey,
|
||||
userElevenLabsApiKey,
|
||||
setShowSettingsModal,
|
||||
hasOpenAIAPIKey,
|
||||
language,
|
||||
setLanguage,
|
||||
handleClickSaveOpenAIApiKey,
|
||||
isEditingOpenAIKey,
|
||||
setIsEditingOpenAIKey,
|
||||
}: SettingsVoiceModalProps) => {
|
||||
const popupRef = useRef<HTMLDivElement>(null);
|
||||
const [voice, setVoice] = useState<string>("alloy");
|
||||
const [open, setOpen] = useState<boolean>(false);
|
||||
const voices = useVoiceStore((state) => state.voices);
|
||||
const shouldFetchVoices = voices.length === 0;
|
||||
const [openaiApiKey, setOpenaiApiKey] = useState<string>(
|
||||
userOpenaiApiKey ?? "",
|
||||
);
|
||||
const [elevenLabsApiKey, setElevenLabsApiKey] = useState<string>(
|
||||
userElevenLabsApiKey ?? "",
|
||||
);
|
||||
|
||||
const globalVariables = useGlobalVariablesStore(
|
||||
(state) => state.globalVariablesEntries,
|
||||
);
|
||||
|
||||
const openaiVoices = useVoiceStore((state) => state.openaiVoices);
|
||||
const [allVoices, setAllVoices] = useState<
|
||||
{
|
||||
name: string;
|
||||
value: string;
|
||||
}[]
|
||||
>([]);
|
||||
|
||||
const saveButtonClicked = useRef(false);
|
||||
|
||||
const {
|
||||
data: voiceList,
|
||||
isFetched,
|
||||
refetch,
|
||||
} = useGetVoiceList({
|
||||
enabled: shouldFetchVoices,
|
||||
refetchOnMount: shouldFetchVoices,
|
||||
refetchOnWindowFocus: shouldFetchVoices,
|
||||
staleTime: Infinity,
|
||||
});
|
||||
|
||||
const [microphones, setMicrophones] = useState<MediaDeviceInfo[]>([]);
|
||||
const [selectedMicrophone, setSelectedMicrophone] = useState<string>("");
|
||||
|
||||
const [currentLanguage, setCurrentLanguage] = useState(
|
||||
localStorage.getItem("lf_preferred_language") || "en-US",
|
||||
);
|
||||
|
||||
useEffect(() => {
|
||||
if (isFetched) {
|
||||
if (voiceList) {
|
||||
const allVoicesMerged = [...openaiVoices, ...voiceList];
|
||||
|
||||
voiceList.length > 0
|
||||
? setAllVoices(allVoicesMerged)
|
||||
: setAllVoices(openaiVoices);
|
||||
} else {
|
||||
setAllVoices(openaiVoices);
|
||||
}
|
||||
}
|
||||
}, [voiceList, isFetched, userElevenLabsApiKey]);
|
||||
|
||||
useEffect(() => {
|
||||
const audioSettings = JSON.parse(
|
||||
getLocalStorage("lf_audio_settings_playground") || "{}",
|
||||
);
|
||||
if (isFetched) {
|
||||
if (audioSettings.provider) {
|
||||
setVoice(audioSettings.voice);
|
||||
} else {
|
||||
setVoice(openaiVoices[0].value);
|
||||
}
|
||||
} else {
|
||||
setVoice(openaiVoices[0].value);
|
||||
}
|
||||
}, [isFetched]);
|
||||
|
||||
const handleSetVoice = (value: string) => {
|
||||
setVoice(value);
|
||||
const isOpenAiVoice = openaiVoices.some((voice) => voice.value === value);
|
||||
if (isOpenAiVoice) {
|
||||
setLocalStorage(
|
||||
"lf_audio_settings_playground",
|
||||
JSON.stringify({
|
||||
provider: "openai",
|
||||
voice: value,
|
||||
}),
|
||||
);
|
||||
} else {
|
||||
setLocalStorage(
|
||||
"lf_audio_settings_playground",
|
||||
JSON.stringify({
|
||||
provider: "elevenlabs",
|
||||
voice: value,
|
||||
}),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
const onOpenChangeDropdownMenu = (open: boolean) => {
|
||||
setOpen(open);
|
||||
setShowSettingsModal(open, openaiApiKey, elevenLabsApiKey);
|
||||
};
|
||||
|
||||
const checkIfGlobalVariableExists = (variable: string) => {
|
||||
return globalVariables?.map((variable) => variable).includes(variable);
|
||||
};
|
||||
|
||||
const handleSetMicrophone = (deviceId: string) => {
|
||||
setSelectedMicrophone(deviceId);
|
||||
localStorage.setItem("lf_selected_microphone", deviceId);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
setOpenaiApiKey(userOpenaiApiKey ?? "");
|
||||
}, [userOpenaiApiKey]);
|
||||
|
||||
useEffect(() => {
|
||||
setElevenLabsApiKey(userElevenLabsApiKey ?? "");
|
||||
|
||||
if (!userElevenLabsApiKey) {
|
||||
handleSetVoice(openaiVoices[0].value);
|
||||
setAllVoices(openaiVoices);
|
||||
return;
|
||||
}
|
||||
|
||||
refetch();
|
||||
}, [userElevenLabsApiKey]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!hasOpenAIAPIKey) {
|
||||
setOpen(true);
|
||||
}
|
||||
}, [hasOpenAIAPIKey]);
|
||||
|
||||
const handleSetLanguage = (value: string) => {
|
||||
setCurrentLanguage(value);
|
||||
localStorage.setItem("lf_preferred_language", value);
|
||||
if (setLanguage) {
|
||||
setLanguage(value);
|
||||
}
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
if (language) {
|
||||
setCurrentLanguage(language);
|
||||
}
|
||||
}, [language]);
|
||||
|
||||
const handleClickSaveApiKey = (value: string) => {
|
||||
if (!value) return;
|
||||
if (value === "OPENAI_API_KEY") {
|
||||
setIsEditingOpenAIKey(false);
|
||||
return;
|
||||
}
|
||||
handleClickSaveOpenAIApiKey(value);
|
||||
saveButtonClicked.current = true;
|
||||
};
|
||||
|
||||
const handleOpenAIKeyChange = (value: string) => {
|
||||
if (!value) return;
|
||||
setOpenaiApiKey(value);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
setOpenaiApiKey("");
|
||||
}, [isEditingOpenAIKey]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!open) {
|
||||
setIsEditingOpenAIKey(false);
|
||||
}
|
||||
}, [open]);
|
||||
|
||||
const showAddOpenAIKeyButton = !hasOpenAIAPIKey || isEditingOpenAIKey;
|
||||
const showAllSettings = hasOpenAIAPIKey && !isEditingOpenAIKey;
|
||||
|
||||
return (
|
||||
<>
|
||||
<DropdownMenu open={open} onOpenChange={onOpenChangeDropdownMenu}>
|
||||
<DropdownMenuTrigger data-dropdown-trigger="true">
|
||||
{children}
|
||||
</DropdownMenuTrigger>
|
||||
<DropdownMenuContent
|
||||
className="w-[324px] rounded-xl shadow-lg"
|
||||
sideOffset={18}
|
||||
alignOffset={-54}
|
||||
align="end"
|
||||
>
|
||||
<div ref={popupRef} className="rounded-3xl">
|
||||
<div>
|
||||
<AudioSettingsHeader />
|
||||
<Separator className="w-full" />
|
||||
|
||||
<div className="w-full space-y-4 p-4">
|
||||
<div className="grid w-full items-center gap-2">
|
||||
<span className="flex items-center text-sm">
|
||||
OpenAI API Key
|
||||
<span className="ml-1 text-destructive">*</span>
|
||||
<ShadTooltip content="OpenAI API key is required to use the voice assistant.">
|
||||
<div>
|
||||
<IconComponent
|
||||
name="Info"
|
||||
strokeWidth={2}
|
||||
className="relative -top-[3px] left-1 h-[14px] w-[14px] text-placeholder"
|
||||
/>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
</span>
|
||||
|
||||
{showAddOpenAIKeyButton && (
|
||||
<>
|
||||
<InputComponent
|
||||
isObjectOption={false}
|
||||
password={false}
|
||||
nodeStyle
|
||||
popoverWidth="16rem"
|
||||
placeholder={getPlaceholder(
|
||||
false,
|
||||
"Enter your OpenAI API key",
|
||||
)}
|
||||
id="openai-api-key"
|
||||
options={
|
||||
globalVariables?.map((variable) => variable) ?? []
|
||||
}
|
||||
optionsPlaceholder={"Global Variables"}
|
||||
optionsIcon="Globe"
|
||||
optionsButton={<GeneralGlobalVariableModal />}
|
||||
optionButton={(option) => (
|
||||
<GeneralDeleteConfirmationModal
|
||||
option={option}
|
||||
onConfirmDelete={() => {}}
|
||||
/>
|
||||
)}
|
||||
value={openaiApiKey}
|
||||
onChange={handleOpenAIKeyChange}
|
||||
selectedOption={
|
||||
checkIfGlobalVariableExists(openaiApiKey)
|
||||
? openaiApiKey
|
||||
: ""
|
||||
}
|
||||
commandWidth="11rem"
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
|
||||
{showAllSettings && (
|
||||
<>
|
||||
<Button
|
||||
variant="primary"
|
||||
className="w-full"
|
||||
onClick={() => setIsEditingOpenAIKey(true)}
|
||||
size="md"
|
||||
>
|
||||
Edit
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{!showAllSettings && (
|
||||
<div className="flex gap-2">
|
||||
<Button
|
||||
onClick={() => setIsEditingOpenAIKey(false)}
|
||||
variant="primary"
|
||||
size="md"
|
||||
className="w-full"
|
||||
data-testid="voice-assistant-settings-modal-cancel-button"
|
||||
>
|
||||
Cancel
|
||||
</Button>
|
||||
<Button
|
||||
onClick={() => handleClickSaveApiKey(openaiApiKey)}
|
||||
className="w-full"
|
||||
disabled={!openaiApiKey}
|
||||
size="md"
|
||||
data-testid="voice-assistant-settings-modal-save-button"
|
||||
>
|
||||
{isEditingOpenAIKey ? "Update" : "Save"}
|
||||
</Button>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{showAllSettings && (
|
||||
<>
|
||||
<div className="grid w-full items-center gap-2">
|
||||
<span className="flex items-center text-sm">
|
||||
ElevenLabs API Key
|
||||
<ShadTooltip content="If you have an ElevenLabs API key, you can select ElevenLabs voices.">
|
||||
<div>
|
||||
<IconComponent
|
||||
name="Info"
|
||||
strokeWidth={2}
|
||||
className="relative -top-[3px] left-1 h-[14px] w-[14px] text-placeholder"
|
||||
/>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
</span>
|
||||
|
||||
<InputComponent
|
||||
isObjectOption={false}
|
||||
password
|
||||
nodeStyle
|
||||
popoverWidth="16rem"
|
||||
placeholder={getPlaceholder(
|
||||
false,
|
||||
"Enter your ElevenLabs API key",
|
||||
)}
|
||||
id="eleven-labs-api-key"
|
||||
options={
|
||||
globalVariables?.map((variable) => variable) ?? []
|
||||
}
|
||||
optionsPlaceholder={"Global Variables"}
|
||||
optionsIcon="Globe"
|
||||
optionsButton={<GeneralGlobalVariableModal />}
|
||||
optionButton={(option) => (
|
||||
<GeneralDeleteConfirmationModal
|
||||
option={option}
|
||||
onConfirmDelete={() => {}}
|
||||
/>
|
||||
)}
|
||||
value={elevenLabsApiKey}
|
||||
onChange={(value) => {
|
||||
setElevenLabsApiKey(value);
|
||||
}}
|
||||
selectedOption={
|
||||
checkIfGlobalVariableExists(elevenLabsApiKey)
|
||||
? elevenLabsApiKey
|
||||
: ""
|
||||
}
|
||||
setSelectedOption={setElevenLabsApiKey}
|
||||
commandWidth="11rem"
|
||||
blockAddNewGlobalVariable
|
||||
/>
|
||||
</div>
|
||||
|
||||
<VoiceSelect
|
||||
voice={voice}
|
||||
handleSetVoice={handleSetVoice}
|
||||
allVoices={allVoices}
|
||||
/>
|
||||
|
||||
<MicrophoneSelect
|
||||
selectedMicrophone={selectedMicrophone}
|
||||
handleSetMicrophone={handleSetMicrophone}
|
||||
microphones={microphones}
|
||||
setMicrophones={setMicrophones}
|
||||
setSelectedMicrophone={setSelectedMicrophone}
|
||||
/>
|
||||
|
||||
<LanguageSelect
|
||||
language={currentLanguage}
|
||||
handleSetLanguage={handleSetLanguage}
|
||||
allLanguages={ALL_LANGUAGES}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default SettingsVoiceModal;
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
import React from "react";
|
||||
import IconComponent from "../../../../../../../../../../components/common/genericIconComponent";
|
||||
import { ICON_STROKE_WIDTH } from "../../../../../../../../../../constants/constants";
|
||||
|
||||
const AudioSettingsHeader = () => {
|
||||
return (
|
||||
<div
|
||||
className="grid gap-1 p-4"
|
||||
data-testid="voice-assistant-settings-modal-header"
|
||||
>
|
||||
<p className="flex items-center gap-2 text-sm text-primary">
|
||||
<IconComponent
|
||||
name="Settings"
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
className="h-4 w-4 text-muted-foreground hover:text-foreground"
|
||||
/>
|
||||
Voice settings
|
||||
</p>
|
||||
<p className="text-[13px] leading-4 text-muted-foreground">
|
||||
Voice chat is powered by OpenAI. You can also add more voices with
|
||||
ElevenLabs.
|
||||
</p>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default AudioSettingsHeader;
|
||||
|
|
@ -0,0 +1,58 @@
|
|||
import IconComponent from "../../../../../../../../../../components/common/genericIconComponent";
|
||||
import ShadTooltip from "../../../../../../../../../../components/common/shadTooltipComponent";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectGroup,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "../../../../../../../../../../components/ui/select";
|
||||
|
||||
interface LanguageSelectProps {
|
||||
language: string;
|
||||
handleSetLanguage: (value: string) => void;
|
||||
allLanguages: { value: string; name: string }[];
|
||||
}
|
||||
|
||||
const LanguageSelect = ({
|
||||
language,
|
||||
handleSetLanguage,
|
||||
allLanguages,
|
||||
}: LanguageSelectProps) => {
|
||||
return (
|
||||
<div className="grid w-full items-center gap-2">
|
||||
<span className="flex w-full items-center text-sm">
|
||||
Preferred Language
|
||||
<ShadTooltip content="Select the language for speech recognition">
|
||||
<div>
|
||||
<IconComponent
|
||||
name="Info"
|
||||
strokeWidth={2}
|
||||
className="relative -top-[3px] left-1 h-[14px] w-[14px] text-placeholder"
|
||||
/>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
</span>
|
||||
|
||||
<Select value={language} onValueChange={handleSetLanguage}>
|
||||
<SelectTrigger className="h-9 w-full">
|
||||
<SelectValue placeholder="Select language" />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="max-h-[200px]">
|
||||
<SelectGroup>
|
||||
{allLanguages.map((lang) => (
|
||||
<SelectItem key={lang?.value} value={lang?.value}>
|
||||
<div className="max-w-[220px] truncate text-left">
|
||||
{lang?.name}
|
||||
</div>
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectGroup>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default LanguageSelect;
|
||||
|
|
@ -0,0 +1,115 @@
|
|||
import { useEffect } from "react";
|
||||
import IconComponent from "../../../../../../../../../../components/common/genericIconComponent";
|
||||
import ShadTooltip from "../../../../../../../../../../components/common/shadTooltipComponent";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectGroup,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "../../../../../../../../../../components/ui/select";
|
||||
|
||||
interface MicrophoneSelectProps {
|
||||
selectedMicrophone: string;
|
||||
handleSetMicrophone: (value: string) => void;
|
||||
microphones: MediaDeviceInfo[];
|
||||
setMicrophones: (microphones: MediaDeviceInfo[]) => void;
|
||||
setSelectedMicrophone: (microphone: string) => void;
|
||||
}
|
||||
|
||||
const MicrophoneSelect = ({
|
||||
selectedMicrophone,
|
||||
handleSetMicrophone,
|
||||
microphones,
|
||||
setMicrophones,
|
||||
setSelectedMicrophone,
|
||||
}: MicrophoneSelectProps) => {
|
||||
useEffect(() => {
|
||||
const getMicrophones = async () => {
|
||||
try {
|
||||
await navigator?.mediaDevices?.getUserMedia({ audio: true });
|
||||
|
||||
const devices = await navigator?.mediaDevices?.enumerateDevices();
|
||||
const audioInputDevices = devices?.filter(
|
||||
(device) => device.kind === "audioinput",
|
||||
);
|
||||
setMicrophones(audioInputDevices);
|
||||
|
||||
if (audioInputDevices.length > 0 && !selectedMicrophone) {
|
||||
const savedMicrophoneId = localStorage.getItem(
|
||||
"lf_selected_microphone",
|
||||
);
|
||||
if (
|
||||
savedMicrophoneId &&
|
||||
audioInputDevices.some(
|
||||
(device) => device.deviceId === savedMicrophoneId,
|
||||
)
|
||||
) {
|
||||
setSelectedMicrophone(savedMicrophoneId);
|
||||
} else {
|
||||
setSelectedMicrophone(audioInputDevices[0].deviceId);
|
||||
}
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error accessing media devices:", error);
|
||||
}
|
||||
};
|
||||
|
||||
getMicrophones();
|
||||
|
||||
navigator?.mediaDevices?.addEventListener("devicechange", getMicrophones);
|
||||
|
||||
return () => {
|
||||
navigator?.mediaDevices?.removeEventListener(
|
||||
"devicechange",
|
||||
getMicrophones,
|
||||
);
|
||||
};
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<div
|
||||
className="grid w-full items-center gap-2"
|
||||
data-testid="voice-assistant-settings-modal-microphone-select"
|
||||
>
|
||||
<span className="flex w-full items-center text-sm">
|
||||
Audio Input
|
||||
<ShadTooltip content="Select which microphone to use for voice input">
|
||||
<div>
|
||||
<IconComponent
|
||||
name="Info"
|
||||
strokeWidth={2}
|
||||
className="relative -top-[3px] left-1 h-[14px] w-[14px] text-placeholder"
|
||||
/>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
</span>
|
||||
|
||||
<Select value={selectedMicrophone} onValueChange={handleSetMicrophone}>
|
||||
<SelectTrigger className="h-9 w-full">
|
||||
<SelectValue placeholder="Select microphone" />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="max-h-[200px]">
|
||||
<SelectGroup>
|
||||
{microphones?.map((device) => (
|
||||
<SelectItem key={device?.deviceId} value={device?.deviceId}>
|
||||
<div className="max-w-[220px] truncate text-left">
|
||||
{device?.label ||
|
||||
`Microphone ${device?.deviceId?.slice(0, 5)}...`}
|
||||
</div>
|
||||
</SelectItem>
|
||||
))}
|
||||
{microphones?.length === 0 && (
|
||||
<SelectItem value="no-microphones" disabled>
|
||||
No microphones found
|
||||
</SelectItem>
|
||||
)}
|
||||
</SelectGroup>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default MicrophoneSelect;
|
||||
|
|
@ -0,0 +1,59 @@
|
|||
import IconComponent from "../../../../../../../../../../components/common/genericIconComponent";
|
||||
import ShadTooltip from "../../../../../../../../../../components/common/shadTooltipComponent";
|
||||
import {
|
||||
Select,
|
||||
SelectContent,
|
||||
SelectGroup,
|
||||
SelectItem,
|
||||
SelectTrigger,
|
||||
SelectValue,
|
||||
} from "../../../../../../../../../../components/ui/select";
|
||||
import { toTitleCase } from "../../../../../../../../../../utils/utils";
|
||||
|
||||
interface VoiceSelectProps {
|
||||
voice: string;
|
||||
handleSetVoice: (value: string) => void;
|
||||
allVoices: { value: string; name: string }[];
|
||||
}
|
||||
|
||||
const VoiceSelect = ({
|
||||
voice,
|
||||
handleSetVoice,
|
||||
allVoices,
|
||||
}: VoiceSelectProps) => {
|
||||
return (
|
||||
<div className="grid w-full items-center gap-2">
|
||||
<span className="flex w-full items-center text-sm">
|
||||
Voice
|
||||
<ShadTooltip content="You can select ElevenLabs voices if you have an ElevenLabs API key. Otherwise, you can only select OpenAI voices.">
|
||||
<div>
|
||||
<IconComponent
|
||||
name="Info"
|
||||
strokeWidth={2}
|
||||
className="relative -top-[3px] left-1 h-[14px] w-[14px] text-placeholder"
|
||||
/>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
</span>
|
||||
|
||||
<Select value={voice} onValueChange={handleSetVoice}>
|
||||
<SelectTrigger className="h-9 w-full">
|
||||
<SelectValue placeholder="Select" />
|
||||
</SelectTrigger>
|
||||
<SelectContent className="max-h-[200px]">
|
||||
<SelectGroup>
|
||||
{allVoices?.map((voice, index) => (
|
||||
<SelectItem value={voice?.value} key={index}>
|
||||
<div className="max-w-[220px] truncate text-left">
|
||||
{toTitleCase(voice?.name)}
|
||||
</div>
|
||||
</SelectItem>
|
||||
))}
|
||||
</SelectGroup>
|
||||
</SelectContent>
|
||||
</Select>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
export default VoiceSelect;
|
||||
|
|
@ -0,0 +1,35 @@
|
|||
import ForwardedIconComponent from "@/components/common/genericIconComponent";
|
||||
import ShadTooltip from "@/components/common/shadTooltipComponent";
|
||||
import { Button } from "@/components/ui/button";
|
||||
|
||||
interface SettingsVoiceButtonProps {
|
||||
isRecording: boolean;
|
||||
setShowSettingsModal: (value: boolean) => void;
|
||||
}
|
||||
|
||||
const SettingsVoiceButton = ({
|
||||
isRecording,
|
||||
setShowSettingsModal,
|
||||
}: SettingsVoiceButtonProps) => {
|
||||
return (
|
||||
<>
|
||||
<ShadTooltip content="Audio Settings" side="top">
|
||||
<div>
|
||||
<Button
|
||||
className={`btn-playground-actions cursor-pointer text-muted-foreground hover:text-primary`}
|
||||
unstyled
|
||||
disabled={isRecording}
|
||||
onClick={() => setShowSettingsModal(true)}
|
||||
>
|
||||
<ForwardedIconComponent
|
||||
className={`h-[18px] w-[18px]`}
|
||||
name={"Wrench"}
|
||||
/>
|
||||
</Button>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default SettingsVoiceButton;
|
||||
|
|
@ -0,0 +1,32 @@
|
|||
import ForwardedIconComponent from "@/components/common/genericIconComponent";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { ICON_STROKE_WIDTH } from "@/constants/constants";
|
||||
|
||||
interface VoiceButtonProps {
|
||||
toggleRecording: () => void;
|
||||
}
|
||||
|
||||
const VoiceButton = ({ toggleRecording }: VoiceButtonProps) => {
|
||||
return (
|
||||
<>
|
||||
<div>
|
||||
<Button
|
||||
onClick={toggleRecording}
|
||||
className="btn-playground-actions group"
|
||||
unstyled
|
||||
data-testid="voice-button"
|
||||
>
|
||||
<ForwardedIconComponent
|
||||
className={
|
||||
"icon-size text-muted-foreground group-hover:text-primary"
|
||||
}
|
||||
name={"Mic"}
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
/>
|
||||
</Button>
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default VoiceButton;
|
||||
|
|
@ -0,0 +1,90 @@
|
|||
class StreamProcessor extends AudioWorkletProcessor {
|
||||
constructor(options) {
|
||||
super();
|
||||
this.bufferSize = 4096;
|
||||
this.buffer = new Float32Array(this.bufferSize);
|
||||
this.bufferIndex = 0;
|
||||
// Increase threshold for much less sensitivity
|
||||
this.noiseThreshold = 0.15;
|
||||
// Require more consecutive frames above threshold to trigger speech detection
|
||||
this.activationThreshold = 8;
|
||||
this.silenceFrameCount = 0;
|
||||
this.activationCount = 0;
|
||||
this.isSpeaking = false;
|
||||
this.port.onmessage = this.handleMessage.bind(this);
|
||||
}
|
||||
|
||||
handleMessage(event) {
|
||||
if (event.data.type === "updateNoiseGate") {
|
||||
this.noiseThreshold = event.data.threshold;
|
||||
}
|
||||
}
|
||||
|
||||
calculateRMS(buffer) {
|
||||
let sum = 0;
|
||||
for (let i = 0; i < buffer.length; i++) {
|
||||
sum += buffer[i] * buffer[i];
|
||||
}
|
||||
return Math.sqrt(sum / buffer.length);
|
||||
}
|
||||
|
||||
process(inputs, outputs) {
|
||||
const input = inputs[0];
|
||||
if (!input || !input.length) return true;
|
||||
|
||||
const channel = input[0];
|
||||
|
||||
// Calculate RMS volume
|
||||
const rms = this.calculateRMS(channel);
|
||||
|
||||
// Scale the RMS value to match the scale used in use-bar-controls.ts
|
||||
// This makes the threshold more comparable to the "3" value
|
||||
const scaledRMS = rms * 10;
|
||||
|
||||
// Use scaled value for detection
|
||||
const isSilent = scaledRMS < 3;
|
||||
|
||||
// Voice activity detection logic with stricter requirements
|
||||
if (isSilent) {
|
||||
this.activationCount = 0;
|
||||
this.silenceFrameCount++;
|
||||
// Require more silent frames before deciding speech has ended
|
||||
if (this.silenceFrameCount > 20 && this.isSpeaking) {
|
||||
this.isSpeaking = false;
|
||||
}
|
||||
} else {
|
||||
this.silenceFrameCount = 0;
|
||||
if (!this.isSpeaking) {
|
||||
// Require multiple consecutive frames above threshold to start speech
|
||||
this.activationCount++;
|
||||
if (this.activationCount >= this.activationThreshold) {
|
||||
this.isSpeaking = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Fill buffer with audio data
|
||||
for (let i = 0; i < channel.length; i++) {
|
||||
if (this.bufferIndex < this.bufferSize) {
|
||||
// Apply noise gate - zero out audio when not speaking
|
||||
this.buffer[this.bufferIndex++] = !this.isSpeaking ? 0 : channel[i];
|
||||
}
|
||||
}
|
||||
|
||||
// When buffer is full, send it to the main thread
|
||||
if (this.bufferIndex >= this.bufferSize) {
|
||||
const audioData = this.buffer.slice(0);
|
||||
this.port.postMessage({
|
||||
type: "input",
|
||||
audio: audioData,
|
||||
isSilent: !this.isSpeaking,
|
||||
volume: scaledRMS, // Send scaled volume for consistency
|
||||
});
|
||||
this.bufferIndex = 0;
|
||||
}
|
||||
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
registerProcessor("stream_processor", StreamProcessor);
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
import { getLocalStorage, setLocalStorage } from "@/utils/local-storage-util";
|
||||
|
||||
export const checkProvider = () => {
|
||||
const audioSettings = JSON.parse(
|
||||
getLocalStorage("lf_audio_settings_playground") || "{}",
|
||||
);
|
||||
if (!audioSettings.provider) {
|
||||
setLocalStorage(
|
||||
"lf_audio_settings_playground",
|
||||
JSON.stringify({ provider: "openai", voice: "alloy" }),
|
||||
);
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,5 @@
|
|||
export const formatTime = (timeInSeconds: number): string => {
|
||||
const minutes = Math.floor(timeInSeconds / 60);
|
||||
const seconds = Math.floor(timeInSeconds % 60);
|
||||
return `${minutes.toString().padStart(2, "0")}:${seconds.toString().padStart(2, "0")}s`;
|
||||
};
|
||||
|
|
@ -0,0 +1,91 @@
|
|||
export const workletCode = `
|
||||
class StreamProcessor extends AudioWorkletProcessor {
|
||||
constructor() {
|
||||
super();
|
||||
// manipulate this to change the buffer size, 24000 is one per second
|
||||
// this.inputBuffer = new Float32Array(24000);
|
||||
this.inputBuffer = new Float32Array(128);
|
||||
this.inputOffset = 0;
|
||||
this.outputBuffers = [];
|
||||
this.isPlaying = false;
|
||||
|
||||
this.port.onmessage = (event) => {
|
||||
if (event.data.type === 'playback') {
|
||||
this.outputBuffers.push(event.data.audio);
|
||||
this.isPlaying = true;
|
||||
}
|
||||
else if (event.data.type === 'stop_playback') {
|
||||
// Immediately stop playback and clear any queued audio
|
||||
this.outputBuffers = [];
|
||||
this.isPlaying = false;
|
||||
// Optionally notify main thread if you want
|
||||
this.port.postMessage({ type: 'done' });
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
process(inputs, outputs, parameters) {
|
||||
const input = inputs[0];
|
||||
if (input && input.length > 0) {
|
||||
const inputData = input[0];
|
||||
for (let i = 0; i < inputData.length; i++) {
|
||||
this.inputBuffer[this.inputOffset++] = inputData[i];
|
||||
|
||||
if (this.inputOffset >= this.inputBuffer.length) {
|
||||
const outputData = new Int16Array(this.inputBuffer.length);
|
||||
for (let j = 0; j < this.inputBuffer.length; j++) {
|
||||
outputData[j] = Math.max(-1, Math.min(1, this.inputBuffer[j])) * 0x7FFF;
|
||||
}
|
||||
this.port.postMessage({
|
||||
type: 'input',
|
||||
audio: outputData
|
||||
});
|
||||
// manipulate this to change the buffer size, 24000 is one per second
|
||||
// this.inputBuffer = new Float32Array(24000);
|
||||
this.inputBuffer = new Float32Array(128);
|
||||
this.inputOffset = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const output = outputs[0];
|
||||
if (output && output.length > 0 && this.isPlaying) {
|
||||
if (this.outputBuffers.length > 0) {
|
||||
const currentBuffer = this.outputBuffers[0];
|
||||
const chunkSize = Math.min(output[0].length, currentBuffer.length);
|
||||
|
||||
const gain = 0.8;
|
||||
for (let channel = 0; channel < output.length; channel++) {
|
||||
const outputChannel = output[channel];
|
||||
for (let i = 0; i < chunkSize; i++) {
|
||||
outputChannel[i] = currentBuffer[i] * gain;
|
||||
}
|
||||
}
|
||||
|
||||
if (chunkSize === currentBuffer.length) {
|
||||
this.outputBuffers.shift();
|
||||
} else {
|
||||
this.outputBuffers[0] = currentBuffer.slice(chunkSize);
|
||||
}
|
||||
}
|
||||
|
||||
if (this.outputBuffers.length === 0) {
|
||||
this.isPlaying = false;
|
||||
this.port.postMessage({ type: 'done' });
|
||||
}
|
||||
}
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
try {
|
||||
registerProcessor('stream_processor', StreamProcessor);
|
||||
} catch (e) {
|
||||
// Check for registration error without relying on DOMException
|
||||
if (e && e.message && e.message.includes('is already registered')) {
|
||||
// Processor already registered, ignore the error
|
||||
} else {
|
||||
throw e;
|
||||
}
|
||||
}
|
||||
`;
|
||||
|
|
@ -0,0 +1,17 @@
|
|||
export function base64ToFloat32Array(base64String: string): Float32Array {
|
||||
const binaryString = atob(base64String);
|
||||
const pcmData = new Int16Array(binaryString.length / 2);
|
||||
|
||||
for (let i = 0; i < binaryString.length; i += 2) {
|
||||
const lsb = binaryString.charCodeAt(i);
|
||||
const msb = binaryString.charCodeAt(i + 1);
|
||||
pcmData[i / 2] = (msb << 8) | lsb;
|
||||
}
|
||||
|
||||
const float32Data = new Float32Array(pcmData.length);
|
||||
for (let i = 0; i < pcmData.length; i++) {
|
||||
float32Data[i] = pcmData[i] / 32768.0;
|
||||
}
|
||||
|
||||
return float32Data;
|
||||
}
|
||||
|
|
@ -0,0 +1,172 @@
|
|||
import { useEffect, useRef } from "react";
|
||||
|
||||
export const useBarControls = (
|
||||
isRecording: boolean,
|
||||
setRecordingTime: React.Dispatch<React.SetStateAction<number>>,
|
||||
setBarHeights: React.Dispatch<React.SetStateAction<number[]>>,
|
||||
analyserRef?: React.MutableRefObject<AnalyserNode | null>,
|
||||
setSoundDetected?,
|
||||
) => {
|
||||
const animationFrameRef = useRef<number | null>(null);
|
||||
const timeDataRef = useRef<Uint8Array | null>(null);
|
||||
const baseHeightsRef = useRef<number[]>([]);
|
||||
const lastRandomizeTimeRef = useRef<number>(0);
|
||||
const minHeightRef = useRef<number>(20);
|
||||
const analyzerInitializedRef = useRef<boolean>(false);
|
||||
|
||||
useEffect(() => {
|
||||
if (isRecording) {
|
||||
analyzerInitializedRef.current = false;
|
||||
|
||||
if (analyserRef?.current) {
|
||||
const analyser = analyserRef.current;
|
||||
analyser.fftSize = 256;
|
||||
timeDataRef.current = new Uint8Array(analyser.fftSize);
|
||||
analyzerInitializedRef.current = true;
|
||||
}
|
||||
|
||||
const interval = setInterval(() => {
|
||||
setRecordingTime((prev) => prev + 1);
|
||||
}, 1000);
|
||||
return () => clearInterval(interval);
|
||||
} else {
|
||||
setBarHeights(Array(30).fill(minHeightRef.current));
|
||||
if (setSoundDetected) setSoundDetected(false);
|
||||
if (animationFrameRef.current) {
|
||||
cancelAnimationFrame(animationFrameRef.current);
|
||||
animationFrameRef.current = null;
|
||||
}
|
||||
timeDataRef.current = null;
|
||||
}
|
||||
}, [
|
||||
isRecording,
|
||||
setRecordingTime,
|
||||
setBarHeights,
|
||||
setSoundDetected,
|
||||
analyserRef,
|
||||
]);
|
||||
|
||||
useEffect(() => {
|
||||
const staticHeights = Array(30)
|
||||
.fill(0)
|
||||
.map((_, i) => {
|
||||
const position = i / 30;
|
||||
const height = 50 + Math.sin(position * Math.PI) * 30;
|
||||
return Math.max(minHeightRef.current, Math.min(80, height));
|
||||
});
|
||||
|
||||
setBarHeights(staticHeights);
|
||||
baseHeightsRef.current = staticHeights;
|
||||
}, [setBarHeights]);
|
||||
|
||||
useEffect(() => {
|
||||
if (
|
||||
analyserRef?.current &&
|
||||
!analyzerInitializedRef.current &&
|
||||
isRecording
|
||||
) {
|
||||
const analyser = analyserRef.current;
|
||||
analyser.fftSize = 256;
|
||||
timeDataRef.current = new Uint8Array(analyser.fftSize);
|
||||
analyzerInitializedRef.current = true;
|
||||
}
|
||||
}, [analyserRef?.current, isRecording]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!isRecording) return;
|
||||
|
||||
if (
|
||||
analyserRef?.current &&
|
||||
(!timeDataRef.current || !analyzerInitializedRef.current)
|
||||
) {
|
||||
const analyser = analyserRef.current;
|
||||
analyser.fftSize = 256;
|
||||
timeDataRef.current = new Uint8Array(analyser.fftSize);
|
||||
analyzerInitializedRef.current = true;
|
||||
}
|
||||
|
||||
const animate = (timestamp: number) => {
|
||||
let soundDetected = false;
|
||||
let scaledVolume = 0;
|
||||
|
||||
if (analyserRef?.current && timeDataRef.current) {
|
||||
try {
|
||||
const analyser = analyserRef.current;
|
||||
|
||||
if (timeDataRef.current.length !== analyser.fftSize) {
|
||||
timeDataRef.current = new Uint8Array(analyser.fftSize);
|
||||
}
|
||||
|
||||
analyser.getByteTimeDomainData(timeDataRef.current);
|
||||
|
||||
let sum = 0;
|
||||
let max = 0;
|
||||
for (let i = 0; i < timeDataRef.current.length; i++) {
|
||||
const deviation = Math.abs(timeDataRef.current[i] - 128);
|
||||
sum += deviation;
|
||||
max = Math.max(max, deviation);
|
||||
}
|
||||
|
||||
const volumeLevel =
|
||||
(sum / (timeDataRef.current.length * 128)) * 0.5 +
|
||||
(max / 128) * 0.5;
|
||||
|
||||
scaledVolume = volumeLevel * 10;
|
||||
|
||||
soundDetected = scaledVolume > 0.3;
|
||||
|
||||
if (setSoundDetected) {
|
||||
setSoundDetected(soundDetected);
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error detecting sound:", error);
|
||||
if (setSoundDetected) {
|
||||
setSoundDetected(false);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
if (setSoundDetected) {
|
||||
setSoundDetected(false);
|
||||
}
|
||||
}
|
||||
|
||||
const shouldRandomize =
|
||||
soundDetected && timestamp - lastRandomizeTimeRef.current > 100;
|
||||
|
||||
if (shouldRandomize) {
|
||||
lastRandomizeTimeRef.current = timestamp;
|
||||
}
|
||||
|
||||
setBarHeights((prevHeights) => {
|
||||
return prevHeights.map((height, index) => {
|
||||
if (soundDetected) {
|
||||
const baseHeight = baseHeightsRef.current[index] || 50;
|
||||
|
||||
const volumeFactor = 1.0 + Math.min(1.5, scaledVolume);
|
||||
|
||||
const randomFactor = shouldRandomize
|
||||
? 0.7 + Math.random() * 0.6
|
||||
: 0.85 + Math.random() * 0.3;
|
||||
|
||||
const newHeight = baseHeight * volumeFactor * randomFactor;
|
||||
|
||||
return Math.max(minHeightRef.current, Math.min(120, newHeight));
|
||||
} else {
|
||||
return height + (minHeightRef.current - height) * 0.2;
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
animationFrameRef.current = requestAnimationFrame(animate);
|
||||
};
|
||||
|
||||
animationFrameRef.current = requestAnimationFrame(animate);
|
||||
|
||||
return () => {
|
||||
if (animationFrameRef.current) {
|
||||
cancelAnimationFrame(animationFrameRef.current);
|
||||
animationFrameRef.current = null;
|
||||
}
|
||||
};
|
||||
}, [isRecording, analyserRef, setSoundDetected, setBarHeights]);
|
||||
};
|
||||
|
|
@ -0,0 +1,140 @@
|
|||
import { BuildStatus } from "@/constants/enums";
|
||||
import { base64ToFloat32Array } from "../helpers/utils";
|
||||
|
||||
export const useHandleWebsocketMessage = (
|
||||
event: MessageEvent,
|
||||
interruptPlayback: () => void,
|
||||
audioContextRef: React.MutableRefObject<AudioContext | null>,
|
||||
audioQueueRef: React.MutableRefObject<AudioBuffer[]>,
|
||||
isPlayingRef: React.MutableRefObject<boolean>,
|
||||
playNextAudioChunk: () => void,
|
||||
setIsBuilding: (isBuilding: boolean) => void,
|
||||
revertBuiltStatusFromBuilding: () => void,
|
||||
clearEdgesRunningByNodes: () => void,
|
||||
setMessage: React.Dispatch<React.SetStateAction<string>>,
|
||||
edges,
|
||||
setStatus: React.Dispatch<React.SetStateAction<string>>,
|
||||
messagesStore,
|
||||
setEdges,
|
||||
addDataToFlowPool: (data: any, nodeId: string) => void,
|
||||
updateEdgesRunningByNodes: (nodeIds: string[], isRunning: boolean) => void,
|
||||
updateBuildStatus: (nodeIds: string[], status: BuildStatus) => void,
|
||||
hasOpenAIAPIKey: boolean,
|
||||
showErrorAlert: (title: string, list: string[]) => void,
|
||||
) => {
|
||||
const data = JSON.parse(event.data);
|
||||
|
||||
switch (data.type) {
|
||||
case "response.content_part.added":
|
||||
if (data.part?.type === "text" && data.part.text) {
|
||||
setMessage((prev) => prev + data.part.text);
|
||||
}
|
||||
break;
|
||||
|
||||
case "response.done":
|
||||
if (data.response?.status_details?.error?.code) {
|
||||
const errorCode =
|
||||
data.response?.status_details?.error?.code?.replaceAll("_", " ");
|
||||
setStatus(`API key error: ${errorCode}`);
|
||||
showErrorAlert("API key error: " + errorCode, [
|
||||
"Please check your API key and try again",
|
||||
]);
|
||||
}
|
||||
break;
|
||||
|
||||
case "response.cancelled":
|
||||
interruptPlayback();
|
||||
break;
|
||||
|
||||
case "response.audio.delta":
|
||||
if (data.delta && audioContextRef.current) {
|
||||
try {
|
||||
const float32Data = base64ToFloat32Array(data.delta);
|
||||
const audioBuffer = audioContextRef.current.createBuffer(
|
||||
2,
|
||||
float32Data.length,
|
||||
24000,
|
||||
);
|
||||
audioBuffer.copyToChannel(float32Data, 0);
|
||||
audioBuffer.copyToChannel(float32Data, 1);
|
||||
audioQueueRef.current.push(audioBuffer);
|
||||
if (!isPlayingRef.current) {
|
||||
playNextAudioChunk();
|
||||
}
|
||||
} catch (error) {
|
||||
console.error("Error processing audio response:", error);
|
||||
}
|
||||
}
|
||||
break;
|
||||
|
||||
case "flow.build.progress":
|
||||
const buildData = data.data;
|
||||
switch (buildData.event) {
|
||||
case "start":
|
||||
setIsBuilding(true);
|
||||
break;
|
||||
|
||||
case "start_vertex":
|
||||
updateBuildStatus([buildData.vertex_id], BuildStatus.BUILDING);
|
||||
const newEdges = edges.map((edge) => {
|
||||
if (buildData.vertex_id === edge.data?.targetHandle?.id) {
|
||||
edge.animated = true;
|
||||
edge.className = "running";
|
||||
}
|
||||
return edge;
|
||||
});
|
||||
setEdges(newEdges);
|
||||
break;
|
||||
|
||||
case "end_vertex":
|
||||
updateBuildStatus([buildData.vertex_id], BuildStatus.BUILT);
|
||||
addDataToFlowPool(
|
||||
{
|
||||
...buildData.data.build_data,
|
||||
run_id: buildData.run_id,
|
||||
id: buildData.vertex_id,
|
||||
valid: true,
|
||||
},
|
||||
buildData.vertex_id,
|
||||
);
|
||||
updateEdgesRunningByNodes([buildData.vertex_id], false);
|
||||
break;
|
||||
|
||||
case "error":
|
||||
updateBuildStatus([buildData.vertex_id], BuildStatus.ERROR);
|
||||
updateEdgesRunningByNodes([buildData.vertex_id], false);
|
||||
break;
|
||||
|
||||
case "end":
|
||||
setIsBuilding(false);
|
||||
revertBuiltStatusFromBuilding();
|
||||
clearEdgesRunningByNodes();
|
||||
break;
|
||||
|
||||
case "add_message":
|
||||
messagesStore.addMessage(buildData.data);
|
||||
break;
|
||||
}
|
||||
break;
|
||||
|
||||
case "error":
|
||||
if (data.code === "api_key_missing") {
|
||||
setStatus("Error: " + "API key is missing");
|
||||
showErrorAlert("API key not valid", [
|
||||
"Please check your API key and try again",
|
||||
]);
|
||||
return;
|
||||
}
|
||||
if (data.error.message.toLowerCase().includes("api key")) {
|
||||
setStatus("Error: " + "API key is missing");
|
||||
showErrorAlert("API key not valid", [
|
||||
"Please check your API key and try again",
|
||||
]);
|
||||
return;
|
||||
}
|
||||
data.error.message === "Cancellation failed: no active response found"
|
||||
? interruptPlayback()
|
||||
: setStatus("Error: " + data.error);
|
||||
break;
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,29 @@
|
|||
import { MutableRefObject } from "react";
|
||||
|
||||
export const useInitializeAudio = async (
|
||||
audioContextRef: MutableRefObject<AudioContext | null>,
|
||||
setStatus: (status: string) => void,
|
||||
startConversation: () => void,
|
||||
): Promise<void> => {
|
||||
try {
|
||||
if (audioContextRef.current?.state === "closed") {
|
||||
audioContextRef.current = null;
|
||||
}
|
||||
|
||||
if (!audioContextRef.current) {
|
||||
audioContextRef.current = new (window.AudioContext ||
|
||||
(window as any).webkitAudioContext)({
|
||||
sampleRate: 24000,
|
||||
});
|
||||
}
|
||||
|
||||
if (audioContextRef.current.state === "suspended") {
|
||||
await audioContextRef.current.resume();
|
||||
}
|
||||
|
||||
startConversation();
|
||||
} catch (error) {
|
||||
console.error("Failed to initialize audio:", error);
|
||||
setStatus("Error: Failed to initialize audio");
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,13 @@
|
|||
import { MutableRefObject } from "react";
|
||||
|
||||
export const useInterruptPlayback = (
|
||||
audioQueueRef: MutableRefObject<AudioBuffer[]>,
|
||||
isPlayingRef: MutableRefObject<boolean>,
|
||||
processorRef: MutableRefObject<AudioWorkletNode | null>,
|
||||
) => {
|
||||
audioQueueRef.current.splice(0, audioQueueRef.current.length);
|
||||
isPlayingRef.current = false;
|
||||
if (processorRef.current) {
|
||||
processorRef.current.port.postMessage({ type: "stop_playback" });
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,27 @@
|
|||
import { MutableRefObject } from "react";
|
||||
|
||||
export const usePlayNextAudioChunk = (
|
||||
audioQueueRef: MutableRefObject<AudioBuffer[]>,
|
||||
isPlayingRef: MutableRefObject<boolean>,
|
||||
processorRef: MutableRefObject<AudioWorkletNode | null>,
|
||||
) => {
|
||||
if (audioQueueRef.current.length === 0) {
|
||||
isPlayingRef.current = false;
|
||||
return;
|
||||
}
|
||||
|
||||
isPlayingRef.current = true;
|
||||
const audioBuffer = audioQueueRef.current.shift();
|
||||
|
||||
if (audioBuffer && processorRef.current) {
|
||||
try {
|
||||
processorRef.current.port.postMessage({
|
||||
type: "playback",
|
||||
audio: audioBuffer.getChannelData(0),
|
||||
});
|
||||
} catch (error) {
|
||||
console.error("Error playing audio:", error);
|
||||
isPlayingRef.current = false;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,80 @@
|
|||
import { getLocalStorage } from "@/utils/local-storage-util";
|
||||
|
||||
export const useStartConversation = (
|
||||
flowId: string,
|
||||
wsRef: React.MutableRefObject<WebSocket | null>,
|
||||
setStatus: (status: string) => void,
|
||||
startRecording: () => void,
|
||||
handleWebSocketMessage: (event: MessageEvent) => void,
|
||||
stopRecording: () => void,
|
||||
currentSessionId: string,
|
||||
) => {
|
||||
const currentHost = window.location.hostname;
|
||||
const currentPort = window.location.port;
|
||||
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
|
||||
const url = `${protocol}//${currentHost}:${currentPort}/api/v1/voice/ws/flow_as_tool/${flowId}/${currentSessionId}`;
|
||||
|
||||
try {
|
||||
if (wsRef.current?.readyState === WebSocket.CONNECTING) {
|
||||
return;
|
||||
}
|
||||
|
||||
if (wsRef.current?.readyState === WebSocket.OPEN) {
|
||||
wsRef.current.close();
|
||||
}
|
||||
|
||||
const audioSettings = JSON.parse(
|
||||
getLocalStorage("lf_audio_settings_playground") || "{}",
|
||||
);
|
||||
|
||||
wsRef.current = new WebSocket(url);
|
||||
|
||||
wsRef.current.onopen = () => {
|
||||
setStatus("Connected");
|
||||
if (wsRef.current?.readyState === WebSocket.OPEN) {
|
||||
wsRef.current.send(
|
||||
JSON.stringify({
|
||||
type: "langflow.elevenlabs.config",
|
||||
enabled: audioSettings.provider === "elevenlabs",
|
||||
voice_id:
|
||||
audioSettings.provider === "elevenlabs"
|
||||
? audioSettings.voice
|
||||
: "",
|
||||
}),
|
||||
);
|
||||
if (audioSettings.provider !== "elevenlabs") {
|
||||
wsRef.current.send(
|
||||
JSON.stringify({
|
||||
type: "session.update",
|
||||
session: {
|
||||
voice: audioSettings.voice,
|
||||
},
|
||||
}),
|
||||
);
|
||||
}
|
||||
startRecording();
|
||||
}
|
||||
};
|
||||
|
||||
wsRef.current.onmessage = handleWebSocketMessage;
|
||||
|
||||
wsRef.current.onclose = (event) => {
|
||||
if (event.code !== 1000) {
|
||||
// 1000 is normal closure
|
||||
console.warn(`WebSocket closed with code ${event.code}`);
|
||||
}
|
||||
setStatus(`Disconnected (${event.code})`);
|
||||
stopRecording();
|
||||
};
|
||||
|
||||
wsRef.current.onerror = (error) => {
|
||||
console.error("WebSocket Error:", error);
|
||||
setStatus("Connection error");
|
||||
stopRecording();
|
||||
};
|
||||
} catch (error) {
|
||||
console.error("Failed to create WebSocket:", error);
|
||||
setStatus("Connection failed");
|
||||
stopRecording();
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,103 @@
|
|||
import { MutableRefObject } from "react";
|
||||
|
||||
export const useStartRecording = async (
|
||||
audioContextRef: MutableRefObject<AudioContext | null>,
|
||||
microphoneRef: MutableRefObject<MediaStreamAudioSourceNode | null>,
|
||||
analyserRef: MutableRefObject<AnalyserNode | null>,
|
||||
wsRef: MutableRefObject<WebSocket | null>,
|
||||
setIsRecording: (isRecording: boolean) => void,
|
||||
playNextAudioChunk: () => void,
|
||||
isPlayingRef: MutableRefObject<boolean>,
|
||||
audioQueueRef: MutableRefObject<AudioBuffer[]>,
|
||||
workletCode: string,
|
||||
processorRef: MutableRefObject<AudioWorkletNode | null>,
|
||||
setStatus: (status: string) => void,
|
||||
) => {
|
||||
try {
|
||||
const selectedMicrophone = localStorage.getItem("lf_selected_microphone");
|
||||
const preferredLanguage =
|
||||
localStorage.getItem("lf_preferred_language") || "en-US";
|
||||
|
||||
const stream = await navigator?.mediaDevices?.getUserMedia({
|
||||
audio: {
|
||||
noiseSuppression: true,
|
||||
echoCancellation: true,
|
||||
autoGainControl: true,
|
||||
sampleRate: 48000,
|
||||
deviceId: selectedMicrophone
|
||||
? { exact: selectedMicrophone }
|
||||
: undefined,
|
||||
},
|
||||
});
|
||||
if (!audioContextRef.current) return;
|
||||
|
||||
microphoneRef.current =
|
||||
audioContextRef?.current?.createMediaStreamSource(stream);
|
||||
analyserRef.current = audioContextRef?.current?.createAnalyser();
|
||||
analyserRef.current.fftSize = 2048;
|
||||
microphoneRef.current.connect(analyserRef.current);
|
||||
|
||||
const blob = new Blob([workletCode], { type: "application/javascript" });
|
||||
const workletUrl = URL.createObjectURL(blob);
|
||||
|
||||
try {
|
||||
try {
|
||||
await audioContextRef.current.audioWorklet.addModule(workletUrl);
|
||||
} catch (err) {
|
||||
// Check if the error is because the processor is already registered
|
||||
if (
|
||||
err instanceof DOMException &&
|
||||
err.message.includes("already been loaded")
|
||||
) {
|
||||
console.log("AudioWorklet module already loaded, continuing...");
|
||||
} else {
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
processorRef.current = new AudioWorkletNode(
|
||||
audioContextRef.current,
|
||||
"stream_processor",
|
||||
);
|
||||
|
||||
analyserRef.current.connect(processorRef.current);
|
||||
processorRef.current.connect(audioContextRef.current.destination);
|
||||
|
||||
processorRef.current.port.onmessage = (event) => {
|
||||
if (event.data.type === "input" && event.data.audio && wsRef.current) {
|
||||
// Only send audio if it's not detected as silence
|
||||
const base64Audio = btoa(
|
||||
String.fromCharCode.apply(
|
||||
null,
|
||||
Array.from(new Uint8Array(event.data.audio.buffer)),
|
||||
),
|
||||
);
|
||||
|
||||
wsRef.current.send(
|
||||
JSON.stringify({
|
||||
type: "input_audio_buffer.append",
|
||||
audio: base64Audio,
|
||||
language: preferredLanguage,
|
||||
}),
|
||||
);
|
||||
} else if (event.data.type === "done") {
|
||||
if (audioQueueRef.current.length > 0) {
|
||||
playNextAudioChunk();
|
||||
} else {
|
||||
isPlayingRef.current = false;
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
setIsRecording(true);
|
||||
} catch (err) {
|
||||
console.error("AudioWorklet failed to load:", err);
|
||||
setStatus("Error initializing audio: " + (err as Error).message);
|
||||
} finally {
|
||||
URL.revokeObjectURL(workletUrl);
|
||||
}
|
||||
} catch (err) {
|
||||
console.error("Error accessing microphone:", err);
|
||||
setStatus("Error: " + (err as Error).message);
|
||||
}
|
||||
};
|
||||
|
|
@ -0,0 +1,25 @@
|
|||
export const useStopRecording = (
|
||||
microphoneRef,
|
||||
processorRef: React.MutableRefObject<AudioWorkletNode | null>,
|
||||
analyserRef: React.MutableRefObject<AnalyserNode | null>,
|
||||
wsRef: React.MutableRefObject<WebSocket | null>,
|
||||
setIsRecording: (isRecording: boolean) => void,
|
||||
) => {
|
||||
if (microphoneRef.current) {
|
||||
microphoneRef.current.disconnect();
|
||||
microphoneRef.current = null;
|
||||
}
|
||||
if (processorRef.current) {
|
||||
processorRef.current.disconnect();
|
||||
processorRef.current = null;
|
||||
}
|
||||
if (analyserRef.current) {
|
||||
analyserRef.current.disconnect();
|
||||
analyserRef.current = null;
|
||||
}
|
||||
if (wsRef.current) {
|
||||
wsRef.current.close();
|
||||
wsRef.current = null;
|
||||
}
|
||||
setIsRecording(false);
|
||||
};
|
||||
|
|
@ -0,0 +1,482 @@
|
|||
import ShadTooltip from "@/components/common/shadTooltipComponent";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { ICON_STROKE_WIDTH, SAVE_API_KEY_ALERT } from "@/constants/constants";
|
||||
import { useGetMessagesPollingMutation } from "@/controllers/API/queries/messages/use-get-messages-polling";
|
||||
import {
|
||||
useGetGlobalVariables,
|
||||
usePatchGlobalVariables,
|
||||
usePostGlobalVariables,
|
||||
} from "@/controllers/API/queries/variables";
|
||||
import useAlertStore from "@/stores/alertStore";
|
||||
import useFlowStore from "@/stores/flowStore";
|
||||
import { useGlobalVariablesStore } from "@/stores/globalVariablesStore/globalVariables";
|
||||
import { useMessagesStore } from "@/stores/messagesStore";
|
||||
import { useUtilityStore } from "@/stores/utilityStore";
|
||||
import { useVoiceStore } from "@/stores/voiceStore";
|
||||
import { cn } from "@/utils/utils";
|
||||
import { AxiosError } from "axios";
|
||||
import { useEffect, useMemo, useRef, useState } from "react";
|
||||
import IconComponent from "../../../../../../../components/common/genericIconComponent";
|
||||
import SettingsVoiceModal from "./components/audio-settings/audio-settings-dialog";
|
||||
import { checkProvider } from "./helpers/check-provider";
|
||||
import { formatTime } from "./helpers/format-time";
|
||||
import { workletCode } from "./helpers/streamProcessor";
|
||||
import { useBarControls } from "./hooks/use-bar-controls";
|
||||
import { useHandleWebsocketMessage } from "./hooks/use-handle-websocket-message";
|
||||
import { useInitializeAudio } from "./hooks/use-initialize-audio";
|
||||
import { useInterruptPlayback } from "./hooks/use-interrupt-playback";
|
||||
import { usePlayNextAudioChunk } from "./hooks/use-play-next-audio-chunk";
|
||||
import { useStartConversation } from "./hooks/use-start-conversation";
|
||||
import { useStartRecording } from "./hooks/use-start-recording";
|
||||
import { useStopRecording } from "./hooks/use-stop-recording";
|
||||
|
||||
interface VoiceAssistantProps {
|
||||
flowId: string;
|
||||
setShowAudioInput: (value: boolean) => void;
|
||||
}
|
||||
|
||||
export function VoiceAssistant({
|
||||
flowId,
|
||||
setShowAudioInput,
|
||||
}: VoiceAssistantProps) {
|
||||
const [recordingTime, setRecordingTime] = useState(0);
|
||||
const [isRecording, setIsRecording] = useState(false);
|
||||
const [status, setStatus] = useState("");
|
||||
const [message, setMessage] = useState("");
|
||||
const [showSettingsModal, setShowSettingsModal] = useState(false);
|
||||
const [addKey, setAddKey] = useState(false);
|
||||
const [barHeights, setBarHeights] = useState<number[]>(Array(30).fill(20));
|
||||
const [preferredLanguage, setPreferredLanguage] = useState(
|
||||
localStorage.getItem("lf_preferred_language") || "en-US",
|
||||
);
|
||||
const [isEditingOpenAIKey, setIsEditingOpenAIKey] = useState<boolean>(false);
|
||||
|
||||
const waveformRef = useRef<HTMLDivElement>(null);
|
||||
const audioContextRef = useRef<AudioContext | null>(null);
|
||||
const wsRef = useRef<WebSocket | null>(null);
|
||||
const processorRef = useRef<AudioWorkletNode | null>(null);
|
||||
const audioQueueRef = useRef<AudioBuffer[]>([]);
|
||||
const isPlayingRef = useRef(false);
|
||||
const microphoneRef = useRef<MediaStreamAudioSourceNode | null>(null);
|
||||
const analyserRef = useRef<AnalyserNode | null>(null);
|
||||
|
||||
const soundDetected = useVoiceStore((state) => state.soundDetected);
|
||||
const setSoundDetected = useVoiceStore((state) => state.setSoundDetected);
|
||||
const messagesStore = useMessagesStore();
|
||||
const setIsBuilding = useFlowStore((state) => state.setIsBuilding);
|
||||
const edges = useFlowStore((state) => state.edges);
|
||||
const setEdges = useFlowStore((state) => state.setEdges);
|
||||
const updateBuildStatus = useFlowStore((state) => state.updateBuildStatus);
|
||||
const addDataToFlowPool = useFlowStore((state) => state.addDataToFlowPool);
|
||||
const updateEdgesRunningByNodes = useFlowStore(
|
||||
(state) => state.updateEdgesRunningByNodes,
|
||||
);
|
||||
const revertBuiltStatusFromBuilding = useFlowStore(
|
||||
(state) => state.revertBuiltStatusFromBuilding,
|
||||
);
|
||||
const clearEdgesRunningByNodes = useFlowStore(
|
||||
(state) => state.clearEdgesRunningByNodes,
|
||||
);
|
||||
const variables = useGlobalVariablesStore(
|
||||
(state) => state.globalVariablesEntries,
|
||||
);
|
||||
const createVariable = usePostGlobalVariables();
|
||||
const updateVariable = usePatchGlobalVariables();
|
||||
const setSuccessData = useAlertStore((state) => state.setSuccessData);
|
||||
const currentSessionId = useUtilityStore((state) => state.currentSessionId);
|
||||
const setErrorData = useAlertStore((state) => state.setErrorData);
|
||||
const { data: globalVariables } = useGetGlobalVariables();
|
||||
|
||||
const hasOpenAIAPIKey = useMemo(() => {
|
||||
return (
|
||||
variables?.find((variable) => variable === "OPENAI_API_KEY")?.length! > 0
|
||||
);
|
||||
}, [variables, open, addKey]);
|
||||
|
||||
const hasElevenLabsApiKey = useMemo(() => {
|
||||
return (
|
||||
variables?.find((variable) => variable === "ELEVENLABS_API_KEY")
|
||||
?.length! > 0
|
||||
);
|
||||
}, [variables, addKey, open]);
|
||||
|
||||
const openaiApiKey = useMemo(() => {
|
||||
return variables?.find((variable) => variable === "OPENAI_API_KEY");
|
||||
}, [variables, addKey]);
|
||||
|
||||
const openaiApiKeyGlobalVariable = useMemo(() => {
|
||||
return globalVariables?.find(
|
||||
(variable) => variable.name === "OPENAI_API_KEY",
|
||||
);
|
||||
}, [globalVariables]);
|
||||
|
||||
const elevenLabsApiKeyGlobalVariable = useMemo(() => {
|
||||
return globalVariables?.find(
|
||||
(variable) => variable.name === "ELEVENLABS_API_KEY",
|
||||
);
|
||||
}, [globalVariables]);
|
||||
|
||||
const hasElevenLabsApiKeyEnv = useMemo(() => {
|
||||
return Boolean(process.env?.ELEVENLABS_API_KEY);
|
||||
}, [variables, addKey]);
|
||||
|
||||
useEffect(() => {
|
||||
if (!isRecording && hasOpenAIAPIKey && !showSettingsModal) {
|
||||
setIsRecording(true);
|
||||
initializeAudio();
|
||||
} else {
|
||||
stopRecording();
|
||||
}
|
||||
}, []);
|
||||
|
||||
const getMessagesMutation = useGetMessagesPollingMutation();
|
||||
|
||||
const initializeAudio = async () => {
|
||||
useInitializeAudio(audioContextRef, setStatus, startConversation);
|
||||
};
|
||||
|
||||
const startRecording = async () => {
|
||||
useStartRecording(
|
||||
audioContextRef,
|
||||
microphoneRef,
|
||||
analyserRef,
|
||||
wsRef,
|
||||
setIsRecording,
|
||||
playNextAudioChunk,
|
||||
isPlayingRef,
|
||||
audioQueueRef,
|
||||
workletCode,
|
||||
processorRef,
|
||||
setStatus,
|
||||
);
|
||||
};
|
||||
|
||||
const stopRecording = () => {
|
||||
useStopRecording(
|
||||
microphoneRef,
|
||||
processorRef,
|
||||
analyserRef,
|
||||
wsRef,
|
||||
setIsRecording,
|
||||
);
|
||||
};
|
||||
|
||||
const playNextAudioChunk = () => {
|
||||
usePlayNextAudioChunk(audioQueueRef, isPlayingRef, processorRef);
|
||||
};
|
||||
|
||||
const handleWebSocketMessage = (event: MessageEvent) => {
|
||||
useHandleWebsocketMessage(
|
||||
event,
|
||||
interruptPlayback,
|
||||
audioContextRef,
|
||||
audioQueueRef,
|
||||
isPlayingRef,
|
||||
playNextAudioChunk,
|
||||
setIsBuilding,
|
||||
revertBuiltStatusFromBuilding,
|
||||
clearEdgesRunningByNodes,
|
||||
setMessage,
|
||||
edges,
|
||||
setStatus,
|
||||
messagesStore,
|
||||
setEdges,
|
||||
addDataToFlowPool,
|
||||
updateEdgesRunningByNodes,
|
||||
updateBuildStatus,
|
||||
hasOpenAIAPIKey,
|
||||
showErrorAlert,
|
||||
);
|
||||
};
|
||||
|
||||
const startConversation = () => {
|
||||
useStartConversation(
|
||||
flowId,
|
||||
wsRef,
|
||||
setStatus,
|
||||
startRecording,
|
||||
handleWebSocketMessage,
|
||||
stopRecording,
|
||||
currentSessionId,
|
||||
);
|
||||
};
|
||||
|
||||
const interruptPlayback = () => {
|
||||
useInterruptPlayback(audioQueueRef, isPlayingRef, processorRef);
|
||||
};
|
||||
|
||||
useBarControls(
|
||||
isRecording,
|
||||
setRecordingTime,
|
||||
setBarHeights,
|
||||
analyserRef,
|
||||
setSoundDetected,
|
||||
);
|
||||
|
||||
const handleGetMessagesMutation = () => {
|
||||
getMessagesMutation.mutate({
|
||||
mode: "union",
|
||||
id: currentSessionId,
|
||||
});
|
||||
};
|
||||
|
||||
const showErrorAlert = (title: string, list: string[]) => {
|
||||
setErrorData({
|
||||
title,
|
||||
list,
|
||||
});
|
||||
setIsRecording(false);
|
||||
};
|
||||
|
||||
const handleSaveApiKey = async (
|
||||
apiKey: string,
|
||||
variableName: string,
|
||||
elevenLabsKey: boolean,
|
||||
) => {
|
||||
const updateOpenAiKey =
|
||||
isEditingOpenAIKey && openaiApiKeyGlobalVariable?.id;
|
||||
const updateElevenLabsApiKey =
|
||||
elevenLabsApiKeyGlobalVariable?.id && elevenLabsKey;
|
||||
|
||||
if (updateOpenAiKey || updateElevenLabsApiKey) {
|
||||
await updateVariable.mutateAsync(
|
||||
{
|
||||
name: variableName,
|
||||
value: apiKey,
|
||||
id: elevenLabsKey
|
||||
? elevenLabsApiKeyGlobalVariable?.id!
|
||||
: openaiApiKeyGlobalVariable?.id!,
|
||||
},
|
||||
{
|
||||
onSuccess: () => {
|
||||
setSuccessData({
|
||||
title: SAVE_API_KEY_ALERT,
|
||||
});
|
||||
setAddKey(!addKey);
|
||||
setIsEditingOpenAIKey(false);
|
||||
},
|
||||
},
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
await createVariable.mutateAsync(
|
||||
{
|
||||
name: variableName,
|
||||
value: apiKey,
|
||||
type: "secret",
|
||||
default_fields: ["voice_mode"],
|
||||
},
|
||||
{
|
||||
onSuccess: () => {
|
||||
setSuccessData({
|
||||
title: SAVE_API_KEY_ALERT,
|
||||
});
|
||||
setAddKey(!addKey);
|
||||
},
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
checkProvider();
|
||||
handleGetMessagesMutation();
|
||||
|
||||
return () => {
|
||||
stopRecording();
|
||||
if (audioContextRef.current) {
|
||||
audioContextRef.current.close();
|
||||
audioContextRef.current = null;
|
||||
}
|
||||
};
|
||||
}, []);
|
||||
|
||||
const handleCloseAudioInput = () => {
|
||||
setIsRecording;
|
||||
stopRecording();
|
||||
setShowAudioInput(false);
|
||||
};
|
||||
|
||||
const handleSetShowSettingsModal = async (
|
||||
open: boolean,
|
||||
openaiApiKey: string,
|
||||
elevenLabsApiKey: string,
|
||||
) => {
|
||||
const saveApiKey = openaiApiKey && openaiApiKey !== "OPENAI_API_KEY";
|
||||
const saveElevenLabsApiKey =
|
||||
elevenLabsApiKey && elevenLabsApiKey !== "ELEVENLABS_API_KEY";
|
||||
|
||||
if (open) {
|
||||
stopRecording();
|
||||
if (audioContextRef.current) {
|
||||
audioContextRef.current.close();
|
||||
audioContextRef.current = null;
|
||||
}
|
||||
setIsRecording(false);
|
||||
} else {
|
||||
setRecordingTime(0);
|
||||
setBarHeights(Array(30).fill(20));
|
||||
|
||||
if (hasOpenAIAPIKey) {
|
||||
if (audioContextRef.current) {
|
||||
audioContextRef.current.close();
|
||||
audioContextRef.current = null;
|
||||
}
|
||||
analyserRef.current = null;
|
||||
|
||||
setTimeout(() => {
|
||||
initializeAudio();
|
||||
startRecording();
|
||||
setIsRecording(true);
|
||||
}, 100);
|
||||
}
|
||||
}
|
||||
|
||||
if (saveApiKey) {
|
||||
await handleSaveApiKey(openaiApiKey, "OPENAI_API_KEY", false);
|
||||
}
|
||||
|
||||
if (saveElevenLabsApiKey && !open) {
|
||||
await handleSaveApiKey(elevenLabsApiKey, "ELEVENLABS_API_KEY", true);
|
||||
}
|
||||
};
|
||||
|
||||
const handleToggleRecording = () => {
|
||||
if (isRecording) {
|
||||
if (microphoneRef?.current && microphoneRef?.current?.mediaStream) {
|
||||
microphoneRef.current.mediaStream.getAudioTracks().forEach((track) => {
|
||||
track.enabled = false;
|
||||
});
|
||||
}
|
||||
setBarHeights(Array(30).fill(20));
|
||||
setIsRecording(false);
|
||||
} else {
|
||||
if (microphoneRef?.current && microphoneRef?.current?.mediaStream) {
|
||||
microphoneRef.current.mediaStream.getAudioTracks().forEach((track) => {
|
||||
track.enabled = true;
|
||||
});
|
||||
} else {
|
||||
startRecording();
|
||||
}
|
||||
setIsRecording(true);
|
||||
}
|
||||
};
|
||||
|
||||
useEffect(() => {
|
||||
if (preferredLanguage) {
|
||||
localStorage.setItem("lf_preferred_language", preferredLanguage);
|
||||
}
|
||||
}, [preferredLanguage]);
|
||||
|
||||
const handleClickSaveOpenAIApiKey = async (openaiApiKey: string) => {
|
||||
await handleSaveApiKey(openaiApiKey, "OPENAI_API_KEY", false);
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<div
|
||||
data-testid="voice-assistant-container"
|
||||
className="mx-auto flex w-full max-w-[324px] items-center justify-center rounded-md border bg-background px-4 py-2 shadow-xl"
|
||||
>
|
||||
<div
|
||||
className={cn(
|
||||
"flex items-center",
|
||||
hasOpenAIAPIKey ? "gap-3" : "gap-2",
|
||||
)}
|
||||
>
|
||||
<ShadTooltip
|
||||
content={isRecording ? "Mute" : "Unmute"}
|
||||
delayDuration={500}
|
||||
>
|
||||
<Button unstyled onClick={handleToggleRecording}>
|
||||
<IconComponent
|
||||
name={isRecording ? "Mic" : "MicOff"}
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
className="h-4 w-4 text-placeholder-foreground"
|
||||
/>
|
||||
</Button>
|
||||
</ShadTooltip>
|
||||
|
||||
<div
|
||||
ref={waveformRef}
|
||||
className="flex h-5 flex-1 items-center justify-center"
|
||||
>
|
||||
{barHeights.map((height, index) => (
|
||||
<div
|
||||
key={index}
|
||||
className={cn(
|
||||
"mx-[1px] w-[2px] rounded-sm transition-all duration-200",
|
||||
isRecording && soundDetected
|
||||
? "bg-red-foreground"
|
||||
: "bg-placeholder-foreground",
|
||||
)}
|
||||
style={{ height: `${height}%` }}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
<div className="min-w-[50px] cursor-default text-center font-mono text-sm font-medium text-placeholder-foreground">
|
||||
{hasOpenAIAPIKey ? formatTime(recordingTime) : "--:--s"}
|
||||
</div>
|
||||
|
||||
<div>
|
||||
<SettingsVoiceModal
|
||||
userOpenaiApiKey={openaiApiKey}
|
||||
userElevenLabsApiKey={elevenLabsApiKeyGlobalVariable?.name}
|
||||
hasElevenLabsApiKeyEnv={hasElevenLabsApiKeyEnv}
|
||||
setShowSettingsModal={handleSetShowSettingsModal}
|
||||
hasOpenAIAPIKey={hasOpenAIAPIKey}
|
||||
language={preferredLanguage}
|
||||
setLanguage={setPreferredLanguage}
|
||||
handleClickSaveOpenAIApiKey={handleClickSaveOpenAIApiKey}
|
||||
isEditingOpenAIKey={isEditingOpenAIKey}
|
||||
setIsEditingOpenAIKey={setIsEditingOpenAIKey}
|
||||
>
|
||||
{hasOpenAIAPIKey ? (
|
||||
<>
|
||||
<Button data-testid="voice-assistant-settings-icon" unstyled>
|
||||
<IconComponent
|
||||
name="Settings"
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
className={cn(
|
||||
"relative top-[2px] h-4 w-4 text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
/>
|
||||
</Button>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<Button
|
||||
variant={"outlineAmber"}
|
||||
size={"icon"}
|
||||
data-testid="voice-assistant-settings-icon-without-openai"
|
||||
className="group h-8 w-8"
|
||||
>
|
||||
<IconComponent
|
||||
name="Settings"
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
className={cn(
|
||||
"h-4 w-4 text-accent-amber-foreground group-hover:text-accent-amber",
|
||||
)}
|
||||
/>
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
</SettingsVoiceModal>
|
||||
</div>
|
||||
|
||||
<Button
|
||||
unstyled
|
||||
onClick={handleCloseAudioInput}
|
||||
data-testid="voice-assistant-close-button"
|
||||
>
|
||||
<IconComponent
|
||||
name="X"
|
||||
strokeWidth={ICON_STROKE_WIDTH}
|
||||
className="h-4 w-4 text-muted-foreground hover:text-foreground"
|
||||
/>
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
}
|
||||
|
|
@ -9,7 +9,6 @@ import {
|
|||
import useFlowsManagerStore from "@/stores/flowsManagerStore";
|
||||
import useFlowStore from "@/stores/flowStore";
|
||||
import { useUtilityStore } from "@/stores/utilityStore";
|
||||
import { ChatMessageType } from "@/types/chat";
|
||||
import Convert from "ansi-to-html";
|
||||
import { useEffect, useRef, useState } from "react";
|
||||
import Robot from "../../../../../assets/robot.png";
|
||||
|
|
@ -54,6 +53,8 @@ export default function ChatMessage({
|
|||
const [showError, setShowError] = useState(false);
|
||||
const isBuilding = useFlowStore((state) => state.isBuilding);
|
||||
|
||||
const isAudioMessage = chat.category === "audio";
|
||||
|
||||
useEffect(() => {
|
||||
const chatMessageString = chat.message ? chat.message.toString() : "";
|
||||
setChatMessage(chatMessageString);
|
||||
|
|
@ -306,7 +307,17 @@ export default function ChatMessage({
|
|||
"sender_name_" + chat.sender_name?.toLocaleLowerCase()
|
||||
}
|
||||
>
|
||||
{chat.sender_name}
|
||||
<span className="flex items-center gap-2">
|
||||
{chat.sender_name}
|
||||
{isAudioMessage && (
|
||||
<div className="flex h-5 w-5 items-center justify-center rounded-sm bg-muted">
|
||||
<ForwardedIconComponent
|
||||
name="mic"
|
||||
className="h-3 w-3 text-muted-foreground"
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
</span>
|
||||
{chat.properties?.source && !playgroundPage && (
|
||||
<div className="text-[13px] font-normal text-muted-foreground">
|
||||
{chat.properties?.source.source}
|
||||
|
|
@ -379,6 +390,7 @@ export default function ChatMessage({
|
|||
/>
|
||||
) : (
|
||||
<MarkdownField
|
||||
isAudioMessage={isAudioMessage}
|
||||
chat={chat}
|
||||
isEmpty={isEmpty}
|
||||
chatMessage={chatMessage}
|
||||
|
|
@ -407,9 +419,10 @@ export default function ChatMessage({
|
|||
) : (
|
||||
<>
|
||||
<div
|
||||
className={`w-full items-baseline whitespace-pre-wrap break-words text-[14px] font-normal ${
|
||||
isEmpty ? "text-muted-foreground" : "text-primary"
|
||||
}`}
|
||||
className={cn(
|
||||
"w-full items-baseline whitespace-pre-wrap break-words text-[14px] font-normal",
|
||||
isEmpty ? "text-muted-foreground" : "text-primary",
|
||||
)}
|
||||
data-testid={`chat-message-${chat.sender_name}-${chatMessage}`}
|
||||
>
|
||||
{isEmpty ? EMPTY_INPUT_SEND_MESSAGE : decodedMessage}
|
||||
|
|
@ -441,6 +454,7 @@ export default function ChatMessage({
|
|||
isBotMessage={!chat.isSend}
|
||||
onEvaluate={handleEvaluateAnswer}
|
||||
evaluation={chat.properties?.positive_feedback}
|
||||
isAudioMessage={isAudioMessage}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
|
|
|
|||
|
|
@ -11,6 +11,7 @@ type MarkdownFieldProps = {
|
|||
isEmpty: boolean;
|
||||
chatMessage: string;
|
||||
editedFlag: React.ReactNode;
|
||||
isAudioMessage?: boolean;
|
||||
};
|
||||
|
||||
// Function to replace <think> tags with a placeholder before markdown processing
|
||||
|
|
@ -26,6 +27,7 @@ export const MarkdownField = ({
|
|||
isEmpty,
|
||||
chatMessage,
|
||||
editedFlag,
|
||||
isAudioMessage,
|
||||
}: MarkdownFieldProps) => {
|
||||
// Process the chat message to handle <think> tags
|
||||
const processedChatMessage = preprocessChatMessage(chatMessage);
|
||||
|
|
|
|||
|
|
@ -10,6 +10,7 @@ export function EditMessageButton({
|
|||
onEvaluate,
|
||||
isBotMessage,
|
||||
evaluation,
|
||||
isAudioMessage,
|
||||
}: ButtonHTMLAttributes<HTMLButtonElement> & {
|
||||
onEdit: () => void;
|
||||
onCopy: () => void;
|
||||
|
|
@ -17,6 +18,7 @@ export function EditMessageButton({
|
|||
onEvaluate?: (value: boolean | null) => void;
|
||||
isBotMessage?: boolean;
|
||||
evaluation?: boolean | null;
|
||||
isAudioMessage?: boolean;
|
||||
}) {
|
||||
const [isCopied, setIsCopied] = useState(false);
|
||||
|
||||
|
|
@ -32,18 +34,20 @@ export function EditMessageButton({
|
|||
|
||||
return (
|
||||
<div className="flex items-center rounded-md border border-border bg-background">
|
||||
<ShadTooltip styleClasses="z-50" content="Edit message" side="top">
|
||||
<div className="p-1">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={onEdit}
|
||||
className="h-8 w-8"
|
||||
>
|
||||
<IconComponent name="Pen" className="h-4 w-4" />
|
||||
</Button>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
{!isAudioMessage && (
|
||||
<ShadTooltip styleClasses="z-50" content="Edit message" side="top">
|
||||
<div className="p-1">
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
onClick={onEdit}
|
||||
className="h-8 w-8"
|
||||
>
|
||||
<IconComponent name="Pen" className="h-4 w-4" />
|
||||
</Button>
|
||||
</div>
|
||||
</ShadTooltip>
|
||||
)}
|
||||
|
||||
<ShadTooltip
|
||||
styleClasses="z-50"
|
||||
|
|
|
|||
|
|
@ -1,20 +1,26 @@
|
|||
import LangflowLogo from "@/assets/LangflowLogo.svg?react";
|
||||
import ForwardedIconComponent from "@/components/common/genericIconComponent";
|
||||
import { ProfileIcon } from "@/components/core/appHeaderComponent/components/ProfileIcon";
|
||||
import { TextEffectPerChar } from "@/components/ui/textAnimation";
|
||||
import { CustomProfileIcon } from "@/customization/components/custom-profile-icon";
|
||||
import { ENABLE_DATASTAX_LANGFLOW } from "@/customization/feature-flags";
|
||||
import { track } from "@/customization/utils/analytics";
|
||||
import { useMessagesStore } from "@/stores/messagesStore";
|
||||
import { useUtilityStore } from "@/stores/utilityStore";
|
||||
import { useVoiceStore } from "@/stores/voiceStore";
|
||||
import { cn } from "@/utils/utils";
|
||||
import { memo, useEffect, useMemo, useRef, useState } from "react";
|
||||
import { v5 as uuidv5 } from "uuid";
|
||||
import useTabVisibility from "../../../../shared/hooks/use-tab-visibility";
|
||||
import useFlowsManagerStore from "../../../../stores/flowsManagerStore";
|
||||
import useFlowStore from "../../../../stores/flowStore";
|
||||
import { ChatMessageType } from "../../../../types/chat";
|
||||
import { chatViewProps } from "../../../../types/components";
|
||||
import FlowRunningSqueleton from "../flow-running-squeleton";
|
||||
import ChatInput from "./chatInput/chat-input";
|
||||
import useDragAndDrop from "./chatInput/hooks/use-drag-and-drop";
|
||||
import { useFileHandler } from "./chatInput/hooks/use-file-handler";
|
||||
import ChatMessage from "./chatMessage/chat-message";
|
||||
import useTabVisibility from "../../../../../shared/hooks/use-tab-visibility";
|
||||
import useFlowsManagerStore from "../../../../../stores/flowsManagerStore";
|
||||
import useFlowStore from "../../../../../stores/flowStore";
|
||||
import { ChatMessageType } from "../../../../../types/chat";
|
||||
import { chatViewProps } from "../../../../../types/components";
|
||||
import FlowRunningSqueleton from "../../flow-running-squeleton";
|
||||
import ChatInput from "../chatInput/chat-input";
|
||||
import useDragAndDrop from "../chatInput/hooks/use-drag-and-drop";
|
||||
import { useFileHandler } from "../chatInput/hooks/use-file-handler";
|
||||
import ChatMessage from "../chatMessage/chat-message";
|
||||
|
||||
const MemoizedChatMessage = memo(ChatMessage, (prevProps, nextProps) => {
|
||||
return (
|
||||
|
|
@ -157,6 +163,7 @@ export default function ChatView({
|
|||
};
|
||||
|
||||
const flowRunningSkeletonMemo = useMemo(() => <FlowRunningSqueleton />, []);
|
||||
const soundDetected = useVoiceStore((state) => state.soundDetected);
|
||||
|
||||
return (
|
||||
<div
|
||||
|
|
@ -182,27 +189,29 @@ export default function ChatView({
|
|||
))}
|
||||
</>
|
||||
) : (
|
||||
<div className="flex h-full w-full flex-col items-center justify-center">
|
||||
<div className="flex flex-col items-center justify-center gap-4 p-8">
|
||||
<LangflowLogo
|
||||
title="Langflow logo"
|
||||
className="h-10 w-10 scale-[1.5]"
|
||||
/>
|
||||
<div className="flex flex-col items-center justify-center">
|
||||
<h3 className="mt-2 pb-2 text-2xl font-semibold text-primary">
|
||||
New chat
|
||||
</h3>
|
||||
<p
|
||||
className="text-lg text-muted-foreground"
|
||||
data-testid="new-chat-text"
|
||||
>
|
||||
<TextEffectPerChar>
|
||||
Test your flow with a chat prompt
|
||||
</TextEffectPerChar>
|
||||
</p>
|
||||
<>
|
||||
<div className="flex h-full w-full flex-col items-center justify-center">
|
||||
<div className="flex flex-col items-center justify-center gap-4 p-8">
|
||||
<LangflowLogo
|
||||
title="Langflow logo"
|
||||
className="h-10 w-10 scale-[1.5]"
|
||||
/>
|
||||
<div className="flex flex-col items-center justify-center">
|
||||
<h3 className="mt-2 pb-2 text-2xl font-semibold text-primary">
|
||||
New chat
|
||||
</h3>
|
||||
<p
|
||||
className="text-lg text-muted-foreground"
|
||||
data-testid="new-chat-text"
|
||||
>
|
||||
<TextEffectPerChar>
|
||||
Test your flow with a chat prompt
|
||||
</TextEffectPerChar>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
))}
|
||||
<div
|
||||
className={
|
||||
|
|
@ -217,6 +226,7 @@ export default function ChatView({
|
|||
flowRunningSkeletonMemo}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="m-auto w-full max-w-[768px] md:w-5/6">
|
||||
<ChatInput
|
||||
playgroundPage={!!playgroundPage}
|
||||
|
|
@ -144,6 +144,10 @@ export default function IOModal({
|
|||
),
|
||||
);
|
||||
const [sessionId, setSessionId] = useState<string>(currentFlowId);
|
||||
const setCurrentSessionId = useUtilityStore(
|
||||
(state) => state.setCurrentSessionId,
|
||||
);
|
||||
|
||||
const { isFetched: messagesFetched } = useGetMessagesQuery(
|
||||
{
|
||||
mode: "union",
|
||||
|
|
@ -213,8 +217,10 @@ export default function IOModal({
|
|||
setSessionId(
|
||||
`Session ${new Date().toLocaleString("en-US", { day: "2-digit", month: "short", hour: "2-digit", minute: "2-digit", hour12: false, second: "2-digit", timeZone: "UTC" })}`,
|
||||
);
|
||||
setCurrentSessionId(currentFlowId);
|
||||
} else if (visibleSession) {
|
||||
setSessionId(visibleSession);
|
||||
setCurrentSessionId(visibleSession);
|
||||
if (selectedViewField?.type === "Session") {
|
||||
setSelectedViewField({
|
||||
id: visibleSession,
|
||||
|
|
|
|||
|
|
@ -0,0 +1,81 @@
|
|||
import ForwardedIconComponent from "@/components/common/genericIconComponent";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { ICON_STROKE_WIDTH } from "@/constants/constants";
|
||||
import {
|
||||
useDeleteGlobalVariables,
|
||||
useGetGlobalVariables,
|
||||
} from "@/controllers/API/queries/variables";
|
||||
import DeleteConfirmationModal from "@/modals/deleteConfirmationModal";
|
||||
import useAlertStore from "@/stores/alertStore";
|
||||
import { cn } from "@/utils/utils";
|
||||
|
||||
interface GeneralDeleteConfirmationModalProps {
|
||||
option: string;
|
||||
onConfirmDelete: () => void;
|
||||
}
|
||||
|
||||
const GeneralDeleteConfirmationModal = ({
|
||||
option,
|
||||
onConfirmDelete,
|
||||
}: GeneralDeleteConfirmationModalProps) => {
|
||||
const setErrorData = useAlertStore((state) => state.setErrorData);
|
||||
const { mutate: mutateDeleteGlobalVariable } = useDeleteGlobalVariables();
|
||||
const { data: globalVariables } = useGetGlobalVariables();
|
||||
|
||||
async function handleDelete(key: string) {
|
||||
if (!globalVariables) return;
|
||||
const id = globalVariables.find((variable) => variable.name === key)?.id;
|
||||
if (id !== undefined) {
|
||||
mutateDeleteGlobalVariable(
|
||||
{ id },
|
||||
{
|
||||
onSuccess: () => {
|
||||
onConfirmDelete();
|
||||
},
|
||||
onError: () => {
|
||||
setErrorData({
|
||||
title: "Error deleting variable",
|
||||
list: [cn("ID not found for variable: ", key)],
|
||||
});
|
||||
},
|
||||
},
|
||||
);
|
||||
} else {
|
||||
setErrorData({
|
||||
title: "Error deleting variable",
|
||||
list: [cn("ID not found for variable: ", key)],
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<DeleteConfirmationModal
|
||||
onConfirm={(e) => {
|
||||
e.stopPropagation();
|
||||
e.preventDefault();
|
||||
handleDelete(option);
|
||||
}}
|
||||
description={'variable "' + option + '"'}
|
||||
asChild
|
||||
>
|
||||
<button
|
||||
onClick={(e) => {
|
||||
e.stopPropagation();
|
||||
}}
|
||||
className="pr-1"
|
||||
>
|
||||
<ForwardedIconComponent
|
||||
name="Trash2"
|
||||
className={cn(
|
||||
"h-4 w-4 text-primary opacity-0 hover:text-status-red group-hover:opacity-100",
|
||||
)}
|
||||
aria-hidden="true"
|
||||
/>
|
||||
</button>
|
||||
</DeleteConfirmationModal>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default GeneralDeleteConfirmationModal;
|
||||
25
src/frontend/src/shared/components/global-variable-modal.tsx
Normal file
25
src/frontend/src/shared/components/global-variable-modal.tsx
Normal file
|
|
@ -0,0 +1,25 @@
|
|||
import ForwardedIconComponent from "@/components/common/genericIconComponent";
|
||||
import GlobalVariableModal from "@/components/core/GlobalVariableModal/GlobalVariableModal";
|
||||
import { CommandItem } from "@/components/ui/command";
|
||||
import { cn } from "@/utils/utils";
|
||||
|
||||
interface GeneralGlobalVariableModalProps {}
|
||||
|
||||
const GeneralGlobalVariableModal = ({}: GeneralGlobalVariableModalProps) => {
|
||||
return (
|
||||
<>
|
||||
<GlobalVariableModal disabled={false}>
|
||||
<CommandItem value="doNotFilter-addNewVariable">
|
||||
<ForwardedIconComponent
|
||||
name="Plus"
|
||||
className={cn("mr-2 h-4 w-4 text-primary")}
|
||||
aria-hidden="true"
|
||||
/>
|
||||
<span>Add New Variable</span>
|
||||
</CommandItem>
|
||||
</GlobalVariableModal>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
export default GeneralGlobalVariableModal;
|
||||
|
|
@ -40,4 +40,7 @@ export const useUtilityStore = create<UtilityStoreType>((set, get) => ({
|
|||
webhookPollingInterval: 5000,
|
||||
setWebhookPollingInterval: (webhookPollingInterval: number) =>
|
||||
set({ webhookPollingInterval }),
|
||||
currentSessionId: "",
|
||||
setCurrentSessionId: (sessionId: string) =>
|
||||
set({ currentSessionId: sessionId }),
|
||||
}));
|
||||
|
|
|
|||
40
src/frontend/src/stores/voiceStore.ts
Normal file
40
src/frontend/src/stores/voiceStore.ts
Normal file
|
|
@ -0,0 +1,40 @@
|
|||
import { VoiceStoreType } from "@/types/zustand/voice/voice.types";
|
||||
import { create } from "zustand";
|
||||
|
||||
export const useVoiceStore = create<VoiceStoreType>((set, get) => ({
|
||||
voices: [],
|
||||
setVoices: (
|
||||
voices: {
|
||||
name: string;
|
||||
voice_id: string;
|
||||
}[],
|
||||
) => set({ voices }),
|
||||
providersList: [
|
||||
{ name: "OpenAI", value: "openai" },
|
||||
{ name: "ElevenLabs", value: "elevenlabs" },
|
||||
],
|
||||
setProvidersList: (
|
||||
providersList: {
|
||||
name: string;
|
||||
value: string;
|
||||
}[],
|
||||
) => set({ providersList }),
|
||||
openaiVoices: [
|
||||
{ name: "alloy", value: "alloy" },
|
||||
{ name: "ash", value: "ash" },
|
||||
{ name: "ballad", value: "ballad" },
|
||||
{ name: "coral", value: "coral" },
|
||||
{ name: "echo", value: "echo" },
|
||||
{ name: "sage", value: "sage" },
|
||||
{ name: "shimmer", value: "shimmer" },
|
||||
{ name: "verse", value: "verse" },
|
||||
],
|
||||
setOpenaiVoices: (
|
||||
openaiVoices: {
|
||||
name: string;
|
||||
value: string;
|
||||
}[],
|
||||
) => set({ openaiVoices }),
|
||||
soundDetected: false,
|
||||
setSoundDetected: (soundDetected: boolean) => set({ soundDetected }),
|
||||
}));
|
||||
|
|
@ -1280,6 +1280,10 @@
|
|||
.btn-add-input-list {
|
||||
@apply flex h-8 w-full items-center justify-center rounded-md p-2 text-sm hover:bg-muted;
|
||||
}
|
||||
|
||||
.btn-playground-actions {
|
||||
@apply flex h-[32px] w-[32px] items-center justify-center rounded-md bg-muted font-bold transition-all;
|
||||
}
|
||||
}
|
||||
|
||||
/* Gradient background */
|
||||
|
|
|
|||
|
|
@ -187,6 +187,12 @@
|
|||
--slider-input-border: #d4d4d8;
|
||||
|
||||
--zinc-foreground: 240 5.9% 90%;
|
||||
|
||||
|
||||
|
||||
--accent-amber: 21.7 77.8% 26.5%;
|
||||
--accent-amber-foreground: 26 90.5% 37.1%;
|
||||
--red-foreground: 0 90.6% 70.8%;
|
||||
}
|
||||
|
||||
.dark {
|
||||
|
|
@ -214,8 +220,7 @@
|
|||
--placeholder-foreground: 240 4% 46%; /* hsl(240, 4%, 46%) */
|
||||
--canvas: 0 0% 0%; /* hsl(0, 0%, 0%) */
|
||||
--canvas-dot: 240 5.3% 26.1%; /* hsl(240, 5.3%, 26.1%) */
|
||||
--accent-amber: 26 90% 37%; /* hsl(26, 90%, 37%) */
|
||||
--accent-amber-foreground: 26 90% 37%; /* hsl(26, 90%, 37%) */
|
||||
|
||||
--accent-emerald: 164 86% 16%; /* hsl(164, 86%, 16%) */
|
||||
--accent-emerald-foreground: 158 64% 52%; /* hsl(158, 64%, 52%) */
|
||||
--accent-emerald-hover: 163.1 88.1% 19.8%; /* hsl(163.1, 88.1%, 19.8%) */
|
||||
|
|
@ -439,5 +444,10 @@
|
|||
--slider-input-border: #d4d4d8;
|
||||
|
||||
--zinc-foreground: 240 5.2% 33.9%;
|
||||
|
||||
|
||||
--accent-amber: 48 96.5% 88.8%;
|
||||
--accent-amber-foreground: 45.9 96.7% 64.5%;
|
||||
--red-foreground: 0 72.2% 50.6%;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -47,6 +47,8 @@ export type InputComponentType = {
|
|||
nodeStyle?: boolean;
|
||||
isToolMode?: boolean;
|
||||
popoverWidth?: string;
|
||||
commandWidth?: string;
|
||||
blockAddNewGlobalVariable?: boolean;
|
||||
};
|
||||
export type DropDownComponent = {
|
||||
disabled?: boolean;
|
||||
|
|
|
|||
|
|
@ -21,6 +21,8 @@ export type UtilityStoreType = {
|
|||
setChatValueStore: (value: string) => void;
|
||||
dismissAll: boolean;
|
||||
setDismissAll: (dismissAll: boolean) => void;
|
||||
currentSessionId: string;
|
||||
setCurrentSessionId: (sessionId: string) => void;
|
||||
setClientId: (clientId: string) => void;
|
||||
clientId: string;
|
||||
};
|
||||
|
|
|
|||
34
src/frontend/src/types/zustand/voice/voice.types.ts
Normal file
34
src/frontend/src/types/zustand/voice/voice.types.ts
Normal file
|
|
@ -0,0 +1,34 @@
|
|||
export type VoiceStoreType = {
|
||||
voices: {
|
||||
name: string;
|
||||
voice_id: string;
|
||||
}[];
|
||||
setVoices: (
|
||||
voices: {
|
||||
name: string;
|
||||
voice_id: string;
|
||||
}[],
|
||||
) => void;
|
||||
providersList: {
|
||||
name: string;
|
||||
value: string;
|
||||
}[];
|
||||
setProvidersList: (
|
||||
providersList: {
|
||||
name: string;
|
||||
value: string;
|
||||
}[],
|
||||
) => void;
|
||||
openaiVoices: {
|
||||
name: string;
|
||||
value: string;
|
||||
}[];
|
||||
setOpenaiVoices: (
|
||||
openaiVoices: {
|
||||
name: string;
|
||||
value: string;
|
||||
}[],
|
||||
) => void;
|
||||
soundDetected: boolean;
|
||||
setSoundDetected: (soundDetected: boolean) => void;
|
||||
};
|
||||
|
|
@ -70,6 +70,10 @@ function getInactiveVertexData(vertexId: string): VertexBuildTypeAPI {
|
|||
return inactiveVertexData;
|
||||
}
|
||||
|
||||
function logFlowLoad(message: string, data?: any) {
|
||||
console.log(`[FlowLoad] ${message}`, data || "");
|
||||
}
|
||||
|
||||
export async function updateVerticesOrder(
|
||||
flowId: string,
|
||||
startNodeId?: string | null,
|
||||
|
|
@ -82,6 +86,7 @@ export async function updateVerticesOrder(
|
|||
runId?: string;
|
||||
verticesToRun: string[];
|
||||
}> {
|
||||
logFlowLoad("Updating vertices order");
|
||||
return new Promise(async (resolve, reject) => {
|
||||
const setErrorData = useAlertStore.getState().setErrorData;
|
||||
let orderResponse;
|
||||
|
|
@ -93,7 +98,9 @@ export async function updateVerticesOrder(
|
|||
nodes,
|
||||
edges,
|
||||
);
|
||||
logFlowLoad("Got vertices order response:", orderResponse);
|
||||
} catch (error: any) {
|
||||
logFlowLoad("Error getting vertices order:", error);
|
||||
setErrorData({
|
||||
title: MISSED_ERROR_ALERT,
|
||||
list: [error.response?.data?.detail ?? "Unknown Error"],
|
||||
|
|
@ -130,6 +137,7 @@ export async function updateVerticesOrder(
|
|||
export async function buildFlowVerticesWithFallback(
|
||||
params: BuildVerticesParams,
|
||||
) {
|
||||
logFlowLoad("Starting flow load");
|
||||
try {
|
||||
// Use shouldUsePolling() to determine stream mode
|
||||
return await buildFlowVertices({ ...params });
|
||||
|
|
|
|||
11
src/frontend/src/utils/local-storage-util.ts
Normal file
11
src/frontend/src/utils/local-storage-util.ts
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
export const getLocalStorage = (key: string) => {
|
||||
return localStorage.getItem(key);
|
||||
};
|
||||
|
||||
export const setLocalStorage = (key: string, value: string) => {
|
||||
localStorage.setItem(key, value);
|
||||
};
|
||||
|
||||
export const removeLocalStorage = (key: string) => {
|
||||
localStorage.removeItem(key);
|
||||
};
|
||||
|
|
@ -1698,17 +1698,49 @@ export function templatesGenerator(data: APIObjectType) {
|
|||
|
||||
export function extractFieldsFromComponenents(data: APIObjectType) {
|
||||
const fields = new Set<string>();
|
||||
|
||||
// Check if data exists
|
||||
if (!data) {
|
||||
console.warn("[Types] Data is undefined in extractFieldsFromComponenents");
|
||||
return fields;
|
||||
}
|
||||
|
||||
Object.keys(data).forEach((key) => {
|
||||
// Check if data[key] exists
|
||||
if (!data[key]) {
|
||||
console.warn(
|
||||
`[Types] data["${key}"] is undefined in extractFieldsFromComponenents`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
Object.keys(data[key]).forEach((kind) => {
|
||||
// Check if data[key][kind] exists
|
||||
if (!data[key][kind]) {
|
||||
console.warn(
|
||||
`[Types] data["${key}"]["${kind}"] is undefined in extractFieldsFromComponenents`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
// Check if template exists
|
||||
if (!data[key][kind].template) {
|
||||
console.warn(
|
||||
`[Types] data["${key}"]["${kind}"].template is undefined in extractFieldsFromComponenents`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
|
||||
Object.keys(data[key][kind].template).forEach((field) => {
|
||||
if (
|
||||
data[key][kind].template[field].display_name &&
|
||||
data[key][kind].template[field].show
|
||||
data[key][kind].template[field]?.display_name &&
|
||||
data[key][kind].template[field]?.show
|
||||
)
|
||||
fields.add(data[key][kind].template[field].display_name!);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
return fields;
|
||||
}
|
||||
/**
|
||||
|
|
|
|||
11
src/frontend/src/utils/session-storage-util.ts
Normal file
11
src/frontend/src/utils/session-storage-util.ts
Normal file
|
|
@ -0,0 +1,11 @@
|
|||
export const getSessionStorage = (key: string) => {
|
||||
return sessionStorage.getItem(key);
|
||||
};
|
||||
|
||||
export const setSessionStorage = (key: string, value: string) => {
|
||||
sessionStorage.setItem(key, value);
|
||||
};
|
||||
|
||||
export const removeSessionStorage = (key: string) => {
|
||||
sessionStorage.removeItem(key);
|
||||
};
|
||||
|
|
@ -28,11 +28,11 @@ function toKebabCase(str: string): string {
|
|||
}
|
||||
|
||||
function toLowerCase(str: string): string {
|
||||
return str.toLowerCase();
|
||||
return str?.toLowerCase();
|
||||
}
|
||||
|
||||
function toUpperCase(str: string): string {
|
||||
return str.toUpperCase();
|
||||
return str?.toUpperCase();
|
||||
}
|
||||
|
||||
function noBlank(str: string): string {
|
||||
|
|
|
|||
|
|
@ -33,6 +33,7 @@ import {
|
|||
ArrowRightLeft,
|
||||
ArrowUpRight,
|
||||
ArrowUpToLine,
|
||||
AudioLines,
|
||||
Bell,
|
||||
Binary,
|
||||
Blocks,
|
||||
|
|
@ -154,6 +155,9 @@ import {
|
|||
MessageSquare,
|
||||
MessageSquareMore,
|
||||
MessagesSquare,
|
||||
Mic,
|
||||
Mic2,
|
||||
MicOff,
|
||||
Minimize2,
|
||||
Minus,
|
||||
Monitor,
|
||||
|
|
@ -1031,6 +1035,10 @@ export const nodeIconsLucide: iconsType = {
|
|||
ScrapeGraph: ScrapeGraph,
|
||||
ScrapeGraphSmartScraperApi: ScrapeGraph,
|
||||
ScrapeGraphMarkdownifyApi: ScrapeGraph,
|
||||
Mic,
|
||||
MicOff,
|
||||
Mic2,
|
||||
DollarSign,
|
||||
BookOpenText,
|
||||
AudioLines,
|
||||
};
|
||||
|
|
|
|||
|
|
@ -262,6 +262,7 @@ const config = {
|
|||
"cosmic-void": "hsl(var(--cosmic-void))",
|
||||
"slider-input-border": "var(--slider-input-border)",
|
||||
"zinc-foreground": "hsl(var(--zinc-foreground))",
|
||||
"red-foreground": "hsl(var(--red-foreground))",
|
||||
},
|
||||
borderRadius: {
|
||||
lg: `var(--radius)`,
|
||||
|
|
|
|||
60
src/frontend/tests/core/features/voice-assistant.spec.ts
Normal file
60
src/frontend/tests/core/features/voice-assistant.spec.ts
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
import { expect, test } from "@playwright/test";
|
||||
import { awaitBootstrapTest } from "../../utils/await-bootstrap-test";
|
||||
|
||||
test(
|
||||
"should able to see and interact with voice assistant",
|
||||
{ tag: ["@release", "@workspace", "@api"] },
|
||||
|
||||
async ({ page }) => {
|
||||
test.skip(
|
||||
!process?.env?.OPENAI_API_KEY,
|
||||
"OPENAI_API_KEY required to run this test",
|
||||
);
|
||||
|
||||
await awaitBootstrapTest(page);
|
||||
|
||||
await page.getByTestId("side_nav_options_all-templates").click();
|
||||
await page.getByRole("heading", { name: "Basic Prompting" }).click();
|
||||
await page.getByTestId("playground-btn-flow-io").click();
|
||||
|
||||
await expect(page.getByTestId("voice-button")).toBeVisible();
|
||||
|
||||
await page.getByTestId("voice-button").click();
|
||||
|
||||
try {
|
||||
const apiKeyInput = page.getByTestId("popover-anchor-openai-api-key");
|
||||
|
||||
const isVisible = await apiKeyInput
|
||||
.isVisible({ timeout: 2000 })
|
||||
.catch(() => false);
|
||||
|
||||
if (isVisible) {
|
||||
await apiKeyInput.fill(process.env.OPENAI_API_KEY || "");
|
||||
await page
|
||||
.getByTestId("voice-assistant-settings-modal-save-button")
|
||||
.click();
|
||||
}
|
||||
} catch (e) {
|
||||
console.log(e);
|
||||
}
|
||||
|
||||
await expect(page.getByTestId("voice-assistant-container")).toBeVisible();
|
||||
await page.getByTestId("voice-assistant-settings-icon").click();
|
||||
await expect(
|
||||
page.getByTestId("voice-assistant-settings-modal-microphone-select"),
|
||||
).toBeVisible();
|
||||
await expect(
|
||||
page.getByTestId("voice-assistant-settings-modal-header"),
|
||||
).toBeVisible();
|
||||
|
||||
await page.keyboard.press("Escape");
|
||||
|
||||
await page.getByTestId("voice-assistant-close-button").click();
|
||||
|
||||
await expect(
|
||||
page.getByTestId("voice-assistant-settings-modal-microphone-select"),
|
||||
).not.toBeVisible();
|
||||
|
||||
await expect(page.getByTestId("input-wrapper")).toBeVisible();
|
||||
},
|
||||
);
|
||||
|
|
@ -72,11 +72,15 @@ test(
|
|||
|
||||
await page.getByText("openai").last().click();
|
||||
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
await page.getByPlaceholder("Fields").waitFor({
|
||||
state: "visible",
|
||||
timeout: 30000,
|
||||
});
|
||||
|
||||
await page.waitForTimeout(1000);
|
||||
|
||||
await page.getByPlaceholder("Fields").fill("ollama");
|
||||
|
||||
await page.keyboard.press("Escape");
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue