1. Voice Input for Chat

2. Revamped Notification Screens
3. Updated AI SQL Functions
4. Fixed Issues on the Streamlit

Author: sgsshankar

README.md

@@ -7,6 +7,7 @@ Snowflake AI Toolkit is an AI Accelerator and Playground for enabling AI in Snow
 ### Playground
 
 An interactive environment where users can chat and experiment with Snowflake Cortex functions, test prompts, and and play around with cortex functions.
+Supports both manual text entry and voice-to-text input for prompts wherever needed, across multiple functions including Complete, Translate, Extract, Sentiment Analysis, and AI Complete (debug mode only)
 
 ### Build
 
@@ -32,6 +33,25 @@ Cortex Agents orchestrate across both structured and unstructured data sources t
 - Test the Agent against the Data
 - Utilize Cortex Search and Cortex Analyst through Agents
 
+### Audio Input Integration
+
+The Snowflake AI Toolkit now supports voice input across multiple functionalities, making it easier to interact with AI functions through speech (available in debug mode only):
+
+- **Voice-to-Text Conversion**: Convert spoken words directly into text inputs for various AI functions
+- **Multi-Function Support**: Available for Complete, Complete Multimodal, Translate, Extract, Sentiment Analysis, AI Classify (Image), and AI Complete functions
+- **Real-Time Processing**: Instant speech recognition with immediate text population in input fields
+- **User-Friendly Interface**: Microphone button (🎙️) positioned conveniently next to the Run button for seamless workflow
+- **Debug Mode Only**: Audio input feature is enabled only when `mode` is set to `"debug"` in `src/settings_config.json`
+
+### Notification
+
+- **Real-Time Updates**: Receive instant notifications for every function call and completion.
+- **Synchronous & Asynchronous Support**: Stay informed about both immediate and long-running processes, including RAG and fine-tuning tasks.
+- **Computation Details**: View token usage and token credit consumption for each operation.
+- **In-App Delivery**: All notifications appear directly within the application interface.
+- **Enhanced Transparency**: Get clear, timely feedback throughout all workflows.
+
+
 ## Prerequisites
 
 Before you begin, ensure you have met the following requirements:

requirements.txt

@@ -5,4 +5,6 @@ langchain
 pandas
 pyarrow
 snowflake
-PyJWT
+PyJWT
+streamlit-mic-recorder
+snowflake.core

snowflake.yml

@@ -1,25 +1,48 @@
-definition_version: '1'
-streamlit:
-  name: snowflake_ai_toolkit
-  stage: sf_ai_stage
-  query_warehouse: compute_wh
-  main_file: streamlit_app.py
-  env_file: environment.yml
-  #pages_dir: pages/
-  additional_source_files:
-    - src/cortex_functions.py
-    - src/cortex_agent.py
-    - src/utils.py
-    - src/html_snippets.py
-    - src/styles.css
-    - src/home.py
-    - src/playground.py
-    - src/build.py
-    - src/rag.py
-    - src/fine_tune.py
-    - src/setup.py
-    - src/notification.py
-    - src/settings_config.json
-    - src/logo.png
-    - src/query_result_builder.py
-    - src/search.py
+definition_version: '2'
+
+entities:
+  streamlit:
+    type: streamlit
+    identifier: snowflake_ai_toolkit_demo
+    stage: sf_ai_stage
+    query_warehouse: compute_wh
+    main_file: streamlit_app.py
+    # pages_dir: pages/
+    # external_access_integrations:
+    #   - test_egress
+    # secrets:
+    #   dummy_secret: "db.schema.dummy_secret"
+    # imports:
+    #   - '@sf_ai_stage/streamlit_mic_recorder.zip'
+    artifacts:
+      - streamlit_app.py
+      - src/cortex_functions.py
+      - src/cortex_agent.py
+      - src/utils.py
+      - src/html_snippets.py
+      - src/styles.css
+      - src/home.py
+      - src/playground.py
+      - src/build.py
+      - src/rag.py
+      - src/fine_tune.py
+      - src/setup.py
+      - src/notification.py
+      - src/settings_config.json
+      - src/logo.png
+      - src/query_result_builder.py
+      - src/search.py
+      - environment.yml
+
+  # audio_to_text:
+  #   handler: 'src.functions.audio_to_text'
+  #   stage: sf_ai_stage
+  #   signature: ''
+  #   returns: string
+  #   artifacts: ['src/']
+  #   type: function
+  #   imports:
+  #     - '@sf_ai_stage/streamlit_mic_recorder.zip'
+  #   meta:
+  #     use_mixins:
+  #       - snowpark_shared