Merge pull request #50 from neo4j-product-examples/talent-workshop-addotions

Talent workshop additions

Author: zach-blumenfeld

.github/scripts/aura.py

@@ -81,7 +81,7 @@ def wait_for_status(self, instance_id, status=None, time_out=300):
         start = time.time()
         current_status = self.status(instance_id)
         while current_status != status and time.time() - start <= time_out:
-            time.sleep(20)
+            time.sleep(5)
             current_status = self.status(instance_id)
             logger.info("Waiting: {} {}".format(instance_id, current_status))
         return current_status

.github/workflows/run-notebooks.yml

@@ -41,26 +41,33 @@ jobs:
         env:
            ENV_FILE: ws.env
 
-      - name: Run data loading notebook
+      - name: Copy env file to each workshop subdir
         run: |
-          jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=1200 --execute data-load.ipynb
-          rm data-load.nbconvert.ipynb
+          cp ws.env customers-and-products/ws.env
+          cp ws.env talent/ws.env
+        env:
+           ENV_FILE: ws.env
+
+      - name: Run customer and products data loading notebook
+        run: |
+          jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=1200 --execute customers-and-products/data-load.ipynb
+          rm customers-and-products/data-load.nbconvert.ipynb
         env:
           ENV_FILE: ws.env
 
-      - name: Run and save workshop notebook
+      - name: Run and save customer and products workshop notebook
         run: |
           export AUTOMATED_RUN=true
-          jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=1200 --execute genai-workshop.ipynb
-          mv genai-workshop.nbconvert.ipynb genai-workshop-w-outputs.ipynb
+          jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=1200 --execute customers-and-products/genai-workshop.ipynb
+          mv customers-and-products/genai-workshop.nbconvert.ipynb customers-and-products/genai-workshop-w-outputs.ipynb
         env:
           ENV_FILE: ws.env
 
-      - name: Run example-app-only notebook
+      - name: Run customer and products example-app-only notebook
         run: |
           export AUTOMATED_RUN=true
-          jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=1200 --execute genai-example-app-only.ipynb
-          rm genai-example-app-only.nbconvert.ipynb
+          jupyter nbconvert --to notebook --ExecutePreprocessor.timeout=1200 --execute customers-and-products/genai-example-app-only.ipynb
+          rm customers-and-products/genai-example-app-only.nbconvert.ipynb
         env:
           ENV_FILE: ws.env
 
@@ -79,6 +86,6 @@ jobs:
         run: |
           git config --global user.name 'GitHub Action'
           git config --global user.email '[email protected]'
-          git add genai-workshop-w-outputs.ipynb
+          git add customers-and-products/genai-workshop-w-outputs.ipynb
           git commit -m "Auto-commit: Run notebook and update notebook with output file"
           git push

.gitignore

@@ -2,7 +2,6 @@ embedding_model/
 .idea/
 __pycache__/
 scratch/
-data/
-*.csv
 *.env
 .DS_Store
+.ipynb_checkpoints/

README.md

@@ -1,100 +1,16 @@
-# Neo4j GenAI Workshop
+# GraphRAG Workshops
 
-Please see [`genai-workshop.ipynb`](genai-workshop.ipynb) which serves as the self-contained workshop. 
+This repository contains two GraphRAG workshops: **Customers and Products** and **Talent**. Each workshop provides notebooks, data, and resources for hands-on exercises.
 
-The other companion notebooks contain code for staging data, building the Neo4j Graph, and providing easy access to demos:
-1. [`data-prep.ipynb`](data-prep.ipynb) stages the workshop data, sampling and formatting data sourced from the [H&M Personalized Fashion Recommendations Dataset](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/data).
-2. [`data-load.ipynb`](data-load.ipynb) loads the staged data into Neo4j, performs text embedding, and creates a vector index.
-3. [`genai-workshop-w-outputs.ipynb`](genai-workshop-w-outputs.ipynb) is a reference notebook for following along and checking outputs. It is simply `genai-workshop.ipynb`with cell outputs intact.
-4. [`genai-example-app-only.ipynb`](genai-example-app-only.ipynb) is a copy of `genai-workshop.ipynb` that contains only the final section: the demo LLM GraphRAG app for content generation. It assumes you have already run [`genai-workshop.ipynb`](genai-workshop.ipynb) and exists only for instructor demo purposes.
+## Workshops
 
 
-## Changelog
+1. **[Talent](talent)**  
+Combine both structured and unstructured data about employees and their technical skills into a knowledge graph. Perform graph pattern matching, vector search, and graph analytics to find similar skill sets and cohorts/clusters.  Use the knowledge graph to power a GraphRAG talent agent that can search and respond to inquiries about people, their skills, and their similarities.
 
-### v4 (Sep 2nd, 2024 - Present)
 
-------------
-- Transition from using Neo4j Sandbox to AuraDS
+2. **[Customers & Products](customers-and-products)**  
+Uses real-world customer and product data from a fashion, style, and beauty retailer. Learn how to use a knowledge graph to ground an LLM with GraphRAG, enabling AI to build tailored marketing content personalized to each customer based on their interests and shared purchase histories. Learn about retrieval strategies leveraging vector search, graph pattern matching, and graph machine learning.
 
 
-- Split out data loading
-  - Split out data load into a separate notebook
-  - Live workshops now begin with the dataset pre-loaded to cut down on time and spend more of the course walking through GraphRAG. the data-load.ipynb is kept for reference and replication. 
-  - Remove `neo4j_tools` Python package. the functions/utilities are now included in data-load.ipynb
 
-
-- Updated Workshop Slides
-
-
-- Added more query exploration & improved explainer queries
-  - Add Browser-based graph exploration in beginning of workshop
-  - Include database tips & more Cypher queries in multiple steps
-  - Update explainer markdown and code cells for graph patterns and GDS for clarity
-  - Various other minor adjustments to markdown and code to improve course quality
-
-
-- Added `genai-workshop-w-outputs.ipynb` and GitHub Actions Workflow
-  - `genai-workshp.ipynb` is now maintained with cleared outputs for better workshop experience and easier PR review
-  - A GitHub actions workflow automatically tests data loading, workshop, and example-only notebooks and auto-commits `genai-workshop-w-outputs.ipynb` file for each PR. 
-
-### [v3 (June 25th, 2024 - Sep 1st, 2024)](https://github.com/neo4j-product-examples/genai-workshop/releases/tag/v3.0)
-
-------------
-- improve LLM response quality and cleaned up code for LLM chains and vector stores
-  - parameterizing customer id so don't need to recreate chains & stores for each customer
-  - updated prompts to better account for seasonality and use all retrieved data
-  - update to use gpt-4o
-
-
-- Improve text embedding speed and reduce code by transitioning to native `genai.vector` Cypher functions
-
-
-- Updated slides
-
-
-- Various other minor adjustments to markdown and code to improve course quality
-
-### [v2 (Feb 20th, 2024 - June 24th, 2024)](https://github.com/neo4j-product-examples/genai-workshop/releases/tag/v2.0)
-
-------------
-
-- (fix) Add `langchain_community` to the libraries that are pip installed in the notebooks
-
-
-- Simplify and Shorten Course
-  - Shortened GDS section to just three cells to run 
-  - Condensed Vector Search Section
-  - Condensed Loading to Single Notebook Cell
-  - Switched Recommendation Retriever to a Simple KG Query
-  - Adding `neo4j_tools` Package to hold convenience functions for loading data and reduce code footprint in main workshop notebook
-  - Updated to GPT-4 throughout
-  - General Notebook Cleaning - Removed duplicate load statements, updating to newest llm packages, etc. 
-
-
-- Provide Better Explainers & Examples
-  - Add A Chain for Printing Final Prompt to LLM with retrieval data to better explain process. 
-  - Added Differentiated Names for Customer Examples in Demo App.
-
-
-- Added Additional Resources
-  - Added workshop slides
-  - Added "demo only" notebook
-
-
-### [v1 (Nov 13th, 2023 - Feb 19th, 2024)](https://github.com/neo4j-product-examples/genai-workshop/releases/tag/v1.0)
-
-------------
-
-- Initial 5-part course with
-  - Building the knowledge graph
-  - Vector search & text embedding
-  - Graph patterns to improve semantic search
-  - knowledge graph inference & ML
-  - Building the LLM chain and demo app for generating content
-
-
-## Contributing
-Contributions are welcome!  To contribute please:
-1. Make a PR with a descriptive name
-2. If you are updating [`genai-workshop.ipynb`](genai-workshop.ipynb) please ensure to clear all outputs before committing. 
-3. Do not alter the [`genai-workshop-w-outputs.ipynb`](genai-workshop-w-outputs.ipynb) file.  This file is autogenerated upon creating/updating PRs. 

customers-and-products/README.md

@@ -0,0 +1,100 @@
+# Customer & Products GraphRAG Workshop
+
+Please see [`genai-workshop.ipynb`](genai-workshop.ipynb) which serves as the self-contained workshop. 
+
+The other companion notebooks contain code for staging data, building the Neo4j Graph, and providing easy access to demos:
+1. [`data-prep.ipynb`](data-prep.ipynb) stages the workshop data, sampling and formatting data sourced from the [H&M Personalized Fashion Recommendations Dataset](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/data).
+2. [`data-load.ipynb`](data-load.ipynb) loads the staged data into Neo4j, performs text embedding, and creates a vector index.
+3. [`genai-workshop-w-outputs.ipynb`](genai-workshop-w-outputs.ipynb) is a reference notebook for following along and checking outputs. It is simply `genai-workshop.ipynb`with cell outputs intact.
+4. [`genai-example-app-only.ipynb`](genai-example-app-only.ipynb) is a copy of `genai-workshop.ipynb` that contains only the final section: the demo LLM GraphRAG app for content generation. It assumes you have already run [`genai-workshop.ipynb`](genai-workshop.ipynb) and exists only for instructor demo purposes.
+
+
+## Changelog
+
+### v4 (Sep 2nd, 2024 - Present)
+
+------------
+- Transition from using Neo4j Sandbox to AuraDS
+
+
+- Split out data loading
+  - Split out data load into a separate notebook
+  - Live workshops now begin with the dataset pre-loaded to cut down on time and spend more of the course walking through GraphRAG. the data-load.ipynb is kept for reference and replication. 
+  - Remove `neo4j_tools` Python package. the functions/utilities are now included in data-load.ipynb
+
+
+- Updated Workshop Slides
+
+
+- Added more query exploration & improved explainer queries
+  - Add Browser-based graph exploration in beginning of workshop
+  - Include database tips & more Cypher queries in multiple steps
+  - Update explainer markdown and code cells for graph patterns and GDS for clarity
+  - Various other minor adjustments to markdown and code to improve course quality
+
+
+- Added `genai-workshop-w-outputs.ipynb` and GitHub Actions Workflow
+  - `genai-workshp.ipynb` is now maintained with cleared outputs for better workshop experience and easier PR review
+  - A GitHub actions workflow automatically tests data loading, workshop, and example-only notebooks and auto-commits `genai-workshop-w-outputs.ipynb` file for each PR. 
+
+### [v3 (June 25th, 2024 - Sep 1st, 2024)](https://github.com/neo4j-product-examples/genai-workshop/releases/tag/v3.0)
+
+------------
+- improve LLM response quality and cleaned up code for LLM chains and vector stores
+  - parameterizing customer id so don't need to recreate chains & stores for each customer
+  - updated prompts to better account for seasonality and use all retrieved data
+  - update to use gpt-4o
+
+
+- Improve text embedding speed and reduce code by transitioning to native `genai.vector` Cypher functions
+
+
+- Updated slides
+
+
+- Various other minor adjustments to markdown and code to improve course quality
+
+### [v2 (Feb 20th, 2024 - June 24th, 2024)](https://github.com/neo4j-product-examples/genai-workshop/releases/tag/v2.0)
+
+------------
+
+- (fix) Add `langchain_community` to the libraries that are pip installed in the notebooks
+
+
+- Simplify and Shorten Course
+  - Shortened GDS section to just three cells to run 
+  - Condensed Vector Search Section
+  - Condensed Loading to Single Notebook Cell
+  - Switched Recommendation Retriever to a Simple KG Query
+  - Adding `neo4j_tools` Package to hold convenience functions for loading data and reduce code footprint in main workshop notebook
+  - Updated to GPT-4 throughout
+  - General Notebook Cleaning - Removed duplicate load statements, updating to newest llm packages, etc. 
+
+
+- Provide Better Explainers & Examples
+  - Add A Chain for Printing Final Prompt to LLM with retrieval data to better explain process. 
+  - Added Differentiated Names for Customer Examples in Demo App.
+
+
+- Added Additional Resources
+  - Added workshop slides
+  - Added "demo only" notebook
+
+
+### [v1 (Nov 13th, 2023 - Feb 19th, 2024)](https://github.com/neo4j-product-examples/genai-workshop/releases/tag/v1.0)
+
+------------
+
+- Initial 5-part course with
+  - Building the knowledge graph
+  - Vector search & text embedding
+  - Graph patterns to improve semantic search
+  - knowledge graph inference & ML
+  - Building the LLM chain and demo app for generating content
+
+
+## Contributing
+Contributions are welcome!  To contribute please:
+1. Make a PR with a descriptive name
+2. If you are updating [`genai-workshop.ipynb`](genai-workshop.ipynb) please ensure to clear all outputs before committing. 
+3. Do not alter the [`genai-workshop-w-outputs.ipynb`](genai-workshop-w-outputs.ipynb) file.  This file is autogenerated upon creating/updating PRs. 

data-load.ipynb renamed to customers-and-products/data-load.ipynb

@@ -6,7 +6,7 @@
    "metadata": {},
    "source": [
     "# Neo4j Generative AI - Data Loading\n",
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neo4j-product-examples/genai-workshop/blob/main/data-load.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neo4j-product-examples/genai-workshop/blob/main/customers-and-products/data-load.ipynb)\n",
     "\n",
     "This workshop will teach you how to use Neo4j for Graph-Powered Retrieval-Augmented Generation (GraphRAG) to enhance GenAI and improve response quality for real-world applications.\n",
     "\n",
@@ -18,7 +18,7 @@
     "- Building the knowledge graph and\n",
     "- generating text embeddings from scratch\n",
     "\n",
-    "[genai-workshop.ipynb](https://github.com/neo4j-product-examples/genai-workshop/blob/main/genai-workshop.ipynb) contains the rest of the workshop including\n",
+    "[genai-workshop.ipynb](https://github.com/neo4j-product-examples/genai-workshop/blob/main/customers-and-products/genai-workshop.ipynb) contains the rest of the workshop including\n",
     "    - Vector search\n",
     "    - Graph patterns to improve semantic search\n",
     "    - Augmenting semantic search with graph data science\n",
@@ -236,7 +236,7 @@
     "This workshop will leverage the [H&M Personalized Fashion Recommendations Dataset](https://www.kaggle.com/competitions/h-and-m-personalized-fashion-recommendations/data), a sample of real customer purchase data that includes rich information around products including names, types, descriptions, department sections, etc.\n",
     "\n",
     "*Bonus!*\n",
-    "The data we use is a sampled and preformatted version of the Kaggle data. If you are interested in what we did, you can find the details [here](https://github.com/neo4j-product-examples/genai-workshop/blob/main/data-prep.ipynb)"
+    "The data we use is a sampled and preformatted version of the Kaggle data. If you are interested in what we did, you can find the details [here](https://github.com/neo4j-product-examples/genai-workshop/blob/main/customers-and-products/data-prep.ipynb)"
    ]
   },
   {
@@ -540,8 +540,16 @@
    "metadata": {},
    "source": [
     "## Next Steps\n",
-    "Analyze the graph, try out a vector search, and learn how to enhance search with graphs and graph data science in [genai-workshop.ipynb](https://github.com/neo4j-product-examples/genai-workshop/blob/main/genai-workshop.ipynb)\n"
+    "Analyze the graph, try out a vector search, and learn how to enhance search with graphs and graph data science in [genai-workshop.ipynb](https://github.com/neo4j-product-examples/genai-workshop/blob/main/customers-and-products/genai-workshop.ipynb)\n"
    ]
+  },
+  {
+   "metadata": {},
+   "cell_type": "code",
+   "outputs": [],
+   "execution_count": null,
+   "source": "",
+   "id": "fceb214524a7764a"
   }
  ],
  "metadata": {

data-prep.ipynb renamed to customers-and-products/data-prep.ipynb

@@ -8,7 +8,7 @@
    },
    "source": [
     "# Neo4j Generative AI Workshop Example Application\n",
-    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neo4j-product-examples/genai-workshop/blob/main/genai-example-app-only.ipynb)\n",
+    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/neo4j-product-examples/genai-workshop/blob/main/customers-and-products/genai-example-app-only.ipynb)\n",
     "\n",
     "__This notebook is a copy of `genai-workshop.ipynb` that contains only the final section: the example application for the LLM content generator. This notebook assumes you have already run `genai-workshop.ipynb`.__\n",
     "\n",
@@ -33,7 +33,7 @@
    },
    "source": [
     "### Some Logistics\n",
-    "1. Make a copy of this notebook in Colab by [clicking here](https://colab.research.google.com/github/neo4j-product-examples/genai-workshop/blob/main/genai-workshop.ipynb).\n",
+    "1. Make a copy of this notebook in Colab by [clicking here](https://colab.research.google.com/github/neo4j-product-examples/genai-workshop/blob/main/customers-and-products/genai-workshop.ipynb).\n",
     "2. Run the pip install below to get the necessary dependencies.  this can take a while. Then run the following cell to import relevant libraries\n"
    ]
   },