Create a search based document for graph-of-thoughts

.fdignore

@@ -0,0 +1,4 @@
+.*
+*.{bz2,svg,csv}
+docs
+LICENSE

docs/.gitignore

@@ -0,0 +1,6 @@
+!.gitignore
+!*
+!*/*
+cache_db.json
+cache_tree.json
+vector_cache

docs/data/6.json

@@ -0,0 +1,24 @@
+{
+    "600": {
+        "file_id": 36,
+        "content": "[build-system]\nrequires = [\"hatchling\"]\nbuild-backend = \"hatchling.build\"\n[project]\nname = \"graph_of_thoughts\"\nversion = \"0.0.3\"\nauthors = [\n  { name=\"Maciej Besta\", email=\"[email protected]\" },\n  { name=\"Nils Blach\", email=\"[email protected]\" },\n  { name=\"Ales Kubicek\", email=\"[email protected]\" },\n  { name=\"Robert Gerstenberger\", email=\"[email protected]\" },\n]\ndescription = \"Python package for Graph of Thoughts that enables solving elaborate problems with Large Language Models\"\nreadme = \"README.md\"\nlicense = {file = \"LICENSE\"}\nrequires-python = \">=3.8\"\nclassifiers = [\n  \"Programming Language :: Python :: 3\",\n  \"Operating System :: OS Independent\",\n]\ndependencies = [\n  \"backoff>=2.2.1,<3.0.0\",\n  \"openai>=1.0.0,<2.0.0\",\n  \"matplotlib>=3.7.1,<4.0.0\",\n  \"numpy>=1.24.3,<2.0.0\",\n  \"pandas>=2.0.3,<3.0.0\",\n  \"sympy>=1.12,<2.0\",\n  \"torch>=2.0.1,<3.0.0\",\n  \"transformers>=4.31.0,<5.0.0\",\n  \"accelerate>=0.21.0,<1.0.0\",\n  \"bitsandbytes>=0.41.0,<1.0.0\",\n  \"scipy>=1.10.1,<2.0.0\",\n]\n[project.urls]\nHomepage = \"https://github.com/spcl/graph-of-thoughts\"",
+        "type": "code",
+        "location": "/pyproject.toml:1-37"
+    },
+    "601": {
+        "file_id": 36,
+        "content": "This code defines the project settings for a Python package called \"graph_of_thoughts\" using Hatchling as the build system. It specifies the package name, version, authors, description, dependencies, and URLs for further information.",
+        "type": "comment"
+    },
+    "602": {
+        "file_id": 36,
+        "content": "[project.scripts]",
+        "type": "code",
+        "location": "/pyproject.toml:39-39"
+    },
+    "603": {
+        "file_id": 36,
+        "content": "The code snippet is a part of a TOML configuration file, specifically defining the \"scripts\" section within the \"project\" block. It sets up an entry point for executable scripts under the project's namespace.",
+        "type": "comment"
+    }
+}

docs/doc/0cea9af3-6d7e-464b-9c98-67d890a84704.json

@@ -0,0 +1,10 @@
+{
+    "summary": "This directory contains scripts for running various examples using the Graph of Thoughts package. Each script is a standalone Python program that sets up and runs a particular example, with prompt files available to test prompts manually in a console. Individual example directories provide more information on specific examples.",
+    "details": [
+        {
+            "comment": "This directory contains scripts for running various examples using the Graph of Thoughts package. Each script is a standalone Python program that sets up and runs a particular example, with prompt files available to test prompts manually in a console. Individual example directories provide more information on specific examples.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/examples/README.md\":0-6",
+            "content": "# Examples\nThis directory contains scripts for running various examples using the Graph of Thoughts package. Each script is a standalone Python program that sets up and runs a particular example.\nWe further include prompt files for each example that can be used to test prompts manually in a console.\nPlease refer to the individual example directories for more information on the specific example."
+        }
+    ]
+}

docs/doc/33c839ec-9562-4993-8007-5179237aa1f5.json

@@ -0,0 +1,50 @@
+{
+    "summary": "The Language Models module supports GPT-4/GPT-3.5 and Llama-2, with functionality for instantiating LLMs, adding new ones, and using OpenAI API features like pricing and response_token_cost. It is implemented in a base class for building language models that allows for querying and retrieving response texts.",
+    "details": [
+        {
+            "comment": "This code introduces the Language Models module and explains its purpose. It currently supports GPT-4/GPT-3.5 (Remote - OpenAI API) and Llama-2 (Local - HuggingFace Transformers). The following sections describe how to instantiate individual LLMs and add new ones to the framework. The LLM instantiation process involves creating a copy of `config_template.json`, filling in configuration details based on the used model, and adjusting predefined configurations or creating a new one with an unique key for GPT-4/GPT-3.5.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":0-17",
+            "content": "# Language Models\nThe Language Models module is responsible for managing the large language models (LLMs) used by the Controller.\nCurrently, the framework supports the following LLMs:\n- GPT-4 / GPT-3.5 (Remote - OpenAI API)\n- Llama-2 (Local - HuggingFace Transformers) \nThe following sections describe how to instantiate individual LLMs and how to add new LLMs to the framework.\n## LLM Instantiation\n- Create a copy of `config_template.json` named `config.json`.\n- Fill configuration details based on the used model (below).\n### GPT-4 / GPT-3.5\n- Adjust predefined `chatgpt`,  `chatgpt4` or create new configuration with an unique key.\n| Key                 | Value                                                                                                                                                                                                                                                                                                                                                               |"
+        },
+        {
+            "comment": "This table maps model IDs to their respective OpenAI names and calculates prompt token costs based on OpenAI pricing, which is used for determining cumulative prices per language modeling (LLM) instance.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":18-20",
+            "content": "|---------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| model_id            | Model name based on [OpenAI model overview](https://platform.openai.com/docs/models/overview).                                                                                                                                                                                                                                                                      |\n| prompt_token_cost   | Price per 1000 prompt tokens based on [OpenAI pricing](https://openai.com/pricing), used for calculating cumulative price per LLM instance.                                                                           "
+        },
+        {
+            "comment": "The code defines 'response_token_cost', a variable representing the price per 1000 response tokens, which follows OpenAI's pricing. It also includes 'temperature', a parameter controlling randomness and creativity in responses. The value is between 0.0 and 2.0, defaulting to 1.0, with further details available in the OpenAI API reference. Lastly, 'max_tokens' sets the maximum number of tokens generated in chat completions.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":20-23",
+            "content": "                                                                                                                                              |\n| response_token_cost | Price per 1000 response tokens based on [OpenAI pricing](https://openai.com/pricing), used for calculating cumulative price per LLM instance.                                                                                                                                                                                                                       |\n| temperature         | Parameter of OpenAI models that controls randomness and the creativity of the responses (higher temperature = more diverse and unexpected responses). Value between 0.0 and 2.0, default is 1.0. More information can be found in the [OpenAI API reference](https://platform.openai.com/docs/api-reference/completions/create#completions/create-temperature).     |\n| max_tokens          | The maximum number of tokens to generate in the chat completion. Value "
+        },
+        {
+            "comment": "This code defines three input parameters for the OpenAI API's chat creation endpoint: \"model\", \"stop\", and \"organization\". The model parameter specifies the language model to use, with its maximum context size determined by the OpenAI model overview. The stop parameter identifies a sequence of characters that halt further token generation, referencing the OpenAI API reference for more information. Lastly, organization is an optional field used for API requests, which can be left empty.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":23-25",
+            "content": "depends on the maximum context size of the model specified in the [OpenAI model overview](https://platform.openai.com/docs/models/overview). More information can be found in the [OpenAI API reference](https://platform.openai.com/docs/api-reference/chat/create#chat/create-max_tokens). |\n| stop                | String or array of strings specifying sequence of characters which if detected, stops further generation of tokens. More information can be found in the [OpenAI API reference](https://platform.openai.com/docs/api-reference/chat/create#chat/create-stop).                                                                                                       |\n| organization        | Organization to use for the API requests (may be empty).                                                                                                                                                                                                                                                                                                            |"
+        },
+        {
+            "comment": "The code snippet is initializing a language model controller using the ChatGPT class. It takes in the path to a configuration file and a model name corresponding to the selected configuration key. The model can be predefined (llama7b-hf, llama13b-hf, llama70b-hf) or custom with a unique key.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":26-41",
+            "content": "| api_key             | Personal API key that will be used to access OpenAI API.                                                                                                                                                                                                                                                                                                            |\n- Instantiate the language model based on the selected configuration key (predefined / custom).\n```\nlm = controller.ChatGPT(\n    \"path/to/config.json\", \n    model_name=<configuration key>\n)\n```\n### Llama-2\n- Requires local hardware to run inference and a HuggingFace account.\n- Adjust predefined `llama7b-hf`, `llama13b-hf`, `llama70b-hf` or create a new configuration with an unique key.\n| Key                 | Value                                                                                                                                                                           |\n|---------------------|----------------"
+        },
+        {
+            "comment": "This code block is defining the parameters for a language model, including the Llama 2 model identifier (`model_id`), the local directory where the model will be stored and accessed (`cache_dir`), the price per 1000 prompt tokens (`prompt_token_cost`), the price per 1000 response tokens (`response_token_cost`), and a parameter for temperature control. Note that currently, these costs are not used due to the local model being cost-free.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":41-46",
+            "content": "-----------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| model_id            | Specifies HuggingFace Llama 2 model identifier (`meta-llama/<model_id>`).                                                                                                       |\n| cache_dir           | Local directory where model will be downloaded and accessed.                                                                                                                    |\n| prompt_token_cost   | Price per 1000 prompt tokens (currently not used - local model = no cost).                                                                                                      |\n| response_token_cost | Price per 1000 response tokens (currently not used - local model = no cost).                                                                                                    |\n| temperature         | Parameter "
+        },
+        {
+            "comment": "The code initializes a language model (Llama2HF) with a specified configuration key, which determines the randomness and creativity of responses. It also sets top-K sampling method from Transformers tutorial and maximum tokens to generate in chat completion. Access to Llama-2 is requested via Meta form using the same email as HuggingFace account, then access HuggingFace Llama-2 model page.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":46-58",
+            "content": "that controls randomness and the creativity of the responses (higher temperature = more diverse and unexpected responses). Value between 0.0 and 1.0, default is 0.6. |\n| top_k               | Top-K sampling method described in [Transformers tutorial](https://huggingface.co/blog/how-to-generate). Default value is set to 10.                                            |\n| max_tokens          | The maximum number of tokens to generate in the chat completion. More tokens require more memory.                                                                               |\n- Instantiate the language model based on the selected configuration key (predefined / custom).\n```\nlm = controller.Llama2HF(\n    \"path/to/config.json\", \n    model_name=<configuration key>\n)\n```\n- Request access to Llama-2 via the [Meta form](https://ai.meta.com/resources/models-and-libraries/llama-downloads/) using the same email address as for the HuggingFace account.\n- After the access is granted, go to [HuggingFace Llama-2 model ca"
+        },
+        {
+            "comment": "This code provides instructions for adding a new LLM (Language Language Model) to the existing model. To do so, create a subclass of `AbstractLanguageModel` and use the constructor to load configuration and instantiate the language model if needed. The model is downloaded from HuggingFace into the cache directory specified in the config.json. Running queries with larger models may require multiple GPUs, which will be automatically split by the Transformers library.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":58-75",
+            "content": "rd](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf), log in and accept the license (_\"You have been granted access to this model\"_ message should appear).\n- Generate HuggingFace access token.\n- Log in from CLI with: `huggingface-cli login --token <your token>`.\nNote: 4-bit quantization is used to reduce the model size for inference. During instantiation, the model is downloaded from HuggingFace into the cache directory specified in the `config.json`. Running queries using larger models will require multiple GPUs (splitting across many GPUs is done automatically by the Transformers library).\n## Adding LLMs\nMore LLMs can be added by following these steps:\n- Create new class as a subclass of `AbstractLanguageModel`.\n- Use the constructor for loading configuration and instantiating the language model (if needed). \n```\nclass CustomLanguageModel(AbstractLanguageModel):\n    def __init__(\n        self,\n        config_path: str = \"\",\n        model_name: str = \"llama7b-hf\",\n        cache: bool = False\n    ) -> None:"
+        },
+        {
+            "comment": "The code is a part of a class that serves as a base for building language models. It loads configuration and initializes the model. The `query` method calls the LLM to get responses based on a query, while `get_response_texts` retrieves raw texts from the response structure produced by `query`. These methods are abstract and need to be implemented in child classes.",
+            "location": "\"/media/root/Toshiba XG3/works/graph-of-thoughts/docs/src/graph_of_thoughts/language_models/README.md\":76-94",
+            "content": "        super().__init__(config_path, model_name, cache)\n        self.config: Dict = self.config[model_name]\n        # Load data from configuration into variables if needed\n        # Instantiate LLM if needed\n```\n- Implement `query` abstract method that is used to get a list of responses from the LLM (call to remote API or local model inference).\n```\ndef query(self, query: str, num_responses: int = 1) -> Any:\n    # Support caching \n    # Call LLM and retrieve list of responses - based on num_responses    \n    # Return LLM response structure (not only raw strings)    \n```\n- Implement `get_response_texts` abstract method that is used to get a list of raw texts from the LLM response structure produced by `query`.\n```\ndef get_response_texts(self, query_response: Union[List[Dict], Dict]) -> List[str]:\n    # Retrieve list of raw strings from the LLM response structure    \n```"
+        }
+    ]
+}