|
7 | 7 | ### Control Your Computer Using LLMs
|
8 | 8 |
|
9 | 9 | Open Interface
|
10 |
| -- Self-drives your computer by sending your requests to an LLM backend (GPT-4o, etc) to figure out the required steps. |
| 10 | +- Self-drives your computer by sending your requests to an LLM backend (GPT-4o, Gemini, etc) to figure out the required steps. |
11 | 11 | - Automatically executes these steps by simulating keyboard and mouse input.
|
12 | 12 | - Course-corrects by sending the LLM backend updated screenshots of the progress as needed.
|
13 | 13 |
|
@@ -162,17 +162,29 @@ Open Interface
|
162 | 162 |
|
163 | 163 | </details>
|
164 | 164 |
|
| 165 | +<details> |
| 166 | + <summary><b>Set up the Google Gemini API key</b></summary> |
| 167 | + |
| 168 | +- Go to Settings -> Advanced Settings and select the Gemini model you wish to use. |
| 169 | +- Get your Google Gemini API key from https://aistudio.google.com/app/apikey. |
| 170 | +- Save the API key in Open Interface settings. |
| 171 | +- Save the settings and <b>restart the app</b>. |
| 172 | + |
| 173 | +</details> |
| 174 | + |
165 | 175 | <details>
|
166 | 176 | <summary><b>Optional: Setup a Custom LLM</b></summary>
|
167 | 177 |
|
168 | 178 | - Open Interface supports using other OpenAI API style LLMs (such as Llava) as a backend and can be configured easily in the Advanced Settings window.
|
169 | 179 | - Enter the custom base url and model name in the Advanced Settings window and the API key in the Settings window as needed.
|
170 |
| -- You may need to enter a random string like "xxx" in the API key input box. |
171 |
| - <br> |
172 |
| - <picture> |
173 |
| - <img src="assets/advanced_settings.png" align="middle" alt="Set API key in settings" width="400"> |
174 |
| - </picture><br> |
175 |
| - <br> |
| 180 | +- NOTE - If you're using Llama: |
| 181 | + - You may need to enter a random string like "xxx" in the API key input box. |
| 182 | + - You may need to append /v1/ to the base URL. |
| 183 | + <br> |
| 184 | + <picture> |
| 185 | + <img src="assets/advanced_settings.png" align="middle" alt="Set API key in settings" width="400"> |
| 186 | + </picture><br> |
| 187 | + <br> |
176 | 188 | - If your LLM does not support an OpenAI style API, you can use a library like [this](https://github.com/BerriAI/litellm) to convert it to one.
|
177 | 189 | - You will need to restart the app after these changes.
|
178 | 190 |
|
|
0 commit comments