With Ollama you can run popular LLMs like Llama, Gemma, Qwen and dozens of others locally for free!

Ollama Setup

  1. Open AI Toolbox Settings window (Edit ▶︎ Project Settings ▶︎ AI Toolbox ▶︎ ‘Ollama’ tab).
  2. Install the Ollama application locally on your machine: use Download at ollama.com button to download the Ollama app. Make sure the app is running after installation.
  3. Install at least one Ollama model, either manually from the Ollama website, or using the Terminal command ollama run <model_name>. For example, to install the llama 3.2 model, use the command ollama run llama3.2.
  4. Test the connection using the Test Connection button.

Once you add any additional model, please click the Refresh installed models button to update the list of available models in the dropdown.

To remove a model, use the Terminal command ollama rm <model_name>.

Selecting Ollama Chatbot in AI Toolbox

Selecting Ollama Chatbot in the Code Generation Window

  1. Open the Generate Component window: Window ▶︎ AI Toolbox ▶︎ Generate …. To open Generate Script window, you can also use the shortcut: Ctrl+Shift+S (Windows) or Cmd+Shift+S (Mac).
  2. Expand Settings menu on the bottom of the window.
  3. Select Ollama in the Chatbot dropdown.
  4. If you have more than one Ollama model installed, you can select the desired model in the AI Toolbox Settings (Edit → Project Settings → AI Toolbox → ‘Ollama’ tab).

Ollama selected as the backend in the ‘Generate Component’ window.

Selecting Ollama Chatbot in the Chat with AI Window

  1. Open the Chat with AI window: Window ▶︎ AI Toolbox ▶︎ Chat with AI, or use the shortcut: Alt+Shift+C (Windows) or Option+Shift+C (Mac).
  2. Click the Settings button on the bottom of the window.
  3. Select Ollama in the Chatbot dropdown.
  4. If you have more than one Ollama model installed, you can select the desired model in the Model dropdown.

Ollama selected as the backend in the ‘Chat with AI’ window.

Ollama Models

The wide range of LLMs available in Ollama can be used for code generation, code explanation, and chatting with your AI assistant. The list of the available LLMs is available at the Ollama website.

Comments