We're thrilled to announce that the much-anticipated On-Device LLMs are now live in the Pieces for Developers Suite! With the ability to seamlessly switch between different LLMs, whether they're on your device or in the cloud, you now have more power and flexibility at your fingertips. On-Device LLM support is also available in our VS Code and Obsidian plugins, and coming soon to JetBrains and JupyterLab.
We support some of the most well-known LLMs, including OpenAI’s GPT and Meta’s Llama 2, so that you can leverage the strengths of these models directly in your development workflow. This means you can choose the LLM that best fits your coding style and project requirements, which enhances the Pieces Copilot's ability to provide relevant, context-aware code suggestions from your code snippets.
Let's explore this exciting update in detail!
When chatting with the Pieces Copilot in our Desktop App, you can select either On-Device or Cloud LLMs. To do so:
Currently, you can choose either the 7B CPU or 7B GPU Llama 2 model to be installed on your device. Please note: these models are large, and installation time will depend on your download speed. You can stop the download at any time by hitting the Stop button.
Once you select a model, you can set any local files, websites, or saved snippets as context and then interact with the Pieces Copilot.
Our Cloud LLM options include GPT 3.5 Turbo, GPT 3.5 Turbo 16k, and GPT 4.
Support for more LLMs is coming soon, including:
When selecting an on-device LLM, consider the following guidelines:
If you are concerned about large consumption of memory by the local LLM, don’t worry. We provide ways to unload this memory consumption.
Just like in our Desktop App, you can now select the Copilot Runtime in Pieces for VS Code. This is a significant enhancement for developers who prefer to work within their VS Code IDE instead of utilizing the Pieces Desktop App.
To choose your Copilot Runtime:
Note: The dialog box also has suggestions on what runtime might be ideal based on the device.
Additionally, you can set the context for the Pieces Copilot. Whether it's a specific file, code snippet, or entire project, you can select context to receive optimized responses from the Copilot, even offline.
Data privacy is absolutely critical to the Obsidian community. We understand that you typically do not want your vault contents to co-mingle with large companies’ training data sets. Now that On-Device LLMs are available in the Pieces Copilot, Obsidian users can interact with a Copilot contextualized by your vault without your notes leaving your device. This is all possible due to Pieces’ commitment to developing on-device, air-gapped AI features.
To access On-Device LLMs in Obsidian:
Pro tip: If you would like to use some files or folders as context within the chat, open the context selector by clicking on the settings icon in the chat, and choose which files you want to use as context for your conversation!
Do you love Pieces? Stop sending us carrier pigeons 🐦 and join our Discord Server to chat with our team, other power users, get support, and more. 🤝
As always, if you run into issues or have feedback, please fill out this quick form or email us at firstname.lastname@example.org and we’ll be in touch as soon as possible!