This is Tim from AnythingLLM (https://github.com/Mintplex-Labs/anything-llm). AnythingLLM is an open-source desktop assistant that brings together RAG (Retrieval-Augmented Generation), agents, embeddings, vector databases, and more—all in one seamless package.
We built AnythingLLM over the last year iterating and iterating from user feedback. Our primary mission is to enable people with a layperson understanding of AI to be able to use AI with little to no setup for either themselves, their jobs, or just to try out using AI as an assistant but with *privacy by default*.
From these iterations & feedback, we have a couple of key learnings I wanted to share:
- "Chat with your docs" solutions are a dime-a-dozen
- Agent frameworks require knowing how to code or are too isolated from other tools
- Users do not care about benchmarks, only outputs. The magic box needs to be magic to them.
- Asking Consumers to start a docker container or open a terminal is a non-starter for most.
- Privacy by default is non-negotiable. Either by personal preference or legal constraints
- Everything needs to be in one place
From these ideas, we landed on the current state of AnythingLLM:
- Everything in AnythingLLM is private by default, but fully customizable for advanced users.
- Built-in LLM provider, but can swap at any time to the hundreds of other local or cloud LLM providers & models.
- Built-in Vector Database, most users don't even know that it is there.
- Built-in Embedding model, but of course can change if the user wants to.
- Scrape websites, import Github/GitLab repos, YouTube Transcripts, Confluence spaces - all of this is already built in for the user.
- An entire baked-in agent framework that works seamlessly within the app. We even pre-built a handful of agent skills for customers. Custom plugins are in the next update and will be able to be built with code, or a no-code builder.
- All of this just works out of the box in a single installable app that can run on any consumer-grade laptop. Everything a user does, chats, or configures is stored on the user's device. Available for Mac, Windows, and Linux
We have been actively maintaining and working on AnythingLLM via our open-source repo for a while now and welcome contributors as we hopefully launch a Community Hub soon to really proliferate users' abilities to add more niche agent skills, data connectors, and more.
*But there is even more*
We view the desktop app as a hyper-accessible single-player version of AnythingLLM. We publish a Docker image too (https://hub.docker.com/r/mintplexlabs/anythingllm) that supports multi-user management with permissioning so that you can easily bring AnythingLLM into an organization with all of the same features with minimal headache or lift.
The Docker image is for those more adept with a CLI, but being able to comfortably go from a single-user to a multi-user version of the same familiar app was very important for us.
AnythingLLM aims to be more than a UI for LLMs, we are building a comprehensive tool to leverage LLMs and all that they can do while maintaining user privacy and not needing to be an expert on AI to do it.