Private LLM - Local AI Chat 12+

I purchased also for the idea of having a repository for shortcuts and extensions to lunch the Private LLM directly from Finder or inside a document. It's incomplete without that functionality. It would be nice to have the following "extensions" directly in Finder:
- analyze the document
- summerize document
- chat with the document
and inside the file
- translate
- summerize the text
- rephase
- elaborate

Last but not least it would be nice to add the DocsGPT model and the possibility of dragging an drop one or more document directly in the Private LLM app.

Thanks for the review! We don't ship any shortcuts with the app, but you can download popular user contributed shortcuts from our website (privatellm.app). Also, we have macOS services for bulletizing text, grammar correction, rephrasing and shortening text. You can right click on editable text within any app and access them from the services sub-menu. Additionally, users can also assign keyboard shortcuts to these services from System Settings -> Keyboard -> Keyboard Shortcuts -> Services -> Text. Also, users can add their own services for translation, or anything else they'd like to wish for by creating a macOS Shortcut that uses Private LLM and checking the checkbox within the Shortcuts app to make it accessible from the Services menu. Finally, We're working on implementing Retrieval augmented generation (RAG). As you know, given our commitment to privacy, we need to implement completely on-device and there are some technical challenges that need to be overcome before we can ship that feature. If you have any feature requests, please feel free to email us (support@numen.ie) or DM us on Twitter (@private_llm) or join our Discord.

I use LMStudio, anythingLLM, Ollama and open web ui on daily basis. I have a M3MAX/64GB and unfortunately this software doesn’t have anything of useful: 1) if you need more control on models and parameters, use LMStudio; 2) if you need something to analyze your own documents use Dot (it’s free); 3) if you need something to transcribe use Whisper; 4) if you need something to Upscale images use Upscaly; 5) If you need to create images use DiffusionBee. Unfortunately this chatbot is completely useless since you have to stick to their models and nothing else. And the models are freely available on Hugginface via LMStudio or Ollama. So it is 6 bucks wasted. Sorry devs, but someone has to say this. This app is useful only if you don’t want to read anything online to make it completely for free.

Dear shaice, it looks like you aren't following your own advice to do some reading online. Please go to Youtube, Google, or any other search engine, and search for "LM Studio vs Private LLM", "Ollama vs Private LLM", etc. While you're at it, you might also want to go read the FAQ on our website (privatellm.app/faq). What you'll learn, might change your mind. If not, go to reportaproblem.apple.com and request a refund. One more thing: two of the three VC funded llama.cpp wrappers that you're shilling are bloated electron.js apps (LMStudio, anythingLLM), the third does not have a UI (Ollama). Private LLM is a fully native SwiftUI app, with deep macOS and iOS integrations. Also, none of the apps you're shilling run on iOS and iPadOS, Private LLM does.

Your application is paid, but it should allow users to upload their own personal model via file. I use Llama3, 4Q, which works everywhere, while your model does not function correctly as it responds without understanding. You have only provided Llama3, 3Q. We request that you add a feature to select and upload a GGUF file that we have already downloaded.

One of the reasons why our app is far better than other GGUF based apps is because we quantize models with the state of the art OmniQuant algorithm. It takes many hours of NVIDIA GPU time to quantize each model with OmniQuant, while it only takes a few minutes of CPU time to RTN quantize GGUF models. But the resulting OmniQuant quantized models are superior to RTN quantized models, in terms of perplexity. You'll find more details about this on our website (privatellm.app). If you have any requests for models, please write to us and we'll quantize and include the option to download it in an upcoming release.