I was disappointed to see that I could not save a conversation other than copying one response at time. When I put app to background then returned the conversation closed and I got back to main screen. When going to conversation screen I find the app was reset and my earlier data is not present. Still, I’m impressed to have the chance to play with models on my phone.

Definitely not as good as GPT4 models, but it runs offline locally, and it’s free. Pretty amazing.*feature request* it would be really nice if I could delete models that I’ve added. The Huggingface site for MLC AI doesn’t explain the vram requirements per model, so my current workaround is to delete the app and all data then reinstall it every time I try a new model. So far it looks like only the preinstalled model is compatible with iPad mini 6 and iPhone 13 mini based on requested VRAM requirements, but reinstalling a 1.6GB app every time seems a little unnecessary. Would also be nice if we could change the settings per model for things like repetition penalties, temperature, etc. Finally, a how to for how to import any other LLM (or recompile them for use if necessary) would be really awesome.

I love that it’s completely offline, with local inference and that there’s no subscription. But, it’d be a lot cooler with two additional features: (a) history of conversations and (b) ability to use the LLMs in iOS shortcuts

I got the latest iPhone 15 with 8GB ram but was only able to use the 7b 3bit model which uses less than 4gb of ram. Program crash consistently when I try to run the 4bit model which should only use a bit more than 4bit ram.I have to say the result from the 3bit model is comically bad, which is expected according to latest literature.