Interesting. You're using a model without special finetuning for this specific purpose and managed to get it to work with just giving it a prompt. I didn't think that was possible. How would you piece together something like this? Can I just ask AI to give me a prompt which I can use on it/another AI?
How much of VRAM does your GPU have?
Thank you. I was going to try and host Ollama and Open WebUI. I think the problem is to find a source for pretrained/finetuned models which provide such.... Interaction. Does huggingface have such pre-trained models? Any suggestions?
Is there a hardened version of void? I'm interested in hardened distributions and like that Void has a musl build, but is there any dialogue from the devs or the community in using void as a hardened server OS?
Assuming NGINX is terminating SSL, I think the problem is ports.
I don't think OP made two A records here. He simply configured the reverse proxy to point to the VM and the A record to point to the reverse proxy. In my mind, if NGINX is terminating SSL then the only problem could be ports.
OP you NEED to tell me how you did this. I want this. I want to host something like character.ai on my own hardware. If you have a guide on this I'd love it.
Are there guides on this?
I'll take it. Reality isn't very fun and hasn't been good for a while for me. This is a very good escape. I'll take this over drugs
I had never heard of Kobold AI. I was going to self-host Ollama and try with it but I'll take a look at Kobold. I had never heard about controls on world-building and dialogue triggers either; there's a lot to learn.
Will more VRAM solve the problem of not retaining context? Can I throw 48GB of VRAM towards an 8B model to help it remember stuff?
Yes, I'm looking at image generation (stable diffusion) too. Thanks