Post
2036
The best quality public GGUFs and experimental MLX inference for open LLMs!
Collection: https://huggingface.co/collections/TheStageAI/edge-lm
Collection: https://huggingface.co/collections/TheStageAI/edge-lm
alkinun
AtAndDev
AI & ML interests
decentralize
Recent Activity
liked a dataset about 7 hours ago
Luoberta/cve_train liked a model about 8 hours ago
thinkingmachines/Inkling-Small liked a dataset about 8 hours ago
deepreinforce-ai/CUDA-L2Organizations
reacted to hypothetical's post with π€ about 18 hours ago
reacted to danielhanchen's post with β€οΈπ₯ 1 day ago
Post
2471
Kimi K3 can now be run locally! β¨
The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).
Run on a Mac Studio connected with 128GB RAM device. Kimi K3 is the strongest open model to date.
GGUF: unsloth/Kimi-K3-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k3
The 1-bit model retains ~78.9% accuracy after we shrunk it from 1.56TB to 594GB (-62% size).
Run on a Mac Studio connected with 128GB RAM device. Kimi K3 is the strongest open model to date.
GGUF: unsloth/Kimi-K3-GGUF
Guide: https://unsloth.ai/docs/models/kimi-k3
- 2 replies
reacted to ProCreations's post with π₯π 4 days ago
Post
2049
what should i do for 200 followers... hmmmm... top reply to this post (most emojis) wins, just not anything too crazy
- 11 replies
reacted to appvoid's post with π 11 days ago
Post
258
small reasoning models are overrated, these little ones just doom loop a lot by default. good data will always be the moat when training or finetuning small models and latest sota models like fable 5 and gpt 5.6 are increasingly making this a lot easier to do.
- 3 replies
reacted to ProCreations's post with π 12 days ago
Post
2387
grug-v2 here. grug v1 good but lost tool use brain. grug v2 RL fix. grug v2 more SFT to make more grug.
Try grug v2 here:
ProCreations/grug-v2-9b-gguf
ProCreations/grug-v2-9b
Demo space here:
ProCreations/grug-v2-9b-demo
Try grug v2 here:
ProCreations/grug-v2-9b-gguf
ProCreations/grug-v2-9b
Demo space here:
ProCreations/grug-v2-9b-demo
reacted to danielhanchen's post with π€πππ 12 days ago
Post
5710
Gemma 4 is now faster and much more accurate! π
Google made huge improvements to tool-calling and chat accuracy, reliability + speed.
To get fixes, re-download our updated GGUF, MLX, NVFP4 quants!
Unsloth quants: https://huggingface.co/collections/unsloth/gemma-4
Gemma 4 Guide: https://unsloth.ai/docs/models/gemma-4
Google made huge improvements to tool-calling and chat accuracy, reliability + speed.
To get fixes, re-download our updated GGUF, MLX, NVFP4 quants!
Unsloth quants: https://huggingface.co/collections/unsloth/gemma-4
Gemma 4 Guide: https://unsloth.ai/docs/models/gemma-4
- 8 replies
reacted to s3nh's post with πβ€οΈ 13 days ago
Post
403
Existing methods β GPTQ, AWQ, llama.cpp's k-quants β minimize empirical loss heuristically. None of them prove they are optimal in any information-theoretic sense. ICRB-Q builds a quantization scheme that is provably optimal via the CramΓ©r-Rao lower bound (CRB): no unbiased estimator of a weight can have lower variance than [F(ΞΈ)]β»ΒΉ, where F is the Fisher information matrix.
- 1 reply
replied to ProCreations's post 13 days ago
man.. someone get this man a sponsor
reacted to ProCreations's post with ππ₯π€ 13 days ago
Post
3211
sad grug news:
I have burnt a lot of GPU credits already just making grug and itβs variants (I do not own a workstation yet) and grug 35b has a pretty bad issue but if I continue burning GPU credits to fix it then it will take even longer to get the workstation I am working forβ¦ so the fix may take a few days or never happen.
The error:
I tested it in opencode after benchmarks were good and after release (my mistake) and it had pretty bad repetition failure and simply didnβt work. so.. yea.
I have burnt a lot of GPU credits already just making grug and itβs variants (I do not own a workstation yet) and grug 35b has a pretty bad issue but if I continue burning GPU credits to fix it then it will take even longer to get the workstation I am working forβ¦ so the fix may take a few days or never happen.
The error:
I tested it in opencode after benchmarks were good and after release (my mistake) and it had pretty bad repetition failure and simply didnβt work. so.. yea.
- 4 replies
reacted to s3nh's post with π 18 days ago
Post
344
Uncensoring Mistral,
give it a try
s3nh/Ministral-3-14B-Instruct-2512-BF16-abliterated
give it a try
s3nh/Ministral-3-14B-Instruct-2512-BF16-abliterated
reacted to ProCreations's post with π€π 18 days ago
Post
535
I won 4th place on the huggingface hackathon, genuinely so happy. Congrats to everyone that won, this has been super fun