Unsloth AI @unsloth.ai 路 Jun 15 You can now run Kimi K2.7 Code locally! 馃寴 We shrank the 1T model to 325GB (-48%) via Dynamic 2-bit where important layers are upcasted. Run at >40 tok/s on 330GB RAM/VRAM setups. Run full precision on 610 GB. Guide: unsloth.ai/docs/models/... GGUF: huggingface.co/unsloth/Kimi...
120 likes 4 replies
Replies
Man, that's quite a stretch of the term "locally"....
Now all I have to do is rob a bank and build a massive home rig.
I wanted to jokingly post a screenshot of me adding 660 GB RAM to my shopping cart, but I literally couldn't find a shop that would even let me add them to the cart 馃槀
Path to AI riches: pretend to make a model so big that nobody can run it except datacenters already running SOTA. Nobody tests it. Make wild claims about how good it is. Sell company to some other sucker. Pivot.