So his site that already doesn’t make money with let subscribers make porn at an astonishing rate of electricity. He has no clue how to run a business.

  • BreadstickNinja@lemmy.world
    link
    fedilink
    English
    arrow-up
    2
    ·
    4 months ago

    There are a series of quants available at much smaller sizes such that you don’t need the full size model. The full fp16 model is 23 GB but the fp8 quant is half that, and there are GGUF quants down to 6.8 GB.