• xthexder@l.sw0.com
    link
    fedilink
    English
    arrow-up
    8
    ·
    2 days ago

    they built a model specifically to work well on the benchmarks.

    To be fair, I’m pretty sure that’s what everyone is doing. If you’re not measuring against something, there’s no way to tell if you’re doing anything at all.

    • jacksilver@lemmy.world
      link
      fedilink
      English
      arrow-up
      1
      ·
      2 days ago

      My point was a mixture of Experts model could suffer from generalization. Although in reading more I’m not sure if it’s the newer R model that had the MoE element.