Google rolled out AI overviews across the United States this month, exposing its flagship product to the hallucinations of large language models.

Google rolled out AI overviews across the United States this month, exposing its flagship product to the hallucinations of large language models.

  • adam_y@lemmy.worldEnglish
    17511·
    6 months ago

    Can we swap out the word “hallucinations” for the word “bullshit”?

    I think all AI/LLM stuf should be prefaced as “someone down the pub said…”

    So, “someone down the pub said you can eat rocks” or, “someone down the pub said you should put glue on your pizza”.

    Hallucinations are cool, shit like this is worthless.

    • Eheran@lemmy.worldEnglish
      8011·
      6 months ago

      No, hallucination is a really good term. It can be super confident and seemingly correct but still completely made up.

      • kbin_space_program@kbin.run
        36·
        6 months ago

        It is, but it isnt applicable in at least the glue-pizza situation as the probable source comment has been found on reddit.

        A better use of the term might be how when you try to get Bing’s image creator to make “Battletech” art, you just mostly get really obvious Warhammer 40k Space Marines and occasionally Iron Maiden album art.

        • barsoap@lemm.eeEnglish
          5·
          6 months ago

          That’s not hallucinations (in particular), that’s concept bleed. Try the following:

          1. Acquire a human experimental subject. Ask them:
          2. What colour is snow?
          3. What colour is the fridge (point to a white fridge)?
          4. What do cows drink?

          …and hear them answer “milk”. “White, cold, drink, cow” are all wired to “milk” in our heads logic comes later. It’s quite a bit harder to trick humans with this than AIs because we do have the capacity to double-check but if you simply want to bend an answer, not have it be completely nonsensical, it’s quite easy.

          Also your 40k or Iron Maiden result might very well still be Battletech. E.g. when it comes to image composition. Another explanation would be low resolution in the prompt encoding, that’d be similar to boomers calling your PS5 a Nintendo. Most likely though it has only seen two or three Battletech images (face it, it’s not that popular in comparison) and thought “eh looks like a Nintendo that’s where I’ll store it”, Humans and current-gen AI are different in principle in that regard as we can come up with encoding strategies, they can’t. Something something T3 systems and need for exponential amounts of data.

      • richieadler@lemmy.myserv.oneEnglish
        104·
        6 months ago

        It’s a really bad term because it’s usually associated with a mind, and LLMs are nothing of the sort.

      • yukijoou@lemmy.blahaj.zoneEnglish
        713·
        6 months ago

        for it to “hallucinate” things, it would have to believe in what it’s saying. ai is unable to think - so it cannot hallucinate

        • Jrockwar@feddit.ukEnglish
          104·
          6 months ago

          Hallucination is a technical term. Nothing to do with thinking. The scientific community could have chosen another term to describe the issue but hallucination explains really well what’s happening.

          • yukijoou@lemmy.blahaj.zoneEnglish
            101·
            6 months ago

            because it’s a text generation machine…? i mean, i wouldn’t say i can prove it, but i don’t think anyone can prove it’s capable of thinking, much less of reasoning

            like, it can string together a coherent sentence thanks to well crafted equations, sure, but i wouldn’t qualify that as “thinking”, though i guess the definition of “thinking” is debatable

            • DominusOfMegadeus@sh.itjust.worksEnglish
              1·
              6 months ago

              It’s an interesting question. I am inclined to believe that the faster it gets at running those equations, over and over and over, reanalysing is data and responses as it goes, that that ultimately leads to some kind of evolution. You know, Vger style.

            • Eheran@lemmy.worldEnglish
              11·
              6 months ago

              It can tell you how to stack things on top of each other the best way to get a high tower. Etc.

              Those are not random sentences. If you can not define thinking in a way this machine fails at, then stop saying it does not think.

              • Aceticon@lemmy.worldEnglish
                2·
                6 months ago

                A parrot can be trained to tell you how to stack things on top of each other the best way to get a high tower.

                This is just an electronic parrot, millions of times faster to train than the biological parrot, specialized in repetition alone (can’t really do anything else a parrot can) and which has been trained on billions of texts.

                You’re confusing one specific form in which humans externally express cogniscence with the actual cogniscence itself: just because intelligence can produce some forms of textual communication doesn’t mean that the relationship holds in the opposite direction and such forms of textual communication require intelligence, or if you will, just because you can photograph a real pizza to get a picture of a pizza doesn’t mean a picture of a pizza is actually of a real pizza and not something with glue to make it look like it has stringy melted cheese.

                • Eheran@lemmy.worldEnglish
                  12·
                  6 months ago

                  Again, it is absolutely capable to come up with it’s own logical stuff, hence my example. Stop saying it just copies existing stuff, that is simply wrong.

                  • yukijoou@lemmy.blahaj.zoneEnglish
                    2·
                    6 months ago

                    it is absolutely capable to come up with it’s own logical stuff

                    interesting, in my experience, it’s only been good at repeating things, and failing on unexpected inputs - it’s able to answer pretty accurately if a small number is even or odd, but not if it’s a large number, which indicates it’s not reasoning but parroting answers to me

                    do you have example prompts where it showed clear logical reasoning?

    • kbin_space_program@kbin.run
      303·
      6 months ago

      Google search isnt a hallucination now though.

      It instead proves that LLMs just reproduce from the model they are supplied with. For example, the “glue on pizza” comment is from a reddit user called FuckSmith roughly 11 years ago.

      • DarkThoughts@fedia.io
        192·
        6 months ago

        It instead proves that LLMs just reproduce from the model they are supplied with.

        What do you mean by that? This isn’t some secret but literally how LLMs work. lol What people mean by hallucinating is when LLMs “create” facts that aren’t any. Be it this genius recipe of glue pizza, or any other wild combination of its model’s source material. The whole cooking thing is a great analogy actually because it’s like all of their fed information are the ingredients, and it just spits out various recipes based on those ingredients, without any guarantee that it is actually edible.

        • kbin_space_program@kbin.run
          101·
          6 months ago

          There are a lot of people, including google itself, claiming that this behaviour is an isolated and basically blamed users for trolling them.

          https://www.bbc.com/news/articles/cd11gzejgz4o

          I was working on the concept of “hallucinations” being things returned that are unrelated to the input query, not directly part of the model as with the glue-pizza.

            • kbin_space_program@kbin.run
              4·
              6 months ago

              A Google spokesperson told the BBC they were “isolated examples”.

              Some of the answers appeared to be based on Reddit comments or articles written by satirical site, The Onion.

              But Google insisted the feature was generally working well.

              “The examples we’ve seen are generally very uncommon queries, and aren’t representative of most people’s experiences,” it said in a statement.

              It said it had taken action where “policy violations” were identified and was using them to refine its systems.

              That’s precisely what they are saying.

              • DarkThoughts@fedia.io
                13·
                6 months ago

                I’m sorry but reading this as “Google blames users for trolling them” is either pure mental gymnastics or mental illness.

        • richieadler@lemmy.myserv.oneEnglish
          1·
          6 months ago

          This isn’t some secret but literally how LLMs work. lol

          Yeah, but John Q. Public reads AI and thinks HAL 9000 and Skynet, and no additional will convince them otherwise.

      • Billiam@lemmy.worldEnglish
        5·
        6 months ago

        Without knowing what specific comment it was, I’m going to guess it was on how advertisers make pizza look better in ads than real life?

    • Delphia@lemm.eeEnglish
      7·
      6 months ago

      I want an AI/LLM that has been trained exclusively on the technical documentation and a haynes manual for a make and model of car.

      “Hey AI, how do I change the fuel filter and what tools will I need?”

    • atrielienz@lemmy.worldEnglish
      4·
      6 months ago

      I don’t even think hallucinations is the right word for this. It’s got a source. It is giving you information from that source. The problem is it’s treating the words at that source as completely factual despite the fact that they are not. Hallucinations from what I’ve read actually is more like when it queries it’s data set, can’t find an answer, and then generates nonsense in order to provide an answer it doesn’t have. Don’t think that’s the same thing.

      • Balder@lemmy.worldEnglish
        3·
        6 months ago

        I don’t even think it’s correct to say it’s querying anything, in the sense of a database. An LLM predicts the next token with no regard for the truth (there’s no sense of factual truth during training to penalize it, since that’s a very hard thing to measure).

        Keep in mind that the same characteristic that allows it to learn the language also allows it to sort of come up with facts, it’s just a statistical distribution based on the whole context, which needs a bit randomness so it can be “creative.” So the ability to come up with facts isn’t something LLMs were designed to do, it’s just something we noticed that happens as it learns the language.

        So it learned from a specific dataset, but the measure of whether it will learn any information depends on how well represented it is in that dataset. Information that appears repeatedly in the web is quite easy for it to answer as it was reinforced during training. Information that doesn’t show up much is just not gonna be learned consistently.[1]

        [1] https://youtu.be/dDUC-LqVrPU

        • atrielienz@lemmy.worldEnglish
          1·
          6 months ago

          I understand the gist but I don’t mean that it’s actively like looking up facts. I mean that it is using bad information to give a result (as in the information it was trained on says 1+1 =5 and so it is giving that result because that’s what the training data had as a result. The hallucinations as they are called by the people studying them aren’t that. They are when the training data doesn’t have an answer for 1+1 so then the LLM can’t do math to say that the next likely word is 2. So it doesn’t have a result at all but it is programmed to give a result so it gives nonsense.

          • Balder@lemmy.worldEnglish
            2·
            6 months ago

            Yeah, I think the problem is really that language is ambiguous and the LLMs can get confused about certain features of it.

            For example, I often ask different models when was the Go programming language created just to compare them. Some say 2007 most of the time and some say 2009 — which isn’t all that wrong, as 2009 is when it was officially announced.

            This gives me a hint that LLMs can mix up things that are “close enough” to the concept we’re looking for.