Last week, the Wall Street Journal published a 10-minute-long interview with OpenAI CTO Mira Murati, with journalist Joanna Stern asking a series of thoughtful yet straightforward questions that Murati failed to satisfactorily answer. When asked about what data was used to train Sora, OpenAI's app for generating video with AI,
    • CeeBee@lemmy.worldEnglish
      278·
      9 months ago

      LLMs as AI is just a marketing term. there’s nothing “intelligent” about “AI”

      Yes there is. You just mean it doesn’t have “high” intelligence. Or maybe you mean to say that there’s nothing sentient or sapient about LLMs.

      Some aspects of intelligence are:

      • Planning
      • Creativity
      • Use of tools
      • Problem solving
      • Pattern recognition
      • Analysts

      LLMs definitely hit basically all of these points.

      Most people have been told that LLMs “simply” provide a result by predicting the next word that’s most likely to come next, but this is a completely reductionist explaining and isn’t the whole picture.

      Edit: yes I did leave out things like “understanding”, "abstract thinking ", and “innovation”.

      • SkybreakerEngineer@lemmy.worldEnglish
        107·
        9 months ago

        Other than maybe pattern recognition, they literally have no mechanism to do any of those things. People say that it recursively spits out the next word, because that is literally how it works on a coding level. It’s called an LLM for a reason.

        • CeeBee@lemmy.worldEnglish
          91·
          9 months ago

          they literally have no mechanism to do any of those things.

          What mechanism does it have for pattern recognition?

          that is literally how it works on a coding level.

          Neural networks aren’t “coded”.

          It’s called an LLM for a reason.

          That doesn’t mean what you think it does. Another word for language is communication. So you could just as easily call it a Large Communication Model.

          Neural networks have hundreds of thousands (at the minimum) of interconnected layers neurons. Llama-2 has 76 billion parameters. The newly released Grok has over 300 billion. And though we don’t have official numbers, ChatGPT 4 is said to be close to a trillion.

          The interesting thing is that when you have neural networks of such a size and you feed large amounts of data into it, emergent properties start to show up. More than just “predicting the next word”, it starts to develop a relational understanding of certain words that you wouldn’t expect. It’s been shown that LLMs understand things like Miami and Houston are closer together than New York and Paris.

          Those kinds of things aren’t programmed, they are emergent from the dataset.

          As for things like creativity, they are absolutely creative. I have asked seemingly impossible questions (like a Harlequin story about the Terminator and Rambo) and the stuff it came up with was actually astounding.

          They regularly use tools. Lang Chain is a thing. There’s a new LLM called Devin that can program, look up docs online, and use a command line terminal. That’s using a tool.

          That also ties in with problem solving. Problem solving is actually one of the benchmarks that researchers use to evaluate LLMs. So they do problem solving.

          To problem solve requires the ability to do analysis. So that check mark is ticked off too.

          Just about anything that’s a neutral network can be called an AI, because the total is usually greater than the sum of its parts.

          Edit: I wrote interconnected layers when I meant neurons

    • FaceDeer@fedia.io
      91·
      9 months ago

      The term “artificial intelligence” was established in 1956 and applies to a broad range of algorithms. You may be thinking of Artificial General Intelligence, AGI, which is the more specific “thinks like we do” sort that you see in science fiction a lot. Nobody is marketing LLMs as AGI.

      • FaceDeer@fedia.io
        172·
        9 months ago

        It’s some weird semantic nitpickery that suddenly became popular for reasons that baffle me. “AI” has been used in videogames for decades and nobody has come out of the woodwork to “um, actually” it until now. I get that people are frightened of AI and would like to minimize it but this is a strange way to do it.

        At least “stochastic parrot” sounded kind of amusing.

        • XTLEnglish
          31·
          9 months ago

          Um, actually clueless people have made “that’s not real AI” and “but computers will never …” complaints about AI as long as it has existed as a computing science topic. (50 years?)

          Chatbots and image generators being in the headlines has made a new loud wave of complainers, but they’ve always been around.

          • FaceDeer@fedia.io
            21·
            9 months ago

            It’s exactly that “new loud wave of complainers” I’m talking about.

            I’ve been in computing and specifically game programming for a long time now, almost two decades, and I can’t recall ever having someone barge in on a discussion of game AI with “that’s not actually AI because it’s not as smart as a human!” If someone privately thought that they at least had the sense not to disrupt a conversation with an irrelevant semantic nitpick that wasn’t going to contribute anything.