Either way, experts think OpenAI should be less opaque about its AI model architecture.
  • The study showed that GPT-4 has become worse at coding and compositing tasks.
  • Researchers Lingjiao Chen, Matei Zachariah, and James Zou tested versions of GPT-3.5 and GPT-4 in March and June 2023.
  • GPT-4 showed a decrease in the accuracy of identifying prime numbers from 97.6% to 2.4%.
  • GPT-3.5 showed improved performance over the same period.
  • Popular theories of GPT-4 performance degradation include model overclocking, fine-tuning, and conspiracy theories.
  • OpenAI denies claims of GPT-4 performance degradation, considering it to be smarter.
  • Professor Arvind Narayanan does not consider the results of the study to be conclusive evidence.