Last week, the Wall Street Journal published a 10-minute-long interview with OpenAI CTO Mira Murati, with journalist Joanna Stern asking a series of thoughtful yet straightforward questions that Murati failed to satisfactorily answer. When asked about what data was used to train Sora, OpenAI's app for generating video with AI,
It takes what, a year minimum to design a chip? I think the iterative hardware-software cycle is just now properly getting a foothold on the architecture. The next few years are going to, at minimum, explore what lavishly-funded, purpose-built hardware can do for the field.
It’ll be years before we reach any kind of maximum. Even if the software doesn’t improve at all, which is unlikely, better utilization alone will make significant improvements on performance.