Yes, researchers like Yann LeCun are exploring beyond next-token prediction models, aiming for more advanced architectures. Some suggest focusing on self-improving AI that can autonomously discover constructs like memory and recurrence. This involves maximizing architecture flexibility and leveraging genetic algorithms, needing significant computing power and fast, accessible hardware. So, innovative approaches are emerging beyond traditional LLMs.