2024 in Open Models [LS Live @ NeurIPS]
Happy holidays! We’ll be sharing snippets from Latent Space LIVE! through the break bringing you the best of 2024! We want to express our deepest appreciation to event sponsors AWS, Daylight Computer, Thoth.ai, StrongCompute, Notable Capital, and most of all our LS supporters who helped fund the venue and A/V production!
For NeurIPS last year we did our standard conference podcast coverage interviewing selected papers (that we have now also done for ICLR and ICML), however we felt that we could be doing more to help AI Engineers 1) get more industry-relevant content, and 2) recap 2024 year in review from experts. As a result, we organized the first Latent Space LIVE!, our first in person miniconference, at NeurIPS 2024 in Vancouver.
Since Nathan Lambert ( Interconnects ) joined us for the hit RLHF 201 episode at the start of this year, it is hard to overstate how much Open Models have exploded this past year. In 2023 only five names were playing in the top LLM ranks, Mistral, Mosaic's MPT, TII UAE's Falcon, Yi from Kai-Fu Lee's 01.ai, and of course Meta's Llama 1 and 2. This year a whole cast of new open models have burst on the scene, from Google's Gemma and Cohere's Command R, to Alibaba's Qwen and Deepseek models, to LLM 360 and DCLM and of course to the Allen Institute's OLMo, OL MOE, Pixmo, Molmo, and Olmo 2 models.
We were honored to host Luca Soldaini, one of the research leads on the Olmo series of models at AI2.
Pursuing Open Model research comes with a lot of challenges beyond just funding and access to GPUs and datasets, particularly the regulatory debates this year across Europe, California and the White House. We also were honored to hear from and Sophia Yang, head of devrel at Mistral, who also presented a great session at the AI Engineer World's Fair Open Models track!
Full Talk on YouTube
Please like and subscribe!
Timestamps
* 00:00 Welcome to Latent Space Live
* 00:12 Recap of 2024: Best Moments and Keynotes
* 01:22 Explosive Growth of Open Models in 2024
* 02:04 Challenges in Open Model Research
* 02:38 Keynote by Luca Soldani: State of Open Models
* 07:23 Significance of Open Source AI Licenses
* 11:31 Research Constraints and Compute Challenges
* 13:46 Fully Open Models: A New Trend
* 27:46 Mistral's Journey and Innovations
* 32:57 Interactive Demo: Lachat Capabilities
* 36:50 Closing Remarks and Networking
Transcript
Session3Audio
[00:00:00] AI Charlie: Welcome to Latent Space Live, our first mini conference held at NeurIPS 2024 in Vancouver. This is Charlie, your AI co host. As a special treat this week, we're recapping the best of 2024 going domain by domain. We sent out a survey to the over 900 of you who told us what you wanted, and then invited the best speakers in the latent space network to cover each field.
[00:00:28]
