(except added links, this post is fully generated)
Welcome to our fourth episode of the DSB Podcast—a candid conversation that bridges the worlds of data science, AI economics, and software development. In this episode, my co-host Jakub Kramata and I dive into a spectrum of topics from the latest trends in large language models (LLMs) to emerging tools for developers, offering insights drawn from our experiences at ČSOB’s Data Science department.
Diminishing Returns in Large Language Models
One of the central themes we explored is the phenomenon of diminishing returns in LLMs—a concept popularized by Gary Marcus in his newsletter. We discussed how, after a certain point, additional investments in training these models yield progressively smaller improvements. This trend, which became apparent as OpenAI’s dominance in 2023 gave way to rising competition in 2024, suggests that while the cost of training ever-larger models continues to rise, the tangible performance gains are not scaling proportionally.
We reflected on our personal experience: transitioning from GPT-3 through GPT-3.5 to GPT-4 showed significant leaps in capability, but more recent iterations seem to offer only marginal improvements. This economic reality has led many industry leaders to adjust expectations, with CEOs from major companies openly acknowledging the current revenue challenges despite the technological hype.
Gary Marcus Newsletter: https://garymarcus.substack.com/p/evidence-that-llms-are-reaching-a https://garymarcus.substack.com/p/confirmed-llms-have-indeed-reached
The Economics of AI: Costs, Revenue, and the Quest for the Killer App
A recurring point in our discussion was the escalating cost of training foundation models. While investments are soaring, the revenue streams generated by these models—whether measured through improved performance on benchmarks or real-world business applications—remain underwhelming. We highlighted examples such as the Stargate program and discussed how generative AI, for now, functions more as a powerful tool rather than a final product that drives direct income.
The conversation took an intriguing turn towards the gaming industry, where innovators like Marek Rossa from Good Game are experimenting with integrating AI into interactive experiences. Could such applications be the long-awaited “killer app” that transforms how we perceive and use AI? Only time will tell.
Good Games “AI revolution” [Czech text]: https://cc.cz/klasicke-hry-vam-budou-pripadat-hloupe-a-omezene-spoustime-ai-revoluci-rika-sef-prazskeho-studia/
Stargate Program: https://openai.com/index/announcing-the-stargate-project/
MSFT controlling expectations: https://www.windowscentral.com/microsoft/microsoft-satya-nadella-dismisses-agi-milestones-as-nonsensical-benchmark-hacking
Beyond Text: Embracing Multimodality
Our podcast also delved into the evolution of LLM inputs. Whereas 2023 was dominated by text-only interactions, the current landscape is rapidly shifting. Today’s models are increasingly multimodal—processing images, audio, PDFs, and other data formats alongside text. We examined how companies like Anthropic and Google (with Gemini) are pushing the envelope by enabling models to interact with digital content in entirely new ways.
This multimodal capability isn’t just a technological novelty; it promises to revolutionize tasks like robotic process automation (RPA) and even everyday interactions, such as controlling devices via voice commands. Imagine a future where you can ask your AI assistant how much seasoning to add to your dish while your hands are busy cooking!
Anthropic computer use: https://www.anthropic.com/research/developing-computer-use
Multimodality: https://magazine.sebastianraschka.com/p/understanding-multimodal-llms
Multimodality chaning computer vision: https://medium.com/@tenyks_blogger/multimodal-large-language-models-mllms-transforming-computer-vision-76d3c5dd267f
Revisiting Transformer Architectures and the Rise of Modern BERT
The evolution of transformer models was another hot topic. We took a closer look at BERT—the bidirectional encoder transformer that once reigned supreme for classification and sentiment analysis. While early versions of BERT set a high bar, the market’s focus shifted towards decoder-only architectures (as seen in models like GPT-4).
Yet, innovation hasn’t stalled. We discussed the release of an updated “modern BERT” by Answer.ai, which leverages advancements such as flash attention to deliver faster, more efficient performance. Although early benchmarks suggest that this new model may not yet outpace its predecessor in every scenario, the development underscores the dynamic and competitive nature of AI research.
Modern BERT: https://huggingface.co/blog/modernbert
The Debate on Retrieval Augmented Generation and Developer Tools
Another engaging segment of the podcast was our conversation about Retrieval Augmented Generation (RAG). With larger context windows now available, some argue that traditional retrieval techniques may become obsolete. However, we maintained that retrieving relevant information remains critical—even when models can process more data—because overloading them with information can actually degrade performance.
Finally, we touched on emerging trends in the software development ecosystem. We discussed a new Package and Project Manager—referred to as “UV”—which is gaining traction as a more efficient and user-friendly alternative to legacy tools like Poetry or Mamba. With a growing community and promising early reviews, this tool might soon become a staple for managing complex projects in Python and beyond.
RAG is alive: https://medium.com/@InferenzTech/why-rag-still-matters-beyond-token-limits-in-llms-289d16a930af
Top python libraries, uv is first: https://tryolabs.com/blog/top-python-libraries-2024
Looking Ahead
As we wrapped up the episode, we hinted at our plans for future shows. Next time, we’ll focus on a “novinky” episode that zeroes in on the latest developments and innovations of the current year. For now, we invite you to catch the full discussion on YouTube and Spotify, and join us as we continue to explore how technology and economics intersect in the fast-evolving world of AI and data science.
Thank you for tuning in, and we look forward to your feedback! (reach us directly or podcast@datasciencebulletin.com)
Be First to Comment