AI took giant strides in 2024, as AGI comes into view

As a seasoned AI researcher who has witnessed the evolution of artificial intelligence since its inception, I can confidently say that 2024 was indeed a pivotal year in the field. The advancements made in various subfields of AI were truly astounding and it’s fascinating to see how close we are getting to achieving human-like intelligence (AGI).

However, as someone who has been involved in this industry for decades, I can also attest to the fact that each new breakthrough comes with its own set of challenges. One such challenge is the issue of training data, which is crucial for developing large language models (LLMs). The increasing scarcity of usable data and the legal implications surrounding its collection are concerns that cannot be ignored.

Why don’t we let cats use computers? Because they can’t type or surf the web—they just sit and watch the “fish” swim by! Jokes aside, it’s essential that as AI developers, we remain vigilant in our pursuit of knowledge while ensuring that our creations are safe, ethical, and beneficial to humanity.

2024 saw significant strides for artificial intelligence, as it not only made headlines but also garnered recognition, attracted substantial investment, impressed financial markets, and demonstrated its ability to solve mathematical problems – including the explanation of differential equations.

Additionally, it caught the notice of international watchdogs, who were anxious about potential privacy and safety issues. Some also fretted over the possibility that AI could rapidly advance to artificial general intelligence (AGI) and then artificial superintelligence, surpassing human cognitive capabilities. Various dire scenarios were contemplated and debated: the use of AI in bioterrorism, autonomous weapon systems, and even events that could potentially lead to extinction-level outcomes.

Here are 10 of 2024’s AI highlights.

#1 GenAI dominates

Artificial Intelligence that generates content, known as Generative AI or GenAI, doesn’t actually create something out of thin air, but rather, it produces new content based on the vast amount of data it has been trained on. If you give it a starting point, like a line of text, it can develop a 500-word ghost story for you.

2024 saw GenAI stepping into the limelight, and it wasn’t merely ChatGPT from OpenAI that was involved. Google’s Gemini, Microsoft’s Copilot, Anthropic’s Claude, and Meta’s Llama 3 series were also part of this advancement, creating software capable of processing and generating not only text but also audio, video, and images.

AI research facilities significantly increased their investments to drive these breakthroughs. Investments in AI soared to a staggering $13.8 billion in 2024, representing over six times the investment made in 2023, as reported by Menlo Ventures. This substantial growth underscores a definitive trend where businesses are transitioning from exploratory phases to practical implementation, integrating AI deeply into their long-term strategies.

#2 AI captures Nobel prizes for physics, chemistry

The announcement of the 2024 Nobel Prizes by the Royal Swedish Academy of Sciences in October serves as more proof that Artificial Intelligence (AI) is not just a passing trend, but a growing part of our future. Geoffrey Hinton and John Hopfield were awarded the physics prize for their groundbreaking work on machine learning with artificial neural networks, which forms a fundamental basis for today’s AI technology.

George Hinton, a British-Canadian scientist specializing in computer science and psychology, has frequently been referred to as the “Father of AI.” His pioneering work on neural networks can be traced back to the 1980s when he applied concepts from statistical physics such as Boltzmann machines to accelerate machine learning.

In another recognition, Demis Hassabis, the co-founder and CEO of Google DeepMind, along with John Jumper, received the Nobel Prize in Chemistry. This was due to their groundbreaking work on creating an AI model capable of forecasting the intricate structures of proteins.

#3 Nvidia overtakes Apple as world’s most valuable company

In 2024, it was the advanced computer chips, particularly Graphics Processing Units (GPUs) from Nvidia, that were crucial for developing and operating the large language models (LLMs) that stood out. Notably, Nvidia manufactured more of these specialized GPUs than any other company globally.

It’s not unexpected that by 2024, Nvidia had become the world’s most valuable company, boasting a market capitalization of $3.53 trillion in late October, surpassing Apple’s value of $3.52 trillion.

As a seasoned researcher with years of observation and analysis under my belt, I have noticed a significant shift in corporate practices: more companies are adopting artificial intelligence (AI) as part of their daily operations. This trend is not surprising given AI’s potential to streamline processes, enhance efficiency, and provide valuable insights that drive growth.

Amidst this surge, the demand for Nvidia chips remains robust. I have witnessed firsthand how these powerful processors enable AI applications to function optimally, handling complex computations with remarkable speed and precision. As a result, companies are eager to invest in Nvidia chips to stay competitive in the rapidly evolving technological landscape.

Russ Mould, investment director at AJ Bell, echoes my observation, underscoring the growing importance of these advanced technologies in today’s business world. The future looks bright for AI and Nvidia, as they continue to shape our economy and reshape industries worldwide.

Is there a chance that Nvidia will continue its strong position as the leading GPU manufacturer up until 2025 and beyond, considering their upcoming Blackwell GPUs have faced delays due to reported design issues? Despite these setbacks, many believe that Nvidia’s substantial market control—holding approximately 98% of the market in 2023—will make it difficult for competitors to challenge them anytime soon.

#4 AI legislation in the EU

It’s desirable for everyone to have an Artificial Intelligence (AI) that is safe, secure, and beneficial for the greater good, but regulating it responsibly isn’t a simple task. However, by 2024, international governing bodies began taking initial steps towards this challenge.

The European Union’s Artificial Intelligence Act became effective in August, establishing protective measures for AI systems designed for general use and addressing certain privacy issues. This act imposes stringent guidelines on the application of AI in facial recognition technology, among other uses, but it also aims to address broader concerns such as job automation, online misinformation dissemination, and threats to national security. The implementation of this law will occur gradually, spanning until 2027.

Despite expectations, controlling AI will prove challenging, as demonstrated in 2024 when California’s proposed SB 1047 legislation was vetoed by the state’s governor in September. This bill, hailed as the “broadest attempt to regulate artificial intelligence” up until that point, garnered support from AI advocates like Geoffrey Hinton and Elon Musk, who believed it offered necessary guidelines for this fast-developing technology.

However, the approach was met with disapproval from fellow tech experts, such as Andrew Ng, the founder of DeepLearning.AI, due to its imposition of liability on AI creators. This could potentially inhibit future advancements in the field by discouraging innovation.

#5 Emergence of small language models (SLMs)

By 2024, it had become standard to use extremely vast AI models that were educated using billions of pieces of data. For example, ChatGPT was trained on 570 gigabytes of text information gathered from the web – roughly equivalent to 300 billion words.

For numerous businesses, the future of AI is found in more compact, sector-focused language models, with some starting to appear as early as 2024.

In the month of April, Microsoft introduced its Phi-3 series of small language models, whereas Apple unveiled eight such models for their portable devices. Currently, Microsoft and Khan Academy are leveraging these Small Language Models (SLMs) to enhance mathematics tutoring for students, as an illustration.

As a technology enthusiast with years of experience in the field, I can attest to the growing trend of making models smaller for specific workloads, which has significantly increased the computational power available at the edge. This shift towards edge computing is particularly exciting because it allows us to take full advantage of this additional power, especially when considering the potential applications in various industries.

In my professional life, I’ve had the opportunity to collaborate with some incredible minds that are pioneering edge computing solutions, and I can confidently say that this development has immense potential for transforming the way we approach data processing and analysis. By bringing computational power closer to the source of the data, we can achieve near real-time insights, reduce latency, and improve overall efficiency – all essential components in today’s fast-paced digital world.

In my opinion, the future of edge computing will be a game-changer for businesses and individuals alike, offering new opportunities for innovation and growth. It’s an exciting time to be part of this evolution and I look forward to seeing how it continues to develop in the coming years.

Small Language Models (SLMs) demand fewer amounts of training data and computing resources for their creation and execution, and they’re quickly closing in on the abilities of larger language models, it was noted.

#6 Agentic AI moved to the forefront

Chatbots such as ChatGPT specialize in answering queries across a vast array of subjects. They’re not just limited to that; they can also compose computer programs, draft emails, produce reports, and even create poems!

Instead of just conversing like chatbots, AI agents take things a step further by making decisions on behalf of users, helping them reach particular objectives. For example, in the healthcare sector, an AI agent could be employed to keep track of patient data and suggest adjustments to treatment plans when needed.

As we move forward, Gartner has identified Agentic AI as one of its key strategic technology trends for the year 2025. Interestingly, by 2028, it is estimated that a third of enterprise software applications will incorporate Agentic AI, a significant increase from less than 1% in 2024.

AI agents may one day be employed to draft blockchain-based smart contracts, using a more intuitive and accessible approach than currently available. Pioneering blockchain platform, Avalanche, is developing a novel virtual machine where AI and blockchains converge, aiming to enable users to compose their smart contract programs in natural languages like English, German, French, Tagalog, Chinese, or any language they learned from their mothers. As stated by Ava Labs founder Emin Gün Sirer, “You can write your [smart contract] programs in the very language your mother taught you.

In simpler terms, Sirer foresees that an intuitive AI tool for smart contract programming might attract “tremendous numbers” or even “billions” of new individuals to the realm of blockchain technology.

#7 Reasoning models for solving ‘hard problems’

As an analyst, I’ve encountered instances where chatbots fall short. For one, they often find it challenging to solve basic mathematical problems or write software code. Additionally, they aren’t particularly adept at providing answers to scientific queries.

In September, OpenAI introduced OpenAI o1, a series of advanced problem-solving models designed to tackle complex issues such as differential equations. This move was generally well-received.

Ultimately, an artificial intelligence model that can manage all the intricate scientific, coding, and mathematical challenges I keep presenting to it was shared by New York Times columnist Kevin Roose on Twitter.

In various exams, student o1 demonstrated comparable skills to the top 500 students in the U.S. who qualified for the USA Math Olympiad. Moreover, they surpassed the precision expected from human PhD candidates on a standard test of physics, biology, and chemistry problems, as reported by OpenAI.

#8 Zeroing in on AGI

Advances in structured problem-solving, as we’ve discussed, are significant because they gradually move Artificial Intelligence (AI) towards mimicking human-like intelligence, also known as Artificial General Intelligence (AGI). This means AI would not only solve specific tasks but could also understand and handle a wide range of intellectual tasks like humans do.

At the end of last year, OpenAI’s o3 models demonstrated superior performance compared to o1, particularly when it came to math and coding examinations. Meanwhile, other initiatives like Google’s Gemini 2.0 also showed advancements in 2024 for solving structured problems, which involves breaking down intricate tasks into smaller, manageable parts.

Nonetheless, achieving Artificial General Intelligence (AGI) remains a future aspiration for numerous specialists. Today’s sophisticated models fall short in terms of having an intuitive grasp of fundamental physical principles such as gravity and causality. Moreover, existing AI systems are unable to formulate questions spontaneously or adapt their learning when faced with unforeseen circumstances.

Essentially, Brian Hopkins, Vice President of Emerging Technology at Forrester, stated that Artificial General Intelligence (AGI) is more about a continuous journey rather than reaching an end point, implying we’ve just started this exciting adventure.

# 9 Signs of a looming training data shortage

2024 proved to be a thrilling year for AI creators and enthusiasts, with many anticipating that AI advancements will continue at a rapid pace. However, some discussions in 2024 hinted that the AI’s Language Learning Model (LLM) sub-era might have already reached its peak.

The issue at hand is an impending scarcity of data. Corporations such as OpenAI and Google could potentially exhaust their data resources, which are essential for the nourishment and development of large-scale artificial intelligence systems.

It’s important to note that not all data can be scraped from the internet, and developers of language models have found out that they can’t always collect publicly available data without consequences. For instance, The New York Times has taken legal action against OpenAI for alleged copyright infringement related to their news content. It’s possible that other significant media organizations might also seek legal remedies in similar situations.

“Everyone in the industry is seeing diminishing returns,” said Google’s Demis Hassabis.

An alternative approach could involve teaching algorithms using simulated data – this is data produced artificially that closely resembles authentic real-world data. For example, AI developer Anthropic’s Claude 3 LLM was educated, at least partially, on synthetic data, which they describe as “data we create internally.

While the phrase “synthetic data” might seem contradictory at first glance, researchers, including some in the medical field, assert that generating data artificially offers potential benefits. It could bolster the performance of AI in healthcare by supplementing sparse data sets, thereby addressing biases towards specific ethnic groups as an example.

#10 Emergence of a more ethical AI

It’s worth mentioning that Anthropic provides a thorough explanation in the referenced paper on how they gather their training data. Notably, their website crawling system operates openly, allowing content providers such as The New York Times to effortlessly recognize Anthropic visits. These providers can also communicate their preferences to Anthropic by signaling them directly.

The business has taken significant steps to ensure its technology isn’t misused, including appointing a Responsible Technology Officer whose role expanded in 2024 with the aim of developing “safe” AI. This commitment didn’t go unrecognized; Time magazine honored it as one of the 100 most impactful companies in 2024, praising it for its approach that safety could be a successful business strategy, often referred to as the “AI Company Staking on Safety.

2025-01-01 02:04