The origins of deep learning (DL), a cornerstone of modern artificial intelligence (AI), can be traced back to the 1940s. This era marked the proposal of the artificial neuron by Warren McCulloch and Walter Pitts—a mathematical construct inspired by the biological neurons in the brain. The perceptron, developed in the 1950s, marked an early milestone, followed by the introduction of backpropagation in the 1980s by Rumelhart, Hinton, and Williams. This algorithm was pivotal in training multi-layered neural networks more efficiently, paving the way for deeper architectures. During the 1980s, Yann LeCun's research on convolutional neural networks (CNNs) proved critical for tasks in image recognition. The 1990s saw the development of recurrent neural networks (RNNs) by Sepp Hochreiter and Jurgen Schmidhuber, which enhanced the model's ability to process sequential data.

Despite these theoretical advancements, progress was impeded in the 1990s and early 2000s due to computational limitations and the scarcity of large datasets. The advent of powerful graphics processing units (GPUs) in the late 2000s, however, provided the necessary boost for training complex deep learning models. The 2010s were characterized by a deep learning explosion, with CNNs achieving remarkable performance in image recognition tasks, such as the ImageNet competition, and RNNs driving progress in natural language processing (NLP) and speech recognition. Today, the field of deep learning is continuously evolving as researchers explore novel architectures, training methodologies, and applications in diverse domains such as healthcare, robotics, and autonomous vehicles.

Training in machine learning

Training deep learning models requires vast datasets containing millions, or even billions, of data points. This scale allows the models to recognize complex patterns and generalize effectively to new, unseen data. A common practice is to pre-train models on large, generic datasets, such as extensive image or text corpora, and then fine-tune them on specific tasks. This strategy benefits from the features learned during the pre-training phase, thereby speeding up the training process for the final task. Given the size and complexity of these models, training typically involves distributing the workload across multiple GPUs or Tensor Processing Units (TPUs), which significantly improves the speed of the process.

Training durations can vary widely—from days or weeks for smaller models to months for the largest and most complex models. Nevertheless, continual advancements in hardware and training techniques are extending the boundaries of what is possible, enabling the development of larger and more sophisticated neural networks.

Focusing on large language models (LLMs), these are trained on vast text corpora, often exceeding 1000 GB—which is roughly equivalent to 1 million 500-page books. The models used for such training have billions of parameters and require substantial infrastructure, typically involving multiple GPUs. For instance, training a model like GPT-3 would take approximately 290 years on a single NVIDIA V100 GPU. Consequently, LLMs are generally trained on thousands of GPUs in parallel. Google, for example, trained a model with 540 billion parameters using 6,144 TPU v4 chips. The immense scale and cost of this infrastructure make it prohibitive for most organizations, compelling even OpenAI to utilize Microsoft’s Azure cloud platform for training.

Training LLMs requires precise coding, careful configuration, and meticulous implementation to ensure accurate and efficient execution. The process is iterative and often involves numerous parallel computing strategies. These are adjusted after experimenting with various configurations to tailor training runs to the specific needs of the model and the available hardware. Selecting an appropriate LLM architecture, such as one with residual connections and transformer architecture with self-attention, is crucial. This choice directly impacts the training complexity and optimizes the balance between computational resources and the model's complexity. Identifying the functional needs of the model—whether for generative modeling, bi-directional/masked language modeling, multi-task learning, or multi-modal analysis—is also vital for successful implementation.

The development of artificial intelligence (AI), especially in the field of deep learning, is the result of the immense dedication and collaborative efforts of tens of thousands of researchers worldwide. Currently, Google’s DeepMind boasts approximately 3,400 employees, while OpenAI has around 1,900 staff members. In terms of financial commitment, DeepMind’s CEO Demis Hassabis revealed in 2024 that Google's investment in AI might ultimately surpass $100 billion. Furthermore, a 2022 report by Statista indicated that global corporate AI investment amounted to $91.9 billion, with Goldman Sachs projecting that it could escalate to nearly $200 billion by 2025.

The human brain as a foundation for AI

The human brain, a marvel of biological intricacy, is foundational to the development of artificial intelligence (AI) and deep learning (DL). This complex organ functions much like a central processing unit, managing a broad spectrum of critical tasks essential for survival and environmental interaction. It processes sensory inputs—such as vision, hearing, touch, taste, and smell—via the nervous system. These inputs are then synthesized by the brain to form our subjective perception of the world.

Additionally, the brain expertly orchestrates motor functions, facilitating actions like walking, speaking, and manipulating objects. It also meticulously regulates vital physiological operations, including breathing, heart rate, digestion, and body temperature, through a sophisticated coordination of neural and hormonal systems.

One of the most fascinating aspects of the human brain is its exceptional plasticity. This feature is crucial as it allows us to learn new information and retain it as long-term memories. This adaptability is fundamental in how we gather life experiences and continuously expand our understanding of the world. Through this capability, the brain supports not only our basic survival instincts but also powers the complex thought processes that machines in the realms of AI and deep learning attempt to mimic.

Emotions and feelings are also governed by the brain, heavily influencing how we think, make decisions, and interact with others. It is the brain's sophisticated system that enables us to understand and use language, whether it's spoken out loud or written down, and to formulate our thoughts into words. Furthermore, the brain manages complex mental functions such as critical thinking, problem-solving, decision-making, creativity, and self-awareness. These skills are essential for us to manage complex challenges and make plans for the future.

It's important to understand that these varied functions do not operate in isolation. Instead, they involve the cooperative and dynamic interaction of different areas of the brain. For example, playing a musical instrument beautifully illustrates this as it requires coordinated motor control, auditory feedback, and emotional engagement.

Neuroscience, the study of the brain, is continually revealing new insights about this incredible organ. We are consistently learning more about its structure, capabilities, and the ways it develops and adapts throughout our lives. It's estimated that the human brain contains about 86 billion neurons, which are specialized cells that communicate with each other using electrical and chemical signals. These neurons are linked by approximately 100 trillion synapses, which are the tiny gaps over which these signals are transmitted. This extensive network of connections allows for the complex processing of information and the learning capabilities that characterize the human brain.

Human intelligence (HI) vs artificial intelligence (AI)

The human brain, equipped with necessary hardware and software, must operate in harmony for optimal functionality. Without the software component, the brain is merely soft tissue composed of gray and white matter. Conversely, without the physical structure of the brain, no software—no matter how sophisticated—can function, demonstrating that the brain with its 86 billion neurons and 100 trillion synapses is perfectly designed to run these complex programs.

Consider the question: "Why can’t the current chess champion, Magnus Carlsen, beat a computer at chess?" This is especially intriguing given that the last known victory of a human under standard tournament conditions was Ponomariov’s win against Fritz in 2005. This question can be likened to asking why a human can't carry loads like a truck, achieve speeds like a Ferrari, calculate the cube root of large prime numbers as quickly as a computer, or why even the combined efforts of math professors from top research universities can't manually compute the inverse of a 10,000 x 10,000 matrix as swiftly as Python on a capable CPU. The answer is simple: the human brain is not designed for these specific functions. Machines may excel in specialized tasks but lack the broader adaptive capabilities of humans. A truck cannot think, a Ferrari cannot cultivate civilization, and a computer does not manage a living body. Similarly, software capable of rapid calculations cannot replicate the nuanced teaching and innovative theoretical work of university professors.

Evidently, the human brain is intricately tailored to navigate the complexities of life. It equips individuals with the ability to make critical decisions, from ethical dilemmas to everyday choices such as providing for one's family, achieving societal status, or engaging in personal relationships like marriage. Despite the remarkable advancements in artificial intelligence, its limitations are evident in practical applications, such as autonomous vehicles. Major automotive manufacturers have invested heavily in this technology: Ford invested $1 billion in the self-driving startup Argo AI in 2017, with Volkswagen later contributing an additional $2.6 billion, aiming to compete with tech giants like Uber, Tesla, and Google. At its peak, Argo AI was valued at $12.4 billion. However, by 2022, the venture was discontinued. Ford acknowledged that it failed to attract further investment and that profitability was still far from reach, highlighting the challenges in aligning AI capabilities with complex real-world applications.

The fact of creation in intelligence

From early childhood, humans embark on an extensive and continuous learning journey, adapting to and mastering the complexities of the surrounding environment. While artificial intelligence (AI) excels at specific tasks like chess and Go, it still falls short of the extraordinary capabilities demonstrated in nature, such as the peregrine falcon’s high-speed dive or the human body’s intricate coordination of 30 trillion cells every millisecond.

Even if AI can be engineered to mimic many daily functions of living beings, this comparison starkly underscores the superior capabilities inherent in life itself, pointing unmistakably to the power revealed in creation.

Our individual brains, alongside the broader universe, serve as unparalleled platforms for continuous learning. Yet, our understanding of the brain's basic features remains rudimentary, and the depth of its operations is still largely uncharted. Current technological applications, such as large language models for language learning and neural networks for image processing, demonstrate our attempts to replicate natural processes. However, these artificial systems require significant resources for tasks that the human brain handles effortlessly and cost-free.

Reflecting on these distinctions, it becomes clear that the intricate intelligence and processes governing life are not the products of an unconscious entity like "mother nature." Instead, they suggest the design of a deliberate and intelligent Creator. This realization invites us to acknowledge the intricate purpose embedded within the human brain and body, pointing us toward the Creator, whose magnificent capabilities and actions are evident in the very fabric of our existence.