Video Transcript (English)

Welcome back! Previously, we talked about Generative AI and Natural Language Processing which are important fields in the AI domain.

In recent years, we have seen huge AI advancements in these fields due to deep learning models, which attempt to replicate human cognitive understanding.

How does the human mind operate, and how does it learn? In this video, we'll go a little deeper into deep learning models.

A full exploration of these deep learning models is beyond the scope of this lesson. However, we will briefly go over the various developments that led to breakthrough models used in tools such as BART and ChatGPT. Let's get started.

Computer scientists have long searched for a way to build artificial intelligence. Their research over the past 50 years explored the biology of the human mind through the learnings from psychologists, neuroscientists, and mathematicians.

Their research gave birth to the neural network: a computational model inspired by the structure and functional aspects of the brain.

The neural network is used to model complex relationships between inputs and outputs and to find patterns in data.

As you learned in the last video, neural networks are made up of multiple layers of nodes and their connections which are used to run different calculations.

Each node is like a neuron in our brain. They receive input through the input layer, process it using hidden layers, and then pass it on to the next layer of nodes via the output layer. This is similar to how our brain works.

An important discovery from all the research is the notion of attention.

A lot of effort focused on how the human mind is able to process large amounts of information in record time.

Their research suggests this is largely due to our ability to focus on the most important information and filter out the rest.

To mimic this ability, the connections between nodes in neural networks use training values known as weights to prioritize certain data.

The weights adjust as the network is exposed to more data and identifies patterns.

This is considered learning.

Now that you're familiar with how neural networks work, let's briefly discuss a few types of neural networks. Each has specific use cases and when combined, produce powerful results.

Feedforward neural networks are the simplest type of neural network. The connections between nodes are linear and move forward. Feedforward neural networks are widely used for tasks like classification.

On the other hand, Recurrent Neural Networks or RNNs are better suited for natural language processing.

You may be surprised but RNNs have been around since the 1980s. As computers became more powerful, so did RNNs.

Similar to a feedforward network, RNNs move forward through nodes, but RNNs have the ability to loop backwards. This allows RNNs to not only leverage the data it’s been given, but it can also use what it has learned from the previous input.

RNNs resulted in huge improvements in AI and powered popular applications such as Siri. But, they weren’t perfect. One thing that plagued RNNs was that they struggled with long input sequences.

Imagine having a conversation with someone, but you can only remember the last four words they said. It would be really difficult to have a conversation, since you lose all context. This was the issue RNNs were encountering.

Later on, we saw an improved version of RNNs using Long-Short-Term-Memory which gave RNNs the ability to deal with long input sequences.

Even with the added improvement to RNNs, they were still computationally intensive and took a long time to train.

By 2017, natural language processing and neural networks were revolutionized with the introduction of the Transformer model.

Before the transformer model, previous models focused on inputs in sequential order. As you can imagine, this can be slow, leading to longer training times. Also, sequential processing hinders capturing the long-range dependencies in the data.

This can impact the accuracy of those models when capturing the meaning and making predictions, especially when you have words with multiple meanings.

The Transformer model improved this by being able to process data in parallel resulting in much quicker training times and a solution that can be scaled on modern hardware.

The transformer model consists of encoders and decoders. Both the encoder and decoder have multiple layers inside that are used to calculate predictions.

Most of these layers consist of things that already existed like feedforward neural networks.

The encoder takes an entire input sequence at once and returns a sequence of numbers or embeddings.

The main job of the encoder is to understand the context of the given input.

At the other end is the decoder which receives the embeddings from the encoder along with the output embeddings it previously generated.

With this data, the decoder makes predictions based on the context provided by the encoder.

Unlike the encoder, the decoder processes data sequentially.

The Transformer model, much like the neural networks before it, is quite complex. This is just to give you an idea of how neural networks have constantly been improving over the decades. It makes you wonder what the future has in store for us in this exciting field.

To recap, we learned that scientists have been tirelessly chasing the prospect of mimicking the way humans process data using machines.

This has resulted in the creation of many different neural networks such as: feedforward, recurrent, and the Transformer model.

We learned that transformer models revolutionized AI by improving the accuracy and speed to train models.

We are not done with the Transformer model yet. Coming up, we’ll see how versatile it truly is.