Transformers Quiz (Easy)

What is the primary innovation that Transformers brought to deep learning models?

In the context of machine translation using Transformers, what is typically the role of the encoder and decoder?

What is the key advantage of the position encodings in the Transformer model?

In the Transformer architecture, where does the decoder’s cross-attention mechanism typically attend to?

How are multi-head attention mechanisms utilized in Transformers?

What is the primary advantage of using the Transformer architecture over traditional RNNs for sequence tasks?

What is one of the main disadvantages of transformer models for language modeling? 

Which aspect of transformer models makes them computationally expensive?