Attention Quiz (Hard)

Quiz Summary

0 of 8 Questions completed

Questions:

You have already completed the quiz before. Hence you can not start it again.

Quiz is loading…

You must sign in or sign up to start the quiz.

You must first complete the following:

Results

Quiz complete. Results are being recorded.

Results

0 of 8 Questions answered correctly

Your time:

You have reached 0 of 0 point(s), (0)

maximum of 8 points
Pos.	Name	Entered on	Points	Result
Table is loading
No data available

Current
Review
Answered
Correct
Incorrect

Question 1 of 8
1. Question
Which type of attention mechanism is often used in sequence-to-sequence tasks to align the elements of the source and target sequences during translation?
- Self-attention
- Cross-attention
- Multi-head attention
- Spatial attention
Correct
Incorrect
Question 2 of 8
2. Question
In the context of the Transformer model, which component uses multi-head self-attention?
- The encoder
- The decoder
- Both the encoder and decoder
- Neither the encoder nor decoder
Correct
Incorrect
Question 3 of 8
3. Question
Which mathematical operation is performed during the computation of attention scores in a typical attention mechanism?
- Matrix multiplication
- Convolution
- Element-wise division
- Principal Component Analysis (PCA)
Correct
Incorrect
Question 4 of 8
4. Question
What is the primary challenge of using self-attention mechanisms in very long sequences?
- Quadratic increase in computational complexity
- The difficulty of training deep networks; this is addressed by using shallower networks
- The risk of overfitting; this is resolved by applying dropout to the attention weights
- The risk of numerical instability; this is addressed by using fixed-length sequences
Correct
Incorrect
Question 5 of 8
5. Question
In multi-head attention, how are the attention weights typically combined from different heads?
- By taking the average of all the attention weights
- By concatenating the attention weights from different heads
- By selecting the attention weights from the head with the highest weight
- By multiplying the attention weights from different heads
Correct
Incorrect
Question 6 of 8
6. Question
How are attention weights typically calculated in an attention mechanism?
- By performing a weighted average of the key vectors
- By concatenating the query and key vectors and then applying a sigmoid activation.
- It is computed as the dot product of the query and key vectors, divided by the square root of the dimension of the key vectors
- By using a fixed set of predefined attention weights
Correct
Incorrect
Question 7 of 8
7. Question
In the context of self-attention mechanisms, what is the purpose of the query, key, and value vectors?
- Query vectors represent the input sequence, key vectors contain positional information, and value vectors store the final predictions
- Query vectors capture the global context, key vectors determine the importance of specific elements, and value vectors store the intermediate feature representations
- Query vectors determine the output sequence, key vectors represent the previous hidden states, and value vectors help with gradient flow
- Query vectors store the attention weights, key vectors contain the input embeddings, and value vectors are learned during training
Correct
Incorrect
Question 8 of 8
8. Question
In a multi-head attention mechanism, what do different attention heads learn?
- Different attention heads learn different aspects of the same data, improving the model's robustness
- Each head is responsible for attending to different positions in the input sequence, improving alignment
- Attention heads focus on different modalities, such as text and images, allowing for cross-modal learning
- All attention heads learn the same attention distribution, but with different parameterizations
Correct
Incorrect

AIML.com

Machine Learning Resources

Attention Quiz (Hard)

Quiz Summary

Information

Results

Results

Categories

1. Question

2. Question

3. Question

4. Question

5. Question

6. Question

7. Question

8. Question

Transformers Quiz (Easy)

Transformers Quiz (Hard)

Attention Quiz (Easy)