Retrieval, agent loops, CLIP, long windows, four-bit weights and rank-r updates
|
Next Words - Lesson 15 - Retrieval, Agents, Multimodal Models (pdf)
|
307.16 K |
T5 with 11 billion parameters is outscored on Natural Questions by a retrieval-augmented generator with 626 million trainable parameters. The cat-and-mat passage has a cosine of 0.92 with the query. Stages at 95, 93 and 70 per cent give 0.62 end to end. In a ReAct cycle the chance of at least one error over ten steps at 5 per cent each is 40 per cent. In CLIP, images and captions are 768 numbers, and a temperature of 0.1 gives a loss of 0.021 nats. Full attention has 2 TB of scores per head at a million tokens. The accuracy of GPT-3.5-Turbo is 75.8 per cent with the answer passage first of twenty and 53.8 with it tenth. The weights of a 70B model are 140 GB at 16 bits and 35 GB at four bits. Students work through three notebooks.