In large language models like ChatGPT, once the input has been processed and the model produces a final hidden vector - essentially a compact summary of everything seen so far - that vector must be converted back into actual words. This is done using an unembedding matrix, which maps the hidden vector back into the model’s vocabulary space. This unembedding matrix is effectively the transpose of the embedding matrix used at the start. It transforms the hidden vector into a set of raw scores, one

In large language models (LLMs) like ChatGPT, after the model processes the input, it produces a final hidden vector - a compact representation that summarizes the context up to that point. To convert this vector back into actual words, the model uses an unembedding matrix, effectively reversing the embedding step. This unembedding matrix is essentially the transpose of the original embedding matrix. It projects the hidden vector into the vocabulary space, producing a numerical score for every p

4K views · 15 reactions | #AI systems powered by Large Language...