🧠 LLMs and Multi-Hop Queries: A Latent Reasoning Analysis cover art

🧠 LLMs and Multi-Hop Queries: A Latent Reasoning Analysis

🧠 LLMs and Multi-Hop Queries: A Latent Reasoning Analysis

Listen for free

View show details

About this listen

One study investigates whether a chess-playing neural network, Leela, learns look-ahead capabilities, finding evidence that it internally represents and uses future moves in its decision-making, employing mechanisms like activation patching and attention analysis to support this. Another paper explores the limitations of large language models (LLMs) in multi-hop reasoning, hypothesising a sequential "knowledge-extraction module" and using a technique called Patchscopes to locate these processes within the network, alongside back-patching experiments to further understand the model's reasoning pathway. Finally, the third source examines if LLMs share representations of grammatical concepts across diverse languages by training sparse autoencoders and using causal interventions, revealing that abstract grammatical concepts are often encoded in shared feature directions, suggesting a degree of language-independent understanding.

No reviews yet