> Where tree search might make more sense applied to LLMs is for more coarser grained reasoning where the branching isn't based on alternate word continuations but on alternate what-if lines of thought...
To do that, the LLM would have to have some notion of "lines of thought". They don't. That is completely foreign to the design of LLMs.
Right - this isn't something that LLMs currently do. Adding search would be a way to add reasoning. Think of it as part of a reasoning agent - external scaffolding similar to tree of thoughts.
To do that, the LLM would have to have some notion of "lines of thought". They don't. That is completely foreign to the design of LLMs.