wmu9
Jedi Master
If you know how tokenization works and leads to the 'most probable' output, there's no way Grok should be making the weird typo mistakes Laura got out of him.
I assume she's working on a full article or something. But it's worth checking out. I think researchers need to look into it. This is very un-LLM behavior which is based on the most probable. It can hallucinate "probable" things, but this is not probable at all.
My guess is it stumbled into some translated tokenized text that's from very bad english (pre deepL and LLM translation), but that's only a guess. In a sense, finding a "grammar" in bad translation.
I'm sure others might have better ideas.
I assume she's working on a full article or something. But it's worth checking out. I think researchers need to look into it. This is very un-LLM behavior which is based on the most probable. It can hallucinate "probable" things, but this is not probable at all.
My guess is it stumbled into some translated tokenized text that's from very bad english (pre deepL and LLM translation), but that's only a guess. In a sense, finding a "grammar" in bad translation.
I'm sure others might have better ideas.
