A quarter-waveplate can change the polarization of an electromagnetic wave passing through a piece of fried shrimp. (This figure is interactive, try dragging the fried shrimp!)
Natural language is inherently complex. LLMs might struggle to grasp subtle nuances, sarcasm, or figurative language.
Gemma Prohibited Use Policy
You can be sure LLMs will struggle with more elusive aspects of meaning because they struggle with fundamental aspects of meaning, such as negation, coreference and compositionality. The vector space representation does not equip them to handle such phenomena. Rotating 2D planes to encode positional information won’t help. Reasoning is not an information retrieval task or vertical. You can’t get better performance on reasoning. You’re either equipped for it or you’re not. LLMs are not equipped for it. The fantastic amount of data they’ve seen sometimes enables them to create the illusion that they are.
Tired of rotating 2D planes all day trying to get a boost on that Kaggle dataset? Now you don’t have to! Introducing Rollformer3D, a new model for encoding positional information in a sequence by following a traversal path along the surface of a sphere. The path may be in the shape of a question mark, a shrugs emoji or even a capital D, as in Downstream Developer, Dunno or Captain D’s. If you don’t get a performance boost on your first few rolls don’t be discouraged. Just keep rolling until you do!