A quarter-waveplate can change the polarization of an electromagnetic wave passing through a piece of fried shrimp. (This figure is interactive, try dragging the fried shrimp!)
As capabilities advance, we may need to explore … alternative access mechanisms.
Gemma: Open Models Based on Gemini Research and TechnologyNatural language is inherently complex. LLMs might struggle to grasp subtle nuances, sarcasm, or figurative language.
Gemma Prohibited Use Policy
You can be sure LLMs will struggle with more elusive aspects of meaning because they struggle with fundamental aspects of meaning, such as negation, compositionality and coreference. A vector space representation of language as token sequences doesn’t equip them to handle such things. Moreover, rotating 2D planes to encode positional information about the tokens will not remedy the situation. Reasoning is not an information retrieval task. It’s not a vertical. You can’t get better performance on reasoning. Either you’re equipped for it or you’re not and LLMs are not equipped for it. It’s the fantastic amount of data they’ve seen that sometimes enables them to create the illusion that they are.
The main problem with these models is that they begin with the assumption that TikTokens are suitable for representing primitives in language. The Gemma paper describes them as a successful research innovation that will enable downstream developers and the next wave of innovations. Word2vec, GloVe, WordPiece, SentencePiece, and GPT-2’s BPE, techniques developed at Google, Stanford (Google) and OpenAI (Google), all do the same thing: they enable text to be mapped into a multimodal embedding space shared, conveniently, with images. This is useful for short-form video content and Google’s knowledge panels. It’s not important that these representations make sense for language. What’s important is that they provide a competitive advantage in the advertising space to their biggest research sponsor. Remember, nothing is more important than scale and nobody can scale like Google.
Tired of rotating 2D planes all day trying to get a boost on that Kaggle dataset? Now you don’t have to! Introducing Rollformer3D, a new model for encoding positional information in a sequence by following a traversal path along the surface of a sphere. The path may be in the shape of a question mark, a shrugs emoji or even a capital D, as in Downstream Developer, Dunno or Captain D’s. If you don’t get a performance boost on your first few rolls don’t be discouraged. Just keep rolling until you do. Leave the discs at home and go for the bowling ball. Bowling is a numbers game. Ultimate is for hippies.