What’s mainly certain, right now, is that linguistic A.I. is changing the relationship between human beings and language. In an age of machine-generated text, terms like “writing,” “understanding,” “meaning,” and “thinking” need to be reconsidered. (View Highlight)
A.I. that can create and comprehend language carries the shock of a category violation; it allows machines to do what we thought only people could. (View Highlight)
For many years, A.I. researchers had experimented with a mechanism called attention, which they hoped might be capable of bridging the divide between efficiency and coherence. Attention allows a neural network to dodge sequentiality by seeking relevance. Instead of looking at each word in order, attention looks at all the words in a piece of text together, evaluating how they are interrelated and which are most important to each of the other words, as it captures the over-all meaning. This is closer to the way people remember a text than the way they read it. If you try to recall the opening paragraph of this article, you might articulate a vaguely connected constellation: Aidan Gomez, couldn’t drink, intern, Google, the uncertain potential of a new technology. Those terms, in any order, might amount to the sense you have retained. (View Highlight)
Una descripción clara y sencilla del mecanismo de “atención” que desarrollaron para incorporar en las redes neurales.