Emergent Adaptive Lexicons

Luc Steels 1996Artificial Intelligence Laboratory

Vrije Universiteit Brussel

Presented by Achim RuoppUniversity of Washington

Agenda

• Introduction to Emergence and Self-Organization

• Steel’s Experiments on the emergence of the lexicon

• Discussion

Excursion: Emergence andSelf-Organization

• A system exhibits emergence when– There are coherent

emergents at the macro-level

– That dynamically arise from interactions between parts a the micro-level

– Emergents are novel w.r.t. individual parts of the system

Excursion: Emergence and Self-Organization

• Definition– A dynamical and adaptive process– Where systems acquire and maintain structure themselves– Without external control

• Combining emergence and self-organization

Origins of Language

• Still unknown• Chomsky’s hypothesis

– Species-specific innate language ability– Refinement by parameter setting process– Some support by experimental simulation via neural

networks• Alternative hypothesis

– Language as an emergent phenomenon• As a mass-phenomenon• Spontaneously forming/becoming more complex

Steel’s Experiments

• Motivated by symbol grounding problem• Experiments on robotic and software

agents– Grounded meaning creation– Lexicon formation ← Focus of paper– Syntax– Emergent phonology

Experimental Model DefinitionsFeatures

• Set of agents A• ∀a A ∈ ∃ set of features Fa = {f0,…,fn}• A feature fi consists of attribute-value pairs

– Examples: (weight light) (size tall)• Distinctive feature set

– Subset of Fa distiguishing agent a from all agents in a group B

• Filtered subset CK,M = {a|K ⊂ Fa}– K is a set of features– M is a set of agents

Experimental Model DefinitionsLexicon

• Word– sequence of letters drawn from a shared alphabet

• Utterance– Set of words– Word order does not play a role

• Lexicon L– A relation between feature sets and words– A single word can have several associated feature

sets– A feature set can have several associated words

Experimental Model DefinitionsLexicon

• Each agent a has lexicon La

– Initially empty

• Feature set of a word: Fw,L

• Cover functions– cover(F,L): set of utterances U: ∀u ∈ U:

{f|f ∈ Fw,Land w ∈ u}– uncover(u,L): set of features F:

F = {f|f ∈ Fw,Land w ∈ u}

Coherence through Self-Organization

• Agents can– Create new words and associate them with a feature

set– Form new associations between a word and a feature

set• Key to self-organized coherence of the lexicon

– Agents participate in communication– Record the success of particular word-meaning pairs– Agents (re-)use words that led to high communication

success in the past

Language Game• Dialog between two agents – Speaker and Hearer• Dialog topic

– Other Agent– Chosen by extra-linguistic means (“pointing”)

• Speaker and hearer identify possible distinctive feature sets of topic

• Speaker– Selects distinctive feature set– Translates to words using cover function

• Hearer– Interprets utterance using uncover function– Compares interpretation to expectation– Uses game to

• Learn part of the language• Check if right meaning is associated with the right words

Language GamePossible Outcomes

1. No differentiation possible2. Speaker does not have a word

– May create new word

3. Hearer does not have a word– Can associate word– Cannot disambiguate when multiple

distinctive feature sets

Language GamePossible Outcomes

4. Speaker and hearer know word• Meanings are compatible with situation

• Sense-ambiguity possible• Meanings not compatible with situation

• No communicative success

Experimental ResultsOne-word Utterances

• Typical experimental setup (5 agents, 10 meanings, 4000 language games)

• Leads to communicative success soonAverage

communicative success

Number of language games(scale 1/20)

Experimental ResultsOne-word Utterances

• Single meanings soon converge on one word form (10 agents, 5 possible words, 1 meaning)

Average communicative

success

Time

Experimental ResultsMultiple Word Utterances

• In case the distinctive feature set of the topic contains multiple features

• Can be used by hearer to “guess” meaning of unknown words

Conclusions

• Self-organization is effective mechanism for achieving coherence

• Side-effects of lexicon formation– Synonymy– Ambiguity– Multiple-word sentences

Discussion• Supports the notion that absolute synonymy does not

exist– “After about 4000 language games the lexicon stabilizes as all

distinctions that need to be made have been lexicalized”• Are the presented “linguistic” results an artifact of the

experimental setup? I.e. in how far does this experiment reflect the real world?– E.g. multi-word sentences

• Can the results just be explained by basic communication theory?

• Language as an emergent phenomenon– Zipf’s law regarding multiple meanings– Self-organized criticality/highly optimized tolerance

fm

ReferencesBrighton, H., Selina, H.; Introducing Artificial Intelligence; 2003; Icon

Books Ltd.; ISBN 1-84046-463-1De Wolf, Tom; Holvoet, Tom; Emergence Versus Self-Organisation:

Different Concepts but Promising When Combined; 2005; In Engineering Self Organising Systems: Methodologies and Applications, Lecture Notes in Computer Science, volume 3464, pp 1-15

Steels, L.; Emergent Adaptive Lexicons; 1996; In: Maes, P. and Mataric, M.J. and Meyer, J.-A. and Pollack, J. and Wilson, S.W. (eds) From Animals To Animats 4: Proceedings of the Forth International Conference on Simulation of Adaptive Behavior, SAB'96, Complex Adaptive Systems, pp. 562-567, Cambridge, MA: The MIT Press

Zipf, G. K.; The Meaning-Frequency Relationship of Words; Journal of General Psychology 33, 251–256 (1945).