Charles Taylor - BiologyTravis CollierYoosook LeeYuan Yao

Ed Stabler - LinguisticsGreg Kobele Jason Riggle

Language and Biology Group

To/from humansTo/from humans

“Explicit” level:well-formed, explicit models

• Are there or were there objects there?

• What kind were they?

• How many were there?

• What did they do?

Requirements at the Implicit Stage

• Robust– changing environments/agents – Wrong information– noisy messages

• Adaptive– unanticipated sources, events– form new concepts– different languages

• Self-configuring– changing situations, goals

Outline

• Solution overview• Partial solution - Evolving language • Partial solution- Intrusion detection• Formal Analysis

• Expressing knowledge with logic• Creating and learning language syntax• Semantics

• Grounding problem• Passing D-structures

External WorldExternal World

Internal Internal RepresentationRepresentation

Logical Logical RepresentationRepresentation

Decisions about Decisions about what/whom to what/whom to communicatecommunicate

English-like English-like LanguageLanguage

HumansHumans

Agent Agent LanguageLanguage

Internal Internal RepresentationRepresentation

Logical Logical RepresentationRepresentation

Decisions about Decisions about what/whom to what/whom to communicatecommunicate

English-like English-like LanguageLanguage

Agent Agent LanguageLanguage

HumansHumans

Compression aids in generalization.

Compression distills experience into a schema or model

“This compressed form can be succinct, right, approximatelycorrect or even wrong, but it can be useful if it can be used to generalize to situations different from previously encountered”.

- Gell-Man

An example of compression:

y = mx + b

Regular Language(Q, , , qo,F)

Q = set of all states (finite)

input alphabet (finite)

qo = initial state

F = set of final states

= transition function (“rewrite rules”)

(Q x ) Q

Example: Rewrite Grammar

S S NP VP NP VPNP NP D N D NNP NP D DVP VP V VVP VP V NP V NPV V loves lovesV V eats eatsD D David DavidD D Mary MaryN N dog dogD D the the

SS

NP VPNP VP

D V NPD V NP

Mary loves DMary loves D

DavidDavid

Minimum Description Length (MDL) Algorithm

Grammar-encoding-length (GEL)the cost of the generalization

Data-encoding-length (DEL)the cost of the compression

MDL-length = grammar-encoding-length + data-encoding-length

- Rissanen & Ristad

Principle of Compression

S2

S3

S S1

S4

Mary likes

Jane

Amy

Caitlin

S5

S6

S2S S1Mary likes

JaneAmyCaitlin

Combine grammatical equivalents

Evolution of Languages

0

50

100

150

200

250

300

350

400

450

1 2 3 4 5 6 7 8 9 10

Generations

Distance 1Distance 2

3) Languages become more smooth?

Evolution of Language with semantics Evolution of Language with semantics (Kirby)(Kirby)

Loves (John, Mary) xxy zzy rrxLoves (John, Mary) xxy zzy rrxLoves (Bill, Mary) xxy aab rrxLoves (Bill, Mary) xxy aab rrxHits (Bill, John) mmn aab zzyHits (Bill, John) mmn aab zzy

aab Billaab Billzzy Johnzzy Johnrrx Maryrrx Maryxxy Lovesxxy Lovesetc.etc.

External WorldExternal World

Internal Internal RepresentationRepresentation

Logical Logical RepresentationRepresentation

Decisions about Decisions about what/whom to what/whom to communicatecommunicate

English-like English-like LanguageLanguage

HumansHumans

Agent Agent LanguageLanguage

Internal Internal RepresentationRepresentation

Logical Logical RepresentationRepresentation

Decisions about Decisions about what/whom to what/whom to communicatecommunicate

English-like English-like LanguageLanguage

Agent Agent LanguageLanguage

HumansHumans

Intrusion Detection Methods

• Specification-based methods(x)[write(x, kernel)]

• Pattern Matching– signature of “red code” worm– (could be specification-based - buffer overflow)

• Anomaly Detection– Scan many ports in short time– analogous to parts of our problem– unanticipated changes in the system

Local Internal Events

• start (Subject Program EventNo Tstamp)• chmod (Subject File Fpermissions EventNo

Tstamp)• open (Subject File Mode EventNo Tstamp)• exec (Subject File Mode EventNo Tstamp)• fork (Subject NewPID EventNo Tstamp)

External WorldExternal World

Internal Internal RepresentationRepresentation

Logical Logical RepresentationRepresentation

Decisions about Decisions about what/whom to what/whom to communicatecommunicate

English-like English-like LanguageLanguage

HumansHumans

1. Trace of activity1. Trace of activity

Computer -linuxComputer -linux

2. C++ objects 2. C++ objects - each file - each file -each process-each process

3. Prolog Environment -3. Prolog Environment -only “interesting” parts,only “interesting” parts,

innate, human told, deducedinnate, human told, deduced